Content on this page requires a newer version of Adobe Flash Player.

Get Adobe Flash player

 

 

 

 

 

 

ALL RESULTS DESCRIBED HERE ARE REVIEWED, INSPECTED AND THEN APPROVED BY THE FDA

Clinical Trial Studies


Overview

The Validation Study period was for approximately 3 years, from late 2006 – late 2009. The initial IRB approval for the validation dataset was received in late 2006 and the study began shortly thereafter. Annual IRB reports were made to document study progress, and the IRB granted continued approval through the study period.

Standalone performance testing was designed to demonstrate how the Breast Companion® CADx Computer-Aided Lesion Assessment ("BC CLA") scoring algorithm performed on the lesions that are outlined (segmented) by the radiologists compared to the confirmed truth of the cases. ROC-AUC, Sensitivity and Specificity were the endpoints of this evaluation.

The diagnostic Sensitivity of the BC is defined as the conditional probability that a person having a disease will be correctly identified: TP/(TP+FN). The diagnostic Specificity of the BC is defined as the conditional probability that a person not having a disease will be correctly identified: TN/(TN+FP). True Positive, True Negative, False Positive, and False Negative definitions are used in accordance with traditional references.

The Validation Case Files cohort was used to produce BC CLA scores that were consequently analyzed with ROC analysis. The lesions assessed by the radiologist were scored using the Reference Library cases in batch processing using the radiologist’s segmentation but without other radiologist interference in the CLA scoring. As a result, BC CLA scores were produced and compared to the confirmed truth of each case. The results are below.


Standalone BC CLA Performance

Standalone performance testing was designed to demonstrate how the BC CLA scoring algorithm performed on the lesions that are outlined (segmented) by the radiologists compared to the confirmed truth of the cases. ROC-AUC, Sensitivity and Specificity were the endpoints of this evaluation.

The diagnostic Sensitivity of the BC is defined as the conditional probability that a person having a disease will be correctly identified: TP/(TP+FN). The diagnostic Specificity of the BC is defined as the conditional probability that a person not having a disease will be correctly identified: TN/(TN+FP). True Positive, True Negative, False Positive, and False Negative definitions are used in accordance with traditional references.

The Validation Case Files cohort was used to produce BC CLA scores that were consequently analyzed with ROC analysis. The Validation Dataset lesions assessed by the radiologist were scored using the Reference Library cases in batch processing using the radiologist’s segmentation but without other radiologist interference in the CLA scoring. As a result, BC CLA scores were produced and compared to the confirmed truth of each case.

Table below presents the ROC AUC analysis of the standalone performance of the BC CLA scoring function.

Test

Area (Az)

(fitted)

Area (Az)

(empirical)

95% CI

SE

Z

p

BC CLA

98.6%

98.2%

0.97 to 0.99

0.006

82.81

<0.0001

 

 

ROC AUC (Az) analysis of standalone performance (BC CLA)

 

 
   
 
 

For AUC ROC, BC CLA reached 98.2% for empirical and 98.6% for fitted method. At traditional BI-RADS Category 3, which corresponds to BC CLA 3 score, Sensitivity reached 95.5% with Specificity is 94.6% while the most effective BC CLA performance was found at 2.5 threshold level.

BC CLA does not produce the numeric estimation on the same scale as radiologists do when they go through the BI-RADS Assessment process and summarize their assessment in selection of one of the BI-RADS Categories. Therefore to illustrate the comparative performance of BC CLA, Table below summarizes the fitted ROC computations (JROCKFIT software, method developed by professor Metz of Chicago University and used in their clinical trial at John Hopkins) based on the “without” reading data of the validation group of radiologists.

596 cases

RADs Average

BC CLA

ROC AUC (Az, fitted)

83.1%

98.7%

Sensitivity

98.7%

98.5%

Specificity

41.5%

89.0%

 

 

Area (Az), Sensitivity and Specificity comparison of BC CLA with FROC results of 4 radiologists reading validation data set of Clinical Trial cases “without” the device
 

 

Clinical Validation Reading Results

 

Summarized ROC analysis results for the validation cohort of the Clinical Trial cases are presented in the following Table. 

 

In addition to effectiveness estimation based on ROC AUC, a traditional accuracy index based on TP, TN, FP and FN differences between “without” and “with” was also computed. Traditional estimation could be computed by using Accuracy index Ac= (TP+TN)/(TP+TN+FP+FN). Using the statistically significant input from the readings ROC results the Ac increase “with” compared to radiologist performance “without” were computed as +6.51% on average.

 

Back to top


Website is designed and developed by Top Apprentice.