Clinical Assessment – Part II William Sacks, PhD, MD

COMPUTERIZED THERMAL IMAGING, INC. Breast Cancer System BCS 2100P010035 Radiological Devices Advisory Panel December 10, 2002 Clinical Assessment – Part II William Sacks, PhD, MD Sacks - Clinical Assessment

Safety of the BCS has two aspects: Adverse events Accuracy of BCS output Sacks - Clinical Assessment

There were only 4 minor adverse events, two of which were thought not to be device related, during a clinical trial involving 2407 subjects. Sacks - Clinical Assessment

From the point of view of the BCS output: Safety is more closely related to sensitivity (cancers), or FNR, but also FPR Effectiveness is more closely related to specificity (benign masses) Sacks - Clinical Assessment

POINTS TO BE COVERED What BCS 2100 (BCS) is and is not intended to do How BCS does it What the clinical trial demonstrated Labeling issues Sacks - Clinical Assessment

CLINICAL SUBMISSIONS PMA Amendment 4 Amendment 5 Amendment 7 Sacks - Clinical Assessment

After reviewing the PMA, the FDA sent a letter to CTI listing a number of deficiencies. CTI’s response was Amendment 4. Sacks - Clinical Assessment

In Amendment 4, for their conclusions concerning BCS effectiveness (p. 11), CTIretrospectively selected from the PMA data one of two analytical indices (Se/Sp, not ROC) and two of three lesion types (masses, and arch. dist., not microcalcifications). Sacks - Clinical Assessment

In Amendment 4 the revised labeling further deleted architectural distortion, and referred to masses alone. Sacks - Clinical Assessment

The FDA sent another deficiency letter to CTI. The response was Amendment 5. Sacks - Clinical Assessment

Amendment 5 was offered as a test of BCS in additional subjects, because Amendment 4 had contained retrospective selections. CTI called the additional dataset PPMA (Post-PMA). Sacks - Clinical Assessment

Amendment 5 confined its analysis of the PPMA data to the newly chosen analytical index (Se/Sp) in the newly chosen subgroup (masses). Sacks - Clinical Assessment

In addition to presenting data on a new set of subjects (PPMA), Amendment 5 also contained analysis of the combined datasets from Amendment 4 and the PPMA. Sacks - Clinical Assessment

Because of the retrospective selections in Amendment 4, FDA asked CTI to justify combining that data with the PPMA. CTI’s response was Amendment 7. Sacks - Clinical Assessment

In Amendment 7 CTI applied a Bonferroni correction in an attempt to compensate for retrospective selection of data in Amendment 4 and the smallness of the additional (PPMA) sample in Amendment 5. Sacks - Clinical Assessment

Two overriding issues: Adequacy of data Interpretation of data, i.e., do the data demonstrate S&E of BCS? Sacks - Clinical Assessment

Adequacy of data Can data from Amendment 4 contribute to judgment of S&E, when it consists of retrospective selection of mammographic masses, because the Se was too low in microcalcifications and in the tested population as a whole? Sacks - Clinical Assessment

Adequacy of data (cont.) Is a Bonferroni correction applicable in this context? Sacks - Clinical Assessment

Adequacy of data (cont.) Are the data from the PPMA alone (in Amendment 5) adequate to judge S&E? Sacks - Clinical Assessment

Interpretation of the data It is noteworthy for the following discussion that no formal hypotheses were explicitly put forward for testing, either in the PMA or in the subsequent amendments. Sacks - Clinical Assessment

To qualify as a testable hypothesis there must be a quantitative criterion, whereby either a point estimate may imply rejection, or a confidence interval may entail exclusion. Sacks - Clinical Assessment

There were two implicit hypotheses: that the ROC area for BCS and mammography combined would exceed that of mammography alone (with statistical significance) [Protocol, Section 6.0 Statistical Analysis] that the point estimate for sensitivity would be at least 99.3% (in at least 75% of simulations with the data) [PMA, Section 5.8.7.2, p. 489]. Sacks - Clinical Assessment

The protocol otherwise contained only non-quantitative statements of what CTI hoped to achieve in the clinical trial. For example, The objective of the study is to determine if the (BCS 2100), when used in conjunction with clinical examination and/or diagnostic mammography, increases the ability of physicians to differentiate benign from malignant, or suspicious, breast abnormalities. [Protocol, Section 3.1 Study Objective] Sacks - Clinical Assessment

In the original PMA submission, the comparison of ROC areas failed to achieve statistical significance, except as an artifact of too few points in the mammography-alone curve, so it was not pursued in any of the amendments. Sacks - Clinical Assessment

In addition, the sensitivities failed to achieve a level of 99.3% with 75% confidence, in any of the datasets. Sacks - Clinical Assessment

Mammography alone PMA (n=187) (Se 97.1%, Sp 14.3%) PPMA (n=15) (Se 93.3%, Sp 25.4%) Combined (n=105) (Se 99.0%, Sp 19.2%) Am. 4 (n=90) (Se 100%, Sp 18.0%) 100% 99.7% 98.8% Reference Se 99.3% 96.7% 95.6% 94.1% Chance Line SENSITIVITY SPECIFICITY 0% 72.1% Sacks - Clinical Assessment

Potential safety and effectiveness in U.S. population: Sacks - Clinical Assessment

Percent of U.S. biopsies potentially obviated by the BCS if used on all eligible women: 1.3 million U.S. women biopsied each year, of which ~45% (585,000) have mammographic masses, of which ~80% (468,000) are benign, of which 15% to 20% (70,000 to 94,000) would be BCS(-) and saved a biopsy. Thus ~5% to 7% of 1.3 million U.S. biopsies would be obviated, and that’s if BCS were used on all 585,000 women with mammographic masses on their way to biopsy. In addition to saving biopsies on these benign masses, approximately 1% to 6% of the malignant masses, and 0.5% to 3% of all breast cancers, might be delayed in diagnosis. ( Sacks - Clinical Assessment

POINTS TO BE COVERED What BCS 2100 (BCS) is and is not intended to do How BCS does it What the clinical trial demonstrated Labeling issues Sacks - Clinical Assessment

Labeling issues Size of mass Depth of mass Sacks - Clinical Assessment

Size of mass Effect of small lesion size on device sensitivity difficult to evaluate, since only 2/105 cancers < 5mm. Sacks - Clinical Assessment

Size of mass (cont.) (Amendment 4, p.8, Table A4.4, Amendment 5, p.23, Table A5.14) Sacks - Clinical Assessment

Depth of mass With the chosen IOS threshold (20.59) there was no definite effect of lesion depth on BCS result, but... Sacks - Clinical Assessment

Depth of mass (cont.) ...the effect of lesion depth was difficult to evaluate, since depth is not easily gauged on mammography. Sacks - Clinical Assessment

Conclusions Sacks - Clinical Assessment

In summary Only 4 out of 2407 subjects had an adverse event, all minor. There were no explicit quantitative hypotheses. There were two implicit quantitative hypotheses. Neither hypothesis was fulfilled. Most data were selected retrospectively. Bonferroni correction is not applicable in this context. Using the trial results, if the BCS were in general use in the U.S., it would obviate 5% to 7% of the 1.3 million biopsies per year. Approximately 1% to 6% of these obviated biopsies would turn out to be malignant, and their diagnoses would thus be delayed. Sacks - Clinical Assessment

Clinical Assessment – Part II William Sacks, PhD, MD