OUP user menu

A UK NEQAS ISH Multicenter Ring Study Using the Ventana HER2 Dual-Color ISH Assay

J.M.S. Bartlett PhD, FRCPath, Fiona M. Campbell MSc, Merdol Ibrahim PhD, Anthony O’Grady PhD, Elaine Kay MD, Catherine Faulkes MSc, Nadine Collins, Jane Starczynski PhD, John M. Morgan PhD, Bharat Jasani PhD, FRCPath, Keith Miller FIBMS
DOI: http://dx.doi.org/10.1309/AJCPVPRKK1ENEDGQ 157-162 First published online: 1 January 2011

Abstract

We performed a multicenter assessment of a new HER2 dual-color chromogenic in situ hybridization (CISH) test and herein report on concordance of CISH data with fluorescence in situ hybridization (FISH) data and intraobserver and interlaboratory scoring consistency. HER2 results were evaluated using duplicate cores from 30 breast cancers in 5 laboratories using the Ventana HER2 dual-color ISH assay (Ventana Medical Systems, Cambridgeshire, England) and in 1 central laboratory using a standard FISH assay.

Overall 93.3% of cases were successfully analyzed by CISH across the 5 participating laboratories. There was excellent concordance (98.0% overall) for diagnosis of HER2 amplification by CISH compared with FISH. Intraobserver variability (7.7%) and intersite variability (9.1%) of absolute HER2/chromosome enumeration probe 17 ratios were tightly controlled across all participating laboratories. The Ventana HER2 dual-color ISH assay is robust and reproducible, shows good concordance with a standard FISH assay, and complies with requirements in national and international guidelines for performance of ISH-based diagnostic tests.

Key Words:
  • HER2
  • Amplification
  • Breast neoplasms
  • Chromogenic in situ hybridization
  • CISH
  • Fluorescence in situ hybridization
  • FISH
  • Consistency
  • Intraobserver
  • Intersite
  • NEQAS

Guidelines for breast cancer management require all patients with breast cancer to be tested for HER2 status at initial diagnosis or at the time of recurrence.1,2 National and international guidelines and external quality assurance schemes aim to ensure accurate and robust diagnostic testing of HER2 expression and amplification required to support treatment decisions.15 Establishing tumor HER2 status is essential for predicting responses to trastuzumab (Herceptin)15 and may inform the treatment choice of endocrine agents or taxanes.24,6

Amplification of the HER2 gene drives overexpression of the oncoprotein,7,8 and in situ hybridization (ISH) is an essential component of HER2 testing in most countries.1,2,912 ISH tests measure HER2 copy number, with or without the chromosome enumeration probe (CEP) 17 number, using fluorescence in situ hybridization (FISH) or chromogenic in situ hybridization (CISH) detection methods, including silver staining (also known as silver ISH, or SISH).1,13 FISH is currently regarded as the most accurate, reproducible, and precise predictor of HER2 overexpression.5,7,8 The PathVysion System (Abbott UK, Kent, England), which comprises 2 fluorescently labeled probes for detection of the HER2 gene and chromosome 17 CEP, is approved by the US Food and Drug Administration and represents the most widely used FISH test in the United Kingdom.1,2,5 However, a number of alternative probes and systems are also available for the detection of HER2 gene amplification. Diagnostic HER2 assay systems must be accurate, reliable, and reproducible across multiple sites. The quality of technical interpretation relevant to routine diagnostic testing is monitored on a quarterly basis by the United Kingdom National External Quality Assessment Scheme for IHC and ISH (UK NEQAS ICC and ISH).5 For ISH, data returned by the participating laboratories are scored against data on sequential sections produced by the UK NEQAS ISH reference laboratories.

The aim of this study was to perform a robust assessment of a new 2-color automated CISH assay, the Ventana HER2 dual-color ISH assay (Ventana Medical Systems, Cambridgeshire, England). We previously reported evaluation of the Ventana INFORM HER2 SISH assay (Ventana Medical Systems), a bright-field fully automated ISH assay performed in approximately 6 hours, which requires staining of separate slides for HER2 and chromosome 17 CEP.14 The technology has been developed further and recently released as the HER2 dual-color ISH assay, which permits bright-field detection of HER2 and chromosome 17 CEP on a single slide. We conducted a multicenter evaluation of 30 breast cancers using this CISH assay and compared the results with those obtained using a FISH assay in a central reference laboratory. We provide data demonstrating that the CISH assay results are equivalent to FISH results and that the assay can be run reliably on the BenchMark series of instruments (Ventana Medical Systems) in line with the quality requirements outlined by the American Society of Clinical Oncology (ASCO)/College of American Pathologists (CAP) and UK HER2 testing guidelines.1,2 We document concordance of CISH data from each laboratory with FISH data from the central reference laboratory in addition to intraobserver and interlaboratory scoring consistency.

Materials and Methods

Study Design

The concordance of HER2 determination by CISH using the Ventana HER2 dual-color ISH assay and by FISH, an established method used routinely for clinical diagnosis,1,9 was determined on the basis of intrasite variation between the 2 assays at a UK NEQAS ISH reference laboratory. In addition, the interlaboratory reproducibility of the HER2 dual-color ISH method was determined across 5 laboratories. A commercially available tissue microarray (TMA; Stretton Scientific, Derbyshire, England) containing 2 replicate cores from 30 breast cancers was circulated to 5 UK NEQAS ISH reference laboratories (randomly numbered 1–5). Each laboratory received the same material and performed independent blinded analysis of HER2 and chromosome 17 CEP on the same slide using the HER2 dual-color ISH assay. In addition, laboratory 1 performed an independent blinded analysis of HER2 and chromosome 17 CEP using FISH.

Determination of HER2 and Chromosome 17 CEP by CISH and FISH

HER2 and chromosome 17 CEP were determined on the same slide by bright-field automated CISH using the Ventana HER2 dual-color ISH assay. The same batch of reagents was used in all participating laboratories. Automated staining was performed using the Ventana BenchMark XT (with reagents [see below] from Ventana Medical Systems as part of the ISH assay), which was installed and validated in all sites, and all staff received appropriate training in assay performance and analysis before the commencement of the study. The assay protocol consisted of extended pretreatment with CC2, pH 6.0, followed by protein digestion with ISH-protease 3 for 4 minutes for the xenograft controls and 12 minutes for the test TMA slides. Initial validation at 1 center demonstrated more consistent staining for HER2 when the digestion time for the test TMA slide was reduced to 8 minutes with ISH-protease 3, whereas the remaining laboratories used the recommended protocol of 12 minutes. This was followed by incubation with the specific dinitrophenyl-labeled DNA probes for 6 hours. Detection was performed with the ultraView SISH Detection Kit and accessory reagents. This consisted of, briefly, incubation with 2 consecutive antibodies followed by the addition of 3 sequential silver reagents. The slides were then incubated with Red ISH V-Probe for 2 hours, followed by addition of more primary antibody and subsequent Red ISH detection reagents. Silver precipitation is deposited in the nuclei, and single copies of the HER2 gene are visualized as single black dots and single copies of chromosome 17 CEP as red dots on the same slide. The slides were then counterstained using Haematoxylin II and a bluing reagent. The numbers of chromosome 17 CEP and HER2 signals were counted in 20 or 40 nonoverlapping nuclei per core.

HER2 and chromosome 17 CEP were determined in a single central laboratory by dual color FISH (PathVysion) using the UK NEQAS scoring guidelines. FISH-stained TMA sections were analyzed at ×1,000 and CISH at ×630 to ×1,000 magnification, and areas of carcinoma within each core were identified. The numbers of chromosome 17 CEP and HER2 signals were counted in 20 nonoverlapping nuclei per core.

The mean HER2/chromosome 17 CEP ratio was calculated per core, and the mean HER2 and mean chromosome 17 CEP number observed were recorded on a core-by-core basis.

Analysis of Results

All data reported were collated centrally and analyzed in the Edinburgh (Scotland) reference laboratory. All 5 participating laboratories obtained satisfactory data from the CISH assay (Ventana HER2 dual-color ISH).

The UK HER2 testing guidelines5 recommend scoring 20 cells per case; however, the protocol for the Ventana HER2 dual-color ISH assay requires that 40 cells be scored for each case. Therefore, we compared the absolute CISH scores obtained from scoring 20 cells with 40 cells in each core across 3 centers.

The success rate, defined as the percentage of cases tested that could be assessed on the TMAs, for the determination of HER2 using the CISH assay was determined for each laboratory by case and by core. The success rate for the determination of HER2 using the FISH assay was also determined for the central reference laboratory (laboratory 1) by case and by core. All data are reported as HER2/chromosome 17 CEP ratios. To evaluate the concordance between FISH and CISH, a regression analysis was performed on CISH results from each site and FISH results from laboratory 1. The correlation of absolute CISH scores from each participating laboratory and FISH scores (all core data) from laboratory 1 was evaluated by regression analysis.

The intraobserver (intrasite) variation at each laboratory was analyzed by determining the variation between duplicate cores for each case and was compared with data obtained centrally for FISH testing. The percentage of intraobserver variation was calculated as the absolute difference between results for duplicate cores divided by the mean value for both cores. Although providing a good estimate of “intraobserver” variation, this measure is compounded by core-to-core variation, and results should be interpreted with this caveat. The intersite variation between each of the individual sites performing the CISH assay was determined and expressed as mean intersite variation for each result reported and the percentage variation. For this calculation, the absolute difference between sites for each core result (in a pairwise comparison) was divided by the mean value for all cores from all sites. An additional regression analysis, using the central laboratory (laboratory 1) as the comparator, investigated the correlation of absolute CISH scores (all core data) between laboratory 1 and other participating laboratories.

Results

Number of Cells Scored in the Ventana HER2 Dual-Color ISH Assay: 20 vs 40

Figure 1 shows a comparison of the absolute CISH scores obtained from scoring 20 cells vs 40 cells. These results, from a total of 180 cores across 3 centers (60 cores per center), show an excellent correlation (R = 0.9906) between absolute CISH scores, even for borderline cases. No significant difference was observed between results for counting 20 vs 40 cells. All subsequent analyses were performed using data from the first 20 cells scored.

Ventana HER2 Dual-Color ISH Assay and Concordance With the Standard FISH Assay

Table 1 shows the rate of successful HER2 determination in the same TMA using the Ventana HER2 dual-color ISH assay across 5 UK NEQAS reference laboratories, compared with the FISH assay in reference laboratory 1. All 5 laboratories obtained satisfactory data from the CISH assay (Ventana HER2 dual-color ISH). Each laboratory determined 30 cases using the CISH assay, and HER2 was successfully determined in 83% to 100% of cases. The overall success rate from 150 cases was 93.3%. Similarly, HER2 was successfully determined by CISH in 67% to 93% of the 60 duplicate cores scored by each laboratory, with an overall success rate of 84.7%. The central laboratory (laboratory 1) also determined HER2 by FISH in the same 30 cases and 60 cores, with a success rate of 93% of cases (n = 28) and 87% of cores (n = 52).

Figure 1

Correlation of absolute chromogenic in situ hybridization (CISH) scores obtained from scoring 20 cells versus 40 cells. The absolute CISH scores obtained from scoring 20 cells from 180 cores across 3 centers were compared with the scores obtained from scoring 40 cells from the same slides. y = 0.9587x + 0.1629; R 2 = 0.9813.

Figure 2 and Table 2 show an evaluation of concordance of the Ventana HER2 dual-color ISH assay with the standard PathVysion FISH assay. Examples of the correlation of absolute CISH scores (all core data) between individual laboratories compared with absolute FISH scores (all core data) from the central laboratory (laboratory 1) are shown in Figure 2. The absolute CISH scores obtained in laboratory 3 were consistent and similar to the scores obtained using the FISH assay in the central laboratory (slope, 1.18; intercept, 0.20; and R = 0.91), and there were no discordant results between the 2 assays (Figure 2A). Figure 2B shows the worst case, laboratory 4, which had 3 discordant results between the 2 assays; however, all of these results were borderline by CISH and FISH (1.53 vs 2.00, 2.05 vs 1.74, and 1.93 vs 2.00), and the overall correlation was adequate (slope, 0.80; intercept, 0.39; and R = 0.92). The slope, intercept, and R values for comparisons between all CISH scores from individual laboratories and FISH scores from the central laboratory (laboratory 1) are shown in Table 2, which also includes the concordance for diagnosis by CISH between each of these laboratories and diagnosis by FISH in laboratory 1. Concordance for the diagnosis of HER2 amplification was excellent: The mean overall concordance was 98.0% (96.7%–100%) using CISH results from all 5 laboratories compared with FISH results from laboratory 1.

View this table:
Table 1
Figure 2

Correlation of absolute chromogenic in situ hybridization (CISH) scores from each laboratory with fluorescence in situ hybridization (FISH) scores obtained from the central laboratory. A, Comparison of laboratory 3 CISH scores with central laboratory (laboratory 1) FISH scores. y = 1.1801x + 0.1969; R 2 = 0.8221. B, Comparison of laboratory 4 CISH scores with central laboratory FISH scores. y = 0.7991x + 0.3919; R 2 = 0.8524. The absolute CISH scores from all core data obtained from each laboratory were compared with FISH scores from all core data obtained from the central laboratory. The points represented by gray squares are discordant results between the CISH and FISH assays. The slope, intercept, and R values were obtained from each plot and are summarized in Table 2.

View this table:
Table 2

Intrasite (Intraobserver) and Intersite Variation in HER2 Testing Using the Ventana HER2 Dual-Color ISH Assay

Intraobserver variation was calculated based on duplicate analysis of both cores for each of the 30 cases. Table 3 shows the mean intraobserver variation for HER2 copy, chromosome 17 CEP, and HER2/chromosome 17 CEP ratio determined by CISH in each center and the data from the FISH assay in laboratory 1. The mean intraobserver variation values for all CISH results were 7.6% for HER2, 7.2% for chromosome 17, and 7.7% for the HER2/chromosome 17 CEP ratio.

The intersite variation represents the compound of technical and observer variation between pairs of sites and observers. Table 4 shows intersite variation for the CISH assay across all participating laboratories. There were no significant differences, and the overall intersite variation (mean ± SD) between laboratories was 9.05% ± 1.17%.

Table 5 shows the correlation of absolute CISH scores (all core data) between individual laboratories and laboratory 1 as the comparator. All laboratories showed satisfactory correlation between CISH results with those obtained from laboratory 1.

Discussion

The rate of successful HER2 determination using the Ventana HER2 dual-color ISH assay in the same TMA construct was consistent with an overall success rate of 93.3% (83%–100%) of cases from the 5 UK NEQAS reference laboratories that participated in this UK NEQAS ISH multicenter “ring” study. This level of performance is excellent for single TMAs, where some fallout is expected owing to the need for a single digestion method for all tissue samples.15 This success rate is consistent with the success rate of 94.8% (89%–100%) previously recorded in a similar 6-center ring study evaluating performance of the Ventana INFORM HER2 SISH assay.14 The overall success rate of 93.3% of cases across all laboratories for HER2 determination using CISH was the same as the success rate of 93.3% of cases for HER2 determination using the standard FISH method in laboratory 1.

There was excellent concordance for the diagnosis of HER2 amplification by CISH compared with FISH. Overall, 98.0% (96.7%–100%) of cases were concordant for the diagnosis of HER2 amplification using CISH results from all 5 laboratories compared with FISH results from the central laboratory. According to the ASCO/CAP guidelines, more than 95% concordance should be achieved to validate novel FISH or immunohistochemical procedures.2 In 7 of 9 discordant cores, the FISH or the CISH results were within the range of 1.80 to 2.20, regarded as “equivocal” by the ASCO/CAP guidelines. Nevertheless, for these 7 cores, the result for FISH or CISH was discordant between amplified and nonamplified cases.2 In the United Kingdom,1 such cases would be revisited with additional cells being counted before a definitive diagnostic report is issued. We therefore did not exclude these cases before analyses of concordance rates between these 2 assays. The excellent level of concordance between FISH and CISH demonstrated in this research study suggests that the Ventana HER2 dual-color ISH assay is robust and provides consistent results across all participating laboratories and that all participating laboratories satisfy the UK and ASCO/CAP guidelines for the validation of novel ISH procedures.5 The 98.0% concordance demonstrated in the present study is comparable to the 96.0% concordance previously recorded in a similar 6-center ring study evaluating performance of the Ventana INFORM HER2 SISH assay.14

When using the Ventana HER2 dual-color ISH assay, intraobserver and intersite variability of absolute HER2/chromosome 17 CEP ratios appears to be tightly controlled across all participating laboratories. The level of intraobserver variability for the CISH assay was consistent across all laboratories, with overall means of 7.6% for HER2, 7.2% for chromosome 17 CEP, and 7.7% for HER2/chromosome 17 CEP ratio. This level of intraobserver variability is similar to the level of 7.8% for HER2/chromosome 17 CEP ratio that we reported in a previous 6-center ring study evaluating performance of the Ventana INFORM HER2 SISH assay.14 The level of intraobserver variability observed with the CISH assay is lower than previously reported interobserver variation for FISH.1619

View this table:
Table 3
View this table:
Table 4
View this table:
Table 5

Site-to-site variation represents a compound of interobserver variation owing to differences in scoring and technical variation between sites. The overall intersite variation between all laboratories of 9.05% ± 1.17% is consistent with that reported in previous research studies (approximately 10%).14,1619 There were no significant differences in intersite variation between any laboratories. Core scoring data from all laboratories were consistent with data obtained in laboratory 1, as demonstrated by analyses of the correlation of absolute CISH scores (all core data) between individual laboratories and laboratory 1 as the comparator. Overall, all laboratories showed excellent performance for diagnostic accuracy and with respect to site-to-site variation.

The Ventana HER2 dual-color ISH assay is robust, provides highly consistent results across all participating UK NEQAS reference laboratories, and complies with requirements in national guidelines for the performance of diagnostic tests. Furthermore, concordance for the diagnosis of HER2 amplification by CISH compared with FISH was excellent. The results of this UK NEQAS ring study support use of the Ventana HER2 dual-color ISH assay as an acceptable alternative for analysis and reporting of HER2 gene status of patients in routine practice. This study has shown close agreement among laboratories using CISH, with excellent intraobserver and intersite consistency in diagnostic ratios for HER2 gene amplification. The high level of consistency underlines the high quality of HER2 testing achievable and demonstrates the potential for extremely robust and quantitatively reproducible CISH in routine practice. Clearly, continued quality assessment is essential to continued good performance.

Footnotes

  • Supported by Roche Diagnostics (West Sussex, England). J. Merritt, PhD (Merritt Science, St Albans, England), a professional medical writer, drafted the manuscript and was paid by Roche Diagnostics.

References

  1. 1.
  2. 2.
  3. 3.
  4. 4.
  5. 5.
  6. 6.
  7. 7.
  8. 8.
  9. 9.
  10. 10.
  11. 11.
  12. 12.
  13. 13.
  14. 14.
  15. 15.
  16. 16.
  17. 17.
  18. 18.
  19. 19.
View Abstract