OUP user menu

Chromogenic In Situ Hybridization
A Multicenter Study Comparing Silver In Situ Hybridization With FISH

J.M.S. Bartlett PhD, FRCPath, Fiona M. Campbell MSc, Merdol Ibrahim PhD, Peter Wencyk, Ian Ellis MD, Elaine Kay, Yvonne Connolly MSc, Anthony O’Grady PhD, Silvana Di Palma MD, Jane Starczynski PhD, John M. Morgan PhD, Bharat Jasani PhD, FRCPath, Keith Miller FIBMS
DOI: http://dx.doi.org/10.1309/AJCPXY3MJ6GSRCYP 514-520 First published online: 1 October 2009

Abstract

Our purposes were to perform a robust assessment of a new HER2 chromogenic in situ hybridization test and report on concordance of silver in situ hybridization (SISH) data with fluorescence in situ hybridization (FISH) data and on intraobserver and interlaboratory scoring consistency. HER2 results were scored from 45 breast cancers in 7 laboratories using the Ventana (Tucson, AZ) INFORM HER-2 SISH assay and in 1 central laboratory using a standard FISH assay. Overall, 94.8% of cases were successfully analyzed by SISH across the 6 participating laboratories that reported data. Concordance for diagnosis of HER2 amplification by SISH compared with FISH was high (96.0% overall). Intraobserver variability (8.0%) and intersite variability (12.66%) of absolute HER2/chromosome 17 ratios appear to be tightly controlled across all 6 participating laboratories. The Ventana INFORM HER-2 SISH assay is robust and reproducible, shows good concordance with a standard FISH assay, and complies with requirements in national guidelines for performance of diagnostic tests.

Key Words:
  • HER2
  • Amplification
  • Breast
  • Chromogenic in situ hybridization
  • Silver in situ hybridization
  • Fluorescence in situ hybridization
  • Consistency
  • Intraobserver
  • Intersite
  • UK National External Quality Assessment Scheme

According to guidelines for breast cancer management, all patients with breast cancer must be tested for HER2 status at initial diagnosis or at the time of recurrence.1,2 Establishing tumor HER2 status supports treatment decisions by predicting responses to trastuzumab (Herceptin)15 and other drugs, including tamoxifen, taxanes, and anthracyclines.24,6 Accurate and robust diagnostic testing of HER2 expression or amplification is supported through additional guidelines and external quality assurance schemes.15

Amplification of the HER2 gene drives overexpression of the oncoprotein,7,8 and in situ hybridization (ISH) is an essential component of HER2 testing in most countries.1,2,912 ISH tests measure HER2 with or without chromosome 17 copy number by using fluorescence in situ hybridization (FISH) or chromogenic in situ hybridization (CISH) detection methods, including silver staining (also known as silver ISH [SISH]).1,13 Currently, FISH is regarded as the most accurate, reproducible, and precise predictor of HER2 overexpression.5,7,8 The PathVysion system (Abbott UK, Kent, England) comprises 2 fluorescently labeled probes for detection of the HER2 gene and chromosome 17. This HER2 test is approved by the US Food and Drug Administration and represents the most widely used FISH test in the United Kingdom.1,2,5 However, a number of alternative probes and systems are also available for the detection of HER2 gene amplification. HER2 assay systems must be reliable and reproducible across multiple sites if they are to be applied within routine diagnostic laboratories for HER2 testing. The UK National External Quality Assessment Scheme (UK NEQAS) for ISH (UK NEQAS ISH)5 monitors the quality of technical interpretation relevant to routine diagnostic testing on a quarterly basis; data returned by the participating laboratories are scored against data on sequential sections produced by the UK NEQAS ISH reference laboratories.

The aim of this study was to perform a robust assessment of a new CISH assay, the Ventana INFORM HER-2 SISH assay (Ventana, West Sussex, England), which detects HER2 and chromosome 17 copy number. The technology has been developed and recently released as an alternative bright-field fully automated ISH assay, which is performed in approximately 6 hours. We provide data demonstrating that the SISH assay results are equivalent to FISH and that the assay can be run reliably on the BenchMark series of instruments (Ventana, Tucson, AZ), in line with the quality requirements outlined by the UK NEQAS-HER2 ISH scheme. We provide data on the multicenter evaluation of 45 breast cancers using the SISH assay compared with evaluation of the same cancers using a FISH assay in a central reference laboratory. We document concordance of SISH data from each laboratory with FISH data from the central reference laboratory and intraobserver and interlaboratory scoring consistency.

Materials and Methods

Study Design

The concordance of HER2 determination by CISH using the Ventana INFORM HER-2 SISH assay and by FISH, an established method used routinely for clinical diagnosis,1,9 was determined on the basis of intrasite variation between the 2 assays at a UK NEQAS ISH reference laboratory. In addition, the interlaboratory reproducibility of the INFORM HER-2 SISH method was determined across 7 laboratories. A commercially available tissue microarray (TMA; Stretton Scientific UK, Stretton, England) containing 2 replicate cores from 45 breast cancers was circulated to 7 laboratories (randomly numbered 1–7) with experience in ISH methods. All laboratories were reference laboratories from the UK NEQAS ISH scheme (performing diagnostic and/or research-based FISH). Each laboratory received the same material and performed independent blinded analysis of HER2 and chromosome 17 using the INFORM HER-2 SISH assay. In addition, laboratory 1, a UK NEQAS ISH reference laboratory performed an independent blinded analysis of HER2 and chromosome 17 using FISH.

Determination of HER2 and Chromosome 17 by SISH and FISH

HER2 and chromosome 17 were determined by bright-field automated CISH using the Ventana INFORM HER-2 SISH assay using the same batch of reagents in all centers. Automated staining was performed on consecutive slides using the Ventana BenchMark XT, which was installed and validated in all sites, and all staff received appropriate training in assay performance and analysis before the commencement of the study. The assay protocol consisted of extended pretreatment with CC2, pH 6.0, followed by protein digestion with ISH protease 3 for 4 minutes for the xenograft control samples and 12 minutes for the test TMA slides. Initial validation at one center demonstrated more consistent staining for HER2 when the digestion time for the test TMA slide was reduced to 8 minutes with ISH protease 3, whereas the remaining laboratories used the recommended protocol of 12 minutes. This procedure was followed by incubation with the specific 2,4-dinotrophenol-labeled DNA probes. Detection was performed with the ultraView SISH Detection Kit and accessory reagents (Ventana, West Sussex, England). This consisted of, briefly, incubation with 2 consecutive antibodies followed by the addition of 3 sequential silver reagents. The silver precipitate is deposited in the nuclei, and a single copy of the chromosome or the HER2 gene is visualized as 1 black dot. The slides were then counterstained using Haematoxylin II (Fisher Scientific, Loughborough, England) and a bluing reagent.

HER2 and chromosome 17 were determined in a single central laboratory by dual-color FISH (PathVysion FISH assay, Abbott UK) using UK NEQAS scoring guidelines. FISH-stained TMA sections were analyzed at ×630 to ×1,000 magnification, and areas of carcinoma within each core were identified. The number of chromosome 17 and HER2 signals was counted in 20 nonoverlapping nuclei per core. The mean HER2/chromosome 17 copy ratio was calculated per core, and the mean HER2 and mean chromosome 17 copy numbers observed were recorded on a core-by-core basis.

Analysis of Results

All data reported were collated centrally and analyzed in the Edinburgh, Scotland, reference laboratory. Satisfactory data were obtained from the CISH assay (INFORM HER-2 SISH) in 6 of 7 participating laboratories; some laboratories repeated the assay once or twice to obtain satisfactory data because of failure to stain either the control or test slides for either chromosome 17 or HER2. Because for some laboratories the Ventana BenchMark XT systems were only installed for this study, the cause of these failures was difficult to ascertain.

The success rate for determination of HER2 using the SISH assay was determined for each laboratory by case and by core. The success rate for determination of HER2 using the FISH assay was also determined for the central reference laboratory (laboratory 1) by case and by core. Data are reported as HER2/chromosome 17 ratios except where intrasite/intersite variability is assessed for HER2 and chromosome 17 copy numbers.

Because 5 of 7 participating laboratories reported SISH data from both cores for each case, the intraobserver variation at each laboratory was analyzed. This analysis determined the variation between duplicate cores for each case and was compared with data obtained centrally for FISH testing.

The intersite variation between each of the individual sites performing the SISH assay was determined: mean intersite variation for each result reported was assessed, the percentage variation was documented, and statistical differences in inter-site variation were determined by using the Student t test. An additional analysis using the central laboratory (laboratory 1) as the comparator investigated the correlation of absolute SISH and FISH scores (all core data) between laboratory 1 and other participating laboratories. Regression analysis was performed on SISH results between each site and FISH results from laboratory 1, and the slopes and intercepts between different pairings were assessed using z statistics. The mean slope and intercept, with SEs of these estimates, were compared with the ideal slope (1.00) and intercept (0.0), with the average of SEs from other slopes being assigned to the test situation.

To evaluate the concordance between FISH and SISH, a similar analysis was performed on SISH results between each site and FISH results from laboratory 1.

Results

Ventana INFORM HER-2 SISH Assay and Concordance With the Standard FISH Assay

Table 1 shows the rate of successful HER2 determination in the same TMA using the Ventana INFORM HER-2 SISH assay across 7 UK NEQAS reference laboratories, compared with the FISH assay in reference laboratory 1. Satisfactory data were obtained from the CISH assay (INFORM HER-2 SISH) in 6 of 7 participating laboratories. In 1 laboratory, the SISH analysis for chromosome 17 failed on the first attempt, and reagents expired before this could be repeated. The data from this laboratory were therefore excluded because complete data could not be provided.

Of the 45 cases determined in each laboratory using the SISH assay (a maximum of 270 results in total), HER2 was successfully determined by SISH in 89% to 100% of cases, with an overall success rate of 94.8%. Similarly, of the 90 duplicate cores scored by each laboratory (except the laboratory that determined only 45 cores, 1 per case), HER2 was successfully determined by SISH in 83% to 96% of cores, with an overall success rate of 89.4%. Of the 45 cases and 90 cores determined by FISH in the central laboratory (laboratory 1), HER2 was successfully determined by FISH in 39 (87%) of 45 cases and 76 (84%) of 90 cores.

Evaluation of the concordance of the Ventana INFORM HER-2 SISH assay with the standard PathVysion FISH assay is shown in Figure 1 and Table 2 . Figure 1 shows examples of the correlation of absolute SISH scores (all core data) between individual laboratories compared with absolute FISH scores (all core data) from the central laboratory (laboratory 1). Figure 1A shows that the absolute HER2/chromosome 17 data from the SISH assay for laboratory 1 are highly consistent and very similar to the data obtained using the FISH assay in the same laboratory (laboratory 1; slope of 1.02, intercept of 0.06, and R value of 0.95). Figure 1B shows that laboratory 5 had a number of cases with poor correlation of absolute SISH scores with the FISH scores obtained using the FISH assay in the central laboratory (laboratory 1), although the overall correlation was adequate (slope of 1.01, intercept of 0.32, and R value of 0.92). The slope, intercept, and R value for comparisons between all SISH scores from individual laboratories and FISH scores from the central laboratory (laboratory 1) are shown in Table 3 , which also includes the concordance for diagnosis by SISH between each of these laboratories and diagnosis by FISH in laboratory 1. Although there were some minor differences between laboratories with some individual cases showing poor correlation, concordance for diagnosis of HER2 amplification was high. Overall, there was 96.0% (range, 88.9%–100%) concordance for diagnosis of HER2 amplification using SISH results from all 6 laboratories that reported data compared with FISH results from laboratory 1. With the exception of laboratory 5, which had some individual cases showing poor correlation, concordance for diagnosis between SISH and FISH was more than 95% in all participating laboratories.

View this table:
Table 1

Intrasite (Intraobserver) and Intersite Variation in HER2 Testing

Table 4 shows mean intraobserver variation for HER2 copy number, chromosome 17 copy number, and HER2/chromosome 17 ratio determined by SISH in each center and the data from the FISH assay in laboratory 1. Intraobserver variation was calculated based on duplicate analysis of both cores for each of the 45 cases at 5 of 7 participating laboratories. Mean intraobserver variation for all SISH results was 5.9% for HER2, 5.4% for chromosome 17, and 8.0% for HER2/chromosome 17 ratio.

Figure 1

Correlation of absolute silver in situ hybridization (SISH) scores from each laboratory with fluorescence in situ hybridization (FISH) scores obtained from the central laboratory. A, Comparison of SISH for laboratory 1 with central laboratory (laboratory 1) FISH. y = 1.019x – 0.0643; R 2 = 0.8998; R = 0.9486. B, Comparison of SISH for laboratory 5 with central laboratory FISH. The absolute SISH scores from all core data obtained from each laboratory were compared with FISH scores from all core data obtained from the central laboratory. The points represented by squares are discordant values between SISH and FISH. The slope, intercept, and R values were obtained from each plot and are summarized in Table 5. Laboratory 5 showed a number of cases with poor correlation with FISH; all other laboratories showed high concordance. y = 1.0103x – 0.3183; R 2 = 0.8484; R = 0.9211.

Table 5 shows intersite variation for the SISH assay across all participating laboratories. This variation represents the compound of technical and observer variation between pairs of sites and observers. Overall intersite variation between laboratories (mean ± SE, 12.66% ± 4.03%) was similar to that observed in previous studies. There were no significant differences in intersite variation.

Figure 2 shows an example of the correlation of absolute SISH scores (all core data) between individual laboratories and laboratory 1 as the comparator. The absolute HER2/chromosome 17 data for laboratory 2 are consistent and very similar to the data obtained from laboratory 1 (slope of 1.22, intercept of 0.05, and R value of 0.95). The slope, intercept, and R value for comparisons between all 6 individual laboratories that reported data and laboratory 1 as the comparator are shown in Table 3, which also includes the concordance for diagnosis between each of these laboratories.

View this table:
Table 2
View this table:
Table 3
View this table:
Table 4
View this table:
Table 5
Figure 2

Correlation of absolute silver in situ hybridization (SISH) scores between laboratories 1 and 2. The absolute SISH scores from all core data obtained from each laboratory were compared. The slope, intercept, and R values were obtained from each plot and are summarized in Table 3. The plot shows an example for the comparison between laboratories 1 and 2. y = 1.2197x – 0.0447; R 2 = 0.9114; R = 0.9548.

Discussion

The results of this UK NEQAS ISH multicenter “ring” study from 7 UK NEQAS reference laboratories show that the rate of successful HER2 determination using the Ventana INFORM HER-2 SISH assay in the same TMA construct was consistent with an overall success rate of 94.8% of cases (89%–100% across all 6 laboratories that reported data). This level of performance is very high for single TMAs, in which some fallout is expected owing to the need for a single digestion method for all tissue samples.14 The overall success rate of 94.8% of cases across all laboratories for HER2 determination using SISH compares with the success rate of 86.7% of cases for HER2 determination using the standard FISH method in laboratory 1. This laboratory had a success rate of 98% of cases for HER2 determination using SISH, which represents a difference of only 5 cases determined successfully by SISH compared with FISH. The success rates between FISH and SISH in this study are comparable and very high for TMA analysis. For both assays, we would expect improved success rates using whole sections. TMAs were used in the current study for ease of comparison of large numbers of samples across sites.

Concordance for diagnosis of HER2 amplification by SISH compared with FISH was high. Overall, there was 96.0% (range, 88.9%–100%) concordance for diagnosis of HER2 amplification using SISH results from all 6 laboratories that reported data compared with FISH results from the central laboratory. With the exception of laboratory 5, which had some individual cases showing poor correlation, concordance for diagnosis between SISH and FISH was more than 95% in all participating laboratories. According to ASCO/CAP guidelines, more than 90% concordance should be achieved to validate novel FISH or immunohistochemical procedures.2 This research study demonstrates a high level of concordance between FISH and SISH, suggesting that the Ventana INFORM HER-2 SISH assay is robust, provides consistent results across all participating laboratories, and the majority of laboratories satisfy UK and ASCO/CAP guidelines for validation of novel FISH procedures.5

The data from this study suggest that intraobserver and intersite variability of absolute HER2/chromosome 17 ratios appears to be tightly controlled across all 6 participating laboratories that reported data using the Ventana INFORM HER-2 SISH assay. The level of intraobserver variability was consistent across all laboratories. Mean intraobserver variability was 5.9% for HER2, 5.4% for chromosome 17, and 8.0% for HER2/chromosome 17 ratio for this SISH assay, which is lower than previously reported interobserver variation for FISH (approximately 10%).8,1517 In laboratory 1, the intraobserver variability was 5.1% for HER2, 3.8% for chromosome 17, and 5.6% for HER2/chromosome 17 ratio for the SISH assay compared with 4.3% for HER2, 3.7% for chromosome 17, and 4.2% for HER2/chromosome 17 ratio for the FISH assay.

Site-to-site variation represents a compound of interobserver variation due to differences in scoring and technical variation between sites. The overall intersite variation (mean ± SE) between all laboratories of 12.66% ± 4.03% is consistent with that reported in previous research studies (approximately 10%).8,1517 There were no significant differences between any laboratories in intersite variation. Analyses of the correlation of absolute SISH scores (all core data) between individual laboratories and laboratory 1 as the comparator showed that core scoring data from all laboratories were consistent with the data obtained in laboratory 1. Overall, all laboratories showed excellent performance in diagnostic accuracy and site-to-site variation.

The Ventana INFORM HER-2 SISH assay is robust, provides highly consistent results across all participating UK NEQAS reference laboratories, and complies with requirements in national guidelines for performance of diagnostic tests. Furthermore, concordance for diagnosis of HER2 amplification by SISH compared with FISH was very high. Evidence from this UK NEQAS ring study provides support for the use of the Ventana INFORM HER-2 SISH assay as a potential alternative for analysis and reporting of HER2 gene status of patients in routine practice. This study has also provided multisite data demonstrating intraobserver and intersite consistency in absolute diagnostic ratios for HER2 using SISH, with very tight practice between laboratories. The high level of consistency underlines the high quality of HER2 testing achievable and demonstrates the potential for extremely robust and quantitatively reproducible SISH in routine practice. Clearly, continued quality assessment is essential to continued good performance.

Footnotes

  • J. Merritt, PhD, Merritt Science, St Albans, England, a professional medical writer, drafted the manuscript and was paid by Ventana Medical Systems.

  • Supported by Ventana (now Ventana Medical Systems, Burgess Hill, England).

References

  1. 1.
  2. 2.
  3. 3.
  4. 4.
  5. 5.
  6. 6.
  7. 7.
  8. 8.
  9. 9.
  10. 10.
  11. 11.
  12. 12.
  13. 13.
  14. 14.
  15. 15.
  16. 16.
  17. 17.
View Abstract