OUP user menu

Special Commentary
Patient Safety and the Next Generation of HPV DNA Tests

Walter Kinney MD, Mark H. Stoler MD, Philip E. Castle PhD, MPH
DOI: http://dx.doi.org/10.1309/AJCPRI8XPQUEAA3K 193-199 First published online: 1 August 2010


Human papillomavirus (HPV) testing is more sensitive for the detection of cervical precancer and cancer than cervical cytology. The increased sensitivity of HPV testing and cytology combined (“cotesting”) compared to cytology alone permitted professional societies to recommend 3-year screening intervals among the cotest-negative results. However, there is an increasing recognition that both clinical sensitivity and specificity of cervical cancer screening are important to patient safety and must be considered in the context of using current and future HPV DNA tests. Exquisite analytic sensitivity for HPV does not increase clinical sensitivity of an HPV test but does result in excessive test positivity and decreased clinical specificity. A recent US Food and Drug Administration (FDA)-approved HPV test, Cervista (Hologic, Bedford, MA), demonstrated excessive test positivity—2 to 4 times more positive than the other FDA-approved HPV test—from its premarketing approval trial. The poor specificity of Cervista raises questions about the safety and applicability of using this test in routine cervical cancer screening. These data provide a didactic example of the potential dangers of mistaking excellent analytic sensitivity and even clinical sensitivity for good clinical performance.

Key Words:
  • Cytology
  • Cervical cancer
  • Human papillomavirus
  • HPV
  • HPV DNA testing

Five years ago, there was a single US Food and Drug Administration (FDA)-approved human papillomavirus (HPV) test, Hybrid Capture 2 (HC2; Qiagen, Gaithersburg, MD). The cutting edge of physician education and guideline writing revolved around recognition of the differences in sensitivity for cervical intraepithelial neoplasia (CIN) grade 2 or more severe (CIN 2+) diagnoses between cytology and HPV testing or the combination of the 2 tests (“cotesting”). Practice recommendations were promulgated in which the “HPV test” was synonymous with HC2 because it was the only test commercially available with a sufficient body of clinical experience to assess performance.

The recognition of the increased sensitivity of cotesting for CIN 2+ vs cytology alone permitted professional societies to recommend screening at 3-year intervals for women 30 years and older with cotest-negative results.1 The sensitivity of HPV tests was therefore perceived as the most important patient safety issue to permit screening interval extension without increasing cancer risk. Specificity was not considered a patient safety issue in the regulatory or the clinical practice setting because the consensus at that time was that the morbidity of excision treatments such as the loop electrosurgical excision procedure was minimal. Given the long time course and huge investment required for test development and approval, it is understandable that new tests have been planned and configured based on the consensus understanding of cervical cancer screening at the outset of their development.

Yet, our understanding of cervical cancer screening and prevention has evolved. Specifically, clinical sensitivity and specificity of cervical cancer screening are important to patient safety and must be considered in the context of using current and future HPV DNA tests.

The guidelines governing the use of HPV DNA testing in the clinical management of cervical screening and treatment are totally dependent on the clinically validated performance of the HPV DNA test. The difficulty in hitting the mark in terms of balancing analytic detection with the clinical tradeoff of sensitivity and specificity for cervical precancer (CIN 3) and cancer (CIN 3+) has been amply documented by the numerous failures in the history of HPV DNA test development, most of which have mistaken high analytic sensitivity for good clinical performance. In contrast with other in vitro diagnostics such as HIV tests in which detection of all HIV is the goal, the objective of clinical HPV DNA detection is not the detection of all HPV; it is the detection of clinically relevant HPV, ie, HPV infections strongly associated with and accounting for the majority (>90%) of CIN 3+. Excessive analytic sensitivity for carcinogenic HPV results only in increasing the number of false-positive results (lowering clinical specificity) without the benefit of increasing clinical sensitivity.

Expert opinions regarding what is or is not a good HPV DNA test in the United States have been published in an article accompanying the American Society for Colposcopy and Cervical Pathology Guideline revisions2 and similar criteria have been adopted by the European testing community3 as well. A recent regulatory approval of a new HPV DNA test provides an instructive lesson on the important distinction between good clinical and analytic performance for clinical HPV DNA testing, reminds us of the importance of clinical sensitivity and specificity in selecting an HPV test, and forces us to rethink the assumption that all HPV tests are and will be created equally, which is implicit in structure of the current guidelines for use of HPV testing in clinical practice.

Cervista: Next Generation of HPV Tests?

On March 12, 2009, the FDA approved 2 assays, Cervista HPV HR (high risk) and Cervista HPV 16/18 (Third Wave Technologies, Madison, WI; now owned by Hologic, Bedford, MA) for use in cervical cancer screening. Cervista HPV HR is a DNA test for 14 carcinogenic HPV genotypes. Cervista HPV HR was approved as a triage test for women with a cervical cytology interpretation of atypical squamous cells of undetermined significance, to determine who needs immediate colposcopy and as an adjunctive test with cervical cytology for routine screening in women 30 years or older. The indicated uses of Cervista HPV HR are in accordance with current national guidelines for use of HPV DNA tests1,4; other accepted applications of validated HPV DNA tests include postcolposcopic and posttreatment surveillance.4 Cervista HPV 16/18 was approved for the DNA detection of HPV-16 and HPV-18, the 2 most carcinogenic HPV types.

At face value, the approval of the Cervista tests would appear to be a boon to cervical cancer screening. First, the Cervista HPV HR test provides the much-needed market competition to HC2, which has been the unopposed, FDA-approved test for the detection of carcinogenic HPV DNA since 2003. In multiple clinical trials conducted throughout the world, HC2 has been shown repeatedly to be a highly sensitive58 and reliable test.9,10 However, HC2 has some limitations, including a turnaround testing time in excess of 5 hours and some degree of cross-reactivity with questionably carcinogenic and noncarcinogenic HPV genotypes.11

In addition, Cervista HPV 16/18 provides a potential solution to the clinical dilemma of the management of carcinogenic HPV+, cytologically negative (HPV+/cyto–) results from women 30 years or older undergoing cotesting. Current guidelines4 recommend the repeating of cotesting 1 year following an index HPV+/cyto– result, with a positive cytology or HPV DNA test at the time of follow-up resulting in referral to colposcopy. However, as noted in the same clinical guidelines,4 “Once such [HPV genotyping] assays are FDA-approved, emerging data support the triage of women 30 years and older with a cytology result of ‘negative for an intraepithelial lesion or malignancy’ but who are HPV positive with HPV genotyping assays to identify those with HPV16 and HPV18.” That is, women with HPV-16+ and/or HPV-18+ results would be referred immediately to colposcopy, whereas women with negative results for both HPV genotypes would be rescreened by cotesting in a year. Recent data from a large clinic-based trial of HPV DNA testing in England12 highlight the need for immediate triage strategies for women with HPV+/cyto– results, whose loss to follow-up at the 1-year interval negated the benefits of adding HPV DNA testing to cytology (because this is the group of women at risk of CIN 3 missed by cytology).13

Careful inspection of the Cervista data included in the manufacturer’s package insert (http://www.accessdata.fda.gov/cdrh_docs/pdf8/P080014c.pdf)14 should raise concerns about test performance, including its use in general screening. As shown in Table 8 in the manufacturers’ package insert for the Cervista HPV HR, 18.5% of women aged 30 years and older with normal cytology had positive results by the Cervista HPV HR.14 Significantly, this high level persists at older ages, with surprisingly 18.4% of women aged 60 years and older with normal cytology still having positive results by the Cervista HPV HR. That is, almost 1 in 5 women who are approximately 40 years past the peak prevalence of HPV infection, approximately 30 years past the peak of cervical precancer, and approximately 15 years past the peak of cervical cancer15,16 had positive results for carcinogenic HPV by the Cervista HPV HR.

To put these data into perspective, we graphed the age-group prevalence data for the Cervista HPV HR against recent reports on the age-group prevalence data for HC217,18 and data from a clinical trial of an HPV genotyping assay (with the data on the carcinogenic HPV genotypes pooled to simulate detection of carcinogenic HPV in aggregate) (unpublished data, 2004–2006, Roche Molecular Systems, Alameda, CA). As shown in Figure 1, the Cervista HPV HR was 2- to 4-fold more likely to give positive results at all age groups compared with the HC2 in women 30 years or older with normal cytology. Strikingly, the Cervista HPV HR was almost twice as likely to give positive results even compared with HC2 results from women recruited at sexually transmitted disease clinics (Sriparna Datta, MD, Centers for Disease Control and Prevention, unpublished results). By comparison, recent meta-analysis found that the adjusted (adjusted for region, study type, study design, publication year, sampling collection device, cell storage medium, HPV assay, primer used, youngest age included, and oldest age included in each study) overall HPV prevalence in women with normal cytology was 11.3% (95% confidence interval [CI], 10.6%–12.1%), and the prevalence in women 54 years or older was approximately 10%.19

Because the data for these tests in Figure 1 were not generated from the same population with paired testing, we considered the possibility that these differences are the result of population-selection biases. However, by using data from Table 16 in the package insert,14 a subset of specimens from the population of women 30 years or older with normal cytology were also tested by HC2 and PCR sequencing, the combined results of which were used as a reference standard. The comparison between the Cervista HPV HR and the composite results of HC2 and PCR sequencing is shown in Table 1. By treating disagreements between HC2 and PCR sequencing as indeterminate values (ie, only positives by both tests being considered positive), the percentage of positive agreement (number of positives by both tests divided by the number of positives by either test) between the Cervista HPV HR and the composite test results was only 20% (95% CI, 12%–30%), and the Cervista HPV HR was more than 4-fold more likely to give positive results than the composite (18% vs 4%; P < .0001; exact McNemar χ2 test). By considering HC2 or PCR sequencing positives as positives (ie, treating the indeterminate values as positives) to maximize the analytic sensitivity of the composite test, the percentage of positive agreement between the Cervista HPV HR and the composite test results was 26% (95% CI, 17%–35%), and the Cervista HPV HR was 2.5-fold more likely to give positive results than the composite result (20% vs 8%; P < .0001; exact McNemar χ2 test).

Figure 1

Percentage of carcinogenic human papillomavirus (HPV) among women who have negative cytology by age group. Data are from Castle et al18 ; Datta et al17 (and S. Deblina Datta, MD, Centers for Disease Control and Prevention, unpublished results) for all clinics, sexually transmitted disease (STD) clinics, and other clinics; Raymond Apple, MD, Roche Molecular Systems, Alameda, CA (unpublished results, 2004–2006); and the Cervista package insert14 for Cervista high-risk (HR) HPV. For the Cervista HR HPV data, we present binomial 95% confidence intervals for each age group as indicated by whiskers. LA, linear array.

View this table:
Table 1

These differences are totally consistent with the data shown in Figure 1. Such disparate HPV detection between the Cervista HPV HR and a composite test that includes the HC2, a well-validated HPV DNA test that on its own provides excellent reassurance against the presence of cervical precancer and cancer for up to 10 years,20,21 raises the question of what the Cervista HPV HR is actually detecting or, more specifically, implies concern over the analytic vs clinical sensitivity cut-point selection process in the Cervista clinical trials.2,3 It also requires reassessment of what the appropriate clinical uses of such a test might be. Laboratory standards for HPV detection have been developed (http://www.who.int/biologicals/areas/human_papillomavirus/WHO_HPV_LabNet/en/index.html),22 which may provide objective standards for evaluating test analytic sensitivity and setting positive cut points.

Given the assumption that the data contained in the package insert represent the actual clinical performance of Cervista, what are the potential implications of using Cervista HPV HR in screening of the general population, the vast majority of whom are healthy? First, 2- to 4-fold more women 30 years or older who have negative cytology will be “labeled” as HPV+, which will commit them to increased surveillance of annual follow-up rather than a 3-year interval. Second, more women will end up going to colposcopy for the following reasons: (1) compliance with cervical cancer screening guidelines,23 especially the use of HPV DNA for approved indications,24,25 is poor and many physicians will refer women to colposcopy after the first HPV+/cyto– result and (2) probabilistically more women with HPV+/cyto– results will be likely to have positive results for HPV at the 1-year follow-up.

Very sensitive screening, such as referring women with HPV+/cyto– results immediately to colposcopy,8 will result in excessive diagnoses of CIN 2,26 which is the clinical threshold for triggering excisional treatment of the cervix. There is an increasing awareness that CIN 2 is an equivocal diagnosis of cervical precancer,27 an admixture of CIN 1, the histopathologic manifestation of productive HPV viral infections, and CIN 3, cervical precancer and the best surrogate for cancer risk. CIN 2 is the most poorly reproducible diagnosis2729 of all the CIN diagnostic categories. Compared with CIN 3, CIN 2 is much more likely to regress.8,30,31 Therefore, if the data in the package insert are representative, using the Cervista for cervical cancer screening will likely result in an increased number of excisional treatments, which is associated with a 2-fold increased risk of preterm delivery and neonatal mortality32 in healthy women.

Applying the Cervista HPV 16/18 to women with HPV+/cyto– results could help reduce the overreferral of the women to colposcopy. However, its use has not been fully validated in this population of women for the detection of CIN 2+ and, more important, CIN 3+. As shown in the manufacturer’s package insert for Cervista HPV 16/18 (http://www.accessdata.fda.gov/cdrh_docs/pdf8/P080015c.pdf; Table 8),33 5% of women with normal cytology had positive results for HPV-16 and/or HPV-18, a greater percentage than the entire fraction of women with HPV+/cyto– results observed to have positive results for all carcinogenic HPV genotypes by HC2 at Kaiser Permanente Northern California.18 Furthermore, approximately 25% of the women with HPV+/cyto– results (by Cervista HPV HR) had positive results for HPV-16 and/or HPV-18 by the Cervista HPV 16/18, rather than the 15% to 20% that has been observed in other studies in screening populations.34 Given that the preceding data imply that the Cervista HPV HR is at least 2- to 3-fold less specific than HC2 and that there is a higher than expected percentage of Cervista HPV HR results that are HPV-16+ and/or HPV-18+ by the Cervista HPV 16/18, it seems highly probable that the Cervista HPV 16/18 positive predictive value for CIN 3+ will be significantly less than observed by Khan et al.34 Therefore, if these data accurately represent the performance of the Cervista HPV 16/18, the test is not applicable to the recommended clinical application for HPV-16/18 detection.4

So what does this mean to conscientious clinicians and laboratory directors? First, if these data are representative, then significant concern about the consequences of the adoption of Cervista products in cervical cancer screening is warranted. Clinicians and laboratory directors may not be intimately familiar with these details and discrepancies in performance when deciding which HPV DNA test to use. If in fact the test performance reported in the package insert represents correctly the performance of the test in clinical practice, then women tested with the Cervista products in screening mode will demonstrate a very different proportion of test positivity, necessitating an increased number of women undergoing surveillance or colposcopy. At a cost of $250 to $500 or more per colposcopic evaluation, the increased patient volume of colposcopy, even if these screening tools are used properly, may quickly offset any minor health care cost savings for an “integrated” cervical cancer screening package (ie, HPV DNA testing by Cervista and ThinPrep cytology [Hologic]) that might be offered from the manufacturer.

Second, regardless of what the actual performance of Cervista may be, we are reminded that as we judge new tests introduced to the market, clinical sensitivity and clinical specificity are essential metrics that are relevant to patient safety. This episode serves notice that if HPV tests with widely differing performance characteristics are introduced to the market, the current system of clinical recommendations will need to be restructured in some way to address or accommodate these differences, if possible, as discussed in the next section.

Final Comments

We acknowledge that we do not know the real-world performance of the Cervista HR and that it is plausible, albeit unlikely, that the data included in the package insert are spurious. In fact, we hope that its true performance and that of any next generation test meets or exceeds that of HC2. Yet, if the clinical specificity of the Cervista now being marketed proves to be dramatically better than represented in the package insert, we also wonder what the true clinical performance of the Cervista HR is and specifically what its real-world clinical sensitivity is, given that any improvement in clinical specificity almost always results in a decrement in clinical sensitivity.

No test will be perfect. Some of the potential advantages of the Cervista HR compared with the HC2 are smaller specimen requirements (2 vs 4 mL) and an internal DNA control, which HC2 does not have. Although the latter has been touted as an important feature for the next generation of HPV tests, it is likely to be a marginal gain at best, given the proven high clinical sensitivity demonstrated by the HC25,8,35 for cervical precancerous lesions, which translates into improved cervical cancer prevention.7,8 Other considerations for choosing a test may include ease of use (HC2 is semiautomated and Cervista is manual) and simplicity in interpretation of results (HC2 uses a single well and the Cervista HR uses 3 wells).

Any new test must demonstrate acceptable clinical performance based on the standards of reliable clinical sensitivity and specificity set by the community of experts.2,3 The data from the package insert of Cervista, whether representative or not, provide a didactic example of the potential dangers of mistaking excellent analytic sensitivity and even clinical sensitivity for good clinical performance. For Cervista, completion of postmarketing surveillance studies will be critical to understanding whether the data included in the Cervista package inserts are representative of the expected performance or are simply an aberration. Until these data are forthcoming, we urge the medical community to take into account the information that is presently available when making decisions concerning the adoption of Cervista into routine cervical cancer screening because of the potential physical32 and psychosocial3639 harm that excessive HPV DNA test positivity poses to women.

An entire generation of new HPV tests will enter the market in the next few years. It is now clear that all HPV tests will not be created equally, and some may even be inappropriate for use in cervical cancer screening. As new HPV DNA tests with differing clinical performance are approved for clinical use, there will be a great need to switch from an algorithm-based model to a risk-based model for clinical management.40,41 As illustrated, the performance of the HC2 and Cervista will likely differ substantially, and management recommendations based on HC2 clinical performance may not be applicable to Cervista. That is, testing positive for HPV DNA by these 2 tests may have very different implications in terms of patient risk and safety. Clinical HPV DNA detection is no longer synonymous with HC2. Amending current algorithms to account for the increasing number of test combinations or developing a separate algorithm for each new test is neither feasible nor practical. Instead, using a risk “calculator,” based on critical parameters (eg, which HPV DNA test, HPV DNA test results, Pap or cytology interpretation, previous test results, screening history, age, and HPV vaccination status) to estimate the risk of CIN 3+ to the next screening visit would help integrate the use of any HPV DNA or other test into routine practice.41 Even so, some tests may not be applicable to cervical cancer screening because of potential overreferral and poor positive predictive values because of excessive test positives. The only rational alternative to this model is for all HPV tests to be held to very similar clinical performance standards with little then to distinguish them other than nonmedical utility factors such as ease of use, automation, and cost.2

Finally, there must be a greater appreciation that both clinical sensitivity and specificity, or their complements, false-negative and false-positive results, are important to patient safety. Perfect clinical sensitivity and, therefore, absolute reassurance following a negative test result are not achievable at any analytic sensitivity because of a myriad of factors that are independent of the actual test performance, including operator error and poor cervical sampling. Misguided attempts to achieve the lofty goal of perfect clinical sensitivity (via increasing analytic sensitivity) have the potential to result in significant patient harm from unnecessary clinical follow-up, unnecessary diagnostic procedures, and unnecessary treatment of healthy women.


We thank S. Deblina Datta, MD, Centers for Disease Control and Prevention, Atlanta, GA; and Raymond Apple, PhD, Roche Molecular Systems, Alameda, CA, for providing unpublished data.


  • Dr Stoler has been a consultant in clinical trial and HPV DNA test development for Third Wave, Hologic, Qiagen, Roche Molecular Systems, and Gen-Probe.

  • Supported by the Intramural Research Program of the National Institutes of Health, National Cancer Institute (Dr Castle).

  • Disclaimer: The opinions expressed do not necessarily represent the opinions or policies of the National Cancer Institute, the National Institutes of Health, the US Department of Health and Human Services, or the US Government.


  1. 1.
  2. 2.
  3. 3.
  4. 4.
  5. 5.
  6. 6.
  7. 7.
  8. 8.
  9. 9.
  10. 10.
  11. 11.
  12. 12.
  13. 13.
  14. 14.
  15. 15.
  16. 16.
  17. 17.
  18. 18.
  19. 19.
  20. 20.
  21. 21.
  22. 22.
  23. 23.
  24. 24.
  25. 25.
  26. 26.
  27. 27.
  28. 28.
  29. 29.
  30. 30.
  31. 31.
  32. 32.
  33. 33.
  34. 34.
  35. 35.
  36. 36.
  37. 37.
  38. 38.
  39. 39.
  40. 40.
  41. 41.
View Abstract