OUP user menu

Effectiveness of Rapid Prescreening and 10% Rescreening in Liquid-Based Papanicolaou Testing

Heather S. Currens CT(SCP), Katharine Nejkauf CT(SCP), Lynn Wagner, Stephen S. Raab MD
DOI: http://dx.doi.org/10.1309/AJCP6LW4SYBTISOW 150-155 First published online: 1 January 2012


Although rapid prescreening (RPS) has been shown to be an effective quality control procedure for detecting false-negative conventional Papanicolaou (Pap) tests, RPS has not been widely implemented in the United States. In our laboratory, cytotechnologists performed RPS in 3,567 liquid-based Pap tests: 1,911 SurePath (BD Diagnostics–TriPath, Burlington, NC) preparations that were manually screened and 1,656 ThinPrep Pap tests (Hologic, Bedford, MA) that were imaged using the ThinPrep Imaging System (Hologic). We compared the sensitivity of RPS, 10% rescreening (R-10%), and routine screening (RS). In contrast with previously published findings, we found that RS + RPS did not improve screening sensitivity compared with RS + R-10%. These results support the following hypotheses: (1) Higher baseline RS sensitivity as a result of Pap test diagnoses standardization implemented for quality improvement purposes decreases the performance impact of RPS. (2) R-10% and RPS quality assurance methods detect diagnostic failures caused by different types of cognitive errors.

Key Words:
  • Quality control
  • Papanicolaou test
  • Rapid prescreening
  • Cervical cancer
  • Diagnostic error
  • Cognitive error

Cytopathology laboratories use several quality control methods to decrease the Papanicolaou (Pap) test false-negative (FN) frequency.129 In the United States, the 1988 Clinical Laboratory Improvement Amendments of 1988 (CLIA ’88)1 regulations require that a minimum of 10% of Pap tests classified as negative for intraepithelial lesion or malignancy (NILM)30 in the routine screening (RS) process be rescreened before case sign-out and verification (R-10% method). For the R-10% method, Pap tests are selected randomly from the total caseload to include negatives and Pap tests from patients or groups of patients identified as having a higher than average probability of developing cervical cancer based on available patient information.1

Several investigators showed that the R-10% method is relatively ineffective and inefficient in reducing the FN frequency.2,3,8,16,17,20 Tabbara and Sidawy2 reported that the R-10% method detected only 0.18% of cases rescreened. Renshaw16 and Renshaw et al17 reported that the R-10% method had a sensitivity of 0% for low-grade squamous intraepithelial lesion (LSIL) or higher.

An alternative quality control method used predominantly in nations outside the United States is rapid prescreening (RPS) of all Pap tests.229 By using RPS, cytotechnologists rapidly review Pap tests for 30 to 120 seconds before RS. The cytotechnologist makes an interpretation of normal or atypical (or worse) without making any markings on the slide before RS. Tavares et al20 showed that RPS detected significantly more FN Pap tests and increased RS sensitivity (71.3% to 92.2%) compared with R-10%, which did not increase RS sensitivity. Other authors also confirmed the benefit of RPS.229

In the United States, RPS has rarely been adopted, with pathologists citing CLIA ’88 regulations that limit individual cytotechnologist daily slide volumes as a limiting factor.1,10 Most published data on the benefit of RPS has been from laboratories that interpret conventional Pap tests, and the effectiveness of RPS in laboratories that use liquid-based and imaging systems is limited.229 The effect of other factors, such as cytotechnologist workload and disease prevalence, on RPS detection frequency also has not been fully evaluated. In this study, we evaluated the effectiveness of RPS and R-10% on the sensitivity of RS in an American laboratory that interpreted liquid-based Pap tests, some with imaging.

Materials and Methods

In 2009, the University of Colorado Denver (UCD) cytopathology laboratory interpreted 14,049 Pap tests: 6,691 (47.6%) ThinPrep Pap tests (TP; Hologic, Bedford, MA) and 7,358 (52.4%) BD SurePath liquid-based Pap tests (SP; BD Diagnostics–TriPath, Burlington, NC) with screening by 1 of 4 full-time equivalent cytotechnologists and final interpretation by 1 of 4 cytopathologists. On August 8, 2008, the ThinPrep Imaging System (Hologic) was implemented.31 In 2009, the UCD cytopathology laboratory percentage of cases according to the 2001 Bethesda System30 diagnostic categories was as follows: 2% unsatisfactory; 83% NILM; 6% atypical squamous cells, undetermined significance (ASC-US); 0.8% atypical squamous cells, cannot rule out high-grade squamous intraepithelial lesion (ASC-H); 5% LSIL; 1% high-grade squamous intraepithelial lesion (HSIL); 0.6% atypical glandular cells; and 0.1% malignant.

The 2 cytotechnologists (H.S.C. and K.N.) who performed RPS had 30 and 10 years of experience, respectively. The 2 cytotechnologists who did not perform RPS had 29 and 5 years of experience, respectively. The 2 cytotechnologists who performed RPS had no prior experience with the method, which was described by Brooke et al3: RPS was performed with a 10× microscope objective lens following a stepwise path. This process lasted approximately 60 seconds, and the primary intent was to detect abnormal cells. The cytotechnologists used an RPS log form to classify all Pap tests with abnormal cells as R for “review” and further classified each Pap test using 2001 Bethesda System terminology. All other Pap tests were classified as “N” for “no review.” The cytotechnologist did not place any marks on the Pap test slides. The log forms were stored securely, pending completion of RS. The RS cytotechnologist was not provided with the RPS categorization.

An individual cytotechnologist did not perform RPS and RS on the same Pap test. The cytotechnologists performed RS in accordance with current practice standards using 2001 Bethesda System terminology. Pap tests designated for review by the RS cytotechnologist were referred to the cytopathologist for final interpretation. Pap tests classified as NILM or unsatisfactory by the RS cytotechnologist were referred to either of the 2 RPS cytotechnologists who compared the RS diagnosis with the RPS diagnosis. Pap tests in which the RS diagnosis was NILM and the RPS diagnosis was atypical were referred to the cytopathologist for final interpretation.

From December 1, 2008, to February 28, 2009, 1 of 2 cytotechnologists (H.S.C. or K.N.) performed RPS of 3,567 consecutive liquid-based Pap tests (1,911 SP and 1,656 TP using the ThinPrep Imaging System). Unsatisfactory Pap tests were not included in the assessment of Pap test performance (ie, sensitivity and specificity).

The UCD cytopathology laboratory R-10% method entailed the review of all Pap tests from women who had a history of an abnormal Pap test or cases initially reported as unsatisfactory. Additional Pap tests were selected so that the entire volume of R-10% cases was at least 10% of the entire laboratory workload. During the RPS interval, 528 Pap tests classified as NILM by both RPS and RS were reviewed by a third cytotechnologist through the R-10% process. Pap tests classified as atypical or worse by the R-10% method were referred to the sign-out pathologist.

For each Pap test with an RPS or R-10% diagnosis of atypical or worse, we recorded the final diagnosis made by the pathologist. We measured the number of RS FN Pap tests detected by the RPS and R-10% methods; FN Pap tests detected by the R-10% method also represented RPS FNs. We calculated the screening sensitivity and specificity of RS, RPS, R-10%, RS + RPS, and RS + R-10% using the methods described by Tavares et al.20 Sensitivity and specificity were calculated using the cytopathologist diagnosis as the “gold standard.” Diagnoses of atypical and worse were considered positive, and diagnoses of NILM were considered negative. Performance measures were calculated with confidence intervals. If the confidence intervals of different screening methods did not overlap, we considered the difference in performance of the methods to be statistically significant.

RS and RS + RPS Pap test FN diagnoses detected by the RPS or R-10% methods were classified as major or minor based on the degree of discordance using a previously published “step” scale of diagnostic discrepancy.32 Minor FN discrepancies were considered 1-step discordant diagnoses (eg, NILM to ASC-US), and major FN discrepancies were considered 2-step or more discordant diagnoses (eg, NILM to LSIL, ASC-H, atypical glandular cells, or HSIL).

High-risk DNA human papillomavirus (HPV) testing (QIAGEN, Hilden, Germany) was performed as a reflex test in the majority of woman who had a Pap test diagnosis of ASC-US. For all RS ASC-US diagnoses with an RPS diagnosis of NILM, we obtained the high-risk DNA HPV result. If the high-risk DNA HPV was negative, we determined that these Pap tests had been overcalled.


The number of diagnoses using the 2001 Bethesda System categories for the RPS, RS, R-10%, and final diagnosis, including the cytopathologist interpretation for atypical or worse cases, is shown in Table 1.

Performance characteristics of RPS, RS, R-10%, RS + RPS, and RS + R-10% are shown in Table 2. RS + RPS and RS + R-10% had higher sensitivity compared with the sensitivity of RS alone, and this difference was statistically significant. RS + RPS did not have a statistically significant higher sensitivity than RS + R-10%. The specificity of RS and RS + R-10% was higher than the specificity of RS + RPS, and this difference was statistically significant.

The RPS method detected 25 RS FN Pap test diagnoses, including 5 major errors (4 TP and 1 SP) and 20 minor errors (15 TP and 5 SP). Four major FN Pap test diagnoses were interpreted as LSIL by the sign-out pathologist and 1 as ASC-H.

The R-10% method detected 26 RS FN Pap test diagnoses, including 6 major errors (5 TP and 1 SP) and 20 minor errors (18 TP and 2 SP). Five major FN Pap test diagnoses were interpreted as LSIL by the sign-out pathologist and 1 as ASC-H.

A comparison of negative RPS diagnoses with RS revealed a 3.4% overcall of ASC-US (on RS) based on high-risk HPV DNA results (123/3,567 cases).


Our data show that RS + RPS did not result in a statistically significant improvement in screening sensitivity compared with RS + R-10%. These findings contrast with much of the previously published literature on RPS.229 We hypothesize that 2 factors may have contributed to our findings: (1) The UCD cytology laboratory had a high baseline RS sensitivity. (2) The UCD cytotechnologists previously had undertaken initiatives to standardize Pap test interpretations and had low interobserver variability.

As calculated using the method reported by Tavares et al,20 the baseline UCD cytology laboratory RS sensitivity was 85.8%, which was considerably higher than the baseline sensitivity in laboratories that previously reported RPS effectiveness data (71.8%).229 A higher baseline would make statistically significant improvement in sensitivity more difficult to show following the implementation of any quality assurance method. We hypothesize that several factors may have contributed to the UCD baseline sensitivity, with one factor being the UCD use of liquid-based and manual screening technologies.3335 Although the effectiveness of these technologies has been controversial, most published RPS data are based on conventional screening practices; this variable cannot be controlled in comparing our data with previously published data, as we do not have pretechnology RS sensitivity data for the UCD laboratory.

View this table:
Table 1
View this table:
Table 2

We propose that cytology process improvements implemented in 2008 as part of a more global anatomic pathology. Lean quality initiative also had an effect in improving baseline RS sensitivity. In 1 initiative, the cytology supervisor and senior cytopathologist led an effort to standardize the use of Pap test diagnoses to decrease interpretation variability among cytotechnologists and cytopathologists. A higher level of interpretive variability results in a higher cytohistologic noncorrelation frequency and clinical outcomes of lower quality.36 Grzybicki et al37 reported methods to lower diagnostic variability among cytology personnel, and we implemented some of these methods in the UCD laboratory.

For example, the UCD laboratory focused on the standardization of ASC-US and setting a specific ASC-US threshold. In setting this threshold, we accepted the trade-off of increased RS sensitivity (calling ASC-US more frequently) at the expense of a lower specificity. The UCD laboratory had a lower frequency of high-risk HPV positivity in ASC-US Pap tests compared with the nationally published median data, indicating the more liberal use of ASC-US in the UCD laboratory.30,38

We believe that other laboratory factors also contributed to fewer FN diagnoses. For example, UCD cytotechnologists screened on average 10.3 slides per hour and averaged approximately 40 to 50 screened slides per day. This lower workload arose partly because of decreased UCD laboratory Pap test volume and partly because we wanted cytotechnologists to have longer RS times to improve vigilance for the detection of rare atypical cells.

We hypothesize that the R-10% and RPS quality assurance methods detect different types of error.6,3941 The R-10% method entails a diligent search that may detect infrequent dysplastic cells; errors detected by this process potentially represent slips in the RS process (eg, not examining all areas of the slide) or faulty rapid cognition processes associated with rare events.4244 The RPS method involves a more rapid low-level search that is less likely to detect rare events and more likely to detect errors secondary to larger gaps in concentration.4244 In addition, we hypothesize that RPS produces an overall gestalt assessment of the slide, which is based on assessing a baseline level of atypia. As RPS focuses on determining if atypical cells are present and the majority of atypical diagnoses are ASC-US, the level of laboratory diagnostic standardization strongly determines the level of RPS and RS agreement. We believe that high levels of diagnostic standardization in the UCD laboratory,37 especially for ASC-US, resulted in a lower frequency of disagreements between RS and RPS.

In the UCD laboratory, a benefit of RPS was in identifying possible cases of overcalled ASC-US, thereby improving specificity. The UCD data showed that RPS identified only 4 cases of LSIL and no cases of HSIL missed by RS in the entire population of 3,567 Pap tests. Although it was possible that our RPS cytotechnologists were not practicing RPS effectively, other authors have reported that the RPS method is easily learned; we have no reason to believe that UCD cytotechnologists lacked skill in performing RPS. The RPS cytotechnologists classified Pap tests as negative that RS classified as ASC-US; 11.8% of these cases had a negative high-risk HPV result. We hypothesize that Pap tests diagnosed as negative by RPS and ASC-US by RS could be referred to a third cytotechnologist who could adjudicate the diagnosis, before the Pap test is referred for a cytopathologist interpretation.

Cytology laboratories are not standardized in many areas of practice, and, consequently, quality improvement initiatives, such as RPS, will have a variable effect that depends on existing practice characteristics. A laboratory that already practices at a high level of RS sensitivity may not gain significant benefit in detecting FN cases by performing RPS, compared with R-10%. However, laboratories with higher baseline RS sensitivity often have lower specificity; in this setting, RPS could be used to identify potential overcalls of ASC-US diagnoses. RPS as a form of double viewing redundancy has different benefits for different types of laboratory practice.


  1. 1.
  2. 2.
  3. 3.
  4. 4.
  5. 5.
  6. 6.
  7. 7.
  8. 8.
  9. 9.
  10. 10.
  11. 11.
  12. 12.
  13. 13.
  14. 14.
  15. 15.
  16. 16.
  17. 17.
  18. 18.
  19. 19.
  20. 20.
  21. 21.
  22. 22.
  23. 23.
  24. 24.
  25. 25.
  26. 26.
  27. 27.
  28. 28.
  29. 29.
  30. 30.
  31. 31.
  32. 32.
  33. 33.
  34. 34.
  35. 35.
  36. 36.
  37. 37.
  38. 38.
  39. 39.
  40. 40.
  41. 41.
  42. 42.
  43. 43.
  44. 44.
View Abstract