Although automation has improved the accuracy and precision of blood cell counts and is more rapid and less labor-intensive, cerebrospinal fluid (CSF) samples are still counted manually. We compared the IRIS iQ200 Body Fluids Module (Iris Diagnostics, Chatsworth, CA) and the Beckman-Coulter LH750 (Beckman-Coulter, Brea, CA) with manual counts and evaluated the impact of automation on the laboratory if clinically acceptable performance was to be maintained.
Automated counts were compared with manual counts on 313 specimens. Clinical reliability was estimated using the weighted κ coefficient and the impact of errors discussed in the context of a historic census of 3,653 samples spanning 19 months.
Nucleated cell counts had a reliability of 0.73 for the LH750 and 0.84 for the iQ200. However, our results showed unacceptable rates of error at counts less than 200/μL (200 × 106/L) for the LH750 and less than 50/μL (50 × 106/L) for the iQ200, representing 94% and 83% of the census specimens, respectively. If clinical reliability is to be maintained, neither the LH750 nor iQ200 would have a significant impact on improving the efficiency of the laboratory because of the high percentage of low CSF cell counts.
Since the inception of laboratory medicine, the goals have been to democratize the use of the laboratory by providing clinically accurate and relevant test results in a cost-effective manner. Cerebrospinal fluid (CSF) cell counts are a challenge because of low cell counts and limited sample volumes. Manual cell counts have provided clinically acceptable results for decades. It is inevitable and desirable that manual methods will be replaced by automated methods. However, it is imperative that this not occur without appropriate validation. Although several studies have attempted to show the usefulness of automated analysis of CSF, these studies also provide data showing unacceptable performance at low counts that are clinically important.1–4 While emphasizing cost savings and enhanced productivity, they fail to provide or discuss the necessary demographic data to show the impact of automation on the workload.
Our study shows the distribution of CSF cell counts in a large hospital laboratory, has a large number of samples, and, in contrast with previous studies, compares manual and automated counts in the context of clinical decision-making thresholds. These directly model actual patient care, a connection that is omitted in parametric methods for assessing reliability, the result being these methods may overestimate the clinical reliability, particularly when the cell counts are low. Two automated instruments having different technology for enumerating cells were compared with duplicate manual counts.
Materials and Methods
Rhode Island Hospital, Providence, is a 500-bed hospital that treats private and welfare inpatients and outpatients and is affiliated with a university medical school. Specimens were received from diverse specialty clinics, the emergency department, and hospitalized patients. Pediatric and adult patient specimens were included in our study.
The study was done in 2 phases. Initially, all CSF samples sent to the clinical laboratory from April 2004 to October 2005 were tabulated to determine the distribution of cell counts. Subsequently, a total of 313 samples were collected for analysis to compare manual cell counts with automated cell counts on the Beckman-Coulter LH750 (Beckman-Coulter, Brea, CA) and Iris iQ200 Body Fluids Module (Iris Diagnostics, Chatsworth, CA). Each automated method was compared with the manual counts; however, this was not always possible because of insufficient sample volume, in which case automated counts were missing. All samples were analyzed at the time they were received on 2 shifts, 7 days a week by selected personnel. Permission for the study was granted by the institutional review board.
Total nucleated cells (TNCs) and RBCs were counted in an improved Neubauer hemocytometer chamber using bright-field microscopy with a 40× objective. A coverslip was placed on a clean, dry hemocytometer chamber that was placed in a moist Petri dish on 2 applicator sticks. Both sides of the chamber were filled and cells allowed to settle for 5 to 10 minutes. RBCs and TNCs were counted on each side of the chamber. If fewer than 200 cells were present, all 9 squares were counted. If more than 200 cells were present, the 3 middle squares were counted unless more than 200 cells were in 1 square. Then only the center square was also counted. The cell count was calculated using the area counted, depth of the chamber, and any corrections for dilution of the sample. The average count of the 2 sides was used for comparison with the automated result.
The specimens were analyzed using the manual aspiration mode. Before analysis, diluent was aspirated 3 times and the background count of the third aspiration was recorded. The CSF sample was then aspirated. Background counts were subtracted to determine the final TNC and RBC counts. Nucleated cells were counted using 2 decimal places and erythrocytes using 3 decimal places.
The iQ200 Body Fluid Module was used to count CSF nucleated cells and erythrocytes. The specimens were divided into 2 aliquots, 1 mixed with lysing reagent to count nucleated cells and 1 mixed with a buffered diluent to count total cells. Clear specimens were diluted 1:5, slightly bloody 1:10, and bloody 1:20.
All analyses and visualizations were conducted using SAS, version 9.1.3 (SAS Institute, Cary, NC), Excel 2003 SP3 (Microsoft, Redmond, WA), and Matlab R2007b (Mathworks, Natick, MA).
TNC counts were grouped into clinically relevant categories.5 The reliabilities of the categories into which counts fell were estimated based on the weighted κ coefficient.6 The Bowker test of symmetry was used to test for systematic overestimation or underestimation between the mean of manual counts and the automated counts.7 Symmetry was not examined for the reliability of manual count duplicates because duplicate assignment was arbitrary.
RBC counts from the automated methods were compared with the mean of the manual counts via regression after logarithmic transformation holding the intercept at zero. Zero counts were excluded rather than adding a constant. A slope of 1.0 represents unbiased estimates. The higher the r2, the less disagreement, and visualization was used in detection of systematic deviations that may occur even when the other parameters appear “good.”
Distribution of CSF Samples by Nucleated Cell Counts
During an 18-month period a total of 3,653 CSF samples were accessioned. Of the 3,653 specimens, nucleated cells ranged from 0 to 10/μL (0–10 × 106/L) in 76.2%. Only 10.4% had nucleated cell counts greater than 100/μL (100 × 106/L) Table 1.
Distribution of Cerebrospinal Fluid Samples by Total Nucleated Cell Counts in 3,653 Specimens
Total Nucleated Cells (/μL)
No. (%) of specimens
Reliability of Manual Nucleated Cell Counts
There were 302 samples for which manual TNC counts were made from duplicate samples from the same patient. These counts showed a weighted κ value of 0.944 with 95% confidence limits of 0.907 and 0.981 Table 2. Where disagreements in clinical category occurred, the differences in the count were generally small relative to the count magnitudes, with one count often lying on the threshold value, particularly for the lower ranges.
↵* Data are given as the number of cases in each clinically relevant group. Numbers in boldface indicate the number of cases in complete agreement by both counts in each clinically relevant group. Weighted κ, 0.944; 95% confidence limits, 0.907, 0.981.
There were 14 patients in whom one count was “normal” (TNC count, ≤5/μL) and the other count was in the category of more than 5 to 10/μL. The mean difference between these counts was 3.8/μL (SD, 1.1/μL), with 7 (50%) of the counts falling on the threshold with values of 6/μL. There were 13 patients in whom one count was in the category of more than 5 to 10/μL and the other was in the category of more than 10 to 50/μL. The mean difference between these counts was 6.2/μL (SD, 6.5/μL), with 4 (31%) of the counts falling on the threshold with values of 11/μL. There were 5 patients in whom one count was in the category of more than 10 to 50/μL and the other was in the category of more than 50 to 200/μL. The mean difference between these counts was 22.4/μL (SD, 8.8/μL). There were 4 patients in whom one count was in the category of more than 50 to 200/μL and the other was more than 200/μL. The mean difference between counts was 77.3/μL (SD, 58.8/μL).
Reliability of LH750 Compared With Manual Nucleated Cell Counts
The reliability of LH750 counts with the mean of the duplicate manual counts of 191 samples was 0.734 with 95% confidence limits of 0.649 and 0.820 Table 3. Of the patient samples, 43.5% were misclassified according to clinical thresholds Table 4, with agreement improving as mean manual cell counts increased to more than 200/μL (200 × 106/L). However, this “good” range would represent only 6.5% of the total specimens based on our hospital census (Table 1).
↵* Data are given as the number of cases in each clinically relevant group. Numbers in boldface indicate the number of cases in complete agreement by both methods in each clinically relevant group. Reliability: weighted κ, 0.734; 95% confidence limits, 0.649, 0.820; symmetry: S = 33.957; df, 10; P = .0002.
Reliability of iQ200 Compared With Manual Nucleated Cell Counts
The reliability of the iQ200 with the mean of duplicate manual counts of 300 samples was 0.836 with 95% confidence limits of 0.779 and 0.894 Table 5. Where the mean manual count was between 0 and 5/μL (0–5 × 106/L), 26.8% were misclassified by the iQ200. Of the normal samples, 11.4% ranged from 11 to 46/μL, a clinically significant misclassification.
↵* Data are given as the number of cases in each clinically relevant group. Numbers in boldface indicate the number of cases in complete agreement by both methods in each clinically relevant group. Reliability: weighted κ, 0.836; 95% confidence limits, 0.779, 0.894; symmetry: S = 45.388; df, 10; P = < .0001.
The comparisons of manual and automated RBC counts for the LH750 and iQ200 are shown in Figure 1. The slope of the LH750 regression function for the nonzero mean ± SE manual counts was 1.04 ± 0.0413 (top left). Although the r2 value was more than 92.6% when all nonzero counts were included, there were clearly systematic deviations whereby the LH750 frequently overestimated the RBC count as 1,000/μL for values less than 1,000/μL. Further difficulties not illustrated here arose when subtraction of background produced negative values (n = 45), which were treated as zero counts. However, when samples having mean manual counts of 1,000/μL or less were excluded from analysis (top right), the systematic variation was removed, the r2 increased to 99.9%, and the slope moved closer to unity, with a reduction in its SE despite the reduction in sample size (0.9911 ± 0.0060). The slopes for both were statistically significantly different from zero, and neither was significantly different from unity.
Automated RBC count as a function of the mean manual RBC count. Upper left, Comparison of the automated LH750 RBC count with the manual RBC count. The LH750 frequently overestimated the RBC count as a single repeating number of 1,000/μL for manual counts ≤1,000/μL. Upper right, Comparison of LH750 RBC counts with manual counts when manual counts ≤1,000/μL are excluded. Lower left, Comparison of iQ200 and manual RBC counts. The iQ200 RBC counts do not show a single repeating value when manual counts are ≤1,000/μL but are overestimated and show an increase in variability. Lower right, Comparison of iQ200 RBC counts with manual counts ≤1,000/μL excluded. The concordance improved with less variability and near perfect agreement.
Similar, but less pronounced results were observed with the iQ200. However, the systematic deviation of the iQ200 from the mean manual counts less than 1,000/μL (bottom left, slope = 1.0047 ± 0.0138, r2 = 96.7%) presented as overestimation and a general increase in variability, rather than a single repeating value. Again, analysis for samples with mean counts more than 1,000/μL improved performance (bottom right), with the slope remaining close to unity and with reduced variability (0.9945 ± 0.0035) and increasing the r2 to near perfect agreement within the reported level of precision (100.0%). The slopes for both were statistically significantly different from zero, and neither was significantly different from unity.
Clinical laboratories are in the unenviable position of having to provide cost-effective, rapid, and accurate test results to clinicians, often with reduced numbers of and/or minimally trained laboratory personnel. Automated methods have helped to successfully meet these goals in many areas of the laboratory, but CSF analysis has remained a challenge because nucleated cell counts that distinguish normal from abnormal test results are ultralow and the sample volume is limited.
Several studies have evaluated the applicability of automated cell counters commonly used in clinical laboratories for the analysis of CSF cell counts. The instruments have diverse principles of operation that include impedance, digital imaging, and uni-fluidics technology.1–4,8 To evaluate the performance of automated counts, statistical analyses have traditionally included parametric methods such as regression analysis, precision, and linearity studies. These are described for the Coulter LH750 and iQ200 in several reports.1,3,4 However, while parametric analyses provide estimates of the degree of concordance between manual and automated methods, they do not adequately determine clinical reliability. Our results highlight that parametric methods may overestimate the reliability of the applied system (ie, automated method) as a whole.
Diseases of the CNS have different patterns of pleocytosis that influence clinical decisions, additional test selection, and follow-up.5,9–12 Thus, to determine the clinical implications of automated compared with manual nucleated cell counts, the weighted κ statistic was used as a measure of clinical reliability, with supplemental information provided as to the types and rates of errors produced.
Our results for the LH750 showed poor reliability of automated nucleated cell counts less than 200/μL (200 × 106/L). In our laboratory, this would exclude about 94% of CSF specimens for analysis. This finding is similar to findings of other studies.1–3 Barnes et al3 reported a cutoff value of 300/μL, excluding more than 90% of the CSF samples and restricting the use of the LH750 to grossly bloody or cloudy fluids.
Results of the iQ200 automated digital imaging system were more encouraging. It is clear that differences between normal and abnormal results determine clinical decisions that, in some cases, would avoid medical misadventures. In the normal range, about 27% of the samples were misclassified. However, interpretation of results also differs among physicians. Whereas Fishman5 considered values greater than 5/μL definitely abnormal, Merritt and Fremont-Smith9 considered values of TNCs in the range of more than 5 to 10/μL (5–10 × 106/L) “suspicious.” By this criterion, only 12% of the samples would be misclassified, a value that is also clinically unacceptable. It is less clear that misclassifications in other groups would alter clinical decisions. In only 1 sample was there a serious misclassification of the TNC count in the more than 50 to 200/μL group. Thus, 50 TNCs/μL (50 × 106/L) is our recommended lower limit of detection for the iQ200, a value similar to the lower limit of detection of TNCs of 30/μL (30 × 106/L) reported by Butch et al.4
There are conflicting data on the Advia 120 hematology analyzer (Bayer HealthCare, Tarrytown, NY). Aulesa et al2 found the lower limit of detection for leukocytes to be 47/μL (47 × 106/L) and established their limit of reliability at 150/μL (150 × 106/L), which was equivalent to 3 SD. Aune et al8 claim accurate results between leukocyte counts of 0 and 10/μL (0–10 × 106/L) in a study supported by Bayer Healthcare. It would be of interest to statistically evaluate data using the weighted κ statistic.
The importance of the clinical laboratory to report accurate CSF nucleated cell counts is concisely summarized by Fishman5: “Finally, clinicians are at the mercy of the clinical laboratories upon which they depend, some of which fail to meet a high standard of performance for the chemical, bacteriological, serological, and cytological study of the CSF. Caveat emptor!”
RBC counts are of importance in diagnosing pathologic bleeding in the central nervous system. This includes cerebral hemorrhage, subarachnoid hemorrhage, cerebral trauma, and subdural hematoma. There are usually numerous RBCs present in these clinical conditions, and their enumeration is not a problem for the clinical laboratory. The counts are also useful in determining if bleeding is secondary to the trauma of the procedure. Decreasing RBC counts in serial tubes are indicative of a traumatic tap. If RBC counts are less than 1,000/μL, the automated instruments would not be able to reliably distinguish RBC counts in serial tubes. If there is a traumatic tap, the RBC count can also be used to calculate the baseline CSF nucleated cell count because 1 leukocyte contaminates the CSF fluid for every 500 to 1,000 RBCs. The LH750 and iQ200 reliably measured RBC counts of more than 1,000/μL.
Clinical laboratories are under intense pressure to replace labor-intensive manual methods with automation. Comparative studies emphasize the benefits of precision, consistency, and reduction in the need for highly trained skilled personnel with automation. As indicated previously, clinicians are held captive to the administrative and scientific decisions implemented by clinical laboratories. Our study demonstrates the importance of using measurements of reliability sensitive to clinical interpretation in addition to parametric analyses to evaluate the needs of clinicians and the impact on patient care. It also demonstrates the importance of evaluating demographic data to determine the actual cost savings of automation, if clinical reliability is to be maintained.