Clinical Lab Quality: CMS and Survey Organization Oversight	 
Should Be Strengthened (16-JUN-06, GAO-06-416). 		 
                                                                 
Improvement Amendments of 1988 (CLIA) strengthened and extended  
quality requirements for labs that perform tests to diagnose or  
treat disease. About 36,000 labs that perform certain complex	 
tests must be surveyed biennially by either a state or one of six
private accrediting organizations. CMS oversees implementation of
CLIA requirements and the activities of survey organizations. GAO
was asked to examine (1) the quality of lab testing; (2) the	 
effectiveness of surveys, complaint investigations, and 	 
enforcement actions in detecting and addressing lab problems; and
(3) the adequacy of CMS's CLIA oversight.			 
-------------------------Indexing Terms------------------------- 
REPORTNUM:   GAO-06-416 					        
    ACCNO:   A55622						        
  TITLE:     Clinical Lab Quality: CMS and Survey Organization	      
Oversight Should Be Strengthened				 
     DATE:   06/16/2006 
  SUBJECT:   Data collection					 
	     Disease detection or diagnosis			 
	     Inspection 					 
	     Laboratories					 
	     Quality assurance					 
	     Standards						 
	     Surveys						 
	     Testing						 

******************************************************************
** This file contains an ASCII representation of the text of a  **
** GAO Product.                                                 **
**                                                              **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced.  Tables are included, but    **
** may not resemble those in the printed version.               **
**                                                              **
** Please see the PDF (Portable Document Format) file, when     **
** available, for a complete electronic file of the printed     **
** document's contents.                                         **
**                                                              **
******************************************************************
GAO-06-416

     

     * Results in Brief
     * Background
          * Surveyed Labs
          * Survey Organizations
          * Surveys and Complaint Investigations
          * CMS Oversight
     * Insufficient Data Exist to Identify Extent of Serious Lab Qu
          * Limited Data Are Available on the Quality of Labs Inspected
          * Quality of Labs Inspected by Other Survey Organizations Is V
          * Proficiency Testing Results Suggest that Quality Has Not Imp
     * Oversight Weaknesses Mask Quality Problems
          * Announced Surveys May Result in Unrealistic Picture of Lab Q
          * Variability in Reported Survey Deficiencies Suggests that La
          * Balance Between Educational and Regulatory Roles by CMS and
          * Use of Volunteer Surveyors by CAP Raises Concerns
          * Lab Workers Who File Complaints About Quality Problems in La
          * Lab Sanctions Are Rarely Imposed
     * CMS Oversight of CLIA Is Inadequate
          * CMS's Implementation of Proficiency Testing Is Inconsistent
          * CMS Is Late in Ensuring CLIA Equivalency of Exempt States' a
          * Many CMS Validation Reviews Lack Independence and Reviews Sk
               * Validation of State Survey Agency Lab Surveys
               * Validation of Accrediting Organizations' Lab Surveys
          * CMS Use of Data for Oversight of CLIA Program Is Limited
          * CMS Implementing Performance Reviews for Survey Organization
     * Conclusions
     * Recommendations for Executive Action
     * Agency and Accrediting Organization Comments and Our Evaluat
          * Assessment of Lab Quality
          * Educational Focus of CLIA
          * Sanctioning Labs with Serious, Repeat Deficiencies
          * Proficiency Testing Frequency
          * Equivalency of Accrediting Organization and Exempt State Pro
          * Accrediting Organization Validation Reviews
          * Additional Comments by Accrediting Organizations
     * GAO Contact
     * Acknowledgments
     * GAO's Mission
     * Obtaining Copies of GAO Reports and Testimony
          * Order by Mail or Phone
     * To Report Fraud, Waste, and Abuse in Federal Programs
     * Congressional Relations
     * Public Affairs

Report to Congressional Requesters

United States Government Accountability Office

GAO

June 2006

CLINICAL LAB QUALITY

CMS and Survey Organization Oversight Should Be Strengthened

GAO-06-416

Contents

Letter 1

Results in Brief 5
Background 7
Insufficient Data Exist to Identify Extent of Serious Lab Quality Problems
15
Oversight Weaknesses Mask Quality Problems 19
CMS Oversight of CLIA Is Inadequate 33
Conclusions 44
Recommendations for Executive Action 46
Agency and Accrediting Organization Comments and Our Evaluation 47
Appendix I Effects of Lab Errors on Patient Health 58
Appendix II Labs Surveyed by State Survey Agencies and the Percentage with
Condition-Level Deficiencies, by State in 2004 61
Appendix III Number of Labs Subject to Surveys by State Survey Agencies in
2005 and Number of Labs with Sanctions, 1998 to 2004 63
Appendix IV Comments from the Centers for Medicare & Medicaid Services 65
Appendix V Comments from the College of American Pathologists 80
Appendix VI Comments from COLA 83
Appendix VII Comments from the Joint Commission on Accreditation of
Healthcare Organizations 89
Appendix VIII GAO Contact and Staff Acknowledgments 93

Tables

Table 1: Percentage of Inspection Requirements Classified as Serious, by
Survey Organization 16
Table 2: Amount of Advance Notice Given to Labs about Upcoming
Inspections, by Survey Organization 21
Table 3: Number of Complaints Received by CAP, 2002-2005 29
Table 4: Number of Labs Inspected with Principal Only, Principal and
Alternative, and Alternative Only Sanctions Imposed, 1998-2004 30
Table 5: Analysis of Results of Simultaneous and Independent Validation
Reviews of State Surveys, Fiscal Years 1999-2003 39
Table 6: Analysis of Results of Simultaneous and Independent Validation
Reviews of Accrediting Organizations' Surveys, Fiscal Years 1999-2003 41
Table 7: Effects of Lab Errors on Patient Health 58

Figures

Figure 1: Types of Survey Organizations, Requirements Used to Survey Labs,
and Percentage of Labs Surveyed by Each Organization, as of December 2005
11
Figure 2: Percentage of Labs with Proficiency Testing Failures from 1999
through 2003, by Survey Organization 19

Abbreviations

AAAB American Association of Blood Banks AOA American Osteopathic
Association ASHI American Society of Histocompatibility and Immunogenetics
CAP College of American Pathologists CLIA Clinical Laboratory Improvement
Amendments of 1988 CMS Centers for Medicare & Medicaid Services CMSO
Center for Medicaid and State Operations ER emergency room JCAHO Joint
Commission on Accreditation of Healthcare Organizations OSCAR On-Line
Survey, Certification, and Reporting system

This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed in
its entirety without further permission from GAO. However, because this
work may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this material
separately.

United States Government Accountability Office

Washington, DC 20548

June 16, 2006

The Honorable Charles E. Grassley Chairman Committee on Finance United
States Senate

The Honorable Mark Souder Chairman The Honorable Elijah E. Cummings
Ranking Minority Member Subcommittee on Criminal Justice, Drug Policy and
Human Resources Committee on Government Reform House of Representatives

Clinical lab tests are one of the most frequently billed Medicare
procedures and, according to the American Clinical Laboratory Association,
affect an estimated 70 percent of medical decisions.1 To improve oversight
of clinical labs, Congress passed legislation in 1967;2 renewed concerns
about quality, including errors in Pap smear tests used to diagnose
cervical cancer, resulted in enactment of the Clinical Laboratory
Improvement Amendments of 1988 (CLIA).3 In recent years, despite CLIA, lab
quality problems in several states have raised questions about the
adequacy of lab oversight. Lab oversight is critical because inaccurate or
unreliable lab tests may lead to improper treatment, unnecessary mental
and physical anguish for patients, and higher health care costs.4

The Centers for Medicare & Medicaid Services (CMS) is responsible for
overseeing compliance with CLIA requirements. As of December 2005, there
were approximately 193,000 labs nationwide, ranging from very small
physician office labs that conduct fewer than 2,000 tests annually to
hospital labs that conduct millions of tests each year. Most clinical labs
regulated under CLIA must obtain a certificate from CMS but only about 19
percent-those that conduct moderate- to high-complexity tests-undergo
biennial inspections, which are also referred to as surveys.5 The surveys
assess lab compliance with mandated personnel and testing standards. In
addition, surveyed labs must participate in proficiency testing, a program
that requires them to test samples with unknown characteristics that are
then graded by an external party. Labs with serious deficiencies may be
sanctioned, e.g., required to cease testing. Labs have a choice of being
surveyed by (1) their state survey agency, under contract with CMS; (2)
their state CLIA-exempt program for labs in New York and Washington; or
(3) one of six private accrediting organizations.6 State survey agency
inspections use CLIA requirements that are intended to help ensure valid
and reliable lab tests; the two state CLIA-exempt programs and six
accrediting organizations survey labs using their own requirements that
CMS has determined to be at least equivalent to CLIA's. Each survey
organization is also responsible for investigating complaints about lab
quality.7 Because of the critical importance of accurate lab test results
and oversight, you asked us to conduct a nationwide assessment of (1) the
quality of lab testing; (2) the effectiveness of surveys, complaint
investigations, and enforcement actions in detecting problems and ensuring
compliance; and (3) the adequacy of CMS oversight of the CLIA program.

1Medicare is a federal health care program serving the elderly and
disabled individuals.

2Clinical Laboratories Improvement Act of 1967, Pub. L. No. 90-174, S:5,
81 Stat. 533, 536.

3Pub. L. No. 100-578, 102 Stat. 2903.

4Appendix I provides examples of the effect of lab errors on patient
health.

To determine what is known about the quality of lab testing, we analyzed
data on serious deficiencies identified during surveys by state survey
agencies using CMS's On-Line Survey, Certification, and Reporting system
(OSCAR).8 The CLIA program inspection requirements are classified as
either "standard-" or "condition-" level. Similarly, deficiencies are also
characterized as standard- or condition-level, based on the requirement in
which the deficiency occurs. Because condition-level requirements
generally consist of one or more standard-level requirements, a deficiency
at the condition-level denotes a serious or systematic problem. We
requested comparable data on serious deficiencies from state CLIA-exempt
programs and the three largest accrediting organizations-the College of
American Pathologists (CAP), COLA, and the Joint Commission on
Accreditation of Healthcare Organizations (JCAHO)-which together survey
about 97 percent of accredited labs.9 CAP, COLA, JCAHO, and exempt-state
programs each maintain their own separate databases. We also analyzed
proficiency testing data-another indicator of a lab's ability to produce
accurate test results. CMS officials generally recognize OSCAR and
proficiency testing data to be reliable. We discussed the OSCAR database
with CMS officials and tailored our analysis to ensure the accuracy of our
findings. Because exempt states and accrediting organizations survey labs
using their own requirements, we worked with them to develop data
comparable to OSCAR deficiency data. We discussed our analyses with CMS
and each of the survey organizations to ensure that we had interpreted the
data correctly. Based on discussions with officials from the three
accrediting organizations, we determined that they take appropriate steps
to ensure the reliability of their data. Because it was not practical to
independently test the reliability of accrediting organization data, we
present these data as reported by those organizations.

5Labs obtain a CLIA certificate that corresponds to the complexity of the
testing they conduct. Generally, each lab has one certificate but a large
hospital with multiple labs may have a corresponding number of
certificates. By regulation, labs that are within a hospital campus and
under common direction are allowed to file either a single application for
a certificate or multiple applications for multiple certificates.

6CMS contracts with state survey agencies in the District of Columbia and
49 states (including New York but not Washington) to survey labs under
CLIA requirements. Labs in Washington are surveyed either by the state's
CLIA-exempt program or by an accrediting organization. Labs in New York
are surveyed either by the state survey agency or New York's CLIA-exempt
program. New York does not authorize accreditation as a basis for lab
licensure.

7Throughout this report, we use the term "survey organizations" when
referring collectively to state survey agencies, the two state CLIA-exempt
programs, and accrediting organizations.

To assess the effectiveness of lab surveys, complaint investigations, and
enforcement mechanisms in detecting problems and securing compliance, we
reviewed the processes used to ensure the quality of clinical lab testing
and analyzed available data related to these issues. We also conducted
structured interviews with officials from (1) CMS, (2) three CMS regional

8In addition to the results of state survey agency lab inspections, CMS's
OSCAR database stores other information on labs registered under the CLIA
program including: (1) labs' CLIA certificate history, such as switching
from being inspected by accrediting organizations to being inspected by
state survey agencies; (2) the results of complaint investigations; (3)
labs' billing history; and (4) proficiency testing enrollment and
performance data.

9COLA was formerly known as the Commission on Office Laboratory
Accreditation.

offices,10 (3) 10 state survey agencies,11 (4) the New York and Washington
CLIA-exempt programs, and (5) the three accrediting organizations. We
judgmentally selected the 10 state survey agencies to include a mixture of
states whose lab inspections identified a range of serious deficiencies
from few to many. We also discussed the quality problems discovered at a
Maryland hospital lab with a Maryland state survey agency official and
interviewed 9 of the 36 CAP surveyors who participated in surveys of this
lab from 1999 through 2003 to obtain a firsthand perspective on the CAP
survey process.12 Based on our review and discussions with CMS and survey
organization officials, we focused on several key issues, including the
rationale for announced surveys, the ability of lab surveys to identify
serious deficiencies, the balance struck between the regulatory and
educational goals of lab surveys, the implications of CAP's use of
volunteer surveyors from neighboring labs to conduct inspections, how
survey organizations facilitate the filing of complaints, and the use of
sanctions to encourage compliance. We analyzed data on the number of
complaints received by each survey organization from 2002 through 2004 and
discussed CAP's initiatives to encourage the filing of complaints. In
addition, we determined the extent to which labs had the same serious
problems on consecutive surveys and discussed with CMS the steps the
agency had taken to deter an inconsistent pattern of compliance.

To assess the effectiveness of CMS oversight of the CLIA program, we
analyzed the laws and regulations that define CMS's role and authority. We
also reviewed CMS's process for determining that the survey requirements
and procedures of state CLIA-exempt programs and accrediting organizations
are at least equivalent to those of CLIA. We analyzed the results of
validation reviews that federal surveyors from CMS regional offices
conducted for state survey agency lab inspections and that state survey
agency staff conducted for accrediting organization inspections from 1999
through 2003. We also examined other mechanisms that CMS uses to hold
survey organizations accountable for their performance: (1) the collection
and analysis of data on surveys, complaints, and enforcement actions,
including steps taken to address communication and coordination issues
that became evident during a complaint investigation at a Maryland
hospital lab; and (2) recently developed annual reviews that measure state
survey agency compliance with CLIA program requirements, such as the
timeliness of surveys. We conducted our work from January 2005 through May
2006 in accordance with generally accepted government auditing standards.

10New York, Philadelphia, and Seattle.

11California, Colorado, Connecticut, Idaho, Louisiana, Michigan, Nebraska,
North Carolina, Pennsylvania, and South Carolina.

12A 2004 complaint investigation conducted by the Maryland state survey
agency found that personnel at this lab falsified records to conceal
problems with HIV and hepatitis testing equipment and that the lab
provided hundreds of patients with potentially erroneous test results.
These problems were not detected during the 2003 CAP inspection or a prior
complaint investigation conducted by the state survey agency in 2002.

                                Results in Brief

Insufficient data exist to identify the extent of serious quality problems
at labs. When CMS implemented revised CLIA survey requirements in 2004, it
modified historical state survey agency findings stored in its OSCAR
database and, as a result, data prior to 2004 no longer reflect key survey
requirements in effect at the time of those surveys. In addition, the lack
of a straightforward method to link similar requirements across survey
organizations makes it virtually impossible to assess lab quality in a
standardized manner, such as identifying the proportion of labs with
condition-level deficiencies, which indicate serious or systemic quality
problems. Although CMS has stated that it believes lab quality has
improved since the early 1990s, the results of proficiency testing-the one
available data source that can be used to uniformly compare lab quality
across survey organizations-suggest that lab quality may not have improved
at hospital labs and that the improvement for physician office labs may be
misleading because a significant number of such labs are no longer
inspected.

Weaknesses in surveys, complaint processes, and enforcement mask potential
quality problems at labs. Lab survey findings may not accurately reflect
the actual quality assurance process in place on a day-to-day basis
because of several shortcomings. First, most survey organizations announce
all surveys, allowing labs to prepare for their inspections. To address
this problem, accrediting organizations that inspect hospital labs began
conducting unannounced surveys in 2006. Second, the limited data available
suggest that state survey agency inspections do not identify all serious
deficiencies. Third, the balance struck between the CLIA program's
educational and regulatory goals is sometimes inappropriately skewed
toward education, which may also result in understatement of survey
findings. For example, CMS instructed state survey agencies not to cite
deficiencies for new lab quality control requirements for 2 years, in part
because of a lack of lab "buy-in" for some of the new policies and
procedures; CMS then extended this period and gave no specific end date.
Regarding complaint processes, complaints are filed by a variety of
sources, including lab workers. Few labs were the subject of a complaint
each year from 2002 through 2004-significantly less than one complaint per
lab per year. Concern that labs can easily identify the lab workers who
file complaints and lab workers' lack of familiarity with how to file a
complaint may explain why so few workers report problems. Since one
accrediting organization required each lab it inspects to display a poster
explaining how to file a complaint, the number of complaints it received
about lab quality has doubled. Finally, based on the large number of labs
with proposed sanctions from 1998 through 2004 that were never
imposed-even for labs with the same serious, condition-level deficiencies
on consecutive surveys-it is unclear how effective CMS's enforcement
process is at motivating labs to consistently comply with CLIA
requirements.

CMS's oversight of clinical lab quality is inadequate to ensure that labs
are meeting CLIA requirements. The agency requires proficiency testing
three times each year instead of on a quarterly basis, as required by
CLIA. Nor is CMS meeting its own requirement to determine in a timely
manner the continued equivalency of accrediting organization and
exempt-state inspection requirements and processes. For example, New
York's and COLA's reviews were about 4 years and 3 years past due,
respectively, as of December 2005. CMS attributed these delays to having
too few staff. Moreover, CMS allows the implementation of changes to
accrediting organization and exempt state inspection requirements between
periodic equivalency determinations before it reviews the proposed
changes. Validation reviews-one of CMS's most important oversight tools-do
not provide an independent assessment of the extent to which surveys
identify all serious deficiencies because many are performed
simultaneously with such surveys. In addition, CMS's requirement for
validating state survey agencies' inspections is vague, resulting in no
validation reviews in some states. Finally, CMS does not effectively use
data to monitor survey organization activities and processes, such as the
proportion of labs with serious deficiencies, proficiency testing results,
or trends in complaints. Realizing that its existing oversight activities
need to be strengthened, CMS has begun instituting performance reviews to
measure survey organization compliance with its requirements and is
developing protocols to ensure improved communication among survey
organizations concerning complaints about lab quality.

We are recommending that the CMS Administrator take actions that will
standardize survey findings across survey organizations, enable it to
compare changes over time, and make meaningful comparisons among
organizations; strengthen survey, complaint, and enforcement processes;
and improve CMS oversight of the CLIA program. In commenting on a draft of
this report, CMS endorsed our overall conclusion that quality assurance
for the nation's clinical labs should be strengthened and said that it
would take actions in response to 11 of our 13 recommendations. CMS
disagreed with our recommendations concerning the frequency of proficiency
testing and the extent of simultaneous accrediting organization validation
reviews. We believe that implementing these recommendations is necessary
to improve oversight of labs and accrediting organizations. CMS also
provided an alternative assessment of lab quality, disagreed with our
conclusion about the educational phase-in periods for new CLIA
requirements, and expressed concern about identifying and sanctioning labs
with repeat condition-level deficiencies. CAP, COLA, and JCAHO also
provided comments on a draft of this report. CAP indicated that it would
identify additional measures it could take to strengthen its own
oversight, and COLA found merit in our recommendations to improve CMS
oversight. CAP, COLA, and JCAHO disagreed with some of our findings and
recommendations to CMS. We incorporated technical comments from CMS and
the three accrediting organizations, as appropriate.

                                   Background

A clinical lab is generally defined as a facility that examines specimens
derived from humans for the purpose of disease diagnosis, prevention, and
treatment, or health assessment of individuals. While hospital and
interstate labs were previously subject to regulation, CLIA strengthened
federal requirements and extended them to most other clinical labs,
including physician office labs. For example, CLIA strengthened personnel
requirements for lab workers, strengthened proficiency testing that
evaluates the accuracy of lab testing between surveys, and created a range
of sanctions to enforce compliance.13

Most clinical labs regulated under CLIA must obtain a certificate from CMS
and pay fees every 2 years to cover the costs of administering the CLIA
program, including surveys and other oversight activities.14 The fees vary
based on the complexity and volume of testing performed. Lab tests are
categorized as waived, moderate, or high complexity.15 Approximately 81
percent of all labs (about 157,000) are not subject to routine biennial
surveys because they perform (1) "waived" tests, which are examinations
and procedures that have an insignificant risk of erroneous results,
including those approved for home use or determined to employ
methodologies so simple or accurate that the likelihood of erroneous
results is negligible;16 or (2) tests performed during the course of a
patient visit with a microscope on specimens that are not easily
transportable.17

13Implementation of CLIA was phased in over a number of years. CLIA does
not apply to forensic laboratories, research laboratories that do not
report patient-specific results, drug testing laboratories certified by
the Substance Abuse and Mental Health Services Administration, and
Veterans Administration laboratories.

14Labs surveyed by either the New York or Washington CLIA-exempt programs
do not obtain a CLIA certificate and do not pay fees to CMS. According to
CMS, labs are billed a year in advance of the surveys to provide adequate
time for them to pay their fees and for states to perform surveys prior to
the expiration of labs' CLIA certificates.

Surveyed Labs

CLIA establishes more stringent requirements for the 19 percent (about
36,000) of labs performing moderate- or high-complexity testing, including
the requirement for a survey and participation in routine proficiency
testing. Since the early 1990s, the number and proportion of labs subject
to surveys have declined, while the number and proportion conducting
waived tests have increased.18 Surveys examine lab compliance with CLIA
program requirements in several areas including: personnel qualifications,
proficiency testing, quality control, quality assurance, and
recordkeeping.

           o  Personnel: CLIA sets minimum qualifications for all persons
           performing or supervising moderate- or high-complexity lab tests
           and specifies responsibilities for each position.
           o  Proficiency testing: Surveyed labs must participate in an
           approved external proficiency testing program, which evaluates the
           accuracy of laboratory testing. Under this requirement, labs
           purchase samples with unknown characteristics several times each
           year from an approved proficiency testing provider.19 The lab is
           required to test the samples with its routine patient testing, and
           the results are returned to the testing provider to be graded. A
           proficiency testing failure is defined as unsatisfactory
           performance on two consecutive or two out of three testing events.
           The results of proficiency testing for all inspected labs are
           transmitted to CMS and maintained in a database.
           o  Quality control: Labs must have a process for routinely
           monitoring personnel, testing equipment, and the testing
           environment to ensure proper operation and accurate results.
           o  Quality assurance: Labs must follow their plan to monitor the
           overall operation of the laboratory on an ongoing basis and must
           resolve identified problems that affect the quality of their
           testing.
           o  Recordkeeping: Labs must maintain an audit trail of testing
           that documents specimen integrity and test performance for all
           phases of the test process from the test order to the test report.

           In general, labs have a choice of who conducts their surveys-state
           survey agencies using CLIA inspection requirements or other survey
           organizations that use requirements CMS has determined to be at
           least equivalent to CLIA's.20 CMS contracts with state survey
           agencies in most states to inspect labs against CLIA
           requirements.21 CLIA established an approval process to allow
           states and private accrediting organizations to use their own
           requirements to survey labs.22 As noted earlier, New York and
           Washington operate CLIA-exempt programs and CMS has approved six
           private, nonprofit accrediting organizations to survey labs-the
           American Association of Blood Banks (AABB), the American
           Osteopathic Association (AOA), the American Society of
           Histocompatibility and Immunogenetics (ASHI), CAP, COLA, and
           JCAHO. The requirements of both state CLIA-exempt programs and
           accrediting organizations must be reviewed by CMS at least every 6
           years to ensure CLIA equivalency, but may be more stringent than
           those of CLIA. For example, when inspecting labs engaged in
           moderate- and high-complexity testing, New York and some
           accrediting organizations also look at the labs' procedures for
           conducting "waived" tests, which is not required under CLIA.
           Figure 1 lists the three types of survey organizations and
           indicates whether they survey labs under CLIA requirements, or use
           their own CLIA-equivalent requirements. It also shows the
           percentage of labs performing moderate- to high-complexity testing
           surveyed by each type of organization. In general, state survey
           agencies, COLA, and Washington's CLIA-exempt program survey
           physician office labs, while New York's CLIA-exempt program, CAP,
           and JCAHO survey hospital labs.

15In January 2000, the Food and Drug Administration assumed responsibility
for categorizing tests conducted by labs from the Centers for Disease
Control and Prevention.

16Pregnancy and blood sugar screenings are examples of such tests. Labs
conducting waived tests are only required to follow manufacturers'
instructions and to limit testing to Food and Drug Administration-approved
or cleared methods.

17Known as provider-performed microscopy procedures, such tests must be
performed by a physician or other qualified provider as defined in CLIA
regulations. Labs conducting such tests are required to have written
procedures for the tests they perform and must also satisfy applicable
proficiency testing requirements and have a system to ensure the
competency of testing personnel. These labs and those performing waived
tests are subject to complaint investigations. See 42 C.F.R. S:
493.1773(f)(2005).

18From 1998 through December 2005, the proportion subject to surveys has
declined from about 30 percent to about 19 percent, while the proportion
of labs that are not surveyed because they perform waived testing has
increased from 70 percent to 81 percent.

Survey Organizations

19Proficiency testing providers are private companies or state lab
departments and are approved by CMS annually.

20Some labs, including Indian Health Service labs, are surveyed by federal
surveyors located in CMS's regional offices.

21CMS contracts with state survey agencies in the District of Columbia and
49 states (including New York but not Washington) to survey labs under
CLIA requirements.

22Prior to CLIA, CMS was not required to routinely determine the
equivalency of accrediting organization and state CLIA-exempt program
requirements.

Figure 1: Types of Survey Organizations, Requirements Used to Survey Labs,
and Percentage of Labs Surveyed by Each Organization, as of December 2005

aWashington is not included as it has only a CLIA-exempt program.

bNew York uses CLIA-equivalent requirements to inspect larger hospital
labs under the state's CLIA-exempt program and CLIA requirements to
inspect smaller labs, including physician office labs. Only the labs in
the CLIA-exempt program are counted here.

cSome labs are counted more than once because labs may be accredited by
more than one organization. While some labs in New York may be accredited,
they are still subject to biennial surveys by the state survey agency or
the state CLIA-exempt program, because New York does not authorize
accreditation as the basis for lab licensure.

Surveys and Complaint Investigations

Survey organizations (1) conduct surveys and complaint investigations, and
(2) monitor proficiency test results submitted by surveyed labs three
times a year. Surveys are typically conducted by former or current lab
workers, who assess lab compliance with CLIA or CLIA-equivalent
requirements. Most lab inspections are announced, that is, the lab has
advance notice of when the survey will occur. Generally, surveyors verify
that lab personnel are appropriately qualified to conduct testing,
evaluate proficiency test records, check equipment and calibration to
ensure that appropriate quality control measures are in place, and
determine whether the lab has a quality assurance plan and uses it to,
among other things, appropriately identify and resolve problems affecting
testing quality. Surveys also include an educational component to assist
labs in understanding how to comply with CLIA requirements. The duration
of a survey generally depends on the size-in terms of the number of tests
conducted-and complexity of a lab as well as the number of surveyors.
Thus, a survey conducted at a small lab may only take a few hours to
complete, while a survey at a large hospital lab may take a survey team a
full week or more.

In addition to inspections, survey organizations are responsible for
determining the seriousness of and investigating all complaints. For those
complaints that are determined to pose immediate jeopardy-an imminent and
serious threat to patient health and a significant hazard to public
health-CMS requires that the investigation be initiated within 2 working
days. Complaints may be investigated on-site or through communications
between the survey organization and the lab. Complaint investigations for
all survey organizations are unannounced.

Lab survey requirements are classified as either "standard-" or
"condition-" level. Generally, condition-level requirements are made up of
one or more related standard-level requirements. For example, the
condition-level requirement on enrollment and testing of samples through a
proficiency testing program has two related standard-level requirements:
(1) enrollment, which includes requirements for the lab to provide the
name of the program it has enrolled in to the Department of Health and
Human Services and authorize the release of testing data to the
department; and (2) testing, which specifies that the samples must be
tested in the same manner as any specimen and prohibits referring the test
samples to another lab for analysis.

Deficiencies are also characterized as standard- or condition-level based
on the requirement in which the deficiency occurs. Deficiencies in
standard-level requirements, that is, standard-level deficiencies, denote
problems that generally are not serious, while condition-level
deficiencies are cited when the problems are serious or systemic in
nature. A serious problem is defined as an inadequacy in a lab's quality
of services that adversely affects, or has the potential to adversely
affect, the accuracy and reliability of patient test results. When
deficiencies are found during surveys or complaint investigations, labs
are required to submit a plan of correction, detailing how and when they
will address the deficiencies. Additionally, CMS can impose principal or
alternative sanctions, or both.23 Principal sanctions include revocation
of a CLIA certificate, cancellation of the right to receive Medicare
payments, or limits on testing. Revocation of a CLIA certificate is
equivalent to termination from the CLIA program. Alternative sanctions are
less severe and include civil money penalties or on-site monitoring.24 For
condition-level deficiencies that do not involve immediate jeopardy, labs
have an opportunity to correct the deficiencies, which we refer to as a
grace period, before the sanctions are imposed. If a lab is unable to
correct a deficiency during this grace period, CMS determines whether to
impose a sanction and the type of sanction.

CMS Oversight

CMS, including its 10 regional offices, oversees state and accrediting
organization survey activities.25 CMS reviews and approves initial and
subsequent applications from exempt-state programs and accrediting
organizations to ensure CLIA equivalency. Validation reviews are one of
CMS's primary oversight tools. Federal surveyors in CMS regional offices
are responsible for conducting validation reviews of state survey agency
and exempt-state program inspections, but state survey agency staff
conduct the validation reviews of accrediting organization inspections.26
An objective of these reviews is to determine if all condition-level
deficiencies were identified.27 These reviews are conducted within 60 days
of a state's or 90 days of an accrediting organization's survey of a lab.
Starting in 1999, CMS required that at least one validation review be
conducted simultaneously with an accrediting organization's survey, a step
intended to encourage an exchange of ideas and approaches among surveyors.
CMS also encourages the use of simultaneous reviews of state survey agency
inspections. By law, the number of labs selected for validation reviews
must be sufficient to allow a reasonable estimate of the performance of
each accrediting organization being assessed.28 CMS requires fewer
validation reviews of state survey agency lab surveys (1 percent) than for
those of exempt-state programs or accrediting organizations (5 percent).

23State survey agencies may propose the imposition of sanctions for
noncompliance but only CMS can impose sanctions. Accrediting organizations
and exempt-state programs may revoke accreditation or remove a lab's state
license, respectively, for noncompliance with their CLIA-equivalent
requirements. While CMS regional office staff determine whether loss of
accreditation should also result in revocation of a lab's CLIA
certificate, loss of state licensure is tantamount to CLIA certificate
revocation.

24Because of congressional concern that available remedies were too
limited and could dissuade CMS from enforcement, CLIA gave CMS additional
tools, called alternative sanctions, to help motivate labs to comply with
quality requirements.

25The Centers for Disease Control and Prevention is also responsible for
carrying out certain CLIA-related tasks, including (1) developing and
evaluating technical standards for lab testing components; (2) working
with CMS and the Food and Drug Administration to determine the regulatory
impact of lab technical standards; (3) conducting lab research and
analysis; and (4) facilitating the CLIA Advisory Committee, which makes
recommendations to improve the CLIA program.

Beginning in 2003, CMS regional offices began reviewing the activities of
state survey agencies against a set of 13 performance standards. The
standards cover areas such as the timeliness of lab inspections, surveyor
personnel qualifications and training, CLIA data management, and the
handling of complaints. CMS's goal is to evaluate each state survey
agency's ability to carry out its CLIA responsibilities and to make
improvements. CMS is also developing performance standards for other
survey organizations that inspect labs using their own CLIA equivalent
requirements.

26Unlike validation reviews of accrediting organization surveys, CMS
refers to the validation of state surveys as Federal Monitoring Surveys.
Because of their similar objective, we refer to all such surveys as
validation reviews in this report. We refer to validation reviews that
occur at the same time as the lab survey as simultaneous. Conversely,
validation reviews that occur after the lab survey are referred to as
independent validations.

27According to CMS, the criterion for identifying a missed deficiency is
the reasonableness of concluding that a condition-level deficiency was
present at the time a survey organization conducted its survey but the
survey organization's findings did not note the deficiency.

28See 42 U.S.C. S: 263a(e)(2)(D)(2000). By regulation, a similar
requirement applies to validation reviews of labs under exempt-state
programs. 42 C.F.R. S: 493.563(b)(2)(2005).

Insufficient Data Exist to Identify Extent of Serious Lab Quality Problems

The extent of serious quality problems at labs is unclear because CMS has
incomplete data on condition-level deficiencies identified by state survey
agencies prior to 2004. We also found that the lack of a straightforward
linkage between CLIA requirements and the CLIA-equivalent requirements of
some survey organizations makes it virtually impossible to assess lab
quality in a standardized manner, such as identifying the proportion of
labs with condition-level deficiencies. Such deficiencies indicate serious
or systemic quality problems. Proficiency testing results-the one
available data source that can be used to uniformly compare lab quality
across survey organizations-raise questions about whether lab quality has
improved in recent years.

Limited Data Are Available on the Quality of Labs Inspected by State Survey
Agencies

CMS's OSCAR database contains limited data on the quality of labs
inspected by state survey agencies and, as a result, it is not possible to
analyze changes in the quality of lab testing over time. In January 2004,
CMS implemented revised CLIA survey requirements and modified the existing
OSCAR data-state survey agency findings-to reflect the changes.29 The
revisions affected approximately two-thirds of the CLIA condition-level
requirements.30 As a result of the data modifications, the findings for
surveys conducted prior to 2004 no longer reflect all key condition-level
requirements in effect at the time of those surveys.31 Based on the
available 2004 OSCAR data (which represent about one half of all labs
surveyed by state survey agencies), we found that 6.3 percent of labs had
condition-level deficiencies (see app. II for data on all state survey
agencies, including the District of Columbia).32 As will be discussed
below, similar data are not available for labs surveyed by other survey
organizations.

29CMS published the regulations for the new requirements in January 2003
and surveyors began using the new requirements on January 12, 2004.

30For example, some condition-level requirements were reorganized and some
were consolidated.

31When we asked for access to backup files, CMS told us that it did not
have backup files of the original pre-2004 survey data.

32We excluded survey results for the period January 1 through January 11,
2004, because CMS modified OSCAR data for findings prior to January 12,
2004, to reflect revised CLIA requirements.

Quality of Labs Inspected by Other Survey Organizations Is Very Difficult to
Measure in a Standardized Manner

Differences between the inspection requirements that state survey agencies
use to measure lab quality and those of exempt-state programs and
accrediting organizations make it virtually impossible to measure lab
quality in a standardized manner. Because exempt-state programs and
accrediting organizations do not classify inspection requirements and
related deficiencies as either standard- or condition-level, they cannot
easily identify the number of CLIA condition-level deficiencies cited at
the labs they survey or the proportion of surveyed labs with
condition-level deficiencies.33

We asked exempt-state programs and accrediting organizations what
percentage of their requirements, and any deficiencies cited for failure
to meet those requirements, indicated serious problems that were
equivalent to CLIA condition-level deficiencies. While only 8 percent of
CLIA requirements used by state survey agencies are classified as
condition-level and therefore serious, the proportion of requirements that
exempt-state programs and accrediting organizations classify as serious
ranged from 20 percent up to 100 percent (see table 1).

Table 1: Percentage of Inspection Requirements Classified as Serious, by
Survey Organization

                                     Percentage of requirements classified as 
Organization                                                       serious 
State survey agencies                                                    8 
New York CLIA-exempt program                                             a 
Washington CLIA-exempt program                                           a 
COLA                                                                    20 
CAP                                                                     80 
JCAHO                                                                  100 

Sources: GAO analysis of information provided by CMS, New York,
Washington, CAP, COLA, and JCAHO.

aThis state's CLIA-exempt program does not distinguish between serious and
nonserious requirements.

33Although CMS reviews the requirements of exempt-state programs and
accrediting organizations to ensure that they are at least equivalent to
CLIA's, there is not necessarily a one-to-one match with CLIA
requirements. Thus, one CLIA condition-level requirement may equal several
accrediting organization requirements or vice versa. For example, CMS's
condition-level requirement for successful lab participation in approved
proficiency testing corresponds to at least 19 CAP, 3 COLA, and 4 JCAHO
requirements.

CAP and COLA crosswalked their recent survey findings to CLIA
condition-level requirements.34 Although their analysis suggested that
from about 56 to 68 percent of labs surveyed during 2004 had a deficiency
in at least one condition-level requirement, they acknowledged that these
proportions overstated the subset of labs with serious problems. JCAHO did
not crosswalk its inspection requirements to those of CLIA because staff
would have had to manually review each survey report to determine which
deficiencies were equivalent to deficiencies in CLIA condition-level
requirements. However, JCAHO did tell us that in 2004, about 90 percent of
the labs it surveyed had a deficiency in at least one requirement and, as
previously noted, JCAHO classifies all its requirements as serious.

Despite the difficulty of identifying CLIA equivalent condition-level
deficiencies, two of the three accrediting organizations we reviewed have
systems to identify labs they survey that have serious quality problems.
COLA estimated that about 9 percent of labs it surveyed in 2004 were
subject to closer scrutiny because of the seriousness of the problems
identified. According to JCAHO, about 5 percent of the labs it surveyed in
2004 were not in compliance with a significant number of requirements. The
third accrediting organization, CAP, has criteria for identifying labs
that warrant greater scrutiny, but CAP officials told us that identifying
such labs had to be accomplished on a case-by-case basis rather than
through a database inquiry. As a result, CAP plans to spend in excess of
$9 million during 2006 and 2007 to develop an integrated data system that
pulls together multiple factors-survey results, complaints, proficiency
testing, findings of other inspection bodies, and changes in lab
directors-to enable it to readily identify problem labs. According to CAP
officials, such labs will be targeted for greater monitoring, and CMS and
other survey organizations will be notified about CAP's actions.

Proficiency Testing Results Suggest that Quality Has Not Improved at Hospital
Labs in Recent Years

Although CMS noted that proficiency testing trend data show a decrease in
failures for labs as a whole, the data suggest that lab quality may not
have improved at hospital labs for the period 1999 through 2003.
Proficiency testing is an important oversight tool for survey
organizations because it is an objective indicator of a lab's ability to
consistently produce accurate test results and is conducted more
frequently than surveys-three times a year versus once every 2 years. In
the absence of comparable survey data, proficiency testing results provide
a uniform way to assess the quality of lab testing across survey
organizations.

34This effort took CAP and COLA about 4 months to complete.

Our analysis of CMS proficiency testing data for 1999 through 2003
suggests that there has been an increase in proficiency testing failures
for labs inspected by CAP and JCAHO, which generally inspect hospital
labs, and a decrease in such failures for labs surveyed by state survey
agencies and COLA, which tend to inspect physician office labs (see fig.
2). CMS defines failures as unsatisfactory performance in two consecutive
or two out of three proficiency testing events. For example, the
percentage of labs with proficiency testing failures surveyed by CAP and
JCAHO from 1999 through 2003 increased from 4.1 percent to 6.8 percent and
from 6.6 percent to 7.8 percent, respectively. It is unclear, however,
whether the decrease in failures for physician office labs represents an
actual improvement in lab quality or reflects the fact that some
problematic labs are no longer surveyed. Specifically, many physician
office labs now perform waived tests and therefore are no longer surveyed
or participate in proficiency testing. Between 1998 and 2005, the
percentage of labs subject to surveys and proficiency testing decreased
from about 30 percent to about 19 percent.

Figure 2: Percentage of Labs with Proficiency Testing Failures from 1999
through 2003, by Survey Organization

Note: Data include labs affiliated with each organization during each
year.

                   Oversight Weaknesses Mask Quality Problems

Weaknesses in surveys, complaint processes, and enforcement mask real and
potential quality problems at labs. Survey weaknesses include: (1)
inspections that most organizations announce ahead of the visit, which
allows labs to prepare for their inspections and portray themselves in a
manner that may not accurately reflect their day-to-day quality assurance
processes; (2) variability in the proportion of labs with condition-level
deficiencies in 2004, which suggests surveys are not conducted in a
consistent manner; and (3) the goal of educating lab workers during
surveys taking precedence over, or precluding, the identification and
reporting of deficiencies. Furthermore, the significant increase in
complaints since CAP took steps to help ensure that lab workers know how
to file a compliant suggests that some quality problems at labs inspected
by other survey organizations may not be reported. Finally, sanctions are
not being used effectively as an enforcement tool to promote labs'
compliance with CLIA requirements, as evidenced by the relatively few labs
with repeat condition-level deficiencies on consecutive surveys from 1998
through 2004 that had sanctions imposed.

Announced Surveys May Result in Unrealistic Picture of Lab Quality

Because labs can and do prepare for surveys, CMS regional office officials
and most of the state survey agencies acknowledged that announced surveys
may not always provide a realistic picture of lab quality.35 As shown in
table 2, the amount of advance notice for surveys varies from as little as
2 weeks up to 12 weeks; until recently, only the New York CLIA-exempt
program conducted unannounced surveys. Survey agency officials in two
states told us that surveyors had inspected labs where records documenting
the implementation of periodic quality control procedures were completed
in the same handwriting using the same colored pen. This degree of
uniformity raises a concern about whether the quality control occurred at
all, or as frequently as the records suggested. A CAP surveyor told us
that the pathologist at one lab had cleaned up, and signed off on, about
3-months worth of quality control records the night before the survey. In
hearings on the questionable test results at a Maryland hospital lab, a
worker testified that lab staff prepared frantically for their announced
inspections.

35CMS does require unannounced surveys for: (1) complaint investigations,
(2) follow-up surveys conducted to verify correction of deficiencies, and
(3) nonroutine surveys conducted when there is reason to believe a lab is
operating in a manner that constitutes a risk to human health. In
contrast, all nursing home surveys are required to be unannounced to help
ensure that homes do not cover up problems that may exist when surveyors
are not present. See GAO, California Nursing Homes: Care Problems Persist
Despite Federal and State Oversight, GAO/HEHS-98-202 (Washington, D.C.:
July 27, 1998).

Table 2: Amount of Advance Notice Given to Labs about Upcoming
Inspections, by Survey Organization

Survey organization            Amount of advance noticea 
New York CLIA-exempt program                        None 
State survey agencies                     Up to 2 weeksb 
Washington CLIA-exempt program                   4 weeks 
JCAHO                                           4 weeksc 
CAP                                             7 weeksd 
COLA                                           12 weekse 

Sources: CMS, New York CLIA-exempt program, Washington CLIA-exempt
program, CAP, COLA, and JCAHO.

aThese numbers reflect stated policy and may not represent actual
practice.

bAdvance notice permitted by CMS guidance.

cIn January 2006, JCAHO stopped providing labs advance notice about
upcoming inspections.

dAverage actual notice provided to CAP labs for 2004. In 2006, CAP began
conducting unannounced inspections.

eCOLA confirms the survey date about 8 weeks in advance.

In 2006, both CAP and JCAHO began conducting unannounced inspections at
most of the hospital labs they survey.36 Both CAP and JCAHO officials told
us that the unannounced surveys will occur as early as 6 months prior to
the anniversary of a lab's prior survey. CMS and survey organizations that
inspect physician office labs provided several justifications for
continuing to announce inspections at such labs, including (1) ensuring
that the lab is open and that appropriate personnel are available to
answer surveyor's questions, and (2) minimizing disruptions to patient
care. These justifications appear to be reasonable because they reflect
the operating tempo at physician office labs. It may not be appropriate,
however, to provide such labs with 4 to 12 weeks advance notice, given
that CMS currently limits the advance notice provided by state survey
agencies to no more than 2 weeks.

36JCAHO implemented its unannounced surveys in January 2006 and CAP began
phasing in unannounced inspections in the spring of 2006. CAP and JCAHO
will continue to provide Department of Defense and prison labs that they
survey with advance notice to enable surveyors to obtain the security
clearances required to enter such facilities.

Variability in Reported Survey Deficiencies Suggests that Labs Are Not Surveyed
Consistently

Variability in OSCAR data for state survey agency inspections conducted in
2004 suggests that labs are not surveyed in a consistent manner, and
interviews with CMS and state survey agency officials confirmed this
hypothesis. As a result, available data likely understate the extent of
serious quality problems at labs. In 2004, the percentage of labs that
state survey agencies reported with condition-level deficiencies varied
considerably by state, ranging from none in 6 states to about 25 percent
of labs in South Carolina. These data only included findings for about
one-half of the labs surveyed by state survey agencies. Of the 33 states
that survey more than 100 labs, 16 found condition-level deficiencies at
fewer than 5 percent of labs, while 6 states identified such serious
deficiencies in more than 10 percent of the labs they surveyed (see app.
II).37

Based on interviews with CMS and 10 state survey agencies, it appears that
at least some of this variability is due to differences in states'
approaches to surveys as opposed to true differences in lab quality. For
example, CMS told us that, because there is not a prescriptive checklist
to guide the survey process, the reliance on state surveyor judgment will
result in variations in the citing of deficiencies. To compensate for the
unstructured nature of the state survey process, officials we interviewed
from 2 state survey agencies told us that they created checklists to help
ensure that surveyors looked at all of the critical elements during lab
surveys.38 Furthermore, while some of the state survey agencies we spoke
with told us that their surveyors always cite condition-level deficiencies
that are identified during surveys, officials in other states said that
there are circumstances under which condition-level deficiencies would not
be cited. For example, according to officials from a state survey agency
we interviewed, surveyors prefer not to cite condition-level deficiencies.
Rather, surveyors in this state prefer to cite multiple standard-level
deficiencies instead of a condition-level deficiency because it allows the
imposition of state law sanctions, avoiding what was characterized as a
less efficient federal sanctions process.39 Additionally, officials from 2
other state survey agencies explained that surveyors consider a lab's
compliance history when determining what deficiencies to cite, while
officials from a third state told us that surveyors will educate lab
workers, particularly new lab workers, about the CLIA requirements rather
than citing CLIA condition-level deficiencies.40

37Our analysis excluded state survey agencies that inspect fewer labs
because even a small change in the number of labs with condition-level
deficiencies can produce a large percentage point change.

38While checklists may be useful, some state survey agencies told us that
use of the checklists may result in insufficient probing and observation.
CAP officials told us that they plan to move beyond their current emphasis
on requiring documentation of lab processes with probing techniques that
require direct interaction with lab staff, observation of testing, and new
survey tools to guide inspectors in assessing compliance with
requirements. COLA's survey process includes a list of questions that
surveyors must answer by asking probing questions of, and interacting
with, lab staff. JCAHO's process, introduced in 2004, uses computer-based
algorithms when making compliance determinations.

Balance Between Educational and Regulatory Roles by CMS and Survey Organizations
Appears to Be Inappropriate

The goal of educating lab workers sometimes takes precedence over, or
precludes, the identification and reporting of deficiencies that affect
the quality of lab testing. As a result, data on the quality of lab
testing and trends in quality over time may be misleading. Although CLIA
neither requires nor precludes an educational role for surveyors, the
preamble to CMS's implementing regulation noted that surveys are intended,
in part, to provide an opportunity for on-site education regarding
accepted laboratory procedures. In addition, CMS guidance and training
encourage state surveyors to play an educational role.41 Many state survey
agency officials we interviewed also told us that their surveyors play a
major educational role. As noted earlier, surveyors from one state survey
agency do not cite condition-level deficiencies when lab workers are new
but prefer to educate the new staff. Because CMS revised its OSCAR
database in 2004, it is not possible to identify states that have
consistently not cited condition-level deficiencies, data that would help
to quantify the extent to which an educational role is substituting for
appropriate regulation of labs.42

39When state survey agencies cite condition-level deficiencies, the CMS
regional office for that state determines what, if any, sanctions should
be imposed.

40One example cited by a state survey agency involved a complaint
investigation of a transfusion-related fatality, the result of a lab
worker mixing up patient samples. Because the lab had already instituted
extensive corrective actions by the time the surveyor arrived, the survey
agency cited a standard-level deficiency for documentation errors rather
than a condition-level deficiency. We discussed this case with CMS
officials who told us that because the problem had been addressed, there
was essentially no condition-level deficiency to cite.

41While combining the roles of educator and regulator may not be unique,
it is different from the exclusively regulatory role state surveyors under
contract with CMS play for other provider groups, such as nursing homes
and home health agencies.

42Using OSCAR trend data, we were able to identify state survey agencies
that educated instead of regulated home health agencies. See Medicare Home
Health Agencies: Weaknesses in Federal and State Oversight Mask Potential
Quality Issues, GAO-02-382 (Washington, D.C.: July 19, 2002).

An inappropriate balance between the educational and regulatory role is
also evident in some accrediting organization practices. One of the CAP
surveyors we interviewed with over 30 years of lab experience estimated
that the majority of pathologists-individuals who generally serve as CAP
survey team leaders-view surveys as educational, rather than as
assessments of compliance with lab requirements. Another surveyor told us
that CAP's survey process focuses heavily on education, and that some
survey team leaders emphasize education more than others. For COLA, the
process of educating labs begins even prior to a survey. For example, COLA
encourages labs to submit a self-assessment for review prior to the
scheduled survey so that the labs can identify COLA requirements with
which they are not in compliance. About 20 percent of all labs surveyed
during 2004 submitted a self-assessment (616 labs) and, compared to labs
that did not submit a self-assessment, fewer deficiencies were identified
at these labs during on-site surveys.43

CMS appears to be inappropriately stressing education over regulation in
its implementation of (1) 2003 lab quality control requirements for the
CLIA program and (2) proficiency testing for lab technicians who interpret
Pap smears, a test for cervical cancer. When state surveyors began
assessing compliance with new lab quality control requirements in January
2004, they were instructed to note deficiencies on a cover letter to labs
rather than on the survey report itself for a period of 2 years. Thus,
such deficiencies are not recorded in the OSCAR database. In part because
of a lack of lab "buy-in" for some of the new policies and procedures, CMS
officials have extended the educational period for about another 2 years.
CMS has taken a similar educational approach to Pap smear proficiency
testing, which began in 2005. CMS will not cite deficiencies or impose
sanctions against labs in which staff fail the new Pap smear proficiency
testing in 2005 or 2006, as long as the labs and individuals involved
complete such testing, including following the regulatory protocol for
subsequent testing in the case of an initial failure. According to CMS,
this educational focus allows labs and their staff to become familiar with
the proficiency testing program and to prepare themselves for such
testing, since there was about a 13-year time lag between the 1992
regulations that implemented CLIA and the 2005 implementation of Pap smear
proficiency testing.44 This educational approach seems questionable given
CMS's concern about some of the high initial proficiency test failure
rates.

43This percentage includes both labs preparing for their initial survey
and those with prior surveys. According to COLA officials, newer labs are
most likely to perform the self-assessment; over time, they believe that
the vast majority of COLA-inspected labs have completed the
self-assessment.

Use of Volunteer Surveyors by CAP Raises Concerns

Although state survey agencies, exempt-state programs, COLA, and JCAHO
employ dedicated staff surveyors, CAP relies primarily on volunteer teams
consisting of lab workers from other CAP-inspected labs to conduct
surveys.45 In contrast to the mandatory training and continuing education
programs in place for the staff surveyors of other survey organizations,
training for CAP's volunteer surveyors is currently optional. CAP plans to
establish a mandatory training program beginning in mid-2006.46 As a
condition of accreditation, labs inspected by CAP must survey another
CAP-accredited lab of similar size and composition at least once every 2
years. According to data provided by CAP, two-thirds of volunteer
surveyors who had recently participated in a survey had no formal training
in the 3 to 5 years preceding the survey. Two CAP surveyors we interviewed
told us that they had not completed any training because it was optional.
Two other surveyors told us that they had never been notified about the
existence of optional CAP training. While full-time surveyors employed by
other survey organizations conduct from 30 to about 200 surveys per year,
CAP volunteer surveyors have much less experience conducting surveys
because they only survey about one lab each year.

44Because of lab testing errors that led to women's deaths from cervical
cancer, Congress required a specific type of proficiency testing for
individuals who interpret the results of Pap smear tests, which requires
examining glass slides under a microscope. Although CLIA was enacted in
1988, CMS told us that cost, the inability to find a national testing
provider, and other technical issues delayed establishing a Pap smear
proficiency testing program until 2005.

45As of November 2005, CAP also employed 11 full-time surveyors.
Historically, CAP staff surveyors were responsible for inspections of
smaller labs that conduct less complex tests. Increasingly, staff
surveyors will (1) accompany survey teams assigned to labs with a large
number of deficiencies on their prior survey, and (2) assist teams in
conducting either a lab's initial survey or the team leader's initial
survey. Staff surveyors will also conduct more nonroutine surveys, such as
investigating a complaint or following up on performance concerns raised
during surveys.

46Currently, CAP volunteer surveyors are encouraged to participate in
surveyor training at least once every 3 years. In July 2006, CAP plans to
begin requiring survey team leaders to complete mandatory training.
Mandatory training for survey team members is targeted to begin in 2007.

Three of the nine CAP surveyors we interviewed stated that they believed
that mandatory training was important, but some surveyors wondered when
lab workers would have time to complete the courses because of their
demanding work schedules. According to CAP officials, however, the
required training will take only 1 to 2 days and surveyors will have a
choice of live seminars and workshops or e-learning completed at their own
computers.47 For ongoing training requirements, CAP plans to give
surveyors a choice of taking additional training or passing a competency
evaluation. CAP will track compliance with its new training requirements
to ensure that surveyors successfully complete training and demonstrate
competency within 2 years of participating in a survey. CAP's required
training is less extensive than that required by other survey
organizations. For example, state survey agency inspectors must complete 5
days of basic training and periodic advanced courses afterwards while COLA
staff inspectors participate in a 5-week orientation program and an annual
20 hours of continuing education.

The use of volunteer inspectors by CAP also raises concerns about the
appearance of a conflict of interest. These concerns arise because of the
way CAP survey teams are structured. CAP's Commission on Laboratory
Accreditation policy manual specifies that the inspection team leader is
the individual responsible for the conduct of an ongoing site inspection,
and must not be in a business, professional, or personal relationship that
would preclude an objective inspection. Furthermore, the manual states
that the inspection team leader is usually responsible for determining the
size of, and assembling, the inspection team. However, until April 2006,
CAP policy did not preclude competing labs from surveying one another or
lab survey team members from soliciting business, such as referrals, from
a lab at the conclusion of the survey.48 The policy was also silent about
survey team members' business, professional, or personal relationships
that could cloud their independence.49 Typically, inspection team leaders
are pathologists who direct other labs in the community, and the
inspection team is comprised of several employees from the team leader's
lab.

47In contrast, CAP staff surveyors complete a 6-month training program
before they are allowed to conduct surveys independently.

48In explaining this policy, CAP notes that it believes team leaders and
inspectors will conduct the inspection of a competing lab professionally
and in an objective manner.

49In April 2006, CAP issued a revised conflict of interest policy that
addresses these concerns.

We believe that the use of volunteers, including those from nearby labs,
and the personal and professional relationships that may exist among lab
staff and survey team members, creates the appearance of a conflict of
interest and could undermine the integrity of the survey process.50
Comments from some CAP surveyors we interviewed raise a concern about
having survey team leaders who are also the day-to-day supervisors of team
members. For example, lack of agreement about the seriousness of a
deficiency could result in the team leader instructing the team to
downgrade the deficiency to a recommendation, a less serious finding that
does not appear in the inspection report. Team members who are
subordinates to the team leader may feel that they have no other recourse
than to follow the team leader's instructions. Recognizing that team
members' objectivity may be compromised in this situation, CAP's revised
conflict of interest policy instructs all parties to be cautious to retain
objectivity in fact finding throughout the inspection process.

In discussing these findings with CAP officials, they told us that they
plan to institute a number of initiatives to help ensure survey
objectivity, including (1) resurveying the same lab by an independent team
to assess the consistency of inspections, (2) centralizing survey team
assignments performed by CAP staff, (3) not announcing surveys, and (4)
not notifying labs of the survey team composition prior to the survey.

Lab Workers Who File Complaints About Quality Problems in Lab Testing Not
Afforded Whistle-blower Protections

Some lab workers may not be filing complaints about quality problems at
their labs because of anonymity concerns or because they may not be
familiar with filing procedures. Complaints about labs can come from a
variety of sources, including lab workers. Complaints are an important
tool in detecting quality problems between lab surveys. For example,
complaints about testing at a hospital lab were crucial because
information had been concealed, complicating the detection of quality
problems during the lab's surveys. As a result of a complaint, surveyors
were able to substantiate inadequate calibration of testing equipment that
could adversely affect patient care.

50According to CAP, 57 percent of surveys in 2004 were conducted by
surveyors who worked in nearby labs. For example, surveyors who inspected
a Maryland hospital lab from 1999 through 2003 worked in labs that were
from 5 to 42 miles away. The remaining 43 percent were conducted by
surveyors who did not work in nearby labs and who therefore required air
travel to carry out the survey.

Based on OSCAR data and data obtained from exempt-state programs and
accrediting organizations for 2002 through 2004, few complaints were
received about lab testing relative to the number of labs-significantly
less than one complaint per lab per year.51 The low volume of lab
complaints may be related to complainants' concerns about anonymity and
fear of retaliation for filing a complaint. It may be easy for a lab to
determine the source of a complaint filed by a lab worker. For example, in
some cases, either the nature of the complaint or the piece of testing
equipment in question could narrow the list of possible complainants. Two
CAP surveyors we interviewed commented that, in their opinion, it would be
easy to determine the identity of a complainant. During congressional
hearings in 2004, a Maryland hospital lab worker testified that she and
her colleagues feared losing their jobs because of the complaints they
filed.

Because of the difficulty of protecting the anonymity of lab workers who
file complaints, whistle-blower protections for such individuals are
particularly important. Two of the three accrediting organizations we
interviewed have whistle-blower protections-CAP and JCAHO.52 For example,
CAP implemented a comprehensive whistle-blower protection policy in July
2004 that includes revocation of accreditation or other appropriate action
for any lab that directly or indirectly threatens, intimidates, or
retaliates against a lab worker. While officials from New York and
Washington's exempt-state programs told us that whistle-blower laws in
their states provide some protection for lab workers who file complaints,
officials in most of the other 10 states we interviewed told us that they
did not have any whistle-blower protections or were unable to identify
specific protections that applied to lab workers in their state.
Currently, there are no federal whistle-blower protections specifically
for workers in labs covered by CLIA. In 2005, legislation was introduced
to provide whistle-blower protections to workers in labs covered by
CLIA.53

We also found that lab workers may not know how to file a complaint. CAP
experienced a significant increase in the number of complaints it received
since October 2004, when it began requiring CAP-inspected labs to display
posters on how to file complaints. Specifically, from October through
December 2004, CAP received an average of 22 complaints per month,
compared to an average of 11 complaints per month in the 9 months
preceding the poster requirement. As a result, the number of complaints
about the quality of lab testing more than doubled in 2004 and the number
substantiated increased by more than 40 percent-even though the poster was
only displayed for the last 3 months of 2004 (see table 3).54 In September
2005, COLA also began requiring labs to display a complaints poster
similar to CAP's. It is too early, however, to determine the impact of
COLA's new complaints poster on the number, type, and substantiation rate
of complaints. Neither CMS nor JCAHO plans to require a similar complaints
poster.55

51The modifications to OSCAR did not affect data on the number of
complaints. The complaint information in OSCAR excludes complaints that do
not require an on-site survey.

52COLA does not have a formal whistle-blower policy. COLA officials told
us that they promptly investigate all complaints, many of them from former
lab employees, and keep the identity of the complainants anonymous.

53H.R. 686, 109th Cong. (2005).

Table 3: Number of Complaints Received by CAP, 2002-2005

Year Received     Substantiated 
2002       82                39 
2003       84                40 
2004      170                70 
2005     290a 74 (preliminary)b 

Source: CAP.

aThis number is as of November 30, 2005, and thus does not include
complaints received in December.

bAs of November 30, 2005, CAP had substantiated 74 complaints; over 100
complaints were still under active investigation.

Lab Sanctions Are Rarely Imposed

Few labs were sanctioned by CMS from 1998 through 2004-even those with the
same condition-level deficiencies on consecutive surveys-because many
proposed sanctions are never imposed. Our analysis of CMS enforcement data
from 1998 through 2004 found that 501 labs were sanctioned, which equates
to less than 3 percent of labs inspected by state survey agencies.56 The
most common were principal sanctions, which may result in suspension or
limitation of testing or termination from the CLIA program; few labs were
subjected to alternative sanctions, such as directed plans of correction
or civil monetary penalties (see table 4). Appendix III shows the number
of labs surveyed by state survey agencies and the number of sanctioned
labs from 1998 through 2004.

54CAP plans to hire an additional staff person to investigate complaints.

55Effective July 2005, JCAHO required labs to educate staff on how to
report concerns about lab quality to the Joint Commission, but does not
specify use of a poster to do so.

56According to CMS, sanctions generally result from deficiencies
identified during an inspection by a state survey agency.

Table 4: Number of Labs Inspected with Principal Only, Principal and
Alternative, and Alternative Only Sanctions Imposed, 1998-2004

Sanction         Description                                Number of labs 
Principal only      o  Revocation of CLIA certificate                  269 
                       (termination)                           
                       o  Cancellation of approval to receive  
                       Medicare payments                       
                       o  Limits placed on testing             
                       o  Suspension of testing                
Principal and       o  At least one principal sanction plus            170 
alternative         at least one alternative sanction       
Subtotal                                                               439 
Alternative only    o  Directed plans of correction                     62 
                       o  Civil money penalties                
                       o  State on-site monitoring             
                       o  Partial or full suspension of        
                       Medicare payments                       
Total                                                                  501 

Source: GAO analysis of CMS lab registries.

Although few labs were sanctioned from 1998 through 2004, over 9,000 labs
had sanctions proposed during that same time period.57 Before sanctions go
into effect, labs are given a grace period to correct condition-level
deficiencies, unless the deficiencies involve immediate jeopardy, that is,
an imminent threat to patient health and significant hazard to public
health. Most labs correct the deficiencies within the grace period. CMS
officials told us that it was appropriate to give labs an opportunity to
correct such deficiencies within a prescribed time frame and thus avoid
sanctions.58 However, a principal objective of the enforcement process-one
reflected in CMS guidance-is to motivate labs to comply with CLIA
requirements, thereby helping to ensure the provision of accurate and
reliable test results. Based on the large number of labs with proposed
sanctions that were never imposed, it is unclear how effective the
enforcement process is at motivating labs to consistently comply with CLIA
requirements.

57Since CMS data lists only the number of labs with proposed sanctions by
year, this number may double-count labs that had proposed sanctions in
multiple years.

58Labs surveyed by states have 23 days to correct immediate jeopardy
deficiencies, 90 days to correct all other condition-level deficiencies,
and up to 12 months to correct standard-level deficiencies. CAP and COLA
give labs 30 days to correct all deficiencies, and effective July 1, 2005,
JCAHO reduced the time labs have to correct deficiencies from 90 to 45
days. Labs may also appeal proposed sanctions, and depending on the
outcome of such appeals, sanctions may be dismissed.

The number of labs with the same repeat condition-level deficiencies from
one survey to the next also raises questions about the overall
effectiveness of the CLIA enforcement process. From 1998 through 2004, 274
labs surveyed by state survey agencies had the same condition-level
deficiency cited on consecutive surveys and 24 of these labs had the same
condition-level deficiency cited on more than two surveys.59 This analysis
may understate the percentage of labs with repeat condition-level
deficiencies because OSCAR data prior to 2004 no longer reflect about
two-thirds of condition-level requirements and associated deficiencies at
the time of those surveys. We found that only 30 of the 274 labs with
repeat condition-level deficiencies had sanctions imposed-either
principal, alternative, or both. According to the CLIA legislative
history, congressional concern about labs with repeat deficiencies led to
alternative sanctions to provide an enforcement option short of principal
sanctions to encourage compliance.

From 1998 through 2004, less than 1 percent of accredited labs (81) lost
their accreditation; few of these labs were subsequently sanctioned by CMS
and many still participate in the CLIA program.60 Our analysis of CMS
reports on sanctioned labs found that only 9 of the 81 labs had either
principal and/or alternative sanctions imposed and that 1 of the 9 still
performs moderate- to high-complexity testing.61 Based on a review of its
CLIA certificate database, CMS officials told us that about half of the 81
labs still perform moderate- to high-complexity testing but could not
describe the actions taken by CMS regional offices in response to the loss
of accreditation. We contacted state survey agencies or CMS regional
office officials to determine why 3 labs that COLA concluded had cheated
on proficiency testing by referring the samples to another lab to be
tested had no sanctions imposed. The purpose of proficiency testing is to
provide an objective, external evaluation of the accuracy of a lab's test
results, which is negated when another lab analyzes the sample. By
statute, the intentional referral of samples to another lab for
proficiency testing is a serious deficiency that should result in
automatic revocation of a lab's CLIA certificate for at least 1 year.62
Based on our interviews, we found that the 3 labs were allowed to continue
testing because they had initiated corrective actions; in effect, these
labs were given an opportunity to correct a deficiency that appears to
have required a loss of their CLIA certificate for at least 1 year. A
fourth lab was ultimately sanctioned for proficiency testing cheating by
CMS but was allowed to continue testing for almost 2 years after having
its accreditation revoked.

59Thirty-three states and the District of Columbia had at least one lab
with the same repeat condition-level deficiency.

60Twenty CAP-, 51 COLA-, and 10 JCAHO-inspected labs had their
accreditation revoked. After notice of revocation of accreditation, a lab
retains its CLIA certificate and may continue to test specimens for 45
days while a state survey agency inspects the lab and makes a
recommendation concerning the lab's continued participation in the CLIA
program to the responsible CMS regional office, unless CMS takes action
sooner. Potential outcomes include (1) termination from the CLIA program;
(2) determination that the lab meets CLIA requirements because they are
less stringent than those of the accrediting organization, resulting in
the lab switching to state survey agency oversight; (3) the lab's return
to compliance and reapplication to be surveyed by an accrediting
organization; or (4) cessation of moderate- to high-complexity testing and
assumption of a CLIA certificate that only allows less complex, waived
testing.

We also attempted to analyze data on other actions, short of revoking
accreditation, used by accrediting organizations to encourage lab
compliance and, in particular, how they respond to labs with serious
repeat deficiencies. According to CMS, this information is dispersed
across CMS regional offices. CAP officials told us that they could
initiate four intermediate actions including probation (lab is closely
watched to ensure correction of problems), accreditation with conditions
(nonroutine inspection to be scheduled), suspension of a lab section, and
cessation of a specific type of testing; suspension and probation were
instituted in 2004. According to CAP, in 2005, 28 labs were on probation,
106 labs were accredited with conditions, 1 lab was suspended, and 7 labs
were required to cease a specific type of test. In 2004, JCAHO awarded
conditional accreditation to 3 percent of the labs it inspects because
they were not in substantial compliance with its survey requirements, as
evidenced by the number of requirements not met; JCAHO conducts an on-site
follow-up survey at such labs. From 2002 through 2004, COLA required about
30 labs per year to cease testing due to issues identified during surveys
and about 217 labs per year to cease testing certain tests or specialties
due to unsuccessful proficiency testing.63

61We created a database for the sanctions data contained in CMS's annual
lab registry reports to Congress from 1998 through 2004 and were able to
identify labs that both lost accreditation and had a sanction imposed.

62Pub. L. No. 100-578, S: 2, 102 Stat. at 2911, codified at 42 U.S.C. S:
263a(i)(4)(2000).

                      CMS Oversight of CLIA Is Inadequate

CMS's oversight is not adequate to help ensure that labs meet CLIA
requirements. While CLIA requires proficiency testing quarterly, CMS only
requires such testing three times each year. In addition, the agency is
not meeting its responsibility to determine that accrediting organization
and exempt-state requirements and processes continue to be at least
equivalent to CLIA's. CMS attributed the delay in making equivalency
determinations to having too few staff. Further, ongoing CMS validation
reviews do not provide an independent assessment of the extent to which
surveys identify all condition-level deficiencies-primarily due to their
timing. Finally, CMS does not adequately use data, such as the results of
surveys, to monitor survey organization activities and processes.
Realizing that its existing oversight activities need to be strengthened,
CMS has begun instituting performance reviews to measure survey
organization compliance with its standards and is developing protocols to
ensure improved communication among survey organizations concerning
complaints about lab quality.

CMS's Implementation of Proficiency Testing Is Inconsistent with CLIA

CMS's decision to require proficiency testing for almost all laboratory
tests only three times a year is inconsistent with the statutory
requirement. CLIA requires that proficiency testing be conducted "on a
quarterly basis, except where the Secretary determines for technical and
scientific reasons that a particular examination or procedure may be
tested less frequently (but not less often than twice per year)."64 The
committee report on the bill that forms the basis for much of CLIA
indicated that "proficiency testing should be the central element in
determining a laboratory's competence, since it purports to measure actual
test outcomes rather than merely gauging the potential for accurate
outcomes."65

63In addition to suspension of or limits on testing, COLA uses directed
plans of corrections, such as requiring a lab to hire a consultant or
participate in specific training.

64Pub. L. No. 100-578, S: 2, 102 Stat. 2903, 2907-08, 42 U.S.C. S:
263a(f)(3)(2000).

65H.R. Rep. 100-899 at 28 (1988) reprinted in 1988 U.S.C.C.A.N. 3828,
3849.

In CMS's 1992 rule implementing CLIA, the agency provided a rationale for
reducing the frequency of proficiency testing, but did not provide a
technical and scientific basis for reducing the frequency for particular
procedures or tests.66 According to CMS's justification, experts were
divided on the appropriate frequency of proficiency testing generally. In
addition, requiring fewer events of proficiency testing would give
laboratories more time to analyze the causes of test failures before the
next event of proficiency testing and also enhance proficiency testing's
value as an educational tool. Because CLIA increased the number of labs
that were required to undergo proficiency testing, CMS believed that the
number of organizations that provided proficiency testing services would
not have been able to meet the anticipated increase in demand for testing
services. To help avoid anticipated delays in completing proficiency
testing and reporting requirements, CMS reduced the frequency of testing
events to three times per year.67

CMS's requirement for proficiency testing does not meet the conditions
specified in the statute that must be satisfied in order to require
testing less frequently than quarterly. The language of the statute, as
well as relevant legislative history, indicate that a decision to reduce
the frequency of testing should be in the nature of an exception made with
regard to a particular test, not the norm for all tests, and must be based
on "technical and scientific" considerations related to that particular
test.68 The reasons that CMS gave for requiring only three events per year
were not based on scientific and technical considerations relevant to
particular tests. Instead, CMS's decision was based on concerns of an
administrative and logistical nature that CMS wanted to alleviate by
reducing the frequency of testing events.

66See 57 Fed. Reg. 7002, 7128-29 (1992). Prior to 1992, CMS required
proficiency testing quarterly. See 55 Fed. Reg. 9538 (1990).

67According to a CMS official, the adoption of less frequent proficiency
testing was accompanied by an increase in the number of specimens subject
to proficiency testing from two every 3 months to five every 4 months.

68The committee report provided examples of technical and scientific
considerations justifying an exception to the quarterly testing
requirement. Those examples also stress the significance of excepting
tests from the quarterly testing requirement on the basis of circumstances
presented by the individual test. See H.R. Rep. 100-899 at 29 (1988), 1988
U.S.C.C.A.N. at 3850. That quarterly testing is intended to be the norm is
further evidenced by the committee report's recommendation that the number
of quarters that a laboratory had failed to pass proficiency testing
determine the severity of sanctions imposed. See id. at 30, 1988
U.S.C.C.A.N. at 3851.

CMS Is Late in Ensuring CLIA Equivalency of Exempt States' and Accrediting
Organizations' Inspection Requirements and Processes

We found that CMS has been late in determining that exempt states' and
accrediting organizations' inspection requirements and processes are at
least equivalent to CLIA's. CMS must verify their equivalency and, by
regulation, CMS requires such survey organizations to seek reapproval at
least once every 6 years, or more frequently if deemed necessary. CMS
establishes the time frames for when the next reapproval should occur,
which have ranged from about 15 months to about 6 years. However, CMS has
not completed its equivalency reviews within these time frames and
accrediting organizations and exempt state programs have continued to
operate without proper approval. Equivalency reviews for CAP, COLA, JCAHO,
and Washington due to be completed between November 1, 1997, and April 30,
2001, were an average of about 40 months late. In August 1995, CMS
determined that New York's next equivalency review should be completed by
June 30, 2001, but was over 4 years past due as of December 2005.
Similarly, COLA's equivalency review was about 3 years past due.

Because accrediting organizations and exempt-state programs may choose to
make changes to their inspection requirements between periodic equivalency
reviews (1) accrediting organizations are required to submit changes to
their inspection requirements and policies 30 days prior to changing their
standards69 and (2) exempt-state programs are required to provide notice
when they change their licensure or inspection requirements. Although
federal regulations require CMS to review equivalency when an accrediting
organization or exempt-state program adopts new requirements, a CMS
official told us that the agency is not required to review such changes
before their implementation to ensure equivalency.70 As a result, such
survey organizations may introduce changes that are inconsistent with CLIA
requirements. For example, JCAHO made a significant change to its
inspection requirements in January 2004 but did not receive CMS approval
until 6 months later; CMS did not begin an in-depth review of JCAHO's
revised requirements until early 2005-over a year after they were
implemented by JCAHO.71 According to CMS, their review has identified
several critical areas where JCAHO standards are less stringent than those
of CLIA. JCAHO acknowledged the need to make some adjustments to its
revised requirements.

69For example, CAP typically reorganized, consolidated, or changed its
requirements two times a year. As a result of these changes, about 1,000
requirements were removed and about 1,200 requirements were added over 5
years. COLA has changed its requirements five times since it was first
approved by CMS in 1993. JCAHO officials stated that they make changes to
their requirements each year.

70See 42 C.F.R. S: 493.573(a)(3)(2005).

CMS officials attributed delays in making equivalency determinations and
reviewing interim changes to having too few staff. The CLIA program,
located in CMS's Center for Medicaid and State Operations (CMSO),
currently has approximately 21 full-time-equivalent positions compared to
a peak of 29 such positions several years ago. The reduction occurred over
time through attrition. As required by statute, the CLIA program is funded
by lab fees and since its inception the program's fees have exceeded
expenses. As of September 30, 2005, the CLIA program had a carryover
balance of about $70 million-far more than required to hire an additional
six to seven staff members. However, CMS officials told us that because
the CLIA program staff are part of CMSO, they are subject to the personnel
limits established for CMSO, regardless of whether or not the program has
sufficient funds to hire more staff. Although CMSO is at its authorized
personnel allocation, the CLIA program could hire additional staff with
approval from the Administrator. We were told that CMSO has not requested
such approval.

We also noted issues that raise a question about the thoroughness of CMS
equivalency reviews because some survey organizations' procedures or
policies appear to be less stringent than those required by CMS for the
CLIA program. For example:

           o  Accrediting organizations provide labs more advance notice
           about upcoming surveys than CMS allows state survey agencies to
           give to the labs they inspect.
           o  JCAHO surveyors focus their review of lab testing on the 12
           months prior to the survey. CMS requires that state surveyors
           review the entire 24 months of testing since the last survey.72 
           o  While CMS requires initial and advanced surveyor training, CAP
           encourages but does not require its volunteer surveyors to
           participate in surveyor training.73 
           o  As of August 2005, CAP's policy manual indicates that complaint
           investigations may be announced or unannounced. CMS guidance
           requires that complaint investigations be unannounced.

           Prior to 2005, CMS's equivalency determination reviews focused on
           the inspection requirements themselves, and not the procedures and
           policies used by accrediting organizations and exempt-state
           programs in carrying out oversight of labs; this focus on
           inspection requirements may explain the divergence from the
           policies and procedures CMS requires for state survey agencies.
           During 2006, CMS is simultaneously reviewing the equivalency of
           COLA and JCAHO inspection requirements and, for the first time,
           incorporating on-site observations of accrediting organization
           policies and systems into the review and approval process. For
           example, CMS is checking to ensure that accrediting organizations
           have adequate systems in place to track such things as (1)
           complaints, (2) correction of deficiencies, and (3) proficiency
           testing.

           CMS validation reviews that are intended to evaluate lab surveys
           conducted by both states and accrediting organizations do not
           provide CMS with an independent assessment of the extent to which
           surveys identify all serious-that is, condition-level or
           condition-level equivalent-deficiencies. CMS requires its regional
           offices to conduct validation reviews of 1 percent of labs
           inspected by state survey agencies in a year. In contrast,
           validation reviews of 5 percent of labs inspected by accrediting
           organizations during a year are conducted by state survey agency
           personnel. CMS does not specifically require that validations
           occur in each state and some states do not have validation reviews
           each year. Furthermore, many validation reviews occur at the same
           time a survey organization conducts its inspection and, in our
           view, the collaboration among the two teams during these
           simultaneous surveys prevents an independent evaluation.74

           The requirement to validate 1 percent of labs surveyed by state
           survey agencies in a year-roughly 100 validation reviews each
           year-does not ensure sufficient oversight of state survey
           agencies. The validation review requirement, which is included in
           CMS regional office annual budget memorandums, does not specify
           how many validation reviews must be conducted in each state. While
           the 10 CMS regional offices generally validated 1 percent of the
           state survey agency inspections within their region, they often
           did not validate 1 percent of inspections within each state and,
           in fact, performed none in some states. From 1999 through 2003,
           federal surveyors:

           o  validated less than 1 percent of labs surveyed by state survey
           agencies in an average of about 25 percent of states, ranging from
           7 states in 2002 to 17 states in 2003; and
           o  did not conduct any validation reviews in an average of 16
           percent of states per year, ranging from 3 states in 2002 to 12
           states in 1999.

           In 11 states, no validation reviews were conducted in multiple
           years. For example, no validation reviews were conducted in
           Michigan and Washington, D.C. during 4 of 5 years from 1999
           through 2003. Without validating at least some surveys in each
           state, CMS is unable to determine if the states are appropriately
           identifying deficiencies.

           Seventy-five percent of validations of state lab surveys were
           conducted simultaneously from fiscal years 1999 through 2003.75
           According to CMS officials, the large proportion of simultaneous
           validation reviews provides an opportunity for federal surveyors
           to share information with state surveyors, monitor their
           conformance with CLIA inspection requirements, and identify
           training and technical assistance needs. However, we found that
           such reviews do not provide an accurate assessment of state
           surveyors' ability to identify condition-level deficiencies. Of
           the 13 validation reviews that identified missed condition-level
           deficiencies, only 1 was a simultaneous review (see table 5).
           Validations of state surveys typically utilize one federal
           surveyor for either independent or simultaneous validation
           reviews;76 therefore, increasing the proportion of independent
           validation reviews to strengthen CMS oversight likely would not
           require additional federal surveyors. Moreover, conducting
           independent validation reviews eliminates the extra effort
           required to coordinate schedules to ensure that the validation
           reviews occur at the same time as the state survey.

71On January 1, 2004, JCAHO launched a new accreditation process, which
included a substantial consolidation of lab requirements. Additionally,
JCAHO began using new methodologies, including a new software program that
analyzes data to help focus on-site surveys on priority areas and a tracer
methodology to track patients and specimens through the continuum of lab
services.

72According to JCAHO, the organization plans to change its survey process
to include a review of a random sample of the other records and documents
over the entire 24-month period.

Many CMS Validation Reviews Lack Independence and Reviews Skip Some State Survey
Agencies

73As noted earlier, CAP plans to begin requiring mandatory surveyor
training in mid-2006.

  Validation of State Survey Agency Lab Surveys

74Simultaneous surveys resemble "observational" federal oversight surveys
conducted at nursing homes. We previously reported that such surveys, in
which federal surveyors accompany and observe state surveyors during the
routine inspection of a nursing home, are not a realistic assessment of
state surveyor performance because CMS's presence may make state surveyors
more attentive to their survey tasks than when they are not being
observed, a phenomenon known as the Hawthorne effect. See GAO, Nursing
Home Care: Enhanced Oversight of State Programs Would Better Ensure
Quality, GAO/HEHS-00-6 (Washington, D.C.: Nov. 4, 1999).

75These validation reviews include both exempt-state and state survey
agency lab surveys.

Table 5: Analysis of Results of Simultaneous and Independent Validation
Reviews of State Surveys, Fiscal Years 1999-2003

                       Conducted        Conducted      
                     simultaneously   independently    
                                                                  Number with 
                                           Number with        condition-level 
           Number of                   condition-level           deficiencies 
Fiscal validation                deficiencies state        state surveyors 
year      reviews         Number   surveyors missed Number          missed 
1999          223            141                  0     82               2 
2000          224            135                  0     89               5 
2001          218            184                  1     34               0 
2002          240            199                  0     41               4 
2003          195            167                  0     28               2 
Total       1,100            826                  1    274              13 

Source: CMS.

  Validation of Accrediting Organizations' Lab Surveys

According to CMS guidance, at least one validation review of an
accrediting organization's survey of labs should be conducted
simultaneously each year, but not all validation reviews should be
simultaneous because a combination of simultaneous and independent reviews
provides a balanced view of surveyor performance. CMS officials were
unable to tell us exactly how many of the roughly 275 validation reviews
conducted each year from fiscal year 1999 through fiscal year 2003 were
simultaneous.77 However, one of the three accrediting organizations we
reviewed told us that a significant proportion of their validation reviews
are conducted simultaneously. JCAHO estimated that 33 percent of its
validation reviews were conducted simultaneously. COLA estimated that 9
percent of validation reviews conducted in 2004 and 2005 were
simultaneous. Finally, CAP officials told us that, from 2002 through 2004,
11 percent of validation reviews of CAP-accredited labs were conducted
simultaneously.

76State survey agencies employ 96 full-time-equivalent staff to survey
labs.

77CMS did not begin tracking this information until August 2003.

Given the limitations of simultaneous reviews, conducting independent
validation reviews are a more effective way of ensuring the equivalency of
accrediting organization inspection requirements and processes between the
equivalency determinations. CMS officials told us that the agency's intent
in instituting simultaneous reviews was for state and accrediting
organization surveyors to share best practices, to promote understanding
of each other's programs, and to foster accrediting organization
improvement. They indicated that they considered it a learning experience
both if an accrediting organization surveyor added a deficiency noted by a
state surveyor to a survey report and vice versa. However, most of the
state survey agency officials we interviewed told us that simultaneous
validation reviews do not provide a realistic evaluation of the adequacy
of accrediting organizations' inspection process. In fact, CMS guidance
encourages surveyors to discuss the survey findings prior to concurrent
conferences with lab personnel to review their findings.

From fiscal years 1999 through 2003, state survey agency surveyors found
condition-level deficiencies missed by accrediting organization surveyors
on 64 validation reviews, but only 6 of these validation reviews were
simultaneous.78 In contrast, 58 (91 percent) of the validation reviews
that identified serious deficiencies missed by accrediting organizations
were independent validation reviews. (See table 6.)

78Both survey organizations submit their findings to CMS, which then
compares the findings to determine whether accrediting organization
surveyors missed any condition-level deficiencies. Examples of
condition-level deficiencies missed by accrediting organizations include:
(1) lab did not correctly calculate the results of tests used to monitor
patients using a blood-thinning medication, which could result in serious
medical complications such as internal bleeding; (2) lab did not follow
manufacturer's instructions for calibrating checks of certain test
equipment; and (3) lab director failed to provide overall management and
direction of lab, such as ensuring timely enrollment in a proficiency
testing program and corrective actions following a proficiency testing
failure.

Table 6: Analysis of Results of Simultaneous and Independent Validation
Reviews of Accrediting Organizations' Surveys, Fiscal Years 1999-2003

                      Validation reviews that  
                       found condition-level   
                      deficiencies missed by   
                            accrediting        
                     organizations' surveyors  
           Number of                                                   Number 
Fiscal validation                           Number conducted     conducted 
year      reviews                     Total   simultaneously independently 
1999          227                         8                0             8 
2000          265                        17                3            14 
2001          214                         8                1             7 
2002          317                        14                1            13 
2003          348                        17                1            16 
Total       1,371                        64                6            58 

Source: CMS.

Note: While our analysis covered validation reviews for all six
accrediting organizations, CAP, COLA, and JCAHO account for the vast
majority of such reviews. CMS officials were unable tell us the total
number of validation reviews conducted simultaneously and independently
during each fiscal year.

CMS Use of Data for Oversight of CLIA Program Is Limited

CMS does not routinely collect and analyze data essential for effective
oversight of the CLIA program but has initiatives to automate some
available data to make them more accessible for analysis. Using data to
analyze activities across survey organizations can be a powerful tool in
improving CMS oversight of the CLIA program. Such analyses include
identifying and addressing inconsistencies in how surveys are conducted.
Although CMS tracks the most frequently cited deficiencies at labs in an
effort to improve quality, it does not routinely track the proportion of
labs, by state, in which state survey agencies identify condition-level
deficiencies-those that denote serious or systemic problems. According to
a CMS official, the agency has not evaluated variability in such
deficiencies since 1999.79 As noted earlier in this report, variability in
survey findings suggests inconsistencies in how surveys are conducted. CMS
also does not require exempt-state programs and accrediting organizations
to routinely submit data on serious deficiencies identified at the labs
they inspect, unless the deficiencies pose immediate jeopardy to the
public or an individual's health.80 As noted earlier, the lack of a common
vocabulary on what constitutes a serious deficiency would make it
virtually impossible for CMS to analyze such data.

79Our nursing home work found a similar absence of analysis by CMS
regarding trends in serious problems identified by state surveyors. See
GAO, Nursing Home Quality: Prevalence of Serious Problems, While
Declining, Reinforces Importance of Enhanced Oversight, GAO-03-561
(Washington, D.C.: July 15, 2003).

We also found that CMS does not effectively use available data to assess
clinical lab quality in areas such as proficiency testing, sanctions, and
complaints. For example, CMS's analysis of proficiency testing data for
all labs showed improvements over time. As reported earlier, proficiency
testing failures have increased for labs surveyed by CAP and JCAHO.
Comprehensive analysis of the proficiency testing database is particularly
valuable because it provides a uniform way to assess the quality of lab
testing across survey organizations, which is not currently available for
survey results. CMS is now in the process of automating the annual
registry of sanctioned labs, which should help it identify important
trends, such as the infrequent use of alternative sanctions. Automating
the registry, however, will not address the lack of data on (1) steps
taken by state survey agencies and regional offices when labs have their
accreditation revoked or (2) interim steps, short of revocation of
accreditation, that accrediting organizations take to help encourage lab
compliance. CMS also lacks a complaints database, and therefore was unable
to assess the impact of CAP's decision to require labs to prominently
display a poster on how to file a complaint. CMS is developing, and plans
to launch, a complaints database in March 2006.81

80Officials from a state survey agency told us that, while they do not
want to receive routine survey reports from accrediting organizations,
they do want to receive information when accrediting organizations
identify problem labs-before a lab loses its accreditation and becomes the
responsibility of the state survey agency. When an accrediting
organization revokes a lab's accreditation, the state survey agency
becomes responsible for determining whether to recommend revocation of the
lab's CLIA certificate to the appropriate CMS regional office.

81In response to communication problems highlighted by the complaint
investigations at a Maryland hospital lab, CMS has been meeting regularly
with officials from exempt-state programs, accrediting organizations,
state survey agencies, and its regional offices to discuss and improve
coordination and data sharing-particularly the handling of complaints. The
Maryland state survey agency identified deficiencies during a 2004
complaint investigation but did not inform CAP, the organization
responsible for surveys of the Maryland hospital lab. CAP officials told
us that they first learned about the 2004 complaint investigation that
revealed problems with HIV and hepatitis testing equipment from
newspapers.

CMS Implementing Performance Reviews for Survey Organizations

CMS has implemented performance reviews for state survey agencies and is
in the process of developing such reviews for accrediting organizations.
First implemented in 2004, the annual CLIA state performance reviews
evaluate each survey agency's ability to accomplish its lab oversight
responsibilities.82 The reviews, conducted on-site by CMS regional office
staff, measure performance in 13 areas, such as the timely conduct of
surveys and the appropriate documentation of any deficiencies
identified.83 According to CMS, the reviews are based on the
performance-improvement model that characterizes much of the
administration of the CLIA program.84 Consequently, the primary role of
regional offices in conducting the reviews is to provide education and
support for state survey agency improvement. For the 2004 reviews, 38
states were required to submit corrective action plans to their respective
CMS regional offices in at least 1 of the 13 areas examined. Three areas
required the most corrective action plans: principles of documentation,
proficiency testing desk reviews, and survey time frames.85

           o  Principles of documentation. CMS found that some state survey
           agencies lacked the supervisory personnel to conduct internal
           reviews intended to ensure the appropriate documentation of
           deficiencies. It also found that some state survey agencies did
           not follow the protocol instructions on how to quantify such
           reviews.
           o  Proficiency testing desk reviews. Because of personnel
           shortages, some state survey agencies were unable to perform
           proficiency testing desk reviews between surveys, waiting instead
           until the next on-site survey to address unsuccessful proficiency
           testing. CMS plans to provide additional surveyor training on desk
           review requirements.
           o  Survey time frames. CMS regional office staff were inconsistent
           in scoring whether state survey agencies met the established time
           frames for initial surveys. While some regions were lenient if a
           state missed the time frame by just 1 day or provided a reasonable
           explanation-such as staff turnover or illness-other regions were
           more stringent in scoring states against the standard.

           However, it is not yet clear to what extent the 2004 scores
           represent state survey agency shortcomings or a learning curve for
           the states in understanding the performance review protocols.

           In partnership with the accrediting organizations, CMS is
           developing performance standards comparable to, but different
           from, those implemented in 2004 for state survey agencies. For
           example, both the state survey agency review protocols and those
           proposed for accrediting organizations measure the timeliness of
           the surveys, but those proposed for the latter would also focus on
           several areas that are unique to accrediting organizations. The
           performance standards would include (1) timely and consistent
           information sharing and (2) alerting CMS about decisions to limit
           or remove accreditation in a timely manner. According to a CMS
           official, the agency plans to phase in the performance standards,
           starting with a standard on complaints. For example, if the CLIA
           complaints database is activated in March 2006, CMS could begin to
           monitor accrediting organization responsiveness to, and outcomes
           of, complaints. Because the database will contain national lab
           complaint data, CMS will be able to compare the volume and outcome
           of complaints across survey organizations. According to CMS,
           implementation of the accrediting organization performance
           standards will be a central-not regional-office responsibility.

           Clinical labs play a pivotal role in the nation's health care
           system by diagnosing many diseases, including potentially
           life-threatening diseases, so that individuals receive appropriate
           medical care. Given this important role, lab tests must be
           accurate and reliable. CMS and survey organization oversight is
           intended to ensure that labs produce reliable test results, a key
           objective of CLIA. Our work demonstrated that the oversight of
           clinical labs needs to be strengthened in several areas.

           Determining the quality of lab testing is difficult because it is
           virtually impossible to crosswalk inspection requirements across
           survey organizations. Without standardized survey findings across
           all survey organizations, CMS cannot tell whether the quality of
           lab testing has improved or worsened over time or whether
           deficiencies are being appropriately identified.

           Lab oversight has weaknesses that make it difficult to determine
           the quality of lab testing because they mask quality problems. To
           help surveys provide a realistic picture of day-to-day operations,
           CAP and JCAHO began unannounced surveys of the labs they
           survey-generally hospital labs-in 2006. While unannounced surveys
           at physician office labs may not be practical, Washington's exempt
           program and COLA currently give such labs more advance notice than
           the 2 weeks CMS prescribes for labs inspected by state survey
           agencies. Similarly, the greater weight that CMS and survey
           organizations sometimes place on their educational, as opposed to
           their regulatory role may lead to survey findings that do not
           accurately reflect lab quality. Educating labs to ensure
           high-quality testing should complement but not replace the
           enforcement of CLIA inspection requirements. The low number of lab
           complaints may be the result of a lack of information about how to
           file a complaint and lab workers' fear of retaliation. Because
           protecting the anonymity of lab workers who file complaints is
           difficult, whistle-blower protections for such individuals are
           particularly important. Finally, labs with the same serious
           deficiencies on consecutive surveys often escape sanctions, even
           though Congress authorized alternative sanctions to give CMS more
           flexibility to achieve lab compliance. Without the threat of real
           consequences, labs may not be sufficiently motivated to comply
           with CLIA inspection requirements.

           CMS's oversight is not adequate to enforce CLIA requirements. The
           agency is not requiring labs to participate in proficiency testing
           on a quarterly basis, as required by CLIA. Furthermore CMS is not
           conducting CLIA-equivalency determinations within the time frames
           it established for such reviews, nor has it always reviewed
           changes to exempt-state and accrediting organizations' inspection
           requirements before their implementation, even though it requires
           their submission to ensure continued CLIA equivalency of their
           requirements. Although the CLIA program has generated funds, CMS
           agencywide staffing limitations have prevented the program from
           hiring sufficient staff to complete equivalency reviews in a
           timely manner. Many validation reviews are conducted at the same
           time a survey organization conducts its survey, and such
           simultaneous reviews may not provide a true assessment of surveyor
           performance. Independent validation reviews of accrediting
           organization surveys are critical because CMS has not conducted
           equivalency reviews within the time frames it established. We also
           found that few validation reviews of state survey agency lab
           inspections are conducted each year and that none occurred in some
           states. Because state surveyors conduct validation reviews of
           accrediting organizations to ensure the continuing CLIA
           equivalency of their inspection requirements, conducting an
           appropriate number of validation reviews of state survey agency
           lab inspections is critical. CMS also has not yet taken the lead
           in ensuring the availability and use of data from survey
           organizations to help it monitor their performance-particularly
           the consistency with which surveys are conducted. CMS is creating
           a new complaint database, but its plan to automate the existing
           sanctions registry will not address the lack of data on
           enforcement actions taken by state survey agencies and regional
           offices when labs have their accreditation revoked.

           To enable CMS to track the nature and extent of lab quality
           problems across survey organizations, we recommend that the CMS
           Administrator take the following action:

           o  Work with exempt-state programs and accrediting organizations
           to standardize their categorization and reporting of survey
           findings in a way that tracks to CLIA inspection requirements and
           allows for meaningful comparisons across organizations, such as
           the analysis of trends in the citation of condition-level
           deficiencies.

           To ensure consistency in the oversight of labs by survey
           organizations, we recommend that the CMS Administrator take the
           following four actions:

           o  Ensure that the advance notice of upcoming surveys provided to
           physician office labs is consistent with CMS's policy for advance
           notice provided by state survey agencies.
           o  Ensure that regulation of labs is the primary goal of survey
           organizations and that education to improve lab quality does not
           preclude the identification and reporting of deficiencies that
           affect lab testing quality.
           o  Impose appropriate sanctions on labs with consecutive
           condition-level deficiencies in the same requirements.
           o  Require all survey organizations to develop, and require labs
           to prominently display, posters instructing lab workers on how to
           file anonymous complaints.

           To improve oversight of labs and survey organizations, we
           recommend that the CMS Administrator take the following eight
           actions:

           o  Consistent with CLIA, require quarterly proficiency testing,
           except when technical and scientific considerations suggest that
           less frequent testing is appropriate for particular examinations
           or procedures.
           o  Ensure that evaluations of exempt-state and accrediting
           organization inspection requirements take place prior to
           expiration of the period for which they are approved in order to
           ensure the continued equivalency of their requirements with
           CLIA's.
           o  Ensure that changes to the inspection requirements of exempt
           states and accrediting organizations be reviewed prior to
           implementation, as required by regulation, to ensure that
           individual changes do not affect the overall CLIA equivalency of
           each organization.
           o  Allow the CLIA program to utilize revenues generated by the
           program to hire sufficient staff to fulfill its statutory
           responsibilities.
           o  Ensure that federal surveyors validate a sufficient number of
           inspections conducted by each state survey agency to allow a
           reasonable estimate of their performance, including a minimum of
           one independent validation review for each state survey agency
           surveyor.
           o  Require that almost all validation reviews of each accrediting
           organizations' surveys be an independent assessment of
           performance.
           o  Collect and routinely review standardized survey findings and
           other available information for all survey organizations to help
           ensure that CLIA requirements are being enforced and to monitor
           the performance of each organization.
           o  Establish an enforcement database to monitor actions taken by
           state survey agencies and regional offices on labs that lose their
           accreditation.

           We provided a draft of this report to CMS, and to CAP, COLA, and
           JCAHO-the three laboratory accrediting organizations included in
           our review. CMS strongly endorsed our overall conclusion that
           quality assurance for the nation's clinical labs should be
           strengthened and noted that the report provided insights into
           areas where it can improve, augment, and reinforce oversight of
           both labs and accrediting organizations to ensure quality testing.
           Overall, CMS concurred with 11 of our 13 recommendations. Despite
           this endorsement, however, CMS (1) provided an alternative
           assessment of lab quality, (2) disagreed that the phase-in of
           certain CLIA requirements inappropriately stressed education as
           opposed to regulation, (3) expressed concern about how to identify
           and sanction labs with repeat condition level deficiencies, (4)
           disagreed with our recommendation regarding the frequency of
           proficiency testing, and (5) stated that it was already meeting
           our recommendation to conduct almost all validation reviews of
           each accrediting organization independently. We continue to
           believe that implementation of these recommendations is necessary
           for the effective oversight of labs. (CMS's comments are
           reproduced in app. IV.) CAP indicated that it took seriously our
           findings and recommendations and intended to determine if there
           were additional measures it could take to strengthen its own
           oversight. COLA said that our recommendations to improve CMS
           oversight of survey organizations had merit. Nonetheless, CAP,
           COLA, and JCAHO disagreed with some of our findings and
           recommendations to CMS. (CAP, COLA, and JCAHO's comments are
           reproduced in app. V, VI, and VII, respectively.) Our evaluation
           first responds to CMS's and related accrediting organizations'
           comments and then addresses additional comments by accrediting
           organizations.

           CMS and COLA commented that lab performance has improved since the
           enactment of CLIA. In particular, CMS pointed to the substantial
           decline-from about 80 percent to about 42 percent-in the
           percentage of labs nationwide with deficiencies between 1994 and
           2004. It is important to note that CMS's data (1) do not
           distinguish between serious condition-level deficiencies and less
           serious standard-level deficiencies,86 (2) include the early
           start-up period when physician office labs were first regulated,87
           and (3) exclude deficiency data on the substantial number of labs
           surveyed by accrediting organizations and state CLIA-exempt
           programs. Due to these shortcomings, we do not believe that CMS's
           data provide an accurate assessment of lab quality nationwide.

           Based on the limited data available on state survey agency
           inspections of labs since 1998 and the lack of any comparable data
           on accrediting organization and exempt-state program survey
           findings, we concluded that insufficient data existed to identify
           the extent of serious quality problems at labs. CMS did not retain
           backup files of pre-2004 data on deficiencies identified by state
           survey agencies. Although CMS has determined that accrediting
           organization and exempt-state program lab requirements are at
           least equivalent to CLIA's, there is no agreement across survey
           organizations on how to distinguish serious from less serious
           deficiencies. While CMS concurred with our recommendation to
           standardize the categorization and reporting of survey findings in
           a way that tracks to CLIA and allows meaningful comparisons across
           survey organizations, it noted that a straightforward linkage of
           requirements is limited by CMS's authority under the statute-that
           is, survey organizations are permitted by statute to have
           different requirements-and that it will approach implementation of
           our recommendation cautiously. JCAHO said that it agreed with the
           need for a common, agreed upon, taxonomy that could be used by all
           survey organizations to track serious deficiencies, but commented
           that it thought CMS's implementation of our recommendation would
           require a revamping of JCAHO's accreditation system. That was not
           the intent of our recommendation and it is clear from CMS's
           comments that its implementation of our recommendation would not
           require an overhaul of accrediting organizations' systems. CAP
           acknowledged the complexity and inherent challenges in measuring
           the quality of lab testing, but noted that it is committed to
           working to develop better systems to detect labs with serious
           quality problems-those that impact patient care.

           The statutory authority that permits standards different from
           CMS's (provided they are at least as stringent) does not impair
           the ability to develop a crosswalk that allows for meaningful
           comparisons across survey organizations-such as an analysis of
           trends in the citation of condition-level deficiencies. In fact,
           CMS regulations already require accrediting organizations and
           exempt-state programs to submit a crosswalk-detailed comparisons
           of their individual accreditation or licensure approval
           requirements with comparable CLIA condition-level
           requirements-when they apply and reapply for approval from CMS.88
           Such a comparison is possible because CMS already identifies
           instances when accrediting organizations have missed
           condition-level requirements during validation reviews. For
           example, CMS should require survey organizations to (1) indicate
           which of their requirements relate to each CLIA condition-level
           requirement, and (2) explain which deficiencies in their
           requirements, if cited, should be considered equivalent to CLIA
           condition-level deficiencies.

           CMS also pointed to the steady increase in successful proficiency
           testing across all labs as an indication of improvements in lab
           quality. Our analysis of proficiency testing results suggested
           that lab quality had not improved at hospital labs in recent
           years. CMS correctly noted that the overall proportion of labs
           with no test failures increased from about 88 percent in 1998 to
           about 93 percent in 2003-that is, fewer labs failed proficiency
           testing. However, by focusing on overall proficiency testing
           results, CMS data mask trends in failure rates for subsets of labs
           such as hospital labs. For example, from 1999 through 2003, the
           percentage of CAP-surveyed labs with proficiency testing failures
           increased from 4.1 percent to 6.8 percent; CAP generally inspects
           hospital labs. CMS also commented that the overall improvement
           cannot be dismissed as a result of some labs being granted waived
           status because the more dramatic improvements predated the recent
           increase in the number of waived labs. It further commented that
           removing waived labs from the data would not result in improved
           performance rates. First, the number of waived labs-those
           performing waived tests or provider-performed microscopy-increased
           by about 26,600 from 1993 though 1998 and then increased by
           another approximately 33,700 labs from 1998 through 2004. Second,
           CMS's comment suggested that it had conducted an analysis of the
           impact of removing waived labs from the proficiency testing data.
           However, it did not provide any data analysis when we subsequently
           asked to see the evidence behind its assertion. COLA also
           addressed this issue, and did not challenge our conclusion that
           the decrease in proficiency testing failures for physician office
           labs might not represent an actual improvement in lab quality, but
           instead could reflect the fact that some problematic labs are no
           longer surveyed.

           CMS agreed that it was important to maintain an appropriate
           balance between its regulatory and educational approaches to CLIA
           implementation. While CMS noted that objective review and feedback
           are the bedrock of education, it emphasized that the educational
           approach does not preclude surveyors from identifying lab
           deficiencies. CAP and COLA offered similar comments. However, we
           found evidence that the goal of educating lab workers sometimes
           takes precedence over, or precludes, the identification and
           reporting of deficiencies and recommended that CMS take steps to
           ensure that regulation remains the primary goal of surveys. To
           address this problem, CMS stated that it will provide additional
           state agency surveyor training, improve guidance, develop an
           action plan to promote greater consistency among surveyors, and
           institute periodic performance and consistency reviews. CMS's
           comments did not address evidence we presented that an educational
           emphasis may also prevent fulfillment of regulatory
           responsibilities by some accrediting organizations.

           CMS disagreed that the extended phase-in periods for new quality
           control requirements and proficiency testing for lab technicians
           who interpret Pap smears were inappropriate. COLA noted that
           federal requirements in many regulated industries are phased in to
           allow them time to understand and effectively implement the
           requirements. CMS reaffirmed that, in the case of significant new
           requirements and for the time period specified by CMS, the
           educational approach may result in identified deficiencies being
           communicated to laboratories without a concomitant citation, as is
           the case with quality control and Pap smear testing requirements.
           As discussed in the report, we believe that CMS's educational
           phase-in periods are excessive. We found that the phase-in period
           for new quality control requirements was extended from 2 years to
           about 4 years, in part because of the lack of lab "buy-in" for
           some of the new policies and procedures. Similarly, the phase-in
           period for Pap smear proficiency testing is 2 years, despite (1)
           CMS's concern about some of the high initial test failure rates,
           (2) the consequences of inaccurate test results on patients'
           diagnoses and treatment, and (3) the approximately 13-year time
           lag between the 1992 implementation of the CLIA regulations and
           the commencement of Pap smear proficiency testing.

           In commenting on our recommendation to appropriately sanction labs
           with repeat condition-level deficiencies, CMS acknowledged the
           need to carefully monitor repeat deficiencies but expressed
           concern that focusing on the condition cited may not indicate a
           true repeat deficiency because the underlying failures could have
           been different in the two consecutive surveys for those labs.89
           CMS's assertion is inconsistent with its own policy on serious,
           repeat deficiencies for other providers, such as nursing homes. In
           general, immediate sanctions must be imposed on nursing homes with
           consecutive serious deficiencies, regardless of whether the
           deficiencies are in the same care area. As we have previously
           reported, allowing providers to avoid sanctions by correcting
           serious deficiencies contributes to an up-and-down pattern of
           compliance and undermines the deterrent effect of sanctions.90
           According to the CLIA legislative history, congressional concern
           about labs with repeat deficiencies led to the introduction of
           alternative sanctions such as civil money penalties as a
           substitute for more severe principal sanctions, which include
           termination from the CLIA program.

           CMS disagreed with our recommendation that it require quarterly
           proficiency testing except when technical and scientific
           considerations indicate that less frequent testing is justified
           for particular tests; CMS insisted that proficiency testing three
           times a year was "appropriate." CMS stated that CMS and the
           Centers for Disease Control and Prevention had together determined
           that the reduced frequency was based on technical and scientific
           grounds. We asked for a record of the agencies' deliberation
           supporting that decision. CMS supplied a brief, undated narrative,
           which it attributed to the Centers for Disease Control and
           Prevention. It was not clear to us that this narrative was
           contemporaneous with the decision to reduce the frequency of
           proficiency testing. Moreover, the narrative focused on the
           relative costs and benefits of proficiency testing at various
           intervals. There was no analysis of technical and scientific
           considerations with regard to particular tests that presented a
           basis for reducing the frequency.

           Based on CMS's response, we maintain that CMS's decision to
           require proficiency testing three times a year is not authorized
           by CLIA. CMS did not dispute that, according to CLIA, it must base
           the decision to reduce the frequency of proficiency testing on
           scientific and technical considerations relevant to particular
           tests, and that the decision to reduce the frequency is in the
           nature of an exception to the norm of quarterly testing. CMS also
           acknowledged that the "public explanation" contained in the
           preamble to the rule setting the proficiency testing requirement
           at three times a year referred only to general concerns about the
           perceived burden associated with quarterly testing. For example,
           CMS stated in its final rule that the prospect of reduced
           frequency would provide a "needed respite" to both laboratories
           and proficiency test providers. In sum, CMS has not adhered to the
           conditions set out in the statute for reducing the frequency of
           proficiency testing and has implemented a policy that is not
           supported by statutory authority.

           CMS acknowledged the need to complete timely equivalency reviews
           of accrediting organization and exempt-state requirements, which
           were an average of about 40 months late for the 3-1/2-year period
           we examined. Regarding interim changes made between periodic
           equivalency reviews, CMS agreed with our recommendation and stated
           it would review such interim changes for both accrediting
           organizations and exempt-state programs prior to implementation,
           as required by regulation.91

           Furthermore, CMS indicated that changes to accrediting
           organization requirements did not necessarily impact CLIA
           equivalency determinations because accrediting organizations may
           have more stringent requirements than CLIA's. While possibly true,
           CMS must review the changes to determine whether CLIA equivalency
           is affected. For example, in 2005, when it reviewed JCAHO's
           revised standards a year after they were implemented, CMS
           identified several critical areas where JCAHO's standards were
           less stringent than those of CLIA.

           CMS acknowledged that a significant increase in workload and the
           decline in CLIA program staff were factors which contributed to
           delays in making equivalency determinations and reviewing interim
           changes. Although CMS stated that it reserved the right to manage
           the work within available resources and its assessment of
           priorities, it also made a commitment to explore our
           recommendation to utilize revenues generated by the CLIA program
           to hire sufficient staff to fulfill its responsibilities. We
           believe that additional staff would not only improve the
           timeliness of equivalency reviews, but also their thoroughness.

           CMS stated that, consistent with our recommendation, 88 percent of
           accrediting organization validation reviews were conducted
           independently in calendar year 2005. However, our recommendation
           was to require that almost all validation reviews of each
           accrediting organizations' surveys be conducted independently.
           CMS's comments do not indicate the proportion of independent
           validation reviews conducted for each accrediting organization.
           Because CMS did not begin collecting data on the number of
           simultaneous accrediting organization validation reviews performed
           until August 2003, we relied on estimates from accrediting
           organizations.92 CMS did not challenge JCAHO's estimate that 33
           percent of its validation reviews were simultaneous, compared to
           about 10 percent for CAP and COLA. We do not believe that
           performing an estimated 33 percent of JCAHO's validation reviews
           simultaneously is consistent with our recommendation.

           COLA commented that simultaneous validation reviews are useful in
           assuring consistency and in providing an understanding of
           processes across survey organizations. It also questioned the
           accuracy of most of the missed condition-level deficiencies
           identified by CMS during independent validation reviews. We did
           not assess the process CMS uses to identify such missed
           deficiencies but based on a discussion with CMS officials it
           appears that the process is thorough and time consuming. JCAHO
           commented that we had misinterpreted the results of simultaneous
           and independent validation reviews of accrediting organizations
           because, by JCAHO's estimate, the proportion of missed
           condition-level deficiencies is roughly equivalent for both types
           of surveys. We did not find the assumptions behind JCAHO's
           estimate convincing, given the lack of data on the actual number
           of simultaneous versus independent validation reviews conducted
           for each accrediting organization. Furthermore, most of the state
           survey agency officials we interviewed, whose inspectors conduct
           accrediting organization validation reviews, told us that
           simultaneous validation reviews do not provide a realistic
           evaluation of the adequacy of accrediting organizations'
           inspection processes.

           Additional CAP comments. CAP commented that we underestimated the
           value of using lab professionals in the inspection process and
           that we provided no factual evidence that their use was less
           effective than other models. In contrast to CAP, other survey
           organizations employ dedicated staff surveyors who have mandatory
           and continuing education requirements. In addition, such dedicated
           surveyors conduct from 30 to about 200 surveys per year compared
           to CAP's lab professionals who volunteer to perform about 1 survey
           per year. CAP partially addressed our concern about the lack of
           mandatory training for its volunteer surveyors. It plans to begin
           requiring training for survey team leaders in July 2006 and for
           survey team members in 2007. However, CAP's proposed new mandatory
           training is much less extensive than that required by other survey
           organizations.

           Moreover, we reported that some CAP surveyors we interviewed
           raised a concern about having survey team leaders who are also the
           day-to-day supervisors of team members. For example, lack of
           agreement about the seriousness of a deficiency could result in
           the team leader instructing the team to downgrade the deficiency
           to a recommendation, a less serious finding that does not appear
           in the inspection report. Team members who are subordinates to the
           team leader may feel that they have no other recourse than to
           follow the team leader's instructions. CAP recently revised its
           conflict-of-interest policy which now instructs all parties to be
           cautious to retain objectivity in fact finding throughout the
           inspection process. We do not believe that this change in CAP's
           conflict-of-interest policy addresses the concerns raised by the
           CAP surveyors we interviewed. In its comments, CAP indicated that
           it would continue to closely monitor this issue to determine if
           further actions were necessary.

           Additional COLA comments. COLA disagreed with our assertion that
           announced surveys may result in an unrealistic picture of lab
           quality-a conclusion supported by CMS regional office staff and
           most state survey agency officials we interviewed. We acknowledged
           that unannounced surveys of the physician office labs typically
           surveyed by COLA and state survey agencies were not practical
           given the unpredictable operating hours of such labs and need to
           minimize disruptions to patient care. However, we recommended that
           the advanced notice be limited to the 2 weeks permitted by CMS for
           state survey agencies. COLA currently provides up to 12 weeks
           advanced notice. COLA contends that providing up to 6 months of
           advanced notice before a survey would only improve the lab's
           operation more quickly if the lab took that opportunity to review
           COLA's self assessment questions and correct any missing or
           incorrect processes or documentation. We believe that COLA's
           example underscores the importance of our recommendation; such
           actions should be an ongoing process at labs-not a reaction to an
           upcoming inspections.

           Additional JCAHO comments. JCAHO said that our recommendation that
           all survey organizations develop and require labs to prominently
           display posters that instruct lab workers on how to file anonymous
           complaints was too narrow and prescriptive and may inadvertently
           limit organizations from using other, more effective ways to
           educate lab workers on this topic. JCAHO did not explain how
           implementing our recommendation would limit other initiatives. In
           fact, CMS's comments identified a number of promising approaches
           that it believed could supplement posters. JCAHO also said that
           our analysis of the increase in CAP complaints after it required
           posters in the labs it inspects failed to recognize a broad
           national trend. JCAHO indicated that it also experienced a
           dramatic increase in lab complaints between 2004 and 2005 without
           the use of posters.93 This increase may be related to JCAHO's July
           2005 requirement for labs to educate staff on how to report
           concerns. CAP told us that during the 3 months after they required
           a poster to be displayed they observed an immediate increase in
           the number of complaints. Thus, CAP lab complaints increased by
           over 100 percent in 2004 compared to 2003 and by another
           approximately 71 percent in 2005. We continue to believe that
           CAP's experience suggests that complaint posters can be an
           important way to encourage lab workers to communicate their
           concerns.

           CMS, CAP, COLA, and JCAHO also provided technical comments which
           we incorporated as appropriate.

           As arranged with your offices, unless you publicly announce its
           contents earlier, we plan no further distribution of this report
           until 30 days after its issue date. At that time, we will send
           copies to the Administrator of the Centers for Medicare & Medicaid
           Services and appropriate congressional committees. We will also
           make copies available to others upon request. In addition, the
           report will be available at no charge on the GAO Web site at
           http://www.gao.gov .

           If you or your staff have any question s about this report, please
           contact me at (312) 220-7600 or [email protected] . Contact
           points for our Offices of Congressional Relations and Public
           Affairs may be found on the last page of this report. GAO staff
           who made major contributions to this report are listed in appendix
           VIII.

           Leslie G. Aronovitz Director, Health Care

           This appendix contains examples of lab errors and their
           consequences, illustrating the importance of the quality of lab
           testing and the effects of lab errors on patient health. The
           examples in table 7 are summarized from case studies in the
           journal Laboratory Errors and Patient Safety.

82According to a CMS official, the Seattle regional office was in the
forefront of developing a set of performance standards for Washington's
CLIA-exempt program. The standards, though not identical to those
implemented for state survey agencies, have been in place for several
years. As of November 2004, the New York regional office is developing
similar performance standards for the New York CLIA-exempt program.

83The 13 areas are personnel qualifications, financial management,
completion of workload targets, survey selection and scheduling,
outcome-oriented survey process, acceptable plan of correction,
complaints, ongoing training activities, data management, survey time
frames, proficiency testing desk review, principles of documentation, and
enforcement.

84In explaining the purpose of the performance reviews to state survey
agencies, CMS noted that they were designed to serve as an additional
opportunity to further the agency's educational and supportive efforts of
state survey agencies. The goal is to promote optimal performance by
identifying areas needing improvement and corrective action. Survey
agencies are expected to have systems in place for monitoring and
evaluating the efficiency of their corrective actions.

85During desk reviews, state survey agency staff track the proficiency
testing results of labs using data reports from proficiency testing
providers and request that labs initiate corrective actions when the
results are below certain thresholds with some frequency.

                                  Conclusions

                      Recommendations for Executive Action

        Agency and Accrediting Organization Comments and Our Evaluation

Assessment of Lab Quality

86We asked CMS officials to provide the data points for fig. 1 in the
agency's comments on our draft report. The data provided by CMS show that
the percentage of labs with condition-level deficiencies remained
relatively constant over time-fluctuating between 6 and 8 percent from
1996 through 2005. However, the pre-2004 and post-2004 data are not
comparable because CMS revised CLIA survey requirements in January 2004.
Additionally, the data likely understate the actual number of
condition-level deficiencies. As noted earlier, state survey agencies do
not consistently cite all condition-level deficiencies identified during
inspections and CMS has instructed states not to include deficiencies
related to the new 2003 lab quality control requirements in a lab's survey
report.

87Generally, our analysis focused on the period 1998 through 2004,
recognizing that the early years of CLIA implementation were probably
atypical because the law expanded oversight to previously unregulated
physician office labs and many labs shifted from more complex testing that
required routine inspections to less complex testing that did not.

88See 42 C.F.R. S: 493.553(a)(1)(2005).

Educational Focus of CLIA

Sanctioning Labs with Serious, Repeat Deficiencies

89JCAHO expressed a similar concern.

Proficiency Testing Frequency

90See GAO, Nursing Homes: Additional Steps Needed to Strengthen
Enforcement of Federal Quality Standards, GAO/HEHS-99-46 (Washington,
D.C.: Mar. 18, 1999).

Equivalency of Accrediting Organization and Exempt State Programs

Accrediting Organization Validation Reviews

91CMS's comments explained that its regulations require 30-day advanced
notice by accrediting organizations but not by exempt-state programs.

Additional Comments by Accrediting Organizations

92Our analysis of accrediting organization validation reviews covered
fiscal year 1999 through fiscal year 2003. We excluded fiscal year 2004
because CMS had not yet completed its analysis of condition-level
deficiencies missed by accrediting organizations.

93JCAHO miscalculated the increase in complaints from 2004 to 2005.
According to the data provided by JCAHO, the increase was 57 percent, not
64 percent. JCAHO's 2005 complaint data were not available when we
initially collected data on complaints received and substantiated by
survey organizations.

Appendix I: Effects of Lab Errors on Patient Health

Table 7: Effects of Lab Errors on Patient Health

                                               Effects of error on patient    
Description of lab error                    health                         
Example 1: Delayed reporting of elevated    
lab value                                   
      o  A 59-year-old woman with a history of    o  There was a delay in the 
      a rapid and irregular heart beat and        diagnosis of a critically   
      stroke is taking coumadin, a blood          elevated blood clotting     
      thinning agent. Her primary care            level.                      
      physician has lab tests completed           o  The patient took         
      regularly to ensure that the coumadin       coumadin inappropriately    
      dose is sufficient to maintain a lab        for 2 days.                 
      value in the range of 2-3. The results      o  The patient experienced  
      of one Friday's test was 5.7, a high        significant bleeding in her 
      value out of her target range,              digestive tract related to  
      indicative that her blood is too thin       her impaired blood clotting 
      and clots too slowly. This lab test         status.                     
      value was documented but the patient's      o  The patient had to be    
      primary care physician was not notified     hospitalized to stabilize   
      about the elevated value. The following     her health condition.       
      Monday morning a lead technologist          o  The patient was exposed  
      noticed that the physician had not been     to blood products, putting  
      called and immediately contacted a nurse    her at risk for a           
      at the physician's office. The nurse,       transfusion reaction and    
      alarmed that both the physician's office    exposure to infectious      
      and the patient had not previously been     agents.                     
      notified, tried to contact the patient   
      at home. The patient had gone to the     
      emergency room (ER) and was admitted to  
      the hospital with an even higher blood   
      clotting value of 7.2. With hospital     
      treatment, her blood clotting value was  
      reduced to 2.3 and she was discharged 3  
      days later.                              
      o  The physician ordering the test did   
      not receive notice of the high blood     
      clotting lab value for 3 days, even      
      though the lab result was documented     
      within an hour of collection. In         
      addition, the lab failed to follow up on 
      the situation within an appropriate time 
      frame.                                   
Example 2: Human recording and data entry   
error                                       
      o  A 60-year-old man with a history of      o  The patient              
      chronic liver disease went to the ER        inappropriately received    
      with a 36-hour history of chest pain.       several unnecessary         
      The ER physician ordered a cardiac blood    medications.                
      test that resulted in the patient's         o  The patient was          
      being diagnosed with a heart attack. The    unnecessarily admitted to   
      patient was started on multiple             the cardiac intensive care  
      medications and admitted to the cardiac     unit, endured the painful   
      intensive care unit at the hospital.        placement of intravenous    
      After 10 hours in intensive care, a         lines, and was attached to  
      second cardiac blood test was run and       several heart monitors.     
      because of significant discrepancy in       o  The patient suffered     
      the results compared to the first test,     significant anxiety related 
      a rapid investigation of the situation      to thinking that he had     
      ensued. The investigation revealed that     suffered a heart attack.    
      the results of the original ER test were    o  During the 2-day delay   
      recorded inaccurately. The patient was      between hospital admission  
      retested by cardiac specialty               and follow-up appointment,  
      physicians, reevaluated, and thought to     the patient remained        
      be stable. He was removed from heart        symptomatic for his actual  
      monitors, taken off all cardiac             condition-an ulcer.         
      medications, discharged from the            o  If the patient's ulcer   
      hospital, and asked to return for a         had been more severe and    
      routine appointment in 2 days. The          had started to bleed        
      patient continued to have mild chest        internally, the             
      pain during the 2 days after he was         inappropriate               
      discharged. During his return visit, the    administration of heart     
      patient was found to have a stomach         attack medications could    
      ulcer. The patient's chest pain was         have had harmful or even    
      really referred pain caused by the          catastrophic effects on the 
      ulcer. The patient was then treated         patient's health.           
      correctly.                               
      o  An investigation of the error         
      revealed that it was a human recording   
      and data entry error. The result from a  
      different patient's blood test had been  
      entered incorrectly into this patient's  
      record because the two patients' blood   
      tests had been run within a few minutes  
      of each other.                           
Example 3: Inaccurate review of lab test    
results                                     
      o  A woman who had been using birth         o  The patient              
      control was referred to a dermatologist     inappropriately received a  
      for severe acne. The dermatologist          drug known to cause birth   
      wanted to prescribe a drug known to         defects.                    
      cause birth defects so she ordered two      o  This necessitated        
      pregnancy tests, one initially and the      increased monitoring of the 
      second 2 weeks later. The first test        patient's pregnancy.        
      came back negative. When the                o  Although the patient's   
      dermatologist called for the results of     pregnancy was ultimately    
      the second test, a lab worker               unaffected, she experienced 
      incorrectly told her that the pregnancy     anxiety throughout. She     
      test was negative. The patient was given    continues to worry that her 
      a prescription for the acne drug and        child is or will be         
      advised to avoid pregnancy while taking     negatively affected in the  
      this medication. Three days after the       future because she took     
      patient started taking the prescription,    this drug early in her      
      the dermatologist saw in the patient's      pregnancy.                  
      record that the second pregnancy test    
      for this patient was actually positive.  
      Upon instruction, the patient stopped    
      taking the medication and her number of  
      prenatal visits was increased so she     
      could be monitored for possible birth    
      defects.                                 
      o  The investigation revealed that the   
      laboratory employee, an experienced      
      technician who was busy when the         
      incident occurred, reported the result   
      of the patient's first pregnancy test    
      when asked for results from the second   
      pregnancy test. Both the physician and   
      the technician neglected to follow lab   
      policies to include the date and time of 
      the test when orally communicating       
      results.                                 
Example 4: Inappropriate judgment concerning a microbiology lab test
result                                      
      o  After returning from a trip to West      o  The patient experienced  
      Africa, a 30-year-old woman went to the     fever and other significant 
      ER after experiencing high fever,           symptoms during the 4-day   
      chills, and a headache. The woman was       delay in receiving          
      tested for malaria, a potentially deadly    appropriate care.           
      disease transmitted via mosquito bites.     o  After the malaria        
      The test result was negative and the        diagnosis was made, the     
      patient was sent home on ibuprofen. Four    patient was admitted to the 
      days later she returned to the ER           hospital and successfully   
      suffering from continued high fever,        treated. There was no       
      listlessness, and a severe headache. She    permanent disability.       
      was tested again for malaria. This time  
      the lab test result was positive for a   
      moderate case of the disease.            
      o  The laboratory director reviewed the  
      testing from the initial ER visit and    
      found that the tests had been positive   
      for malaria. The cause for not           
      identifying malaria initially was        
      inappropriate judgment. Since there was  
      a low pretest probability of a positive  
      result, technicians assumed that there   
      would be a negative result.              

Sources: All examples were summarized from case studies in the journal
Laboratory Errors and Patient Safety. Example 1: Volume 2, Issue 1(2005):
10; Example 2: Volume 1, Issue 4(2005): 6; Example 3: Volume 1, Issue
3(2004): 6; Example 4: Volume 1, Issue 1(2004): 5.

Appendix II: Labs Surveyed by State Survey Agencies and the Percentage
with Condition-Level Deficiencies, by State in 2004

                        Number of labs       Percentage of labs surveyed with 
Statea                     surveyed  reported condition-level deficiencies 
Alabama                         236                                    3.4 
Alaska                           24                                   12.5 
Arizona                         122                                    4.1 
Arkansas                        197                                    9.1 
California                      666                                    3.9 
Colorado                        173                                   10.4 
Connecticut                     140                                    0.0 
Delaware                         18                                    0.0 
District of Columbia             15                                   13.3 
Florida                         664                                    4.1 
Georgia                         369                                    4.3 
Hawaii                           57                                    5.3 
Idaho                            89                                    0.0 
Illinois                        292                                    7.5 
Indiana                         112                                    9.8 
Iowa                            141                                    2.1 
Kansas                          156                                    2.6 
Kentucky                        217                                    3.2 
Louisiana                       157                                   12.1 
Maine                            32                                    0.0 
Maryland                        180                                    3.3 
Massachusetts                   217                                    4.6 
Michigan                        192                                    2.6 
Minnesota                       155                                    3.2 
Mississippi                     202                                    7.9 
Missouri                        238                                    2.9 
Montana                          41                                    7.3 
Nebraska                        127                                    8.7 
Nevada                           98                                    4.1 
New Hampshire                    22                                    9.1 
New Jersey                      302                                    9.9 
New Mexico                       50                                   10.0 
New Yorkb                       545                                    8.1 
North Carolina                  321                                   13.4 
North Dakota                     30                                   10.0 
Ohio                            221                                    5.9 
Oklahoma                        158                                   10.8 
Oregon                          147                                    4.1 
Pennsylvania                    354                                    1.4 
Rhode Island                     36                                    0.0 
South Carolina                  143                                   25.2 
South Dakota                     66                                    1.5 
Tennessee                       403                                   10.7 
Texas                           651                                    6.5 
Utah                             95                                    6.3 
Vermont                          33                                    0.0 
Virginia                        271                                    5.2 
West Virginia                    85                                    7.1 
Wisconsin                       226                                    9.7 
Wyoming                          23                                    8.7 
Nation                        9,509                                    6.3 

Source: GAO analysis of OSCAR data as of May 15, 2006.

aWashington is not included because it operates only a CLIA-exempt
program.

bExcludes labs surveyed under the state's CLIA-exempt program.

Note: Includes surveys conducted from January 12, 2004, through December
31, 2004. We excluded surveys conducted from January 1 to January 11,
2004, because surveyors began using new CLIA inspection requirements on
January 12.

Appendix III: Number of Labs Subject to Surveys by State Survey Agencies
in 2005 and Number of Labs with Sanctions, 1998 to 2004 

                        Number of labs          Number of labs with sanctions 
Statea                       (2005)                            (1998-2004) 
Alabama                         491                                      1 
Alaska                           50                                      0 
Arizona                         265                                      7 
Arkansas                        398                                     14 
California                    1,570                                    134 
Colorado                        310                                      6 
Connecticut                     246                                      0 
Delaware                         46                                      0 
District of Columbia             35                                      6 
Florida                       1,268                                      5 
Georgia                         737                                      5 
Hawaii                           81                                      4 
Idaho                           203                                      0 
Illinois                        517                                     15 
Indiana                         274                                      1 
Iowa                            281                                      1 
Kansas                          279                                      0 
Kentucky                        386                                      0 
Louisiana                       396                                      7 
Maine                            89                                      0 
Maryland                        467                                     28 
Massachusetts                   409                                      0 
Michigan                        387                                     62 
Minnesota                       274                                      1 
Mississippi                     441                                      0 
Missouri                        394                                      5 
Montana                          93                                      4 
Nebraska                        254                                      4 
Nevada                          159                                      3 
New Hampshire                    89                                      1 
New Jersey                      533                                     16 
New Mexico                      113                                      3 
New York                      1,125                                     75 
North Carolina                  676                                      1 
North Dakota                     74                                      0 
Ohio                            428                                     12 
Oklahoma                        292                                      6 
Oregon                          270                                      0 
Pennsylvania                    749                                      7 
Rhode Island                     75                                      1 
South Carolina                  315                                      0 
South Dakota                    116                                      2 
Tennessee                       705                                      0 
Texas                         1,854                                     26 
Utah                            201                                     13 
Vermont                          46                                      0 
Virginia                        540                                     17 
West Virginia                   143                                      6 
Wisconsin                       480                                      1 
Wyoming                          54                                      1 
Total                        19,678                                    501 

Source: GAO analysis of CMS lab registries and CLIA database.

aWashington is not included because it operates only a CLIA-exempt
program.

Appendix IV: Comments from the Centers for Medicare & Medicaid Services


Appendix V: Comments from the College of American Pathologists

Appendix VI: Comments from COLA

Appendix VII: Comments from the Joint Commission on Accreditation of
Healthcare Organizations Appendix

Appendix VIII: GAO Contact and Staff Acknowledgments

                                  GAO Contact

Leslie G. Aronovitz (312) 220-7600 or [email protected]

                                Acknowledgments

In addition to the contact named above, Walter Ochinko, Assistant
Director; Lucia P. Fort; Dan Lee; Kevin Milne; Dean Mohs; Elizabeth T.
Morrison; Michelle Rosenberg; and Elizabeth Scherer made key contributions
to this report.

(290439)

GAO's Mission

The Government Accountability Office, the audit, evaluation and
investigative arm of Congress, exists to support Congress in meeting its
constitutional responsibilities and to help improve the performance and
accountability of the federal government for the American people. GAO
examines the use of public funds; evaluates federal programs and policies;
and provides analyses, recommendations, and other assistance to help
Congress make informed oversight, policy, and funding decisions. GAO's
commitment to good government is reflected in its core values of
accountability, integrity, and reliability.

Obtaining Copies of GAO Reports and Testimony

The fastest and easiest way to obtain copies of GAO documents at no cost
is through GAO's Web site ( www.gao.gov ). Each weekday, GAO posts newly
released reports, testimony, and correspondence on its Web site. To have
GAO e-mail you a list of newly posted products every afternoon, go to
www.gao.gov and select "Subscribe to Updates."

Order by Mail or Phone

The first copy of each printed report is free. Additional copies are $2
each. A check or money order should be made out to the Superintendent of
Documents. GAO also accepts VISA and Mastercard. Orders for 100 or more
copies mailed to a single address are discounted 25 percent. Orders should
be sent to:

U.S. Government Accountability Office 441 G Street NW, Room LM Washington,
D.C. 20548

To order by Phone: Voice: (202) 512-6000 TDD: (202) 512-2537 Fax: (202)
512-6061

To Report Fraud, Waste, and Abuse in Federal Programs

Contact:

Web site: www.gao.gov/fraudnet/fraudnet.htm E-mail: [email protected]
Automated answering system: (800) 424-5454 or (202) 512-7470

Congressional Relations

Gloria Jarmon, Managing Director, [email protected] (202) 512-4400 U.S.
Government Accountability Office, 441 G Street NW, Room 7125 Washington,
D.C. 20548

Public Affairs

Paul Anderson, Managing Director, [email protected] (202) 512-4800 U.S.
Government Accountability Office, 441 G Street NW, Room 7149 Washington,
D.C. 20548

www.gao.gov/cgi-bin/getrpt? GAO-06-416 .

To view the full product, including the scope

and methodology, click on the link above.

For more information, contact Leslie G. Aronovitz at (312) 220-7600 or
[email protected].

Highlights of GAO-06-416 , a report to congressional requesters

June 2006

CLINICAL LAB QUALITY

CMS and Survey Organization Oversight Should Be Strengthened

The Clinical Laboratory Improvement Amendments of 1988 (CLIA) strengthened
and extended quality requirements for labs that perform tests to diagnose
or treat disease. About 36,000 labs that perform certain complex tests
must be surveyed biennially by either a state or one of six private
accrediting organizations. CMS oversees implementation of CLIA
requirements and the activities of survey organizations. GAO was asked to
examine (1) the quality of lab testing; (2) the effectiveness of surveys,
complaint investigations, and enforcement actions in detecting and
addressing lab problems; and (3) the adequacy of CMS's CLIA oversight.

What GAO Recommends

GAO is making recommendations to the CMS Administrator to improve CLIA
oversight including (1) standardizing the reporting of survey deficiencies
to permit meaningful comparisons across survey organizations; (2) working
with survey organizations to ensure that educating lab workers does not
preclude appropriate regulation, such as identifying and reporting
deficiencies that affect lab testing quality; and (3) allowing the CLIA
program to fully use revenues generated by the program to hire sufficient
staff to fulfill its statutory responsibilities. CMS concurred with 11 of
GAO's 13 recommendations and noted that the report provided insights into
areas where it can improve, augment, and reinforce oversight.

Because of limited comparable data from CMS and survey organizations, too
little is known about the quality of lab testing. For example, a
standardized assessment of lab quality across survey organizations is not
possible because of different definitions of what constitutes a serious
quality problem. One survey organization had no systematic way of
identifying the problematic labs it inspects. However, GAO's analysis of
an indicator that measures a lab's ability to consistently produce
accurate test results suggests that lab quality may not have improved at
hospital labs in recent years.

Based on an analysis of available data and interviews with CMS and survey
organizations, real and potential lab quality problems are masked by
survey, complaint, and enforcement weaknesses. Because most survey
organizations announce the timing of biennial surveys, allowing labs to
prepare for inspections, surveys may not provide a realistic picture of
lab quality. Although two survey organizations that generally inspect
hospital labs plan to begin unannounced surveys in 2006, they may not be
possible at physician office labs that have irregular hours. Survey
organizations that typically inspect such labs, however, provide more
advance notice about upcoming inspections than CMS allows states to
provide. Several other factors suggest that surveys and complaints do not
present a realistic picture of lab quality. Interviews with officials from
a sample of states confirmed that some survey organizations do not cite
all serious deficiencies, as evidenced by variability in the limited
available lab survey data. Officials said that surveyors may be reluctant
to cite deficiencies because they view their role as educational, not
regulatory; moreover, CMS has instructed state surveyors not to cite some
deficiencies for over 2 years after implementing new lab requirements.
Finally, lab workers may file complaints infrequently because of concern
about retaliation and a lack of understanding about how to file a
complaint. CMS rarely imposes sanctions, even for labs with the same
repeat deficiencies, a reflection of the educational focus of the CLIA
program.

CMS does not require labs to participate in a key quality assurance test
as frequently as CLIA requires. Although funded by lab fees, CMS officials
indicated that the program has not been allowed to hire sufficient staff
to carry out the agency's oversight responsibilities. Moreover, CMS's
principal oversight tool, intended to determine if all serious
deficiencies were identified, lacks independence because many oversight
reviews are conducted simultaneously with survey organizations. CMS's
presence may make surveyors more attentive to survey tasks than when they
are not being observed. Compared to independent reviews, simultaneous
reviews rarely identify missed deficiencies. Furthermore, CMS does not
collect and analyze data on serious deficiencies identified by each survey
organization and thus, is unable to assess whether lab quality is
improving or declining. Nor does CMS effectively analyze other key data
such as the use of sanctions. To improve oversight, CMS is establishing a
nationwide complaints database. CMS is also instituting annual survey
organization performance reviews.
*** End of document. ***