Clinical Lab Quality: CMS and Survey Organization Oversight
Should Be Strengthened (16-JUN-06, GAO-06-416).
Improvement Amendments of 1988 (CLIA) strengthened and extended
quality requirements for labs that perform tests to diagnose or
treat disease. About 36,000 labs that perform certain complex
tests must be surveyed biennially by either a state or one of six
private accrediting organizations. CMS oversees implementation of
CLIA requirements and the activities of survey organizations. GAO
was asked to examine (1) the quality of lab testing; (2) the
effectiveness of surveys, complaint investigations, and
enforcement actions in detecting and addressing lab problems; and
(3) the adequacy of CMS's CLIA oversight.
-------------------------Indexing Terms-------------------------
REPORTNUM: GAO-06-416
ACCNO: A55622
TITLE: Clinical Lab Quality: CMS and Survey Organization
Oversight Should Be Strengthened
DATE: 06/16/2006
SUBJECT: Data collection
Disease detection or diagnosis
Inspection
Laboratories
Quality assurance
Standards
Surveys
Testing
******************************************************************
** This file contains an ASCII representation of the text of a **
** GAO Product. **
** **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced. Tables are included, but **
** may not resemble those in the printed version. **
** **
** Please see the PDF (Portable Document Format) file, when **
** available, for a complete electronic file of the printed **
** document's contents. **
** **
******************************************************************
GAO-06-416
* Results in Brief
* Background
* Surveyed Labs
* Survey Organizations
* Surveys and Complaint Investigations
* CMS Oversight
* Insufficient Data Exist to Identify Extent of Serious Lab Qu
* Limited Data Are Available on the Quality of Labs Inspected
* Quality of Labs Inspected by Other Survey Organizations Is V
* Proficiency Testing Results Suggest that Quality Has Not Imp
* Oversight Weaknesses Mask Quality Problems
* Announced Surveys May Result in Unrealistic Picture of Lab Q
* Variability in Reported Survey Deficiencies Suggests that La
* Balance Between Educational and Regulatory Roles by CMS and
* Use of Volunteer Surveyors by CAP Raises Concerns
* Lab Workers Who File Complaints About Quality Problems in La
* Lab Sanctions Are Rarely Imposed
* CMS Oversight of CLIA Is Inadequate
* CMS's Implementation of Proficiency Testing Is Inconsistent
* CMS Is Late in Ensuring CLIA Equivalency of Exempt States' a
* Many CMS Validation Reviews Lack Independence and Reviews Sk
* Validation of State Survey Agency Lab Surveys
* Validation of Accrediting Organizations' Lab Surveys
* CMS Use of Data for Oversight of CLIA Program Is Limited
* CMS Implementing Performance Reviews for Survey Organization
* Conclusions
* Recommendations for Executive Action
* Agency and Accrediting Organization Comments and Our Evaluat
* Assessment of Lab Quality
* Educational Focus of CLIA
* Sanctioning Labs with Serious, Repeat Deficiencies
* Proficiency Testing Frequency
* Equivalency of Accrediting Organization and Exempt State Pro
* Accrediting Organization Validation Reviews
* Additional Comments by Accrediting Organizations
* GAO Contact
* Acknowledgments
* GAO's Mission
* Obtaining Copies of GAO Reports and Testimony
* Order by Mail or Phone
* To Report Fraud, Waste, and Abuse in Federal Programs
* Congressional Relations
* Public Affairs
Report to Congressional Requesters
United States Government Accountability Office
GAO
June 2006
CLINICAL LAB QUALITY
CMS and Survey Organization Oversight Should Be Strengthened
GAO-06-416
Contents
Letter 1
Results in Brief 5
Background 7
Insufficient Data Exist to Identify Extent of Serious Lab Quality Problems
15
Oversight Weaknesses Mask Quality Problems 19
CMS Oversight of CLIA Is Inadequate 33
Conclusions 44
Recommendations for Executive Action 46
Agency and Accrediting Organization Comments and Our Evaluation 47
Appendix I Effects of Lab Errors on Patient Health 58
Appendix II Labs Surveyed by State Survey Agencies and the Percentage with
Condition-Level Deficiencies, by State in 2004 61
Appendix III Number of Labs Subject to Surveys by State Survey Agencies in
2005 and Number of Labs with Sanctions, 1998 to 2004 63
Appendix IV Comments from the Centers for Medicare & Medicaid Services 65
Appendix V Comments from the College of American Pathologists 80
Appendix VI Comments from COLA 83
Appendix VII Comments from the Joint Commission on Accreditation of
Healthcare Organizations 89
Appendix VIII GAO Contact and Staff Acknowledgments 93
Tables
Table 1: Percentage of Inspection Requirements Classified as Serious, by
Survey Organization 16
Table 2: Amount of Advance Notice Given to Labs about Upcoming
Inspections, by Survey Organization 21
Table 3: Number of Complaints Received by CAP, 2002-2005 29
Table 4: Number of Labs Inspected with Principal Only, Principal and
Alternative, and Alternative Only Sanctions Imposed, 1998-2004 30
Table 5: Analysis of Results of Simultaneous and Independent Validation
Reviews of State Surveys, Fiscal Years 1999-2003 39
Table 6: Analysis of Results of Simultaneous and Independent Validation
Reviews of Accrediting Organizations' Surveys, Fiscal Years 1999-2003 41
Table 7: Effects of Lab Errors on Patient Health 58
Figures
Figure 1: Types of Survey Organizations, Requirements Used to Survey Labs,
and Percentage of Labs Surveyed by Each Organization, as of December 2005
11
Figure 2: Percentage of Labs with Proficiency Testing Failures from 1999
through 2003, by Survey Organization 19
Abbreviations
AAAB American Association of Blood Banks AOA American Osteopathic
Association ASHI American Society of Histocompatibility and Immunogenetics
CAP College of American Pathologists CLIA Clinical Laboratory Improvement
Amendments of 1988 CMS Centers for Medicare & Medicaid Services CMSO
Center for Medicaid and State Operations ER emergency room JCAHO Joint
Commission on Accreditation of Healthcare Organizations OSCAR On-Line
Survey, Certification, and Reporting system
This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed in
its entirety without further permission from GAO. However, because this
work may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this material
separately.
United States Government Accountability Office
Washington, DC 20548
June 16, 2006
The Honorable Charles E. Grassley Chairman Committee on Finance United
States Senate
The Honorable Mark Souder Chairman The Honorable Elijah E. Cummings
Ranking Minority Member Subcommittee on Criminal Justice, Drug Policy and
Human Resources Committee on Government Reform House of Representatives
Clinical lab tests are one of the most frequently billed Medicare
procedures and, according to the American Clinical Laboratory Association,
affect an estimated 70 percent of medical decisions.1 To improve oversight
of clinical labs, Congress passed legislation in 1967;2 renewed concerns
about quality, including errors in Pap smear tests used to diagnose
cervical cancer, resulted in enactment of the Clinical Laboratory
Improvement Amendments of 1988 (CLIA).3 In recent years, despite CLIA, lab
quality problems in several states have raised questions about the
adequacy of lab oversight. Lab oversight is critical because inaccurate or
unreliable lab tests may lead to improper treatment, unnecessary mental
and physical anguish for patients, and higher health care costs.4
The Centers for Medicare & Medicaid Services (CMS) is responsible for
overseeing compliance with CLIA requirements. As of December 2005, there
were approximately 193,000 labs nationwide, ranging from very small
physician office labs that conduct fewer than 2,000 tests annually to
hospital labs that conduct millions of tests each year. Most clinical labs
regulated under CLIA must obtain a certificate from CMS but only about 19
percent-those that conduct moderate- to high-complexity tests-undergo
biennial inspections, which are also referred to as surveys.5 The surveys
assess lab compliance with mandated personnel and testing standards. In
addition, surveyed labs must participate in proficiency testing, a program
that requires them to test samples with unknown characteristics that are
then graded by an external party. Labs with serious deficiencies may be
sanctioned, e.g., required to cease testing. Labs have a choice of being
surveyed by (1) their state survey agency, under contract with CMS; (2)
their state CLIA-exempt program for labs in New York and Washington; or
(3) one of six private accrediting organizations.6 State survey agency
inspections use CLIA requirements that are intended to help ensure valid
and reliable lab tests; the two state CLIA-exempt programs and six
accrediting organizations survey labs using their own requirements that
CMS has determined to be at least equivalent to CLIA's. Each survey
organization is also responsible for investigating complaints about lab
quality.7 Because of the critical importance of accurate lab test results
and oversight, you asked us to conduct a nationwide assessment of (1) the
quality of lab testing; (2) the effectiveness of surveys, complaint
investigations, and enforcement actions in detecting problems and ensuring
compliance; and (3) the adequacy of CMS oversight of the CLIA program.
1Medicare is a federal health care program serving the elderly and
disabled individuals.
2Clinical Laboratories Improvement Act of 1967, Pub. L. No. 90-174, S:5,
81 Stat. 533, 536.
3Pub. L. No. 100-578, 102 Stat. 2903.
4Appendix I provides examples of the effect of lab errors on patient
health.
To determine what is known about the quality of lab testing, we analyzed
data on serious deficiencies identified during surveys by state survey
agencies using CMS's On-Line Survey, Certification, and Reporting system
(OSCAR).8 The CLIA program inspection requirements are classified as
either "standard-" or "condition-" level. Similarly, deficiencies are also
characterized as standard- or condition-level, based on the requirement in
which the deficiency occurs. Because condition-level requirements
generally consist of one or more standard-level requirements, a deficiency
at the condition-level denotes a serious or systematic problem. We
requested comparable data on serious deficiencies from state CLIA-exempt
programs and the three largest accrediting organizations-the College of
American Pathologists (CAP), COLA, and the Joint Commission on
Accreditation of Healthcare Organizations (JCAHO)-which together survey
about 97 percent of accredited labs.9 CAP, COLA, JCAHO, and exempt-state
programs each maintain their own separate databases. We also analyzed
proficiency testing data-another indicator of a lab's ability to produce
accurate test results. CMS officials generally recognize OSCAR and
proficiency testing data to be reliable. We discussed the OSCAR database
with CMS officials and tailored our analysis to ensure the accuracy of our
findings. Because exempt states and accrediting organizations survey labs
using their own requirements, we worked with them to develop data
comparable to OSCAR deficiency data. We discussed our analyses with CMS
and each of the survey organizations to ensure that we had interpreted the
data correctly. Based on discussions with officials from the three
accrediting organizations, we determined that they take appropriate steps
to ensure the reliability of their data. Because it was not practical to
independently test the reliability of accrediting organization data, we
present these data as reported by those organizations.
5Labs obtain a CLIA certificate that corresponds to the complexity of the
testing they conduct. Generally, each lab has one certificate but a large
hospital with multiple labs may have a corresponding number of
certificates. By regulation, labs that are within a hospital campus and
under common direction are allowed to file either a single application for
a certificate or multiple applications for multiple certificates.
6CMS contracts with state survey agencies in the District of Columbia and
49 states (including New York but not Washington) to survey labs under
CLIA requirements. Labs in Washington are surveyed either by the state's
CLIA-exempt program or by an accrediting organization. Labs in New York
are surveyed either by the state survey agency or New York's CLIA-exempt
program. New York does not authorize accreditation as a basis for lab
licensure.
7Throughout this report, we use the term "survey organizations" when
referring collectively to state survey agencies, the two state CLIA-exempt
programs, and accrediting organizations.
To assess the effectiveness of lab surveys, complaint investigations, and
enforcement mechanisms in detecting problems and securing compliance, we
reviewed the processes used to ensure the quality of clinical lab testing
and analyzed available data related to these issues. We also conducted
structured interviews with officials from (1) CMS, (2) three CMS regional
8In addition to the results of state survey agency lab inspections, CMS's
OSCAR database stores other information on labs registered under the CLIA
program including: (1) labs' CLIA certificate history, such as switching
from being inspected by accrediting organizations to being inspected by
state survey agencies; (2) the results of complaint investigations; (3)
labs' billing history; and (4) proficiency testing enrollment and
performance data.
9COLA was formerly known as the Commission on Office Laboratory
Accreditation.
offices,10 (3) 10 state survey agencies,11 (4) the New York and Washington
CLIA-exempt programs, and (5) the three accrediting organizations. We
judgmentally selected the 10 state survey agencies to include a mixture of
states whose lab inspections identified a range of serious deficiencies
from few to many. We also discussed the quality problems discovered at a
Maryland hospital lab with a Maryland state survey agency official and
interviewed 9 of the 36 CAP surveyors who participated in surveys of this
lab from 1999 through 2003 to obtain a firsthand perspective on the CAP
survey process.12 Based on our review and discussions with CMS and survey
organization officials, we focused on several key issues, including the
rationale for announced surveys, the ability of lab surveys to identify
serious deficiencies, the balance struck between the regulatory and
educational goals of lab surveys, the implications of CAP's use of
volunteer surveyors from neighboring labs to conduct inspections, how
survey organizations facilitate the filing of complaints, and the use of
sanctions to encourage compliance. We analyzed data on the number of
complaints received by each survey organization from 2002 through 2004 and
discussed CAP's initiatives to encourage the filing of complaints. In
addition, we determined the extent to which labs had the same serious
problems on consecutive surveys and discussed with CMS the steps the
agency had taken to deter an inconsistent pattern of compliance.
To assess the effectiveness of CMS oversight of the CLIA program, we
analyzed the laws and regulations that define CMS's role and authority. We
also reviewed CMS's process for determining that the survey requirements
and procedures of state CLIA-exempt programs and accrediting organizations
are at least equivalent to those of CLIA. We analyzed the results of
validation reviews that federal surveyors from CMS regional offices
conducted for state survey agency lab inspections and that state survey
agency staff conducted for accrediting organization inspections from 1999
through 2003. We also examined other mechanisms that CMS uses to hold
survey organizations accountable for their performance: (1) the collection
and analysis of data on surveys, complaints, and enforcement actions,
including steps taken to address communication and coordination issues
that became evident during a complaint investigation at a Maryland
hospital lab; and (2) recently developed annual reviews that measure state
survey agency compliance with CLIA program requirements, such as the
timeliness of surveys. We conducted our work from January 2005 through May
2006 in accordance with generally accepted government auditing standards.
10New York, Philadelphia, and Seattle.
11California, Colorado, Connecticut, Idaho, Louisiana, Michigan, Nebraska,
North Carolina, Pennsylvania, and South Carolina.
12A 2004 complaint investigation conducted by the Maryland state survey
agency found that personnel at this lab falsified records to conceal
problems with HIV and hepatitis testing equipment and that the lab
provided hundreds of patients with potentially erroneous test results.
These problems were not detected during the 2003 CAP inspection or a prior
complaint investigation conducted by the state survey agency in 2002.
Results in Brief
Insufficient data exist to identify the extent of serious quality problems
at labs. When CMS implemented revised CLIA survey requirements in 2004, it
modified historical state survey agency findings stored in its OSCAR
database and, as a result, data prior to 2004 no longer reflect key survey
requirements in effect at the time of those surveys. In addition, the lack
of a straightforward method to link similar requirements across survey
organizations makes it virtually impossible to assess lab quality in a
standardized manner, such as identifying the proportion of labs with
condition-level deficiencies, which indicate serious or systemic quality
problems. Although CMS has stated that it believes lab quality has
improved since the early 1990s, the results of proficiency testing-the one
available data source that can be used to uniformly compare lab quality
across survey organizations-suggest that lab quality may not have improved
at hospital labs and that the improvement for physician office labs may be
misleading because a significant number of such labs are no longer
inspected.
Weaknesses in surveys, complaint processes, and enforcement mask potential
quality problems at labs. Lab survey findings may not accurately reflect
the actual quality assurance process in place on a day-to-day basis
because of several shortcomings. First, most survey organizations announce
all surveys, allowing labs to prepare for their inspections. To address
this problem, accrediting organizations that inspect hospital labs began
conducting unannounced surveys in 2006. Second, the limited data available
suggest that state survey agency inspections do not identify all serious
deficiencies. Third, the balance struck between the CLIA program's
educational and regulatory goals is sometimes inappropriately skewed
toward education, which may also result in understatement of survey
findings. For example, CMS instructed state survey agencies not to cite
deficiencies for new lab quality control requirements for 2 years, in part
because of a lack of lab "buy-in" for some of the new policies and
procedures; CMS then extended this period and gave no specific end date.
Regarding complaint processes, complaints are filed by a variety of
sources, including lab workers. Few labs were the subject of a complaint
each year from 2002 through 2004-significantly less than one complaint per
lab per year. Concern that labs can easily identify the lab workers who
file complaints and lab workers' lack of familiarity with how to file a
complaint may explain why so few workers report problems. Since one
accrediting organization required each lab it inspects to display a poster
explaining how to file a complaint, the number of complaints it received
about lab quality has doubled. Finally, based on the large number of labs
with proposed sanctions from 1998 through 2004 that were never
imposed-even for labs with the same serious, condition-level deficiencies
on consecutive surveys-it is unclear how effective CMS's enforcement
process is at motivating labs to consistently comply with CLIA
requirements.
CMS's oversight of clinical lab quality is inadequate to ensure that labs
are meeting CLIA requirements. The agency requires proficiency testing
three times each year instead of on a quarterly basis, as required by
CLIA. Nor is CMS meeting its own requirement to determine in a timely
manner the continued equivalency of accrediting organization and
exempt-state inspection requirements and processes. For example, New
York's and COLA's reviews were about 4 years and 3 years past due,
respectively, as of December 2005. CMS attributed these delays to having
too few staff. Moreover, CMS allows the implementation of changes to
accrediting organization and exempt state inspection requirements between
periodic equivalency determinations before it reviews the proposed
changes. Validation reviews-one of CMS's most important oversight tools-do
not provide an independent assessment of the extent to which surveys
identify all serious deficiencies because many are performed
simultaneously with such surveys. In addition, CMS's requirement for
validating state survey agencies' inspections is vague, resulting in no
validation reviews in some states. Finally, CMS does not effectively use
data to monitor survey organization activities and processes, such as the
proportion of labs with serious deficiencies, proficiency testing results,
or trends in complaints. Realizing that its existing oversight activities
need to be strengthened, CMS has begun instituting performance reviews to
measure survey organization compliance with its requirements and is
developing protocols to ensure improved communication among survey
organizations concerning complaints about lab quality.
We are recommending that the CMS Administrator take actions that will
standardize survey findings across survey organizations, enable it to
compare changes over time, and make meaningful comparisons among
organizations; strengthen survey, complaint, and enforcement processes;
and improve CMS oversight of the CLIA program. In commenting on a draft of
this report, CMS endorsed our overall conclusion that quality assurance
for the nation's clinical labs should be strengthened and said that it
would take actions in response to 11 of our 13 recommendations. CMS
disagreed with our recommendations concerning the frequency of proficiency
testing and the extent of simultaneous accrediting organization validation
reviews. We believe that implementing these recommendations is necessary
to improve oversight of labs and accrediting organizations. CMS also
provided an alternative assessment of lab quality, disagreed with our
conclusion about the educational phase-in periods for new CLIA
requirements, and expressed concern about identifying and sanctioning labs
with repeat condition-level deficiencies. CAP, COLA, and JCAHO also
provided comments on a draft of this report. CAP indicated that it would
identify additional measures it could take to strengthen its own
oversight, and COLA found merit in our recommendations to improve CMS
oversight. CAP, COLA, and JCAHO disagreed with some of our findings and
recommendations to CMS. We incorporated technical comments from CMS and
the three accrediting organizations, as appropriate.
Background
A clinical lab is generally defined as a facility that examines specimens
derived from humans for the purpose of disease diagnosis, prevention, and
treatment, or health assessment of individuals. While hospital and
interstate labs were previously subject to regulation, CLIA strengthened
federal requirements and extended them to most other clinical labs,
including physician office labs. For example, CLIA strengthened personnel
requirements for lab workers, strengthened proficiency testing that
evaluates the accuracy of lab testing between surveys, and created a range
of sanctions to enforce compliance.13
Most clinical labs regulated under CLIA must obtain a certificate from CMS
and pay fees every 2 years to cover the costs of administering the CLIA
program, including surveys and other oversight activities.14 The fees vary
based on the complexity and volume of testing performed. Lab tests are
categorized as waived, moderate, or high complexity.15 Approximately 81
percent of all labs (about 157,000) are not subject to routine biennial
surveys because they perform (1) "waived" tests, which are examinations
and procedures that have an insignificant risk of erroneous results,
including those approved for home use or determined to employ
methodologies so simple or accurate that the likelihood of erroneous
results is negligible;16 or (2) tests performed during the course of a
patient visit with a microscope on specimens that are not easily
transportable.17
13Implementation of CLIA was phased in over a number of years. CLIA does
not apply to forensic laboratories, research laboratories that do not
report patient-specific results, drug testing laboratories certified by
the Substance Abuse and Mental Health Services Administration, and
Veterans Administration laboratories.
14Labs surveyed by either the New York or Washington CLIA-exempt programs
do not obtain a CLIA certificate and do not pay fees to CMS. According to
CMS, labs are billed a year in advance of the surveys to provide adequate
time for them to pay their fees and for states to perform surveys prior to
the expiration of labs' CLIA certificates.
Surveyed Labs
CLIA establishes more stringent requirements for the 19 percent (about
36,000) of labs performing moderate- or high-complexity testing, including
the requirement for a survey and participation in routine proficiency
testing. Since the early 1990s, the number and proportion of labs subject
to surveys have declined, while the number and proportion conducting
waived tests have increased.18 Surveys examine lab compliance with CLIA
program requirements in several areas including: personnel qualifications,
proficiency testing, quality control, quality assurance, and
recordkeeping.
o Personnel: CLIA sets minimum qualifications for all persons
performing or supervising moderate- or high-complexity lab tests
and specifies responsibilities for each position.
o Proficiency testing: Surveyed labs must participate in an
approved external proficiency testing program, which evaluates the
accuracy of laboratory testing. Under this requirement, labs
purchase samples with unknown characteristics several times each
year from an approved proficiency testing provider.19 The lab is
required to test the samples with its routine patient testing, and
the results are returned to the testing provider to be graded. A
proficiency testing failure is defined as unsatisfactory
performance on two consecutive or two out of three testing events.
The results of proficiency testing for all inspected labs are
transmitted to CMS and maintained in a database.
o Quality control: Labs must have a process for routinely
monitoring personnel, testing equipment, and the testing
environment to ensure proper operation and accurate results.
o Quality assurance: Labs must follow their plan to monitor the
overall operation of the laboratory on an ongoing basis and must
resolve identified problems that affect the quality of their
testing.
o Recordkeeping: Labs must maintain an audit trail of testing
that documents specimen integrity and test performance for all
phases of the test process from the test order to the test report.
In general, labs have a choice of who conducts their surveys-state
survey agencies using CLIA inspection requirements or other survey
organizations that use requirements CMS has determined to be at
least equivalent to CLIA's.20 CMS contracts with state survey
agencies in most states to inspect labs against CLIA
requirements.21 CLIA established an approval process to allow
states and private accrediting organizations to use their own
requirements to survey labs.22 As noted earlier, New York and
Washington operate CLIA-exempt programs and CMS has approved six
private, nonprofit accrediting organizations to survey labs-the
American Association of Blood Banks (AABB), the American
Osteopathic Association (AOA), the American Society of
Histocompatibility and Immunogenetics (ASHI), CAP, COLA, and
JCAHO. The requirements of both state CLIA-exempt programs and
accrediting organizations must be reviewed by CMS at least every 6
years to ensure CLIA equivalency, but may be more stringent than
those of CLIA. For example, when inspecting labs engaged in
moderate- and high-complexity testing, New York and some
accrediting organizations also look at the labs' procedures for
conducting "waived" tests, which is not required under CLIA.
Figure 1 lists the three types of survey organizations and
indicates whether they survey labs under CLIA requirements, or use
their own CLIA-equivalent requirements. It also shows the
percentage of labs performing moderate- to high-complexity testing
surveyed by each type of organization. In general, state survey
agencies, COLA, and Washington's CLIA-exempt program survey
physician office labs, while New York's CLIA-exempt program, CAP,
and JCAHO survey hospital labs.
15In January 2000, the Food and Drug Administration assumed responsibility
for categorizing tests conducted by labs from the Centers for Disease
Control and Prevention.
16Pregnancy and blood sugar screenings are examples of such tests. Labs
conducting waived tests are only required to follow manufacturers'
instructions and to limit testing to Food and Drug Administration-approved
or cleared methods.
17Known as provider-performed microscopy procedures, such tests must be
performed by a physician or other qualified provider as defined in CLIA
regulations. Labs conducting such tests are required to have written
procedures for the tests they perform and must also satisfy applicable
proficiency testing requirements and have a system to ensure the
competency of testing personnel. These labs and those performing waived
tests are subject to complaint investigations. See 42 C.F.R. S:
493.1773(f)(2005).
18From 1998 through December 2005, the proportion subject to surveys has
declined from about 30 percent to about 19 percent, while the proportion
of labs that are not surveyed because they perform waived testing has
increased from 70 percent to 81 percent.
Survey Organizations
19Proficiency testing providers are private companies or state lab
departments and are approved by CMS annually.
20Some labs, including Indian Health Service labs, are surveyed by federal
surveyors located in CMS's regional offices.
21CMS contracts with state survey agencies in the District of Columbia and
49 states (including New York but not Washington) to survey labs under
CLIA requirements.
22Prior to CLIA, CMS was not required to routinely determine the
equivalency of accrediting organization and state CLIA-exempt program
requirements.
Figure 1: Types of Survey Organizations, Requirements Used to Survey Labs,
and Percentage of Labs Surveyed by Each Organization, as of December 2005
aWashington is not included as it has only a CLIA-exempt program.
bNew York uses CLIA-equivalent requirements to inspect larger hospital
labs under the state's CLIA-exempt program and CLIA requirements to
inspect smaller labs, including physician office labs. Only the labs in
the CLIA-exempt program are counted here.
cSome labs are counted more than once because labs may be accredited by
more than one organization. While some labs in New York may be accredited,
they are still subject to biennial surveys by the state survey agency or
the state CLIA-exempt program, because New York does not authorize
accreditation as the basis for lab licensure.
Surveys and Complaint Investigations
Survey organizations (1) conduct surveys and complaint investigations, and
(2) monitor proficiency test results submitted by surveyed labs three
times a year. Surveys are typically conducted by former or current lab
workers, who assess lab compliance with CLIA or CLIA-equivalent
requirements. Most lab inspections are announced, that is, the lab has
advance notice of when the survey will occur. Generally, surveyors verify
that lab personnel are appropriately qualified to conduct testing,
evaluate proficiency test records, check equipment and calibration to
ensure that appropriate quality control measures are in place, and
determine whether the lab has a quality assurance plan and uses it to,
among other things, appropriately identify and resolve problems affecting
testing quality. Surveys also include an educational component to assist
labs in understanding how to comply with CLIA requirements. The duration
of a survey generally depends on the size-in terms of the number of tests
conducted-and complexity of a lab as well as the number of surveyors.
Thus, a survey conducted at a small lab may only take a few hours to
complete, while a survey at a large hospital lab may take a survey team a
full week or more.
In addition to inspections, survey organizations are responsible for
determining the seriousness of and investigating all complaints. For those
complaints that are determined to pose immediate jeopardy-an imminent and
serious threat to patient health and a significant hazard to public
health-CMS requires that the investigation be initiated within 2 working
days. Complaints may be investigated on-site or through communications
between the survey organization and the lab. Complaint investigations for
all survey organizations are unannounced.
Lab survey requirements are classified as either "standard-" or
"condition-" level. Generally, condition-level requirements are made up of
one or more related standard-level requirements. For example, the
condition-level requirement on enrollment and testing of samples through a
proficiency testing program has two related standard-level requirements:
(1) enrollment, which includes requirements for the lab to provide the
name of the program it has enrolled in to the Department of Health and
Human Services and authorize the release of testing data to the
department; and (2) testing, which specifies that the samples must be
tested in the same manner as any specimen and prohibits referring the test
samples to another lab for analysis.
Deficiencies are also characterized as standard- or condition-level based
on the requirement in which the deficiency occurs. Deficiencies in
standard-level requirements, that is, standard-level deficiencies, denote
problems that generally are not serious, while condition-level
deficiencies are cited when the problems are serious or systemic in
nature. A serious problem is defined as an inadequacy in a lab's quality
of services that adversely affects, or has the potential to adversely
affect, the accuracy and reliability of patient test results. When
deficiencies are found during surveys or complaint investigations, labs
are required to submit a plan of correction, detailing how and when they
will address the deficiencies. Additionally, CMS can impose principal or
alternative sanctions, or both.23 Principal sanctions include revocation
of a CLIA certificate, cancellation of the right to receive Medicare
payments, or limits on testing. Revocation of a CLIA certificate is
equivalent to termination from the CLIA program. Alternative sanctions are
less severe and include civil money penalties or on-site monitoring.24 For
condition-level deficiencies that do not involve immediate jeopardy, labs
have an opportunity to correct the deficiencies, which we refer to as a
grace period, before the sanctions are imposed. If a lab is unable to
correct a deficiency during this grace period, CMS determines whether to
impose a sanction and the type of sanction.
CMS Oversight
CMS, including its 10 regional offices, oversees state and accrediting
organization survey activities.25 CMS reviews and approves initial and
subsequent applications from exempt-state programs and accrediting
organizations to ensure CLIA equivalency. Validation reviews are one of
CMS's primary oversight tools. Federal surveyors in CMS regional offices
are responsible for conducting validation reviews of state survey agency
and exempt-state program inspections, but state survey agency staff
conduct the validation reviews of accrediting organization inspections.26
An objective of these reviews is to determine if all condition-level
deficiencies were identified.27 These reviews are conducted within 60 days
of a state's or 90 days of an accrediting organization's survey of a lab.
Starting in 1999, CMS required that at least one validation review be
conducted simultaneously with an accrediting organization's survey, a step
intended to encourage an exchange of ideas and approaches among surveyors.
CMS also encourages the use of simultaneous reviews of state survey agency
inspections. By law, the number of labs selected for validation reviews
must be sufficient to allow a reasonable estimate of the performance of
each accrediting organization being assessed.28 CMS requires fewer
validation reviews of state survey agency lab surveys (1 percent) than for
those of exempt-state programs or accrediting organizations (5 percent).
23State survey agencies may propose the imposition of sanctions for
noncompliance but only CMS can impose sanctions. Accrediting organizations
and exempt-state programs may revoke accreditation or remove a lab's state
license, respectively, for noncompliance with their CLIA-equivalent
requirements. While CMS regional office staff determine whether loss of
accreditation should also result in revocation of a lab's CLIA
certificate, loss of state licensure is tantamount to CLIA certificate
revocation.
24Because of congressional concern that available remedies were too
limited and could dissuade CMS from enforcement, CLIA gave CMS additional
tools, called alternative sanctions, to help motivate labs to comply with
quality requirements.
25The Centers for Disease Control and Prevention is also responsible for
carrying out certain CLIA-related tasks, including (1) developing and
evaluating technical standards for lab testing components; (2) working
with CMS and the Food and Drug Administration to determine the regulatory
impact of lab technical standards; (3) conducting lab research and
analysis; and (4) facilitating the CLIA Advisory Committee, which makes
recommendations to improve the CLIA program.
Beginning in 2003, CMS regional offices began reviewing the activities of
state survey agencies against a set of 13 performance standards. The
standards cover areas such as the timeliness of lab inspections, surveyor
personnel qualifications and training, CLIA data management, and the
handling of complaints. CMS's goal is to evaluate each state survey
agency's ability to carry out its CLIA responsibilities and to make
improvements. CMS is also developing performance standards for other
survey organizations that inspect labs using their own CLIA equivalent
requirements.
26Unlike validation reviews of accrediting organization surveys, CMS
refers to the validation of state surveys as Federal Monitoring Surveys.
Because of their similar objective, we refer to all such surveys as
validation reviews in this report. We refer to validation reviews that
occur at the same time as the lab survey as simultaneous. Conversely,
validation reviews that occur after the lab survey are referred to as
independent validations.
27According to CMS, the criterion for identifying a missed deficiency is
the reasonableness of concluding that a condition-level deficiency was
present at the time a survey organization conducted its survey but the
survey organization's findings did not note the deficiency.
28See 42 U.S.C. S: 263a(e)(2)(D)(2000). By regulation, a similar
requirement applies to validation reviews of labs under exempt-state
programs. 42 C.F.R. S: 493.563(b)(2)(2005).
Insufficient Data Exist to Identify Extent of Serious Lab Quality Problems
The extent of serious quality problems at labs is unclear because CMS has
incomplete data on condition-level deficiencies identified by state survey
agencies prior to 2004. We also found that the lack of a straightforward
linkage between CLIA requirements and the CLIA-equivalent requirements of
some survey organizations makes it virtually impossible to assess lab
quality in a standardized manner, such as identifying the proportion of
labs with condition-level deficiencies. Such deficiencies indicate serious
or systemic quality problems. Proficiency testing results-the one
available data source that can be used to uniformly compare lab quality
across survey organizations-raise questions about whether lab quality has
improved in recent years.
Limited Data Are Available on the Quality of Labs Inspected by State Survey
Agencies
CMS's OSCAR database contains limited data on the quality of labs
inspected by state survey agencies and, as a result, it is not possible to
analyze changes in the quality of lab testing over time. In January 2004,
CMS implemented revised CLIA survey requirements and modified the existing
OSCAR data-state survey agency findings-to reflect the changes.29 The
revisions affected approximately two-thirds of the CLIA condition-level
requirements.30 As a result of the data modifications, the findings for
surveys conducted prior to 2004 no longer reflect all key condition-level
requirements in effect at the time of those surveys.31 Based on the
available 2004 OSCAR data (which represent about one half of all labs
surveyed by state survey agencies), we found that 6.3 percent of labs had
condition-level deficiencies (see app. II for data on all state survey
agencies, including the District of Columbia).32 As will be discussed
below, similar data are not available for labs surveyed by other survey
organizations.
29CMS published the regulations for the new requirements in January 2003
and surveyors began using the new requirements on January 12, 2004.
30For example, some condition-level requirements were reorganized and some
were consolidated.
31When we asked for access to backup files, CMS told us that it did not
have backup files of the original pre-2004 survey data.
32We excluded survey results for the period January 1 through January 11,
2004, because CMS modified OSCAR data for findings prior to January 12,
2004, to reflect revised CLIA requirements.
Quality of Labs Inspected by Other Survey Organizations Is Very Difficult to
Measure in a Standardized Manner
Differences between the inspection requirements that state survey agencies
use to measure lab quality and those of exempt-state programs and
accrediting organizations make it virtually impossible to measure lab
quality in a standardized manner. Because exempt-state programs and
accrediting organizations do not classify inspection requirements and
related deficiencies as either standard- or condition-level, they cannot
easily identify the number of CLIA condition-level deficiencies cited at
the labs they survey or the proportion of surveyed labs with
condition-level deficiencies.33
We asked exempt-state programs and accrediting organizations what
percentage of their requirements, and any deficiencies cited for failure
to meet those requirements, indicated serious problems that were
equivalent to CLIA condition-level deficiencies. While only 8 percent of
CLIA requirements used by state survey agencies are classified as
condition-level and therefore serious, the proportion of requirements that
exempt-state programs and accrediting organizations classify as serious
ranged from 20 percent up to 100 percent (see table 1).
Table 1: Percentage of Inspection Requirements Classified as Serious, by
Survey Organization
Percentage of requirements classified as
Organization serious
State survey agencies 8
New York CLIA-exempt program a
Washington CLIA-exempt program a
COLA 20
CAP 80
JCAHO 100
Sources: GAO analysis of information provided by CMS, New York,
Washington, CAP, COLA, and JCAHO.
aThis state's CLIA-exempt program does not distinguish between serious and
nonserious requirements.
33Although CMS reviews the requirements of exempt-state programs and
accrediting organizations to ensure that they are at least equivalent to
CLIA's, there is not necessarily a one-to-one match with CLIA
requirements. Thus, one CLIA condition-level requirement may equal several
accrediting organization requirements or vice versa. For example, CMS's
condition-level requirement for successful lab participation in approved
proficiency testing corresponds to at least 19 CAP, 3 COLA, and 4 JCAHO
requirements.
CAP and COLA crosswalked their recent survey findings to CLIA
condition-level requirements.34 Although their analysis suggested that
from about 56 to 68 percent of labs surveyed during 2004 had a deficiency
in at least one condition-level requirement, they acknowledged that these
proportions overstated the subset of labs with serious problems. JCAHO did
not crosswalk its inspection requirements to those of CLIA because staff
would have had to manually review each survey report to determine which
deficiencies were equivalent to deficiencies in CLIA condition-level
requirements. However, JCAHO did tell us that in 2004, about 90 percent of
the labs it surveyed had a deficiency in at least one requirement and, as
previously noted, JCAHO classifies all its requirements as serious.
Despite the difficulty of identifying CLIA equivalent condition-level
deficiencies, two of the three accrediting organizations we reviewed have
systems to identify labs they survey that have serious quality problems.
COLA estimated that about 9 percent of labs it surveyed in 2004 were
subject to closer scrutiny because of the seriousness of the problems
identified. According to JCAHO, about 5 percent of the labs it surveyed in
2004 were not in compliance with a significant number of requirements. The
third accrediting organization, CAP, has criteria for identifying labs
that warrant greater scrutiny, but CAP officials told us that identifying
such labs had to be accomplished on a case-by-case basis rather than
through a database inquiry. As a result, CAP plans to spend in excess of
$9 million during 2006 and 2007 to develop an integrated data system that
pulls together multiple factors-survey results, complaints, proficiency
testing, findings of other inspection bodies, and changes in lab
directors-to enable it to readily identify problem labs. According to CAP
officials, such labs will be targeted for greater monitoring, and CMS and
other survey organizations will be notified about CAP's actions.
Proficiency Testing Results Suggest that Quality Has Not Improved at Hospital
Labs in Recent Years
Although CMS noted that proficiency testing trend data show a decrease in
failures for labs as a whole, the data suggest that lab quality may not
have improved at hospital labs for the period 1999 through 2003.
Proficiency testing is an important oversight tool for survey
organizations because it is an objective indicator of a lab's ability to
consistently produce accurate test results and is conducted more
frequently than surveys-three times a year versus once every 2 years. In
the absence of comparable survey data, proficiency testing results provide
a uniform way to assess the quality of lab testing across survey
organizations.
34This effort took CAP and COLA about 4 months to complete.
Our analysis of CMS proficiency testing data for 1999 through 2003
suggests that there has been an increase in proficiency testing failures
for labs inspected by CAP and JCAHO, which generally inspect hospital
labs, and a decrease in such failures for labs surveyed by state survey
agencies and COLA, which tend to inspect physician office labs (see fig.
2). CMS defines failures as unsatisfactory performance in two consecutive
or two out of three proficiency testing events. For example, the
percentage of labs with proficiency testing failures surveyed by CAP and
JCAHO from 1999 through 2003 increased from 4.1 percent to 6.8 percent and
from 6.6 percent to 7.8 percent, respectively. It is unclear, however,
whether the decrease in failures for physician office labs represents an
actual improvement in lab quality or reflects the fact that some
problematic labs are no longer surveyed. Specifically, many physician
office labs now perform waived tests and therefore are no longer surveyed
or participate in proficiency testing. Between 1998 and 2005, the
percentage of labs subject to surveys and proficiency testing decreased
from about 30 percent to about 19 percent.
Figure 2: Percentage of Labs with Proficiency Testing Failures from 1999
through 2003, by Survey Organization
Note: Data include labs affiliated with each organization during each
year.
Oversight Weaknesses Mask Quality Problems
Weaknesses in surveys, complaint processes, and enforcement mask real and
potential quality problems at labs. Survey weaknesses include: (1)
inspections that most organizations announce ahead of the visit, which
allows labs to prepare for their inspections and portray themselves in a
manner that may not accurately reflect their day-to-day quality assurance
processes; (2) variability in the proportion of labs with condition-level
deficiencies in 2004, which suggests surveys are not conducted in a
consistent manner; and (3) the goal of educating lab workers during
surveys taking precedence over, or precluding, the identification and
reporting of deficiencies. Furthermore, the significant increase in
complaints since CAP took steps to help ensure that lab workers know how
to file a compliant suggests that some quality problems at labs inspected
by other survey organizations may not be reported. Finally, sanctions are
not being used effectively as an enforcement tool to promote labs'
compliance with CLIA requirements, as evidenced by the relatively few labs
with repeat condition-level deficiencies on consecutive surveys from 1998
through 2004 that had sanctions imposed.
Announced Surveys May Result in Unrealistic Picture of Lab Quality
Because labs can and do prepare for surveys, CMS regional office officials
and most of the state survey agencies acknowledged that announced surveys
may not always provide a realistic picture of lab quality.35 As shown in
table 2, the amount of advance notice for surveys varies from as little as
2 weeks up to 12 weeks; until recently, only the New York CLIA-exempt
program conducted unannounced surveys. Survey agency officials in two
states told us that surveyors had inspected labs where records documenting
the implementation of periodic quality control procedures were completed
in the same handwriting using the same colored pen. This degree of
uniformity raises a concern about whether the quality control occurred at
all, or as frequently as the records suggested. A CAP surveyor told us
that the pathologist at one lab had cleaned up, and signed off on, about
3-months worth of quality control records the night before the survey. In
hearings on the questionable test results at a Maryland hospital lab, a
worker testified that lab staff prepared frantically for their announced
inspections.
35CMS does require unannounced surveys for: (1) complaint investigations,
(2) follow-up surveys conducted to verify correction of deficiencies, and
(3) nonroutine surveys conducted when there is reason to believe a lab is
operating in a manner that constitutes a risk to human health. In
contrast, all nursing home surveys are required to be unannounced to help
ensure that homes do not cover up problems that may exist when surveyors
are not present. See GAO, California Nursing Homes: Care Problems Persist
Despite Federal and State Oversight, GAO/HEHS-98-202 (Washington, D.C.:
July 27, 1998).
Table 2: Amount of Advance Notice Given to Labs about Upcoming
Inspections, by Survey Organization
Survey organization Amount of advance noticea
New York CLIA-exempt program None
State survey agencies Up to 2 weeksb
Washington CLIA-exempt program 4 weeks
JCAHO 4 weeksc
CAP 7 weeksd
COLA 12 weekse
Sources: CMS, New York CLIA-exempt program, Washington CLIA-exempt
program, CAP, COLA, and JCAHO.
aThese numbers reflect stated policy and may not represent actual
practice.
bAdvance notice permitted by CMS guidance.
cIn January 2006, JCAHO stopped providing labs advance notice about
upcoming inspections.
dAverage actual notice provided to CAP labs for 2004. In 2006, CAP began
conducting unannounced inspections.
eCOLA confirms the survey date about 8 weeks in advance.
In 2006, both CAP and JCAHO began conducting unannounced inspections at
most of the hospital labs they survey.36 Both CAP and JCAHO officials told
us that the unannounced surveys will occur as early as 6 months prior to
the anniversary of a lab's prior survey. CMS and survey organizations that
inspect physician office labs provided several justifications for
continuing to announce inspections at such labs, including (1) ensuring
that the lab is open and that appropriate personnel are available to
answer surveyor's questions, and (2) minimizing disruptions to patient
care. These justifications appear to be reasonable because they reflect
the operating tempo at physician office labs. It may not be appropriate,
however, to provide such labs with 4 to 12 weeks advance notice, given
that CMS currently limits the advance notice provided by state survey
agencies to no more than 2 weeks.
36JCAHO implemented its unannounced surveys in January 2006 and CAP began
phasing in unannounced inspections in the spring of 2006. CAP and JCAHO
will continue to provide Department of Defense and prison labs that they
survey with advance notice to enable surveyors to obtain the security
clearances required to enter such facilities.
Variability in Reported Survey Deficiencies Suggests that Labs Are Not Surveyed
Consistently
Variability in OSCAR data for state survey agency inspections conducted in
2004 suggests that labs are not surveyed in a consistent manner, and
interviews with CMS and state survey agency officials confirmed this
hypothesis. As a result, available data likely understate the extent of
serious quality problems at labs. In 2004, the percentage of labs that
state survey agencies reported with condition-level deficiencies varied
considerably by state, ranging from none in 6 states to about 25 percent
of labs in South Carolina. These data only included findings for about
one-half of the labs surveyed by state survey agencies. Of the 33 states
that survey more than 100 labs, 16 found condition-level deficiencies at
fewer than 5 percent of labs, while 6 states identified such serious
deficiencies in more than 10 percent of the labs they surveyed (see app.
II).37
Based on interviews with CMS and 10 state survey agencies, it appears that
at least some of this variability is due to differences in states'
approaches to surveys as opposed to true differences in lab quality. For
example, CMS told us that, because there is not a prescriptive checklist
to guide the survey process, the reliance on state surveyor judgment will
result in variations in the citing of deficiencies. To compensate for the
unstructured nature of the state survey process, officials we interviewed
from 2 state survey agencies told us that they created checklists to help
ensure that surveyors looked at all of the critical elements during lab
surveys.38 Furthermore, while some of the state survey agencies we spoke
with told us that their surveyors always cite condition-level deficiencies
that are identified during surveys, officials in other states said that
there are circumstances under which condition-level deficiencies would not
be cited. For example, according to officials from a state survey agency
we interviewed, surveyors prefer not to cite condition-level deficiencies.
Rather, surveyors in this state prefer to cite multiple standard-level
deficiencies instead of a condition-level deficiency because it allows the
imposition of state law sanctions, avoiding what was characterized as a
less efficient federal sanctions process.39 Additionally, officials from 2
other state survey agencies explained that surveyors consider a lab's
compliance history when determining what deficiencies to cite, while
officials from a third state told us that surveyors will educate lab
workers, particularly new lab workers, about the CLIA requirements rather
than citing CLIA condition-level deficiencies.40
37Our analysis excluded state survey agencies that inspect fewer labs
because even a small change in the number of labs with condition-level
deficiencies can produce a large percentage point change.
38While checklists may be useful, some state survey agencies told us that
use of the checklists may result in insufficient probing and observation.
CAP officials told us that they plan to move beyond their current emphasis
on requiring documentation of lab processes with probing techniques that
require direct interaction with lab staff, observation of testing, and new
survey tools to guide inspectors in assessing compliance with
requirements. COLA's survey process includes a list of questions that
surveyors must answer by asking probing questions of, and interacting
with, lab staff. JCAHO's process, introduced in 2004, uses computer-based
algorithms when making compliance determinations.
Balance Between Educational and Regulatory Roles by CMS and Survey Organizations
Appears to Be Inappropriate
The goal of educating lab workers sometimes takes precedence over, or
precludes, the identification and reporting of deficiencies that affect
the quality of lab testing. As a result, data on the quality of lab
testing and trends in quality over time may be misleading. Although CLIA
neither requires nor precludes an educational role for surveyors, the
preamble to CMS's implementing regulation noted that surveys are intended,
in part, to provide an opportunity for on-site education regarding
accepted laboratory procedures. In addition, CMS guidance and training
encourage state surveyors to play an educational role.41 Many state survey
agency officials we interviewed also told us that their surveyors play a
major educational role. As noted earlier, surveyors from one state survey
agency do not cite condition-level deficiencies when lab workers are new
but prefer to educate the new staff. Because CMS revised its OSCAR
database in 2004, it is not possible to identify states that have
consistently not cited condition-level deficiencies, data that would help
to quantify the extent to which an educational role is substituting for
appropriate regulation of labs.42
39When state survey agencies cite condition-level deficiencies, the CMS
regional office for that state determines what, if any, sanctions should
be imposed.
40One example cited by a state survey agency involved a complaint
investigation of a transfusion-related fatality, the result of a lab
worker mixing up patient samples. Because the lab had already instituted
extensive corrective actions by the time the surveyor arrived, the survey
agency cited a standard-level deficiency for documentation errors rather
than a condition-level deficiency. We discussed this case with CMS
officials who told us that because the problem had been addressed, there
was essentially no condition-level deficiency to cite.
41While combining the roles of educator and regulator may not be unique,
it is different from the exclusively regulatory role state surveyors under
contract with CMS play for other provider groups, such as nursing homes
and home health agencies.
42Using OSCAR trend data, we were able to identify state survey agencies
that educated instead of regulated home health agencies. See Medicare Home
Health Agencies: Weaknesses in Federal and State Oversight Mask Potential
Quality Issues, GAO-02-382 (Washington, D.C.: July 19, 2002).
An inappropriate balance between the educational and regulatory role is
also evident in some accrediting organization practices. One of the CAP
surveyors we interviewed with over 30 years of lab experience estimated
that the majority of pathologists-individuals who generally serve as CAP
survey team leaders-view surveys as educational, rather than as
assessments of compliance with lab requirements. Another surveyor told us
that CAP's survey process focuses heavily on education, and that some
survey team leaders emphasize education more than others. For COLA, the
process of educating labs begins even prior to a survey. For example, COLA
encourages labs to submit a self-assessment for review prior to the
scheduled survey so that the labs can identify COLA requirements with
which they are not in compliance. About 20 percent of all labs surveyed
during 2004 submitted a self-assessment (616 labs) and, compared to labs
that did not submit a self-assessment, fewer deficiencies were identified
at these labs during on-site surveys.43
CMS appears to be inappropriately stressing education over regulation in
its implementation of (1) 2003 lab quality control requirements for the
CLIA program and (2) proficiency testing for lab technicians who interpret
Pap smears, a test for cervical cancer. When state surveyors began
assessing compliance with new lab quality control requirements in January
2004, they were instructed to note deficiencies on a cover letter to labs
rather than on the survey report itself for a period of 2 years. Thus,
such deficiencies are not recorded in the OSCAR database. In part because
of a lack of lab "buy-in" for some of the new policies and procedures, CMS
officials have extended the educational period for about another 2 years.
CMS has taken a similar educational approach to Pap smear proficiency
testing, which began in 2005. CMS will not cite deficiencies or impose
sanctions against labs in which staff fail the new Pap smear proficiency
testing in 2005 or 2006, as long as the labs and individuals involved
complete such testing, including following the regulatory protocol for
subsequent testing in the case of an initial failure. According to CMS,
this educational focus allows labs and their staff to become familiar with
the proficiency testing program and to prepare themselves for such
testing, since there was about a 13-year time lag between the 1992
regulations that implemented CLIA and the 2005 implementation of Pap smear
proficiency testing.44 This educational approach seems questionable given
CMS's concern about some of the high initial proficiency test failure
rates.
43This percentage includes both labs preparing for their initial survey
and those with prior surveys. According to COLA officials, newer labs are
most likely to perform the self-assessment; over time, they believe that
the vast majority of COLA-inspected labs have completed the
self-assessment.
Use of Volunteer Surveyors by CAP Raises Concerns
Although state survey agencies, exempt-state programs, COLA, and JCAHO
employ dedicated staff surveyors, CAP relies primarily on volunteer teams
consisting of lab workers from other CAP-inspected labs to conduct
surveys.45 In contrast to the mandatory training and continuing education
programs in place for the staff surveyors of other survey organizations,
training for CAP's volunteer surveyors is currently optional. CAP plans to
establish a mandatory training program beginning in mid-2006.46 As a
condition of accreditation, labs inspected by CAP must survey another
CAP-accredited lab of similar size and composition at least once every 2
years. According to data provided by CAP, two-thirds of volunteer
surveyors who had recently participated in a survey had no formal training
in the 3 to 5 years preceding the survey. Two CAP surveyors we interviewed
told us that they had not completed any training because it was optional.
Two other surveyors told us that they had never been notified about the
existence of optional CAP training. While full-time surveyors employed by
other survey organizations conduct from 30 to about 200 surveys per year,
CAP volunteer surveyors have much less experience conducting surveys
because they only survey about one lab each year.
44Because of lab testing errors that led to women's deaths from cervical
cancer, Congress required a specific type of proficiency testing for
individuals who interpret the results of Pap smear tests, which requires
examining glass slides under a microscope. Although CLIA was enacted in
1988, CMS told us that cost, the inability to find a national testing
provider, and other technical issues delayed establishing a Pap smear
proficiency testing program until 2005.
45As of November 2005, CAP also employed 11 full-time surveyors.
Historically, CAP staff surveyors were responsible for inspections of
smaller labs that conduct less complex tests. Increasingly, staff
surveyors will (1) accompany survey teams assigned to labs with a large
number of deficiencies on their prior survey, and (2) assist teams in
conducting either a lab's initial survey or the team leader's initial
survey. Staff surveyors will also conduct more nonroutine surveys, such as
investigating a complaint or following up on performance concerns raised
during surveys.
46Currently, CAP volunteer surveyors are encouraged to participate in
surveyor training at least once every 3 years. In July 2006, CAP plans to
begin requiring survey team leaders to complete mandatory training.
Mandatory training for survey team members is targeted to begin in 2007.
Three of the nine CAP surveyors we interviewed stated that they believed
that mandatory training was important, but some surveyors wondered when
lab workers would have time to complete the courses because of their
demanding work schedules. According to CAP officials, however, the
required training will take only 1 to 2 days and surveyors will have a
choice of live seminars and workshops or e-learning completed at their own
computers.47 For ongoing training requirements, CAP plans to give
surveyors a choice of taking additional training or passing a competency
evaluation. CAP will track compliance with its new training requirements
to ensure that surveyors successfully complete training and demonstrate
competency within 2 years of participating in a survey. CAP's required
training is less extensive than that required by other survey
organizations. For example, state survey agency inspectors must complete 5
days of basic training and periodic advanced courses afterwards while COLA
staff inspectors participate in a 5-week orientation program and an annual
20 hours of continuing education.
The use of volunteer inspectors by CAP also raises concerns about the
appearance of a conflict of interest. These concerns arise because of the
way CAP survey teams are structured. CAP's Commission on Laboratory
Accreditation policy manual specifies that the inspection team leader is
the individual responsible for the conduct of an ongoing site inspection,
and must not be in a business, professional, or personal relationship that
would preclude an objective inspection. Furthermore, the manual states
that the inspection team leader is usually responsible for determining the
size of, and assembling, the inspection team. However, until April 2006,
CAP policy did not preclude competing labs from surveying one another or
lab survey team members from soliciting business, such as referrals, from
a lab at the conclusion of the survey.48 The policy was also silent about
survey team members' business, professional, or personal relationships
that could cloud their independence.49 Typically, inspection team leaders
are pathologists who direct other labs in the community, and the
inspection team is comprised of several employees from the team leader's
lab.
47In contrast, CAP staff surveyors complete a 6-month training program
before they are allowed to conduct surveys independently.
48In explaining this policy, CAP notes that it believes team leaders and
inspectors will conduct the inspection of a competing lab professionally
and in an objective manner.
49In April 2006, CAP issued a revised conflict of interest policy that
addresses these concerns.
We believe that the use of volunteers, including those from nearby labs,
and the personal and professional relationships that may exist among lab
staff and survey team members, creates the appearance of a conflict of
interest and could undermine the integrity of the survey process.50
Comments from some CAP surveyors we interviewed raise a concern about
having survey team leaders who are also the day-to-day supervisors of team
members. For example, lack of agreement about the seriousness of a
deficiency could result in the team leader instructing the team to
downgrade the deficiency to a recommendation, a less serious finding that
does not appear in the inspection report. Team members who are
subordinates to the team leader may feel that they have no other recourse
than to follow the team leader's instructions. Recognizing that team
members' objectivity may be compromised in this situation, CAP's revised
conflict of interest policy instructs all parties to be cautious to retain
objectivity in fact finding throughout the inspection process.
In discussing these findings with CAP officials, they told us that they
plan to institute a number of initiatives to help ensure survey
objectivity, including (1) resurveying the same lab by an independent team
to assess the consistency of inspections, (2) centralizing survey team
assignments performed by CAP staff, (3) not announcing surveys, and (4)
not notifying labs of the survey team composition prior to the survey.
Lab Workers Who File Complaints About Quality Problems in Lab Testing Not
Afforded Whistle-blower Protections
Some lab workers may not be filing complaints about quality problems at
their labs because of anonymity concerns or because they may not be
familiar with filing procedures. Complaints about labs can come from a
variety of sources, including lab workers. Complaints are an important
tool in detecting quality problems between lab surveys. For example,
complaints about testing at a hospital lab were crucial because
information had been concealed, complicating the detection of quality
problems during the lab's surveys. As a result of a complaint, surveyors
were able to substantiate inadequate calibration of testing equipment that
could adversely affect patient care.
50According to CAP, 57 percent of surveys in 2004 were conducted by
surveyors who worked in nearby labs. For example, surveyors who inspected
a Maryland hospital lab from 1999 through 2003 worked in labs that were
from 5 to 42 miles away. The remaining 43 percent were conducted by
surveyors who did not work in nearby labs and who therefore required air
travel to carry out the survey.
Based on OSCAR data and data obtained from exempt-state programs and
accrediting organizations for 2002 through 2004, few complaints were
received about lab testing relative to the number of labs-significantly
less than one complaint per lab per year.51 The low volume of lab
complaints may be related to complainants' concerns about anonymity and
fear of retaliation for filing a complaint. It may be easy for a lab to
determine the source of a complaint filed by a lab worker. For example, in
some cases, either the nature of the complaint or the piece of testing
equipment in question could narrow the list of possible complainants. Two
CAP surveyors we interviewed commented that, in their opinion, it would be
easy to determine the identity of a complainant. During congressional
hearings in 2004, a Maryland hospital lab worker testified that she and
her colleagues feared losing their jobs because of the complaints they
filed.
Because of the difficulty of protecting the anonymity of lab workers who
file complaints, whistle-blower protections for such individuals are
particularly important. Two of the three accrediting organizations we
interviewed have whistle-blower protections-CAP and JCAHO.52 For example,
CAP implemented a comprehensive whistle-blower protection policy in July
2004 that includes revocation of accreditation or other appropriate action
for any lab that directly or indirectly threatens, intimidates, or
retaliates against a lab worker. While officials from New York and
Washington's exempt-state programs told us that whistle-blower laws in
their states provide some protection for lab workers who file complaints,
officials in most of the other 10 states we interviewed told us that they
did not have any whistle-blower protections or were unable to identify
specific protections that applied to lab workers in their state.
Currently, there are no federal whistle-blower protections specifically
for workers in labs covered by CLIA. In 2005, legislation was introduced
to provide whistle-blower protections to workers in labs covered by
CLIA.53
We also found that lab workers may not know how to file a complaint. CAP
experienced a significant increase in the number of complaints it received
since October 2004, when it began requiring CAP-inspected labs to display
posters on how to file complaints. Specifically, from October through
December 2004, CAP received an average of 22 complaints per month,
compared to an average of 11 complaints per month in the 9 months
preceding the poster requirement. As a result, the number of complaints
about the quality of lab testing more than doubled in 2004 and the number
substantiated increased by more than 40 percent-even though the poster was
only displayed for the last 3 months of 2004 (see table 3).54 In September
2005, COLA also began requiring labs to display a complaints poster
similar to CAP's. It is too early, however, to determine the impact of
COLA's new complaints poster on the number, type, and substantiation rate
of complaints. Neither CMS nor JCAHO plans to require a similar complaints
poster.55
51The modifications to OSCAR did not affect data on the number of
complaints. The complaint information in OSCAR excludes complaints that do
not require an on-site survey.
52COLA does not have a formal whistle-blower policy. COLA officials told
us that they promptly investigate all complaints, many of them from former
lab employees, and keep the identity of the complainants anonymous.
53H.R. 686, 109th Cong. (2005).
Table 3: Number of Complaints Received by CAP, 2002-2005
Year Received Substantiated
2002 82 39
2003 84 40
2004 170 70
2005 290a 74 (preliminary)b
Source: CAP.
aThis number is as of November 30, 2005, and thus does not include
complaints received in December.
bAs of November 30, 2005, CAP had substantiated 74 complaints; over 100
complaints were still under active investigation.
Lab Sanctions Are Rarely Imposed
Few labs were sanctioned by CMS from 1998 through 2004-even those with the
same condition-level deficiencies on consecutive surveys-because many
proposed sanctions are never imposed. Our analysis of CMS enforcement data
from 1998 through 2004 found that 501 labs were sanctioned, which equates
to less than 3 percent of labs inspected by state survey agencies.56 The
most common were principal sanctions, which may result in suspension or
limitation of testing or termination from the CLIA program; few labs were
subjected to alternative sanctions, such as directed plans of correction
or civil monetary penalties (see table 4). Appendix III shows the number
of labs surveyed by state survey agencies and the number of sanctioned
labs from 1998 through 2004.
54CAP plans to hire an additional staff person to investigate complaints.
55Effective July 2005, JCAHO required labs to educate staff on how to
report concerns about lab quality to the Joint Commission, but does not
specify use of a poster to do so.
56According to CMS, sanctions generally result from deficiencies
identified during an inspection by a state survey agency.
Table 4: Number of Labs Inspected with Principal Only, Principal and
Alternative, and Alternative Only Sanctions Imposed, 1998-2004
Sanction Description Number of labs
Principal only o Revocation of CLIA certificate 269
(termination)
o Cancellation of approval to receive
Medicare payments
o Limits placed on testing
o Suspension of testing
Principal and o At least one principal sanction plus 170
alternative at least one alternative sanction
Subtotal 439
Alternative only o Directed plans of correction 62
o Civil money penalties
o State on-site monitoring
o Partial or full suspension of
Medicare payments
Total 501
Source: GAO analysis of CMS lab registries.
Although few labs were sanctioned from 1998 through 2004, over 9,000 labs
had sanctions proposed during that same time period.57 Before sanctions go
into effect, labs are given a grace period to correct condition-level
deficiencies, unless the deficiencies involve immediate jeopardy, that is,
an imminent threat to patient health and significant hazard to public
health. Most labs correct the deficiencies within the grace period. CMS
officials told us that it was appropriate to give labs an opportunity to
correct such deficiencies within a prescribed time frame and thus avoid
sanctions.58 However, a principal objective of the enforcement process-one
reflected in CMS guidance-is to motivate labs to comply with CLIA
requirements, thereby helping to ensure the provision of accurate and
reliable test results. Based on the large number of labs with proposed
sanctions that were never imposed, it is unclear how effective the
enforcement process is at motivating labs to consistently comply with CLIA
requirements.
57Since CMS data lists only the number of labs with proposed sanctions by
year, this number may double-count labs that had proposed sanctions in
multiple years.
58Labs surveyed by states have 23 days to correct immediate jeopardy
deficiencies, 90 days to correct all other condition-level deficiencies,
and up to 12 months to correct standard-level deficiencies. CAP and COLA
give labs 30 days to correct all deficiencies, and effective July 1, 2005,
JCAHO reduced the time labs have to correct deficiencies from 90 to 45
days. Labs may also appeal proposed sanctions, and depending on the
outcome of such appeals, sanctions may be dismissed.
The number of labs with the same repeat condition-level deficiencies from
one survey to the next also raises questions about the overall
effectiveness of the CLIA enforcement process. From 1998 through 2004, 274
labs surveyed by state survey agencies had the same condition-level
deficiency cited on consecutive surveys and 24 of these labs had the same
condition-level deficiency cited on more than two surveys.59 This analysis
may understate the percentage of labs with repeat condition-level
deficiencies because OSCAR data prior to 2004 no longer reflect about
two-thirds of condition-level requirements and associated deficiencies at
the time of those surveys. We found that only 30 of the 274 labs with
repeat condition-level deficiencies had sanctions imposed-either
principal, alternative, or both. According to the CLIA legislative
history, congressional concern about labs with repeat deficiencies led to
alternative sanctions to provide an enforcement option short of principal
sanctions to encourage compliance.
From 1998 through 2004, less than 1 percent of accredited labs (81) lost
their accreditation; few of these labs were subsequently sanctioned by CMS
and many still participate in the CLIA program.60 Our analysis of CMS
reports on sanctioned labs found that only 9 of the 81 labs had either
principal and/or alternative sanctions imposed and that 1 of the 9 still
performs moderate- to high-complexity testing.61 Based on a review of its
CLIA certificate database, CMS officials told us that about half of the 81
labs still perform moderate- to high-complexity testing but could not
describe the actions taken by CMS regional offices in response to the loss
of accreditation. We contacted state survey agencies or CMS regional
office officials to determine why 3 labs that COLA concluded had cheated
on proficiency testing by referring the samples to another lab to be
tested had no sanctions imposed. The purpose of proficiency testing is to
provide an objective, external evaluation of the accuracy of a lab's test
results, which is negated when another lab analyzes the sample. By
statute, the intentional referral of samples to another lab for
proficiency testing is a serious deficiency that should result in
automatic revocation of a lab's CLIA certificate for at least 1 year.62
Based on our interviews, we found that the 3 labs were allowed to continue
testing because they had initiated corrective actions; in effect, these
labs were given an opportunity to correct a deficiency that appears to
have required a loss of their CLIA certificate for at least 1 year. A
fourth lab was ultimately sanctioned for proficiency testing cheating by
CMS but was allowed to continue testing for almost 2 years after having
its accreditation revoked.
59Thirty-three states and the District of Columbia had at least one lab
with the same repeat condition-level deficiency.
60Twenty CAP-, 51 COLA-, and 10 JCAHO-inspected labs had their
accreditation revoked. After notice of revocation of accreditation, a lab
retains its CLIA certificate and may continue to test specimens for 45
days while a state survey agency inspects the lab and makes a
recommendation concerning the lab's continued participation in the CLIA
program to the responsible CMS regional office, unless CMS takes action
sooner. Potential outcomes include (1) termination from the CLIA program;
(2) determination that the lab meets CLIA requirements because they are
less stringent than those of the accrediting organization, resulting in
the lab switching to state survey agency oversight; (3) the lab's return
to compliance and reapplication to be surveyed by an accrediting
organization; or (4) cessation of moderate- to high-complexity testing and
assumption of a CLIA certificate that only allows less complex, waived
testing.
We also attempted to analyze data on other actions, short of revoking
accreditation, used by accrediting organizations to encourage lab
compliance and, in particular, how they respond to labs with serious
repeat deficiencies. According to CMS, this information is dispersed
across CMS regional offices. CAP officials told us that they could
initiate four intermediate actions including probation (lab is closely
watched to ensure correction of problems), accreditation with conditions
(nonroutine inspection to be scheduled), suspension of a lab section, and
cessation of a specific type of testing; suspension and probation were
instituted in 2004. According to CAP, in 2005, 28 labs were on probation,
106 labs were accredited with conditions, 1 lab was suspended, and 7 labs
were required to cease a specific type of test. In 2004, JCAHO awarded
conditional accreditation to 3 percent of the labs it inspects because
they were not in substantial compliance with its survey requirements, as
evidenced by the number of requirements not met; JCAHO conducts an on-site
follow-up survey at such labs. From 2002 through 2004, COLA required about
30 labs per year to cease testing due to issues identified during surveys
and about 217 labs per year to cease testing certain tests or specialties
due to unsuccessful proficiency testing.63
61We created a database for the sanctions data contained in CMS's annual
lab registry reports to Congress from 1998 through 2004 and were able to
identify labs that both lost accreditation and had a sanction imposed.
62Pub. L. No. 100-578, S: 2, 102 Stat. at 2911, codified at 42 U.S.C. S:
263a(i)(4)(2000).
CMS Oversight of CLIA Is Inadequate
CMS's oversight is not adequate to help ensure that labs meet CLIA
requirements. While CLIA requires proficiency testing quarterly, CMS only
requires such testing three times each year. In addition, the agency is
not meeting its responsibility to determine that accrediting organization
and exempt-state requirements and processes continue to be at least
equivalent to CLIA's. CMS attributed the delay in making equivalency
determinations to having too few staff. Further, ongoing CMS validation
reviews do not provide an independent assessment of the extent to which
surveys identify all condition-level deficiencies-primarily due to their
timing. Finally, CMS does not adequately use data, such as the results of
surveys, to monitor survey organization activities and processes.
Realizing that its existing oversight activities need to be strengthened,
CMS has begun instituting performance reviews to measure survey
organization compliance with its standards and is developing protocols to
ensure improved communication among survey organizations concerning
complaints about lab quality.
CMS's Implementation of Proficiency Testing Is Inconsistent with CLIA
CMS's decision to require proficiency testing for almost all laboratory
tests only three times a year is inconsistent with the statutory
requirement. CLIA requires that proficiency testing be conducted "on a
quarterly basis, except where the Secretary determines for technical and
scientific reasons that a particular examination or procedure may be
tested less frequently (but not less often than twice per year)."64 The
committee report on the bill that forms the basis for much of CLIA
indicated that "proficiency testing should be the central element in
determining a laboratory's competence, since it purports to measure actual
test outcomes rather than merely gauging the potential for accurate
outcomes."65
63In addition to suspension of or limits on testing, COLA uses directed
plans of corrections, such as requiring a lab to hire a consultant or
participate in specific training.
64Pub. L. No. 100-578, S: 2, 102 Stat. 2903, 2907-08, 42 U.S.C. S:
263a(f)(3)(2000).
65H.R. Rep. 100-899 at 28 (1988) reprinted in 1988 U.S.C.C.A.N. 3828,
3849.
In CMS's 1992 rule implementing CLIA, the agency provided a rationale for
reducing the frequency of proficiency testing, but did not provide a
technical and scientific basis for reducing the frequency for particular
procedures or tests.66 According to CMS's justification, experts were
divided on the appropriate frequency of proficiency testing generally. In
addition, requiring fewer events of proficiency testing would give
laboratories more time to analyze the causes of test failures before the
next event of proficiency testing and also enhance proficiency testing's
value as an educational tool. Because CLIA increased the number of labs
that were required to undergo proficiency testing, CMS believed that the
number of organizations that provided proficiency testing services would
not have been able to meet the anticipated increase in demand for testing
services. To help avoid anticipated delays in completing proficiency
testing and reporting requirements, CMS reduced the frequency of testing
events to three times per year.67
CMS's requirement for proficiency testing does not meet the conditions
specified in the statute that must be satisfied in order to require
testing less frequently than quarterly. The language of the statute, as
well as relevant legislative history, indicate that a decision to reduce
the frequency of testing should be in the nature of an exception made with
regard to a particular test, not the norm for all tests, and must be based
on "technical and scientific" considerations related to that particular
test.68 The reasons that CMS gave for requiring only three events per year
were not based on scientific and technical considerations relevant to
particular tests. Instead, CMS's decision was based on concerns of an
administrative and logistical nature that CMS wanted to alleviate by
reducing the frequency of testing events.
66See 57 Fed. Reg. 7002, 7128-29 (1992). Prior to 1992, CMS required
proficiency testing quarterly. See 55 Fed. Reg. 9538 (1990).
67According to a CMS official, the adoption of less frequent proficiency
testing was accompanied by an increase in the number of specimens subject
to proficiency testing from two every 3 months to five every 4 months.
68The committee report provided examples of technical and scientific
considerations justifying an exception to the quarterly testing
requirement. Those examples also stress the significance of excepting
tests from the quarterly testing requirement on the basis of circumstances
presented by the individual test. See H.R. Rep. 100-899 at 29 (1988), 1988
U.S.C.C.A.N. at 3850. That quarterly testing is intended to be the norm is
further evidenced by the committee report's recommendation that the number
of quarters that a laboratory had failed to pass proficiency testing
determine the severity of sanctions imposed. See id. at 30, 1988
U.S.C.C.A.N. at 3851.
CMS Is Late in Ensuring CLIA Equivalency of Exempt States' and Accrediting
Organizations' Inspection Requirements and Processes
We found that CMS has been late in determining that exempt states' and
accrediting organizations' inspection requirements and processes are at
least equivalent to CLIA's. CMS must verify their equivalency and, by
regulation, CMS requires such survey organizations to seek reapproval at
least once every 6 years, or more frequently if deemed necessary. CMS
establishes the time frames for when the next reapproval should occur,
which have ranged from about 15 months to about 6 years. However, CMS has
not completed its equivalency reviews within these time frames and
accrediting organizations and exempt state programs have continued to
operate without proper approval. Equivalency reviews for CAP, COLA, JCAHO,
and Washington due to be completed between November 1, 1997, and April 30,
2001, were an average of about 40 months late. In August 1995, CMS
determined that New York's next equivalency review should be completed by
June 30, 2001, but was over 4 years past due as of December 2005.
Similarly, COLA's equivalency review was about 3 years past due.
Because accrediting organizations and exempt-state programs may choose to
make changes to their inspection requirements between periodic equivalency
reviews (1) accrediting organizations are required to submit changes to
their inspection requirements and policies 30 days prior to changing their
standards69 and (2) exempt-state programs are required to provide notice
when they change their licensure or inspection requirements. Although
federal regulations require CMS to review equivalency when an accrediting
organization or exempt-state program adopts new requirements, a CMS
official told us that the agency is not required to review such changes
before their implementation to ensure equivalency.70 As a result, such
survey organizations may introduce changes that are inconsistent with CLIA
requirements. For example, JCAHO made a significant change to its
inspection requirements in January 2004 but did not receive CMS approval
until 6 months later; CMS did not begin an in-depth review of JCAHO's
revised requirements until early 2005-over a year after they were
implemented by JCAHO.71 According to CMS, their review has identified
several critical areas where JCAHO standards are less stringent than those
of CLIA. JCAHO acknowledged the need to make some adjustments to its
revised requirements.
69For example, CAP typically reorganized, consolidated, or changed its
requirements two times a year. As a result of these changes, about 1,000
requirements were removed and about 1,200 requirements were added over 5
years. COLA has changed its requirements five times since it was first
approved by CMS in 1993. JCAHO officials stated that they make changes to
their requirements each year.
70See 42 C.F.R. S: 493.573(a)(3)(2005).
CMS officials attributed delays in making equivalency determinations and
reviewing interim changes to having too few staff. The CLIA program,
located in CMS's Center for Medicaid and State Operations (CMSO),
currently has approximately 21 full-time-equivalent positions compared to
a peak of 29 such positions several years ago. The reduction occurred over
time through attrition. As required by statute, the CLIA program is funded
by lab fees and since its inception the program's fees have exceeded
expenses. As of September 30, 2005, the CLIA program had a carryover
balance of about $70 million-far more than required to hire an additional
six to seven staff members. However, CMS officials told us that because
the CLIA program staff are part of CMSO, they are subject to the personnel
limits established for CMSO, regardless of whether or not the program has
sufficient funds to hire more staff. Although CMSO is at its authorized
personnel allocation, the CLIA program could hire additional staff with
approval from the Administrator. We were told that CMSO has not requested
such approval.
We also noted issues that raise a question about the thoroughness of CMS
equivalency reviews because some survey organizations' procedures or
policies appear to be less stringent than those required by CMS for the
CLIA program. For example:
o Accrediting organizations provide labs more advance notice
about upcoming surveys than CMS allows state survey agencies to
give to the labs they inspect.
o JCAHO surveyors focus their review of lab testing on the 12
months prior to the survey. CMS requires that state surveyors
review the entire 24 months of testing since the last survey.72
o While CMS requires initial and advanced surveyor training, CAP
encourages but does not require its volunteer surveyors to
participate in surveyor training.73
o As of August 2005, CAP's policy manual indicates that complaint
investigations may be announced or unannounced. CMS guidance
requires that complaint investigations be unannounced.
Prior to 2005, CMS's equivalency determination reviews focused on
the inspection requirements themselves, and not the procedures and
policies used by accrediting organizations and exempt-state
programs in carrying out oversight of labs; this focus on
inspection requirements may explain the divergence from the
policies and procedures CMS requires for state survey agencies.
During 2006, CMS is simultaneously reviewing the equivalency of
COLA and JCAHO inspection requirements and, for the first time,
incorporating on-site observations of accrediting organization
policies and systems into the review and approval process. For
example, CMS is checking to ensure that accrediting organizations
have adequate systems in place to track such things as (1)
complaints, (2) correction of deficiencies, and (3) proficiency
testing.
CMS validation reviews that are intended to evaluate lab surveys
conducted by both states and accrediting organizations do not
provide CMS with an independent assessment of the extent to which
surveys identify all serious-that is, condition-level or
condition-level equivalent-deficiencies. CMS requires its regional
offices to conduct validation reviews of 1 percent of labs
inspected by state survey agencies in a year. In contrast,
validation reviews of 5 percent of labs inspected by accrediting
organizations during a year are conducted by state survey agency
personnel. CMS does not specifically require that validations
occur in each state and some states do not have validation reviews
each year. Furthermore, many validation reviews occur at the same
time a survey organization conducts its inspection and, in our
view, the collaboration among the two teams during these
simultaneous surveys prevents an independent evaluation.74
The requirement to validate 1 percent of labs surveyed by state
survey agencies in a year-roughly 100 validation reviews each
year-does not ensure sufficient oversight of state survey
agencies. The validation review requirement, which is included in
CMS regional office annual budget memorandums, does not specify
how many validation reviews must be conducted in each state. While
the 10 CMS regional offices generally validated 1 percent of the
state survey agency inspections within their region, they often
did not validate 1 percent of inspections within each state and,
in fact, performed none in some states. From 1999 through 2003,
federal surveyors:
o validated less than 1 percent of labs surveyed by state survey
agencies in an average of about 25 percent of states, ranging from
7 states in 2002 to 17 states in 2003; and
o did not conduct any validation reviews in an average of 16
percent of states per year, ranging from 3 states in 2002 to 12
states in 1999.
In 11 states, no validation reviews were conducted in multiple
years. For example, no validation reviews were conducted in
Michigan and Washington, D.C. during 4 of 5 years from 1999
through 2003. Without validating at least some surveys in each
state, CMS is unable to determine if the states are appropriately
identifying deficiencies.
Seventy-five percent of validations of state lab surveys were
conducted simultaneously from fiscal years 1999 through 2003.75
According to CMS officials, the large proportion of simultaneous
validation reviews provides an opportunity for federal surveyors
to share information with state surveyors, monitor their
conformance with CLIA inspection requirements, and identify
training and technical assistance needs. However, we found that
such reviews do not provide an accurate assessment of state
surveyors' ability to identify condition-level deficiencies. Of
the 13 validation reviews that identified missed condition-level
deficiencies, only 1 was a simultaneous review (see table 5).
Validations of state surveys typically utilize one federal
surveyor for either independent or simultaneous validation
reviews;76 therefore, increasing the proportion of independent
validation reviews to strengthen CMS oversight likely would not
require additional federal surveyors. Moreover, conducting
independent validation reviews eliminates the extra effort
required to coordinate schedules to ensure that the validation
reviews occur at the same time as the state survey.
71On January 1, 2004, JCAHO launched a new accreditation process, which
included a substantial consolidation of lab requirements. Additionally,
JCAHO began using new methodologies, including a new software program that
analyzes data to help focus on-site surveys on priority areas and a tracer
methodology to track patients and specimens through the continuum of lab
services.
72According to JCAHO, the organization plans to change its survey process
to include a review of a random sample of the other records and documents
over the entire 24-month period.
Many CMS Validation Reviews Lack Independence and Reviews Skip Some State Survey
Agencies
73As noted earlier, CAP plans to begin requiring mandatory surveyor
training in mid-2006.
Validation of State Survey Agency Lab Surveys
74Simultaneous surveys resemble "observational" federal oversight surveys
conducted at nursing homes. We previously reported that such surveys, in
which federal surveyors accompany and observe state surveyors during the
routine inspection of a nursing home, are not a realistic assessment of
state surveyor performance because CMS's presence may make state surveyors
more attentive to their survey tasks than when they are not being
observed, a phenomenon known as the Hawthorne effect. See GAO, Nursing
Home Care: Enhanced Oversight of State Programs Would Better Ensure
Quality, GAO/HEHS-00-6 (Washington, D.C.: Nov. 4, 1999).
75These validation reviews include both exempt-state and state survey
agency lab surveys.
Table 5: Analysis of Results of Simultaneous and Independent Validation
Reviews of State Surveys, Fiscal Years 1999-2003
Conducted Conducted
simultaneously independently
Number with
Number with condition-level
Number of condition-level deficiencies
Fiscal validation deficiencies state state surveyors
year reviews Number surveyors missed Number missed
1999 223 141 0 82 2
2000 224 135 0 89 5
2001 218 184 1 34 0
2002 240 199 0 41 4
2003 195 167 0 28 2
Total 1,100 826 1 274 13
Source: CMS.
Validation of Accrediting Organizations' Lab Surveys
According to CMS guidance, at least one validation review of an
accrediting organization's survey of labs should be conducted
simultaneously each year, but not all validation reviews should be
simultaneous because a combination of simultaneous and independent reviews
provides a balanced view of surveyor performance. CMS officials were
unable to tell us exactly how many of the roughly 275 validation reviews
conducted each year from fiscal year 1999 through fiscal year 2003 were
simultaneous.77 However, one of the three accrediting organizations we
reviewed told us that a significant proportion of their validation reviews
are conducted simultaneously. JCAHO estimated that 33 percent of its
validation reviews were conducted simultaneously. COLA estimated that 9
percent of validation reviews conducted in 2004 and 2005 were
simultaneous. Finally, CAP officials told us that, from 2002 through 2004,
11 percent of validation reviews of CAP-accredited labs were conducted
simultaneously.
76State survey agencies employ 96 full-time-equivalent staff to survey
labs.
77CMS did not begin tracking this information until August 2003.
Given the limitations of simultaneous reviews, conducting independent
validation reviews are a more effective way of ensuring the equivalency of
accrediting organization inspection requirements and processes between the
equivalency determinations. CMS officials told us that the agency's intent
in instituting simultaneous reviews was for state and accrediting
organization surveyors to share best practices, to promote understanding
of each other's programs, and to foster accrediting organization
improvement. They indicated that they considered it a learning experience
both if an accrediting organization surveyor added a deficiency noted by a
state surveyor to a survey report and vice versa. However, most of the
state survey agency officials we interviewed told us that simultaneous
validation reviews do not provide a realistic evaluation of the adequacy
of accrediting organizations' inspection process. In fact, CMS guidance
encourages surveyors to discuss the survey findings prior to concurrent
conferences with lab personnel to review their findings.
From fiscal years 1999 through 2003, state survey agency surveyors found
condition-level deficiencies missed by accrediting organization surveyors
on 64 validation reviews, but only 6 of these validation reviews were
simultaneous.78 In contrast, 58 (91 percent) of the validation reviews
that identified serious deficiencies missed by accrediting organizations
were independent validation reviews. (See table 6.)
78Both survey organizations submit their findings to CMS, which then
compares the findings to determine whether accrediting organization
surveyors missed any condition-level deficiencies. Examples of
condition-level deficiencies missed by accrediting organizations include:
(1) lab did not correctly calculate the results of tests used to monitor
patients using a blood-thinning medication, which could result in serious
medical complications such as internal bleeding; (2) lab did not follow
manufacturer's instructions for calibrating checks of certain test
equipment; and (3) lab director failed to provide overall management and
direction of lab, such as ensuring timely enrollment in a proficiency
testing program and corrective actions following a proficiency testing
failure.
Table 6: Analysis of Results of Simultaneous and Independent Validation
Reviews of Accrediting Organizations' Surveys, Fiscal Years 1999-2003
Validation reviews that
found condition-level
deficiencies missed by
accrediting
organizations' surveyors
Number of Number
Fiscal validation Number conducted conducted
year reviews Total simultaneously independently
1999 227 8 0 8
2000 265 17 3 14
2001 214 8 1 7
2002 317 14 1 13
2003 348 17 1 16
Total 1,371 64 6 58
Source: CMS.
Note: While our analysis covered validation reviews for all six
accrediting organizations, CAP, COLA, and JCAHO account for the vast
majority of such reviews. CMS officials were unable tell us the total
number of validation reviews conducted simultaneously and independently
during each fiscal year.
CMS Use of Data for Oversight of CLIA Program Is Limited
CMS does not routinely collect and analyze data essential for effective
oversight of the CLIA program but has initiatives to automate some
available data to make them more accessible for analysis. Using data to
analyze activities across survey organizations can be a powerful tool in
improving CMS oversight of the CLIA program. Such analyses include
identifying and addressing inconsistencies in how surveys are conducted.
Although CMS tracks the most frequently cited deficiencies at labs in an
effort to improve quality, it does not routinely track the proportion of
labs, by state, in which state survey agencies identify condition-level
deficiencies-those that denote serious or systemic problems. According to
a CMS official, the agency has not evaluated variability in such
deficiencies since 1999.79 As noted earlier in this report, variability in
survey findings suggests inconsistencies in how surveys are conducted. CMS
also does not require exempt-state programs and accrediting organizations
to routinely submit data on serious deficiencies identified at the labs
they inspect, unless the deficiencies pose immediate jeopardy to the
public or an individual's health.80 As noted earlier, the lack of a common
vocabulary on what constitutes a serious deficiency would make it
virtually impossible for CMS to analyze such data.
79Our nursing home work found a similar absence of analysis by CMS
regarding trends in serious problems identified by state surveyors. See
GAO, Nursing Home Quality: Prevalence of Serious Problems, While
Declining, Reinforces Importance of Enhanced Oversight, GAO-03-561
(Washington, D.C.: July 15, 2003).
We also found that CMS does not effectively use available data to assess
clinical lab quality in areas such as proficiency testing, sanctions, and
complaints. For example, CMS's analysis of proficiency testing data for
all labs showed improvements over time. As reported earlier, proficiency
testing failures have increased for labs surveyed by CAP and JCAHO.
Comprehensive analysis of the proficiency testing database is particularly
valuable because it provides a uniform way to assess the quality of lab
testing across survey organizations, which is not currently available for
survey results. CMS is now in the process of automating the annual
registry of sanctioned labs, which should help it identify important
trends, such as the infrequent use of alternative sanctions. Automating
the registry, however, will not address the lack of data on (1) steps
taken by state survey agencies and regional offices when labs have their
accreditation revoked or (2) interim steps, short of revocation of
accreditation, that accrediting organizations take to help encourage lab
compliance. CMS also lacks a complaints database, and therefore was unable
to assess the impact of CAP's decision to require labs to prominently
display a poster on how to file a complaint. CMS is developing, and plans
to launch, a complaints database in March 2006.81
80Officials from a state survey agency told us that, while they do not
want to receive routine survey reports from accrediting organizations,
they do want to receive information when accrediting organizations
identify problem labs-before a lab loses its accreditation and becomes the
responsibility of the state survey agency. When an accrediting
organization revokes a lab's accreditation, the state survey agency
becomes responsible for determining whether to recommend revocation of the
lab's CLIA certificate to the appropriate CMS regional office.
81In response to communication problems highlighted by the complaint
investigations at a Maryland hospital lab, CMS has been meeting regularly
with officials from exempt-state programs, accrediting organizations,
state survey agencies, and its regional offices to discuss and improve
coordination and data sharing-particularly the handling of complaints. The
Maryland state survey agency identified deficiencies during a 2004
complaint investigation but did not inform CAP, the organization
responsible for surveys of the Maryland hospital lab. CAP officials told
us that they first learned about the 2004 complaint investigation that
revealed problems with HIV and hepatitis testing equipment from
newspapers.
CMS Implementing Performance Reviews for Survey Organizations
CMS has implemented performance reviews for state survey agencies and is
in the process of developing such reviews for accrediting organizations.
First implemented in 2004, the annual CLIA state performance reviews
evaluate each survey agency's ability to accomplish its lab oversight
responsibilities.82 The reviews, conducted on-site by CMS regional office
staff, measure performance in 13 areas, such as the timely conduct of
surveys and the appropriate documentation of any deficiencies
identified.83 According to CMS, the reviews are based on the
performance-improvement model that characterizes much of the
administration of the CLIA program.84 Consequently, the primary role of
regional offices in conducting the reviews is to provide education and
support for state survey agency improvement. For the 2004 reviews, 38
states were required to submit corrective action plans to their respective
CMS regional offices in at least 1 of the 13 areas examined. Three areas
required the most corrective action plans: principles of documentation,
proficiency testing desk reviews, and survey time frames.85
o Principles of documentation. CMS found that some state survey
agencies lacked the supervisory personnel to conduct internal
reviews intended to ensure the appropriate documentation of
deficiencies. It also found that some state survey agencies did
not follow the protocol instructions on how to quantify such
reviews.
o Proficiency testing desk reviews. Because of personnel
shortages, some state survey agencies were unable to perform
proficiency testing desk reviews between surveys, waiting instead
until the next on-site survey to address unsuccessful proficiency
testing. CMS plans to provide additional surveyor training on desk
review requirements.
o Survey time frames. CMS regional office staff were inconsistent
in scoring whether state survey agencies met the established time
frames for initial surveys. While some regions were lenient if a
state missed the time frame by just 1 day or provided a reasonable
explanation-such as staff turnover or illness-other regions were
more stringent in scoring states against the standard.
However, it is not yet clear to what extent the 2004 scores
represent state survey agency shortcomings or a learning curve for
the states in understanding the performance review protocols.
In partnership with the accrediting organizations, CMS is
developing performance standards comparable to, but different
from, those implemented in 2004 for state survey agencies. For
example, both the state survey agency review protocols and those
proposed for accrediting organizations measure the timeliness of
the surveys, but those proposed for the latter would also focus on
several areas that are unique to accrediting organizations. The
performance standards would include (1) timely and consistent
information sharing and (2) alerting CMS about decisions to limit
or remove accreditation in a timely manner. According to a CMS
official, the agency plans to phase in the performance standards,
starting with a standard on complaints. For example, if the CLIA
complaints database is activated in March 2006, CMS could begin to
monitor accrediting organization responsiveness to, and outcomes
of, complaints. Because the database will contain national lab
complaint data, CMS will be able to compare the volume and outcome
of complaints across survey organizations. According to CMS,
implementation of the accrediting organization performance
standards will be a central-not regional-office responsibility.
Clinical labs play a pivotal role in the nation's health care
system by diagnosing many diseases, including potentially
life-threatening diseases, so that individuals receive appropriate
medical care. Given this important role, lab tests must be
accurate and reliable. CMS and survey organization oversight is
intended to ensure that labs produce reliable test results, a key
objective of CLIA. Our work demonstrated that the oversight of
clinical labs needs to be strengthened in several areas.
Determining the quality of lab testing is difficult because it is
virtually impossible to crosswalk inspection requirements across
survey organizations. Without standardized survey findings across
all survey organizations, CMS cannot tell whether the quality of
lab testing has improved or worsened over time or whether
deficiencies are being appropriately identified.
Lab oversight has weaknesses that make it difficult to determine
the quality of lab testing because they mask quality problems. To
help surveys provide a realistic picture of day-to-day operations,
CAP and JCAHO began unannounced surveys of the labs they
survey-generally hospital labs-in 2006. While unannounced surveys
at physician office labs may not be practical, Washington's exempt
program and COLA currently give such labs more advance notice than
the 2 weeks CMS prescribes for labs inspected by state survey
agencies. Similarly, the greater weight that CMS and survey
organizations sometimes place on their educational, as opposed to
their regulatory role may lead to survey findings that do not
accurately reflect lab quality. Educating labs to ensure
high-quality testing should complement but not replace the
enforcement of CLIA inspection requirements. The low number of lab
complaints may be the result of a lack of information about how to
file a complaint and lab workers' fear of retaliation. Because
protecting the anonymity of lab workers who file complaints is
difficult, whistle-blower protections for such individuals are
particularly important. Finally, labs with the same serious
deficiencies on consecutive surveys often escape sanctions, even
though Congress authorized alternative sanctions to give CMS more
flexibility to achieve lab compliance. Without the threat of real
consequences, labs may not be sufficiently motivated to comply
with CLIA inspection requirements.
CMS's oversight is not adequate to enforce CLIA requirements. The
agency is not requiring labs to participate in proficiency testing
on a quarterly basis, as required by CLIA. Furthermore CMS is not
conducting CLIA-equivalency determinations within the time frames
it established for such reviews, nor has it always reviewed
changes to exempt-state and accrediting organizations' inspection
requirements before their implementation, even though it requires
their submission to ensure continued CLIA equivalency of their
requirements. Although the CLIA program has generated funds, CMS
agencywide staffing limitations have prevented the program from
hiring sufficient staff to complete equivalency reviews in a
timely manner. Many validation reviews are conducted at the same
time a survey organization conducts its survey, and such
simultaneous reviews may not provide a true assessment of surveyor
performance. Independent validation reviews of accrediting
organization surveys are critical because CMS has not conducted
equivalency reviews within the time frames it established. We also
found that few validation reviews of state survey agency lab
inspections are conducted each year and that none occurred in some
states. Because state surveyors conduct validation reviews of
accrediting organizations to ensure the continuing CLIA
equivalency of their inspection requirements, conducting an
appropriate number of validation reviews of state survey agency
lab inspections is critical. CMS also has not yet taken the lead
in ensuring the availability and use of data from survey
organizations to help it monitor their performance-particularly
the consistency with which surveys are conducted. CMS is creating
a new complaint database, but its plan to automate the existing
sanctions registry will not address the lack of data on
enforcement actions taken by state survey agencies and regional
offices when labs have their accreditation revoked.
To enable CMS to track the nature and extent of lab quality
problems across survey organizations, we recommend that the CMS
Administrator take the following action:
o Work with exempt-state programs and accrediting organizations
to standardize their categorization and reporting of survey
findings in a way that tracks to CLIA inspection requirements and
allows for meaningful comparisons across organizations, such as
the analysis of trends in the citation of condition-level
deficiencies.
To ensure consistency in the oversight of labs by survey
organizations, we recommend that the CMS Administrator take the
following four actions:
o Ensure that the advance notice of upcoming surveys provided to
physician office labs is consistent with CMS's policy for advance
notice provided by state survey agencies.
o Ensure that regulation of labs is the primary goal of survey
organizations and that education to improve lab quality does not
preclude the identification and reporting of deficiencies that
affect lab testing quality.
o Impose appropriate sanctions on labs with consecutive
condition-level deficiencies in the same requirements.
o Require all survey organizations to develop, and require labs
to prominently display, posters instructing lab workers on how to
file anonymous complaints.
To improve oversight of labs and survey organizations, we
recommend that the CMS Administrator take the following eight
actions:
o Consistent with CLIA, require quarterly proficiency testing,
except when technical and scientific considerations suggest that
less frequent testing is appropriate for particular examinations
or procedures.
o Ensure that evaluations of exempt-state and accrediting
organization inspection requirements take place prior to
expiration of the period for which they are approved in order to
ensure the continued equivalency of their requirements with
CLIA's.
o Ensure that changes to the inspection requirements of exempt
states and accrediting organizations be reviewed prior to
implementation, as required by regulation, to ensure that
individual changes do not affect the overall CLIA equivalency of
each organization.
o Allow the CLIA program to utilize revenues generated by the
program to hire sufficient staff to fulfill its statutory
responsibilities.
o Ensure that federal surveyors validate a sufficient number of
inspections conducted by each state survey agency to allow a
reasonable estimate of their performance, including a minimum of
one independent validation review for each state survey agency
surveyor.
o Require that almost all validation reviews of each accrediting
organizations' surveys be an independent assessment of
performance.
o Collect and routinely review standardized survey findings and
other available information for all survey organizations to help
ensure that CLIA requirements are being enforced and to monitor
the performance of each organization.
o Establish an enforcement database to monitor actions taken by
state survey agencies and regional offices on labs that lose their
accreditation.
We provided a draft of this report to CMS, and to CAP, COLA, and
JCAHO-the three laboratory accrediting organizations included in
our review. CMS strongly endorsed our overall conclusion that
quality assurance for the nation's clinical labs should be
strengthened and noted that the report provided insights into
areas where it can improve, augment, and reinforce oversight of
both labs and accrediting organizations to ensure quality testing.
Overall, CMS concurred with 11 of our 13 recommendations. Despite
this endorsement, however, CMS (1) provided an alternative
assessment of lab quality, (2) disagreed that the phase-in of
certain CLIA requirements inappropriately stressed education as
opposed to regulation, (3) expressed concern about how to identify
and sanction labs with repeat condition level deficiencies, (4)
disagreed with our recommendation regarding the frequency of
proficiency testing, and (5) stated that it was already meeting
our recommendation to conduct almost all validation reviews of
each accrediting organization independently. We continue to
believe that implementation of these recommendations is necessary
for the effective oversight of labs. (CMS's comments are
reproduced in app. IV.) CAP indicated that it took seriously our
findings and recommendations and intended to determine if there
were additional measures it could take to strengthen its own
oversight. COLA said that our recommendations to improve CMS
oversight of survey organizations had merit. Nonetheless, CAP,
COLA, and JCAHO disagreed with some of our findings and
recommendations to CMS. (CAP, COLA, and JCAHO's comments are
reproduced in app. V, VI, and VII, respectively.) Our evaluation
first responds to CMS's and related accrediting organizations'
comments and then addresses additional comments by accrediting
organizations.
CMS and COLA commented that lab performance has improved since the
enactment of CLIA. In particular, CMS pointed to the substantial
decline-from about 80 percent to about 42 percent-in the
percentage of labs nationwide with deficiencies between 1994 and
2004. It is important to note that CMS's data (1) do not
distinguish between serious condition-level deficiencies and less
serious standard-level deficiencies,86 (2) include the early
start-up period when physician office labs were first regulated,87
and (3) exclude deficiency data on the substantial number of labs
surveyed by accrediting organizations and state CLIA-exempt
programs. Due to these shortcomings, we do not believe that CMS's
data provide an accurate assessment of lab quality nationwide.
Based on the limited data available on state survey agency
inspections of labs since 1998 and the lack of any comparable data
on accrediting organization and exempt-state program survey
findings, we concluded that insufficient data existed to identify
the extent of serious quality problems at labs. CMS did not retain
backup files of pre-2004 data on deficiencies identified by state
survey agencies. Although CMS has determined that accrediting
organization and exempt-state program lab requirements are at
least equivalent to CLIA's, there is no agreement across survey
organizations on how to distinguish serious from less serious
deficiencies. While CMS concurred with our recommendation to
standardize the categorization and reporting of survey findings in
a way that tracks to CLIA and allows meaningful comparisons across
survey organizations, it noted that a straightforward linkage of
requirements is limited by CMS's authority under the statute-that
is, survey organizations are permitted by statute to have
different requirements-and that it will approach implementation of
our recommendation cautiously. JCAHO said that it agreed with the
need for a common, agreed upon, taxonomy that could be used by all
survey organizations to track serious deficiencies, but commented
that it thought CMS's implementation of our recommendation would
require a revamping of JCAHO's accreditation system. That was not
the intent of our recommendation and it is clear from CMS's
comments that its implementation of our recommendation would not
require an overhaul of accrediting organizations' systems. CAP
acknowledged the complexity and inherent challenges in measuring
the quality of lab testing, but noted that it is committed to
working to develop better systems to detect labs with serious
quality problems-those that impact patient care.
The statutory authority that permits standards different from
CMS's (provided they are at least as stringent) does not impair
the ability to develop a crosswalk that allows for meaningful
comparisons across survey organizations-such as an analysis of
trends in the citation of condition-level deficiencies. In fact,
CMS regulations already require accrediting organizations and
exempt-state programs to submit a crosswalk-detailed comparisons
of their individual accreditation or licensure approval
requirements with comparable CLIA condition-level
requirements-when they apply and reapply for approval from CMS.88
Such a comparison is possible because CMS already identifies
instances when accrediting organizations have missed
condition-level requirements during validation reviews. For
example, CMS should require survey organizations to (1) indicate
which of their requirements relate to each CLIA condition-level
requirement, and (2) explain which deficiencies in their
requirements, if cited, should be considered equivalent to CLIA
condition-level deficiencies.
CMS also pointed to the steady increase in successful proficiency
testing across all labs as an indication of improvements in lab
quality. Our analysis of proficiency testing results suggested
that lab quality had not improved at hospital labs in recent
years. CMS correctly noted that the overall proportion of labs
with no test failures increased from about 88 percent in 1998 to
about 93 percent in 2003-that is, fewer labs failed proficiency
testing. However, by focusing on overall proficiency testing
results, CMS data mask trends in failure rates for subsets of labs
such as hospital labs. For example, from 1999 through 2003, the
percentage of CAP-surveyed labs with proficiency testing failures
increased from 4.1 percent to 6.8 percent; CAP generally inspects
hospital labs. CMS also commented that the overall improvement
cannot be dismissed as a result of some labs being granted waived
status because the more dramatic improvements predated the recent
increase in the number of waived labs. It further commented that
removing waived labs from the data would not result in improved
performance rates. First, the number of waived labs-those
performing waived tests or provider-performed microscopy-increased
by about 26,600 from 1993 though 1998 and then increased by
another approximately 33,700 labs from 1998 through 2004. Second,
CMS's comment suggested that it had conducted an analysis of the
impact of removing waived labs from the proficiency testing data.
However, it did not provide any data analysis when we subsequently
asked to see the evidence behind its assertion. COLA also
addressed this issue, and did not challenge our conclusion that
the decrease in proficiency testing failures for physician office
labs might not represent an actual improvement in lab quality, but
instead could reflect the fact that some problematic labs are no
longer surveyed.
CMS agreed that it was important to maintain an appropriate
balance between its regulatory and educational approaches to CLIA
implementation. While CMS noted that objective review and feedback
are the bedrock of education, it emphasized that the educational
approach does not preclude surveyors from identifying lab
deficiencies. CAP and COLA offered similar comments. However, we
found evidence that the goal of educating lab workers sometimes
takes precedence over, or precludes, the identification and
reporting of deficiencies and recommended that CMS take steps to
ensure that regulation remains the primary goal of surveys. To
address this problem, CMS stated that it will provide additional
state agency surveyor training, improve guidance, develop an
action plan to promote greater consistency among surveyors, and
institute periodic performance and consistency reviews. CMS's
comments did not address evidence we presented that an educational
emphasis may also prevent fulfillment of regulatory
responsibilities by some accrediting organizations.
CMS disagreed that the extended phase-in periods for new quality
control requirements and proficiency testing for lab technicians
who interpret Pap smears were inappropriate. COLA noted that
federal requirements in many regulated industries are phased in to
allow them time to understand and effectively implement the
requirements. CMS reaffirmed that, in the case of significant new
requirements and for the time period specified by CMS, the
educational approach may result in identified deficiencies being
communicated to laboratories without a concomitant citation, as is
the case with quality control and Pap smear testing requirements.
As discussed in the report, we believe that CMS's educational
phase-in periods are excessive. We found that the phase-in period
for new quality control requirements was extended from 2 years to
about 4 years, in part because of the lack of lab "buy-in" for
some of the new policies and procedures. Similarly, the phase-in
period for Pap smear proficiency testing is 2 years, despite (1)
CMS's concern about some of the high initial test failure rates,
(2) the consequences of inaccurate test results on patients'
diagnoses and treatment, and (3) the approximately 13-year time
lag between the 1992 implementation of the CLIA regulations and
the commencement of Pap smear proficiency testing.
In commenting on our recommendation to appropriately sanction labs
with repeat condition-level deficiencies, CMS acknowledged the
need to carefully monitor repeat deficiencies but expressed
concern that focusing on the condition cited may not indicate a
true repeat deficiency because the underlying failures could have
been different in the two consecutive surveys for those labs.89
CMS's assertion is inconsistent with its own policy on serious,
repeat deficiencies for other providers, such as nursing homes. In
general, immediate sanctions must be imposed on nursing homes with
consecutive serious deficiencies, regardless of whether the
deficiencies are in the same care area. As we have previously
reported, allowing providers to avoid sanctions by correcting
serious deficiencies contributes to an up-and-down pattern of
compliance and undermines the deterrent effect of sanctions.90
According to the CLIA legislative history, congressional concern
about labs with repeat deficiencies led to the introduction of
alternative sanctions such as civil money penalties as a
substitute for more severe principal sanctions, which include
termination from the CLIA program.
CMS disagreed with our recommendation that it require quarterly
proficiency testing except when technical and scientific
considerations indicate that less frequent testing is justified
for particular tests; CMS insisted that proficiency testing three
times a year was "appropriate." CMS stated that CMS and the
Centers for Disease Control and Prevention had together determined
that the reduced frequency was based on technical and scientific
grounds. We asked for a record of the agencies' deliberation
supporting that decision. CMS supplied a brief, undated narrative,
which it attributed to the Centers for Disease Control and
Prevention. It was not clear to us that this narrative was
contemporaneous with the decision to reduce the frequency of
proficiency testing. Moreover, the narrative focused on the
relative costs and benefits of proficiency testing at various
intervals. There was no analysis of technical and scientific
considerations with regard to particular tests that presented a
basis for reducing the frequency.
Based on CMS's response, we maintain that CMS's decision to
require proficiency testing three times a year is not authorized
by CLIA. CMS did not dispute that, according to CLIA, it must base
the decision to reduce the frequency of proficiency testing on
scientific and technical considerations relevant to particular
tests, and that the decision to reduce the frequency is in the
nature of an exception to the norm of quarterly testing. CMS also
acknowledged that the "public explanation" contained in the
preamble to the rule setting the proficiency testing requirement
at three times a year referred only to general concerns about the
perceived burden associated with quarterly testing. For example,
CMS stated in its final rule that the prospect of reduced
frequency would provide a "needed respite" to both laboratories
and proficiency test providers. In sum, CMS has not adhered to the
conditions set out in the statute for reducing the frequency of
proficiency testing and has implemented a policy that is not
supported by statutory authority.
CMS acknowledged the need to complete timely equivalency reviews
of accrediting organization and exempt-state requirements, which
were an average of about 40 months late for the 3-1/2-year period
we examined. Regarding interim changes made between periodic
equivalency reviews, CMS agreed with our recommendation and stated
it would review such interim changes for both accrediting
organizations and exempt-state programs prior to implementation,
as required by regulation.91
Furthermore, CMS indicated that changes to accrediting
organization requirements did not necessarily impact CLIA
equivalency determinations because accrediting organizations may
have more stringent requirements than CLIA's. While possibly true,
CMS must review the changes to determine whether CLIA equivalency
is affected. For example, in 2005, when it reviewed JCAHO's
revised standards a year after they were implemented, CMS
identified several critical areas where JCAHO's standards were
less stringent than those of CLIA.
CMS acknowledged that a significant increase in workload and the
decline in CLIA program staff were factors which contributed to
delays in making equivalency determinations and reviewing interim
changes. Although CMS stated that it reserved the right to manage
the work within available resources and its assessment of
priorities, it also made a commitment to explore our
recommendation to utilize revenues generated by the CLIA program
to hire sufficient staff to fulfill its responsibilities. We
believe that additional staff would not only improve the
timeliness of equivalency reviews, but also their thoroughness.
CMS stated that, consistent with our recommendation, 88 percent of
accrediting organization validation reviews were conducted
independently in calendar year 2005. However, our recommendation
was to require that almost all validation reviews of each
accrediting organizations' surveys be conducted independently.
CMS's comments do not indicate the proportion of independent
validation reviews conducted for each accrediting organization.
Because CMS did not begin collecting data on the number of
simultaneous accrediting organization validation reviews performed
until August 2003, we relied on estimates from accrediting
organizations.92 CMS did not challenge JCAHO's estimate that 33
percent of its validation reviews were simultaneous, compared to
about 10 percent for CAP and COLA. We do not believe that
performing an estimated 33 percent of JCAHO's validation reviews
simultaneously is consistent with our recommendation.
COLA commented that simultaneous validation reviews are useful in
assuring consistency and in providing an understanding of
processes across survey organizations. It also questioned the
accuracy of most of the missed condition-level deficiencies
identified by CMS during independent validation reviews. We did
not assess the process CMS uses to identify such missed
deficiencies but based on a discussion with CMS officials it
appears that the process is thorough and time consuming. JCAHO
commented that we had misinterpreted the results of simultaneous
and independent validation reviews of accrediting organizations
because, by JCAHO's estimate, the proportion of missed
condition-level deficiencies is roughly equivalent for both types
of surveys. We did not find the assumptions behind JCAHO's
estimate convincing, given the lack of data on the actual number
of simultaneous versus independent validation reviews conducted
for each accrediting organization. Furthermore, most of the state
survey agency officials we interviewed, whose inspectors conduct
accrediting organization validation reviews, told us that
simultaneous validation reviews do not provide a realistic
evaluation of the adequacy of accrediting organizations'
inspection processes.
Additional CAP comments. CAP commented that we underestimated the
value of using lab professionals in the inspection process and
that we provided no factual evidence that their use was less
effective than other models. In contrast to CAP, other survey
organizations employ dedicated staff surveyors who have mandatory
and continuing education requirements. In addition, such dedicated
surveyors conduct from 30 to about 200 surveys per year compared
to CAP's lab professionals who volunteer to perform about 1 survey
per year. CAP partially addressed our concern about the lack of
mandatory training for its volunteer surveyors. It plans to begin
requiring training for survey team leaders in July 2006 and for
survey team members in 2007. However, CAP's proposed new mandatory
training is much less extensive than that required by other survey
organizations.
Moreover, we reported that some CAP surveyors we interviewed
raised a concern about having survey team leaders who are also the
day-to-day supervisors of team members. For example, lack of
agreement about the seriousness of a deficiency could result in
the team leader instructing the team to downgrade the deficiency
to a recommendation, a less serious finding that does not appear
in the inspection report. Team members who are subordinates to the
team leader may feel that they have no other recourse than to
follow the team leader's instructions. CAP recently revised its
conflict-of-interest policy which now instructs all parties to be
cautious to retain objectivity in fact finding throughout the
inspection process. We do not believe that this change in CAP's
conflict-of-interest policy addresses the concerns raised by the
CAP surveyors we interviewed. In its comments, CAP indicated that
it would continue to closely monitor this issue to determine if
further actions were necessary.
Additional COLA comments. COLA disagreed with our assertion that
announced surveys may result in an unrealistic picture of lab
quality-a conclusion supported by CMS regional office staff and
most state survey agency officials we interviewed. We acknowledged
that unannounced surveys of the physician office labs typically
surveyed by COLA and state survey agencies were not practical
given the unpredictable operating hours of such labs and need to
minimize disruptions to patient care. However, we recommended that
the advanced notice be limited to the 2 weeks permitted by CMS for
state survey agencies. COLA currently provides up to 12 weeks
advanced notice. COLA contends that providing up to 6 months of
advanced notice before a survey would only improve the lab's
operation more quickly if the lab took that opportunity to review
COLA's self assessment questions and correct any missing or
incorrect processes or documentation. We believe that COLA's
example underscores the importance of our recommendation; such
actions should be an ongoing process at labs-not a reaction to an
upcoming inspections.
Additional JCAHO comments. JCAHO said that our recommendation that
all survey organizations develop and require labs to prominently
display posters that instruct lab workers on how to file anonymous
complaints was too narrow and prescriptive and may inadvertently
limit organizations from using other, more effective ways to
educate lab workers on this topic. JCAHO did not explain how
implementing our recommendation would limit other initiatives. In
fact, CMS's comments identified a number of promising approaches
that it believed could supplement posters. JCAHO also said that
our analysis of the increase in CAP complaints after it required
posters in the labs it inspects failed to recognize a broad
national trend. JCAHO indicated that it also experienced a
dramatic increase in lab complaints between 2004 and 2005 without
the use of posters.93 This increase may be related to JCAHO's July
2005 requirement for labs to educate staff on how to report
concerns. CAP told us that during the 3 months after they required
a poster to be displayed they observed an immediate increase in
the number of complaints. Thus, CAP lab complaints increased by
over 100 percent in 2004 compared to 2003 and by another
approximately 71 percent in 2005. We continue to believe that
CAP's experience suggests that complaint posters can be an
important way to encourage lab workers to communicate their
concerns.
CMS, CAP, COLA, and JCAHO also provided technical comments which
we incorporated as appropriate.
As arranged with your offices, unless you publicly announce its
contents earlier, we plan no further distribution of this report
until 30 days after its issue date. At that time, we will send
copies to the Administrator of the Centers for Medicare & Medicaid
Services and appropriate congressional committees. We will also
make copies available to others upon request. In addition, the
report will be available at no charge on the GAO Web site at
http://www.gao.gov .
If you or your staff have any question s about this report, please
contact me at (312) 220-7600 or [email protected] . Contact
points for our Offices of Congressional Relations and Public
Affairs may be found on the last page of this report. GAO staff
who made major contributions to this report are listed in appendix
VIII.
Leslie G. Aronovitz Director, Health Care
This appendix contains examples of lab errors and their
consequences, illustrating the importance of the quality of lab
testing and the effects of lab errors on patient health. The
examples in table 7 are summarized from case studies in the
journal Laboratory Errors and Patient Safety.
82According to a CMS official, the Seattle regional office was in the
forefront of developing a set of performance standards for Washington's
CLIA-exempt program. The standards, though not identical to those
implemented for state survey agencies, have been in place for several
years. As of November 2004, the New York regional office is developing
similar performance standards for the New York CLIA-exempt program.
83The 13 areas are personnel qualifications, financial management,
completion of workload targets, survey selection and scheduling,
outcome-oriented survey process, acceptable plan of correction,
complaints, ongoing training activities, data management, survey time
frames, proficiency testing desk review, principles of documentation, and
enforcement.
84In explaining the purpose of the performance reviews to state survey
agencies, CMS noted that they were designed to serve as an additional
opportunity to further the agency's educational and supportive efforts of
state survey agencies. The goal is to promote optimal performance by
identifying areas needing improvement and corrective action. Survey
agencies are expected to have systems in place for monitoring and
evaluating the efficiency of their corrective actions.
85During desk reviews, state survey agency staff track the proficiency
testing results of labs using data reports from proficiency testing
providers and request that labs initiate corrective actions when the
results are below certain thresholds with some frequency.
Conclusions
Recommendations for Executive Action
Agency and Accrediting Organization Comments and Our Evaluation
Assessment of Lab Quality
86We asked CMS officials to provide the data points for fig. 1 in the
agency's comments on our draft report. The data provided by CMS show that
the percentage of labs with condition-level deficiencies remained
relatively constant over time-fluctuating between 6 and 8 percent from
1996 through 2005. However, the pre-2004 and post-2004 data are not
comparable because CMS revised CLIA survey requirements in January 2004.
Additionally, the data likely understate the actual number of
condition-level deficiencies. As noted earlier, state survey agencies do
not consistently cite all condition-level deficiencies identified during
inspections and CMS has instructed states not to include deficiencies
related to the new 2003 lab quality control requirements in a lab's survey
report.
87Generally, our analysis focused on the period 1998 through 2004,
recognizing that the early years of CLIA implementation were probably
atypical because the law expanded oversight to previously unregulated
physician office labs and many labs shifted from more complex testing that
required routine inspections to less complex testing that did not.
88See 42 C.F.R. S: 493.553(a)(1)(2005).
Educational Focus of CLIA
Sanctioning Labs with Serious, Repeat Deficiencies
89JCAHO expressed a similar concern.
Proficiency Testing Frequency
90See GAO, Nursing Homes: Additional Steps Needed to Strengthen
Enforcement of Federal Quality Standards, GAO/HEHS-99-46 (Washington,
D.C.: Mar. 18, 1999).
Equivalency of Accrediting Organization and Exempt State Programs
Accrediting Organization Validation Reviews
91CMS's comments explained that its regulations require 30-day advanced
notice by accrediting organizations but not by exempt-state programs.
Additional Comments by Accrediting Organizations
92Our analysis of accrediting organization validation reviews covered
fiscal year 1999 through fiscal year 2003. We excluded fiscal year 2004
because CMS had not yet completed its analysis of condition-level
deficiencies missed by accrediting organizations.
93JCAHO miscalculated the increase in complaints from 2004 to 2005.
According to the data provided by JCAHO, the increase was 57 percent, not
64 percent. JCAHO's 2005 complaint data were not available when we
initially collected data on complaints received and substantiated by
survey organizations.
Appendix I: Effects of Lab Errors on Patient Health
Table 7: Effects of Lab Errors on Patient Health
Effects of error on patient
Description of lab error health
Example 1: Delayed reporting of elevated
lab value
o A 59-year-old woman with a history of o There was a delay in the
a rapid and irregular heart beat and diagnosis of a critically
stroke is taking coumadin, a blood elevated blood clotting
thinning agent. Her primary care level.
physician has lab tests completed o The patient took
regularly to ensure that the coumadin coumadin inappropriately
dose is sufficient to maintain a lab for 2 days.
value in the range of 2-3. The results o The patient experienced
of one Friday's test was 5.7, a high significant bleeding in her
value out of her target range, digestive tract related to
indicative that her blood is too thin her impaired blood clotting
and clots too slowly. This lab test status.
value was documented but the patient's o The patient had to be
primary care physician was not notified hospitalized to stabilize
about the elevated value. The following her health condition.
Monday morning a lead technologist o The patient was exposed
noticed that the physician had not been to blood products, putting
called and immediately contacted a nurse her at risk for a
at the physician's office. The nurse, transfusion reaction and
alarmed that both the physician's office exposure to infectious
and the patient had not previously been agents.
notified, tried to contact the patient
at home. The patient had gone to the
emergency room (ER) and was admitted to
the hospital with an even higher blood
clotting value of 7.2. With hospital
treatment, her blood clotting value was
reduced to 2.3 and she was discharged 3
days later.
o The physician ordering the test did
not receive notice of the high blood
clotting lab value for 3 days, even
though the lab result was documented
within an hour of collection. In
addition, the lab failed to follow up on
the situation within an appropriate time
frame.
Example 2: Human recording and data entry
error
o A 60-year-old man with a history of o The patient
chronic liver disease went to the ER inappropriately received
with a 36-hour history of chest pain. several unnecessary
The ER physician ordered a cardiac blood medications.
test that resulted in the patient's o The patient was
being diagnosed with a heart attack. The unnecessarily admitted to
patient was started on multiple the cardiac intensive care
medications and admitted to the cardiac unit, endured the painful
intensive care unit at the hospital. placement of intravenous
After 10 hours in intensive care, a lines, and was attached to
second cardiac blood test was run and several heart monitors.
because of significant discrepancy in o The patient suffered
the results compared to the first test, significant anxiety related
a rapid investigation of the situation to thinking that he had
ensued. The investigation revealed that suffered a heart attack.
the results of the original ER test were o During the 2-day delay
recorded inaccurately. The patient was between hospital admission
retested by cardiac specialty and follow-up appointment,
physicians, reevaluated, and thought to the patient remained
be stable. He was removed from heart symptomatic for his actual
monitors, taken off all cardiac condition-an ulcer.
medications, discharged from the o If the patient's ulcer
hospital, and asked to return for a had been more severe and
routine appointment in 2 days. The had started to bleed
patient continued to have mild chest internally, the
pain during the 2 days after he was inappropriate
discharged. During his return visit, the administration of heart
patient was found to have a stomach attack medications could
ulcer. The patient's chest pain was have had harmful or even
really referred pain caused by the catastrophic effects on the
ulcer. The patient was then treated patient's health.
correctly.
o An investigation of the error
revealed that it was a human recording
and data entry error. The result from a
different patient's blood test had been
entered incorrectly into this patient's
record because the two patients' blood
tests had been run within a few minutes
of each other.
Example 3: Inaccurate review of lab test
results
o A woman who had been using birth o The patient
control was referred to a dermatologist inappropriately received a
for severe acne. The dermatologist drug known to cause birth
wanted to prescribe a drug known to defects.
cause birth defects so she ordered two o This necessitated
pregnancy tests, one initially and the increased monitoring of the
second 2 weeks later. The first test patient's pregnancy.
came back negative. When the o Although the patient's
dermatologist called for the results of pregnancy was ultimately
the second test, a lab worker unaffected, she experienced
incorrectly told her that the pregnancy anxiety throughout. She
test was negative. The patient was given continues to worry that her
a prescription for the acne drug and child is or will be
advised to avoid pregnancy while taking negatively affected in the
this medication. Three days after the future because she took
patient started taking the prescription, this drug early in her
the dermatologist saw in the patient's pregnancy.
record that the second pregnancy test
for this patient was actually positive.
Upon instruction, the patient stopped
taking the medication and her number of
prenatal visits was increased so she
could be monitored for possible birth
defects.
o The investigation revealed that the
laboratory employee, an experienced
technician who was busy when the
incident occurred, reported the result
of the patient's first pregnancy test
when asked for results from the second
pregnancy test. Both the physician and
the technician neglected to follow lab
policies to include the date and time of
the test when orally communicating
results.
Example 4: Inappropriate judgment concerning a microbiology lab test
result
o After returning from a trip to West o The patient experienced
Africa, a 30-year-old woman went to the fever and other significant
ER after experiencing high fever, symptoms during the 4-day
chills, and a headache. The woman was delay in receiving
tested for malaria, a potentially deadly appropriate care.
disease transmitted via mosquito bites. o After the malaria
The test result was negative and the diagnosis was made, the
patient was sent home on ibuprofen. Four patient was admitted to the
days later she returned to the ER hospital and successfully
suffering from continued high fever, treated. There was no
listlessness, and a severe headache. She permanent disability.
was tested again for malaria. This time
the lab test result was positive for a
moderate case of the disease.
o The laboratory director reviewed the
testing from the initial ER visit and
found that the tests had been positive
for malaria. The cause for not
identifying malaria initially was
inappropriate judgment. Since there was
a low pretest probability of a positive
result, technicians assumed that there
would be a negative result.
Sources: All examples were summarized from case studies in the journal
Laboratory Errors and Patient Safety. Example 1: Volume 2, Issue 1(2005):
10; Example 2: Volume 1, Issue 4(2005): 6; Example 3: Volume 1, Issue
3(2004): 6; Example 4: Volume 1, Issue 1(2004): 5.
Appendix II: Labs Surveyed by State Survey Agencies and the Percentage
with Condition-Level Deficiencies, by State in 2004
Number of labs Percentage of labs surveyed with
Statea surveyed reported condition-level deficiencies
Alabama 236 3.4
Alaska 24 12.5
Arizona 122 4.1
Arkansas 197 9.1
California 666 3.9
Colorado 173 10.4
Connecticut 140 0.0
Delaware 18 0.0
District of Columbia 15 13.3
Florida 664 4.1
Georgia 369 4.3
Hawaii 57 5.3
Idaho 89 0.0
Illinois 292 7.5
Indiana 112 9.8
Iowa 141 2.1
Kansas 156 2.6
Kentucky 217 3.2
Louisiana 157 12.1
Maine 32 0.0
Maryland 180 3.3
Massachusetts 217 4.6
Michigan 192 2.6
Minnesota 155 3.2
Mississippi 202 7.9
Missouri 238 2.9
Montana 41 7.3
Nebraska 127 8.7
Nevada 98 4.1
New Hampshire 22 9.1
New Jersey 302 9.9
New Mexico 50 10.0
New Yorkb 545 8.1
North Carolina 321 13.4
North Dakota 30 10.0
Ohio 221 5.9
Oklahoma 158 10.8
Oregon 147 4.1
Pennsylvania 354 1.4
Rhode Island 36 0.0
South Carolina 143 25.2
South Dakota 66 1.5
Tennessee 403 10.7
Texas 651 6.5
Utah 95 6.3
Vermont 33 0.0
Virginia 271 5.2
West Virginia 85 7.1
Wisconsin 226 9.7
Wyoming 23 8.7
Nation 9,509 6.3
Source: GAO analysis of OSCAR data as of May 15, 2006.
aWashington is not included because it operates only a CLIA-exempt
program.
bExcludes labs surveyed under the state's CLIA-exempt program.
Note: Includes surveys conducted from January 12, 2004, through December
31, 2004. We excluded surveys conducted from January 1 to January 11,
2004, because surveyors began using new CLIA inspection requirements on
January 12.
Appendix III: Number of Labs Subject to Surveys by State Survey Agencies
in 2005 and Number of Labs with Sanctions, 1998 to 2004
Number of labs Number of labs with sanctions
Statea (2005) (1998-2004)
Alabama 491 1
Alaska 50 0
Arizona 265 7
Arkansas 398 14
California 1,570 134
Colorado 310 6
Connecticut 246 0
Delaware 46 0
District of Columbia 35 6
Florida 1,268 5
Georgia 737 5
Hawaii 81 4
Idaho 203 0
Illinois 517 15
Indiana 274 1
Iowa 281 1
Kansas 279 0
Kentucky 386 0
Louisiana 396 7
Maine 89 0
Maryland 467 28
Massachusetts 409 0
Michigan 387 62
Minnesota 274 1
Mississippi 441 0
Missouri 394 5
Montana 93 4
Nebraska 254 4
Nevada 159 3
New Hampshire 89 1
New Jersey 533 16
New Mexico 113 3
New York 1,125 75
North Carolina 676 1
North Dakota 74 0
Ohio 428 12
Oklahoma 292 6
Oregon 270 0
Pennsylvania 749 7
Rhode Island 75 1
South Carolina 315 0
South Dakota 116 2
Tennessee 705 0
Texas 1,854 26
Utah 201 13
Vermont 46 0
Virginia 540 17
West Virginia 143 6
Wisconsin 480 1
Wyoming 54 1
Total 19,678 501
Source: GAO analysis of CMS lab registries and CLIA database.
aWashington is not included because it operates only a CLIA-exempt
program.
Appendix IV: Comments from the Centers for Medicare & Medicaid Services
Appendix V: Comments from the College of American Pathologists
Appendix VI: Comments from COLA
Appendix VII: Comments from the Joint Commission on Accreditation of
Healthcare Organizations Appendix
Appendix VIII: GAO Contact and Staff Acknowledgments
GAO Contact
Leslie G. Aronovitz (312) 220-7600 or [email protected]
Acknowledgments
In addition to the contact named above, Walter Ochinko, Assistant
Director; Lucia P. Fort; Dan Lee; Kevin Milne; Dean Mohs; Elizabeth T.
Morrison; Michelle Rosenberg; and Elizabeth Scherer made key contributions
to this report.
(290439)
GAO's Mission
The Government Accountability Office, the audit, evaluation and
investigative arm of Congress, exists to support Congress in meeting its
constitutional responsibilities and to help improve the performance and
accountability of the federal government for the American people. GAO
examines the use of public funds; evaluates federal programs and policies;
and provides analyses, recommendations, and other assistance to help
Congress make informed oversight, policy, and funding decisions. GAO's
commitment to good government is reflected in its core values of
accountability, integrity, and reliability.
Obtaining Copies of GAO Reports and Testimony
The fastest and easiest way to obtain copies of GAO documents at no cost
is through GAO's Web site ( www.gao.gov ). Each weekday, GAO posts newly
released reports, testimony, and correspondence on its Web site. To have
GAO e-mail you a list of newly posted products every afternoon, go to
www.gao.gov and select "Subscribe to Updates."
Order by Mail or Phone
The first copy of each printed report is free. Additional copies are $2
each. A check or money order should be made out to the Superintendent of
Documents. GAO also accepts VISA and Mastercard. Orders for 100 or more
copies mailed to a single address are discounted 25 percent. Orders should
be sent to:
U.S. Government Accountability Office 441 G Street NW, Room LM Washington,
D.C. 20548
To order by Phone: Voice: (202) 512-6000 TDD: (202) 512-2537 Fax: (202)
512-6061
To Report Fraud, Waste, and Abuse in Federal Programs
Contact:
Web site: www.gao.gov/fraudnet/fraudnet.htm E-mail: [email protected]
Automated answering system: (800) 424-5454 or (202) 512-7470
Congressional Relations
Gloria Jarmon, Managing Director, [email protected] (202) 512-4400 U.S.
Government Accountability Office, 441 G Street NW, Room 7125 Washington,
D.C. 20548
Public Affairs
Paul Anderson, Managing Director, [email protected] (202) 512-4800 U.S.
Government Accountability Office, 441 G Street NW, Room 7149 Washington,
D.C. 20548
www.gao.gov/cgi-bin/getrpt? GAO-06-416 .
To view the full product, including the scope
and methodology, click on the link above.
For more information, contact Leslie G. Aronovitz at (312) 220-7600 or
[email protected].
Highlights of GAO-06-416 , a report to congressional requesters
June 2006
CLINICAL LAB QUALITY
CMS and Survey Organization Oversight Should Be Strengthened
The Clinical Laboratory Improvement Amendments of 1988 (CLIA) strengthened
and extended quality requirements for labs that perform tests to diagnose
or treat disease. About 36,000 labs that perform certain complex tests
must be surveyed biennially by either a state or one of six private
accrediting organizations. CMS oversees implementation of CLIA
requirements and the activities of survey organizations. GAO was asked to
examine (1) the quality of lab testing; (2) the effectiveness of surveys,
complaint investigations, and enforcement actions in detecting and
addressing lab problems; and (3) the adequacy of CMS's CLIA oversight.
What GAO Recommends
GAO is making recommendations to the CMS Administrator to improve CLIA
oversight including (1) standardizing the reporting of survey deficiencies
to permit meaningful comparisons across survey organizations; (2) working
with survey organizations to ensure that educating lab workers does not
preclude appropriate regulation, such as identifying and reporting
deficiencies that affect lab testing quality; and (3) allowing the CLIA
program to fully use revenues generated by the program to hire sufficient
staff to fulfill its statutory responsibilities. CMS concurred with 11 of
GAO's 13 recommendations and noted that the report provided insights into
areas where it can improve, augment, and reinforce oversight.
Because of limited comparable data from CMS and survey organizations, too
little is known about the quality of lab testing. For example, a
standardized assessment of lab quality across survey organizations is not
possible because of different definitions of what constitutes a serious
quality problem. One survey organization had no systematic way of
identifying the problematic labs it inspects. However, GAO's analysis of
an indicator that measures a lab's ability to consistently produce
accurate test results suggests that lab quality may not have improved at
hospital labs in recent years.
Based on an analysis of available data and interviews with CMS and survey
organizations, real and potential lab quality problems are masked by
survey, complaint, and enforcement weaknesses. Because most survey
organizations announce the timing of biennial surveys, allowing labs to
prepare for inspections, surveys may not provide a realistic picture of
lab quality. Although two survey organizations that generally inspect
hospital labs plan to begin unannounced surveys in 2006, they may not be
possible at physician office labs that have irregular hours. Survey
organizations that typically inspect such labs, however, provide more
advance notice about upcoming inspections than CMS allows states to
provide. Several other factors suggest that surveys and complaints do not
present a realistic picture of lab quality. Interviews with officials from
a sample of states confirmed that some survey organizations do not cite
all serious deficiencies, as evidenced by variability in the limited
available lab survey data. Officials said that surveyors may be reluctant
to cite deficiencies because they view their role as educational, not
regulatory; moreover, CMS has instructed state surveyors not to cite some
deficiencies for over 2 years after implementing new lab requirements.
Finally, lab workers may file complaints infrequently because of concern
about retaliation and a lack of understanding about how to file a
complaint. CMS rarely imposes sanctions, even for labs with the same
repeat deficiencies, a reflection of the educational focus of the CLIA
program.
CMS does not require labs to participate in a key quality assurance test
as frequently as CLIA requires. Although funded by lab fees, CMS officials
indicated that the program has not been allowed to hire sufficient staff
to carry out the agency's oversight responsibilities. Moreover, CMS's
principal oversight tool, intended to determine if all serious
deficiencies were identified, lacks independence because many oversight
reviews are conducted simultaneously with survey organizations. CMS's
presence may make surveyors more attentive to survey tasks than when they
are not being observed. Compared to independent reviews, simultaneous
reviews rarely identify missed deficiencies. Furthermore, CMS does not
collect and analyze data on serious deficiencies identified by each survey
organization and thus, is unable to assess whether lab quality is
improving or declining. Nor does CMS effectively analyze other key data
such as the use of sanctions. To improve oversight, CMS is establishing a
nationwide complaints database. CMS is also instituting annual survey
organization performance reviews.
*** End of document. ***