Clinical Labs: CMS and Survey Organization Oversight Is Not
Sufficient to Ensure Lab Quality (27-JUN-06, GAO-06-879T).
Today's hearing focuses on oversight of clinical labs. The
Clinical Laboratory Improvement Amendments of 1988 (CLIA)
strengthened quality requirements for labs that perform tests to
diagnose or treat disease. About 36,000 labs that perform certain
complex tests must be surveyed biennially by a state survey
agency, a state CLIA-exempt program, or a private accrediting
organization. CMS oversees implementation of CLIA requirements,
which includes determining the CLIA equivalency of the inspection
requirements used by exempt states and accrediting organizations.
GAO was asked to discuss (1) the quality of lab testing and (2)
the adequacy of CLIA oversight. To examine these issues, GAO
analyzed data on lab performance and reviewed the procedures used
by CMS and survey organizations to implement CLIA and oversee lab
performance. This testimony is based on the GAO report, Clinical
Lab Quality: CMS and Survey Organization Oversight Should Be
Strengthened, GAO-06-416 (June 16, 2006).
-------------------------Indexing Terms-------------------------
REPORTNUM: GAO-06-879T
ACCNO: A56000
TITLE: Clinical Labs: CMS and Survey Organization Oversight Is
Not Sufficient to Ensure Lab Quality
DATE: 06/27/2006
SUBJECT: Data collection
Disease detection or diagnosis
Inspection
Laboratories
Quality assurance
Standards
Surveys
Testing
******************************************************************
** This file contains an ASCII representation of the text of a **
** GAO Product. **
** **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced. Tables are included, but **
** may not resemble those in the printed version. **
** **
** Please see the PDF (Portable Document Format) file, when **
** available, for a complete electronic file of the printed **
** document's contents. **
** **
******************************************************************
GAO-06-879T
* Background
* Survey Organizations
* Surveys and Complaint Investigations
* CMS Oversight
* Insufficient Data Exist to Identify Extent of Serious Lab Qu
* Limited Quality Data for Labs Inspected by State Survey Agen
* Quality of Labs Inspected by Survey Organizations Is Very Di
* CMS Use of Data for Monitoring Lab Quality Is Limited
* CLIA Program Oversight Is Inadequate
* Balance Between Educational and Regulatory Roles by CMS and
* Use of Volunteer Surveyors by CAP Raises Concerns
* Lab Workers Who File Complaints About Quality Problems in La
* Lab Sanctions Are Rarely Imposed
* CMS Is Late in Ensuring CLIA Equivalency of Exempt States' a
* CMS Validation Reviews Skip Some State Survey Agencies and M
* Concluding Observations
* Contact and Acknowledgments
* GAO's Mission
* Obtaining Copies of GAO Reports and Testimony
* Order by Mail or Phone
* To Report Fraud, Waste, and Abuse in Federal Programs
* Congressional Relations
* Public Affairs
Testimony
Before the Subcommittee on Criminal Justice, Drug Policy and Human
Resources, Committee on Government Reform, House of Representatives
United States Government Accountability Office
GAO
For Release on Delivery Expected at 2:00 p.m. EDT
Tuesday, June 27, 2006
CLINICAL LABS
CMS and Survey Organization Oversight Is Not Sufficient to Ensure Lab
Quality
Statement of Leslie G. Aronovitz
Director, Health Care
GAO-06-879T
Mr. Chairman and Members of the Subcommittee:
I am pleased to be here today as you discuss oversight of the quality of
testing performed by the nation's clinical laboratories. Clinical lab
tests are one of the most frequently billed Medicare procedures and,
according to the American Clinical Laboratory Association, affect an
estimated 70 percent of medical decisions.1 Ensuring accurate and reliable
lab test results is critical because erroneous results may lead to
improper treatment, unnecessary mental and physical anguish for patients,
and higher health care costs. Concerns about the quality of lab testing
resulted in enactment of the Clinical Laboratory Improvement Amendments of
1988 (CLIA).2 In recent years, despite CLIA, lab quality problems in
several states have raised questions about the adequacy of lab oversight.
The Centers for Medicare & Medicaid Services (CMS) is responsible for
overseeing compliance with clinical lab testing requirements. As of
December 2005, there were approximately 193,000 labs nationwide ranging
from very small physician office labs that conduct fewer than 2,000 tests
annually, to hospital labs that conduct millions of tests each year. Most
clinical labs regulated under CLIA must obtain a certificate from CMS, but
only about 19 percent-those that conduct moderate- to high-complexity
tests-undergo biennial inspections, which are also referred to as surveys.
During the surveys, inspectors assess lab compliance with mandated
personnel and testing standards. In addition, surveyed labs must
participate in proficiency testing, a program that requires them to test
samples with unknown characteristics that are then graded by an external
party. Labs with serious deficiencies may be sanctioned. Labs may choose
to be surveyed by (1) their state survey agency, under contract with CMS;
(2) their state CLIA-exempt program for labs in New York and Washington;
or (3) one of six private accrediting organizations. State survey agency
inspections use CLIA requirements that are intended to help ensure valid
and reliable lab tests; the two state CLIA-exempt programs and six
accrediting organizations survey labs using their own requirements that
CMS has determined to be at least equivalent to CLIA's. Each survey
organization is also responsible for investigating complaints about lab
quality.3
1Medicare is a federal health care program serving elderly and certain
disabled individuals.
2Pub. L. No. 100-578, 102 Stat. 2903.
My remarks today will focus on (1) the quality of lab testing and (2) the
effectiveness of CMS and survey organization oversight of the CLIA
program. My testimony summarizes the findings of a report we released
today that examines these issues in more detail and includes numerous
recommendations to the CMS Administrator for improving the quality of
laboratory testing through closer oversight of clinical labs and the
administration of CLIA standards.
To determine what is known about the quality of lab testing, we analyzed
data on serious deficiencies identified during surveys by state survey
agencies using CMS's On-Line Survey, Certification, and Reporting system
(OSCAR). We requested comparable data on serious deficiencies from state
CLIA-exempt programs and the three largest accrediting organizations-the
College of American Pathologists (CAP), COLA, and the Joint Commission on
Accreditation of Healthcare Organizations (JCAHO)-which together survey
about 97 percent of accredited labs.4 We also analyzed proficiency testing
data-another indicator of a lab's ability to produce accurate test
results. To evaluate the effectiveness of CLIA program oversight, we
reviewed the processes used to ensure the quality of clinical lab testing
and analyzed available data related to these issues. Based on our review
and discussions with CMS and survey organization officials, we focused on
several key issues: (1) the balance struck between the regulatory and
educational goals of lab surveys, (2) the implications of CAP's use of
volunteer surveyors from neighboring labs to conduct inspections, (3) how
survey organizations facilitate the filing of complaints, (4) the use of
sanctions to encourage compliance, (5) CMS's process for determining that
the standards used by state CLIA-exempt programs and accrediting
organizations are at least equivalent to those of CLIA, and (6) the
results of validation reviews that are intended to assess the adequacy of
inspections by survey organizations. In addition, we interviewed officials
from CMS, three CMS regional offices,5 10 state survey agencies,6 the New
York and Washington CLIA-exempt programs, and the three largest
accrediting organizations. We conducted our work from January 2005 through
May 2006 in accordance with generally accepted government auditing
standards.
3We use the term "survey organizations" when referring collectively to
state survey agencies, the two state CLIA-exempt programs, and accrediting
organizations.
4COLA was formerly known as the Commission on Office Laboratory
Accreditation.
5New York, Philadelphia, and Seattle.
In summary, insufficient data exist to identify the extent of serious
quality problems at labs. When CMS implemented revised CLIA survey
requirements in 2004, it modified historical state survey agency findings
stored in its OSCAR database and, as a result, data prior to 2004 no
longer reflect key survey requirements in effect at the time of those
surveys. The limited data available suggest that state survey agency
inspections do not identify all serious deficiencies. In addition, the
lack of a straightforward method to link similar requirements across
survey organizations makes it virtually impossible to assess lab quality
in a standardized manner, such as identifying the proportion of labs with
condition-level deficiencies, which indicate serious or systemic quality
problems. Furthermore, CMS does not effectively use available data-such as
the proportion of labs with serious deficiencies or proficiency testing
results-to monitor lab quality. Proficiency testing is the one available
data source that can be used to uniformly compare lab quality across
survey organizations. Although CMS noted that proficiency testing trend
data show a decrease in failures for labs as a whole, we found that the
data suggest that quality may not have improved at hospital labs in recent
years. Despite the importance of, and the statutory requirement for,
quarterly proficiency testing, CMS requires proficiency testing for almost
all laboratory tests only three times a year.
Regarding oversight of clinical lab quality, we found that it is
inadequate to ensure that labs are meeting CLIA requirements. Weaknesses
in six areas mask real and potential quality problems at labs. First, the
balance struck between the CLIA program's educational and regulatory goals
is sometimes inappropriately skewed toward education, which may result in
understatement of survey findings. In one instance, CMS instructed state
survey agencies not to cite deficiencies for Pap smear proficiency test
results during the first two years of required testing, to allow labs and
their staff to become familiar with the program. Second, the way one
accrediting organization structures its volunteer survey teams raised
concerns about appropriate levels of training and the appearance of a
conflict of interest. Third, although few labs were the subject of a
complaint each year from 2002 through 2004-significantly less than one
complaint per lab per year-concerns about anonymity and lab workers' lack
of familiarity with how to file a complaint suggest that some quality
problems may not be reported. Fourth, based on the large number of labs
with proposed sanctions from 1998 through 2004 that were never
imposed-even for labs with the same serious, condition-level deficiencies
on consecutive surveys-it is unclear how effective CMS's enforcement
process is at motivating labs to consistently comply with CLIA
requirements. Fifth, CMS is not meeting its requirement to determine in a
timely manner the continued equivalency of accrediting organization and
exempt-state inspection requirements and processes. For example, New
York's and COLA's reviews were about 4 years and 3 years past due,
respectively, as of December 2005. Moreover, CMS allows the implementation
of changes to accrediting organization and exempt-state program inspection
requirements between periodic equivalency determinations before it reviews
the proposed changes. CMS attributed these delays to having insufficient
staff. Finally, validation reviews-one of CMS's most important oversight
tools-do not provide an independent assessment of the extent to which
surveys identify all serious deficiencies because many are performed
simultaneously with such surveys.
6California, Colorado, Connecticut, Idaho, Louisiana, Michigan, Nebraska,
North Carolina, Pennsylvania, and South Carolina.
Accordingly, in the report we released today, we made specific
recommendations to the CMS Administrator to standardize survey findings
across survey organizations in order to make meaningful comparisons;
strengthen survey, complaint, and enforcement processes; and improve
oversight of the CLIA program. In its comments on a draft of our report,
CMS endorsed our overall conclusion that quality assurance for the
nation's clinical labs should be strengthened and said that it would take
action in response to 11 of our 13 recommendations. CMS provided an
alternative assessment of lab quality, and disagreed with our
recommendations concerning the frequency of proficiency testing and the
extent of simultaneous accrediting organization validation reviews. CMS
also expressed concern about identifying and sanctioning labs with repeat
condition level deficiencies. After considering CMS's comments, we believe
that implementing our recommendations is necessary to improve oversight of
labs and accrediting organizations.
Background
A clinical lab is generally defined as a facility that examines specimens
derived from humans for the purpose of disease diagnosis, prevention, and
treatment, or health assessment of individuals. Labs conduct a wide range
of tests that are categorized as waived tests or as moderate- or
high-complexity tests. Approximately 81 percent of all labs (about
157,000) are not subject to routine biennial surveys because they perform
(1) "waived" tests, which are generally simple tests that have an
insignificant risk of erroneous results, such as those approved for home
use7 or (2) tests performed during the course of a patient visit with a
microscope on specimens that are not easily transportable. CLIA
establishes more stringent requirements for the 19 percent (about 36,000)
of labs performing moderate- or high-complexity testing, including the
requirement for a survey and participation in routine proficiency testing.
Surveys examine lab compliance with CLIA program requirements in several
areas including: personnel qualifications, proficiency testing,8 quality
control, quality assurance, and recordkeeping.
Survey Organizations
In general, labs have a choice of who conducts their surveys-state survey
agencies using CLIA inspection requirements or other survey organizations
that use requirements CMS has determined to be at least equivalent to
CLIA's. CMS contracts with state survey agencies in most states to inspect
labs against CLIA requirements.9 CLIA established an approval process to
allow states and private accrediting organizations to use their own
requirements to survey labs. As noted earlier, New York and Washington
operate CLIA-exempt programs and CMS has approved six private, nonprofit
accrediting organizations to survey labs-the American Association of Blood
Banks (AABB), the American Osteopathic Association (AOA), the American
Society of Histocompatibility and Immunogenetics (ASHI), CAP, COLA, and
JCAHO. The requirements of both state CLIA-exempt programs and accrediting
organizations must be reviewed by CMS at least every 6 years to ensure
CLIA equivalency, but may be more stringent than those of CLIA. Figure 1
lists the three types of survey organizations and indicates whether they
survey labs under CLIA requirements, or use their own CLIA-equivalent
requirements. It also shows the percentage of labs performing moderate- to
high-complexity testing surveyed by each type of organization. In general,
state survey agencies, COLA, and Washington's CLIA-exempt program survey
physician office labs, while New York's CLIA-exempt program, CAP, and
JCAHO survey hospital labs.
7Pregnancy and blood sugar screenings are examples of such tests.
8Surveyed labs must participate in an approved external proficiency
testing program, which evaluates the accuracy of laboratory testing. Under
this requirement, a lab purchases samples with unknown characteristics
several times each year from an approved proficiency testing provider. The
lab is required to test the samples with its routine patient testing, and
the results are returned to the testing provider to be graded. A
proficiency testing failure is defined as unsatisfactory performance on
two consecutive or two out of three testing events.
9CMS contracts with state survey agencies in the District of Columbia and
49 states (including New York but not Washington) to survey labs under
CLIA requirements.
Figure 1: Types of Survey Organizations, Requirements Used to Survey Labs,
and Percentage of Labs Surveyed by Each Organization, as of December 2005
aWashington is not included as it has only a CLIA-exempt program.
bNew York uses CLIA-equivalent requirements to inspect larger hospital
labs under the state's CLIA-exempt program and CLIA requirements to
inspect smaller labs, including physician office labs. Only the labs in
the CLIA-exempt program are counted here.
cSome labs are counted more than once because labs may be accredited by
more than one organization. While some labs in New York may be accredited,
they are still subject to biennial surveys by the state survey agency or
the state CLIA-exempt program, because New York does not authorize
accreditation as the basis for lab licensure.
Surveys and Complaint Investigations
Survey organizations (1) conduct surveys and complaint investigations and
(2) monitor proficiency test results submitted by surveyed labs three
times a year. Surveys are typically conducted by former or current lab
workers, who assess lab compliance with CLIA or CLIA-equivalent
requirements. Generally, surveyors verify that lab personnel are
appropriately qualified to conduct testing, evaluate proficiency test
records, check equipment and calibration to ensure that appropriate
quality control measures are in place, and determine whether the lab has a
quality assurance plan and uses it to, among other things, appropriately
identify and resolve problems affecting testing quality. Surveys also
include an educational component to assist labs in understanding how to
comply with CLIA requirements.
Lab survey requirements are classified as either "standard-" or
"condition-" level. Deficiencies are also characterized as standard- or
condition-level based on the requirement in which the deficiency occurs.
Standard-level deficiencies denote problems that generally are not
serious, while condition-level deficiencies are cited when the problems
are serious or systemic in nature. When deficiencies are found during
surveys or complaint investigations, labs are required to submit a plan of
correction, detailing how and when they will address the deficiencies.
Additionally, CMS can impose principal or alternative sanctions, or both.
Principal sanctions include revocation of a CLIA certificate, cancellation
of the right to receive Medicare payments, or limits on testing.
Alternative sanctions, authorized by Congress to give CMS more flexibility
to achieve lab compliance, are less severe and include civil money
penalties or on-site monitoring. For condition-level deficiencies that do
not involve an imminent and serious threat to patient health and a
significant hazard to public health, labs have an opportunity to correct
the deficiencies, which we refer to as a grace period, before the
sanctions are imposed. If a lab is unable to correct a deficiency during
this grace period, CMS determines whether to impose sanctions.
CMS Oversight
CMS, including its 10 regional offices, oversees state and accrediting
organization survey activities. CMS reviews and approves initial and
subsequent applications from exempt-state programs and accrediting
organizations to ensure CLIA equivalency. Validation reviews are one of
CMS's primary oversight tools. Federal surveyors in CMS regional offices
are responsible for conducting validation reviews of state survey agency
and exempt-state program inspections, but state survey agency staff
conduct the validation reviews of accrediting organization inspections.10
An objective of these reviews is to determine if all condition-level
deficiencies were identified. These reviews are conducted within 60 days
of a state's, or 90 days of an accrediting organization's, survey of a
lab.
Insufficient Data Exist to Identify Extent of Serious Lab Quality Problems
The extent of serious quality problems at labs is unclear because CMS has
incomplete data on condition-level deficiencies identified by state survey
agencies prior to 2004. Survey results for 2004 show substantial
variability across states, which suggests that state survey agencies do
not conduct surveys in a consistent manner. We also found that the lack of
a straightforward linkage between CLIA requirements and the
CLIA-equivalent requirements of some survey organizations makes it
virtually impossible to assess lab quality in a standardized manner. CMS
does not effectively use available data, such as the results of surveys
and proficiency testing, to monitor and assess lab quality. Although CMS
noted that proficiency testing trend data show a decrease in failures for
labs as a whole, the data suggest that quality may not have improved at
hospital labs for the period 1999 through 2003.
Limited Quality Data for Labs Inspected by State Survey Agencies Suggest Survey
Inconsistencies
CMS's OSCAR database contains limited data on the quality of labs
inspected by state survey agencies and, as a result, it is not possible to
analyze changes in the quality of lab testing over time. In January 2004,
CMS implemented revised CLIA survey requirements and modified the existing
OSCAR data-state survey agency findings-to reflect the changes. The
revisions affected approximately two-thirds of the CLIA condition-level
requirements.11 As a result of the data modifications, the findings for
surveys conducted prior to 2004 no longer reflect all key condition-level
requirements in effect at the time of those surveys.
10Unlike validation reviews of accrediting organization surveys, CMS
refers to the validation of state surveys as Federal Monitoring Surveys.
Because of their similar objective, we refer to all such surveys as
validation reviews in this testimony. We refer to validation reviews that
occur at the same time as the lab survey as simultaneous. Conversely,
validation reviews that occur after the lab survey are referred to as
independent validations.
11For example, some condition-level requirements were reorganized and some
were consolidated.
Based on the available 2004 OSCAR data (which represent about one half of
all labs surveyed by state survey agencies), we found that 6.3 percent of
labs had condition-level deficiencies. However, variability in the OSCAR
data suggests that labs are not surveyed in a consistent manner. In 2004,
the percentage of labs that were reported to have condition-level
deficiencies varied considerably by state, ranging from none in 6 states
to about 25 percent of labs in South Carolina. Based on interviews with
CMS and 10 state survey agencies, it appears that at least some of this
variability is due to differences in states' approaches to conducting
their surveys as opposed to true differences in lab quality. For example,
CMS told us that, because there is not a prescriptive checklist to guide
the survey process, the reliance on state surveyor judgment results in
variations in the citing of deficiencies. In fact, officials in several
states said that there are circumstances under which condition-level
deficiencies would not be cited, such as if the lab staff were new or if
the lab had a good history of compliance. As a result, available data
likely understate the extent of serious quality problems at labs.
Quality of Labs Inspected by Survey Organizations Is Very Difficult to Measure
in a Standardized Manner
Differences in the inspection requirements used by survey organizations
make it virtually impossible to measure lab quality in a standardized
manner. Because exempt-state programs and accrediting organizations do not
classify inspection requirements and related deficiencies with the same
criteria used by state survey agencies-as either standard- or
condition-level-they cannot easily identify the proportion of surveyed
labs with condition-level deficiencies.12
We asked exempt-state programs and accrediting organizations what
percentage of their requirements, and any deficiencies cited for failure
to meet those requirements, indicated serious problems that were
equivalent to CLIA condition-level deficiencies. CAP and COLA crosswalked
their recent survey findings to CLIA condition-level requirements.
Although their analysis suggested that from about 56 to 68 percent of labs
surveyed during 2004 had a deficiency in at least one condition-level
requirement, they acknowledged that these proportions overstated the
subset of labs with serious problems. JCAHO did not crosswalk its
inspection requirements to those of CLIA because staff would have had to
manually review each survey report to determine which deficiencies were
equivalent to deficiencies in CLIA condition-level requirements.13
12Although CMS reviews the requirements of exempt-state programs and
accrediting organizations to ensure that they are at least equivalent to
CLIA's, there is not necessarily a one-to-one match with CLIA
requirements. Thus, one CLIA condition-level requirement may equal several
accrediting organization requirements or vice versa. For example, CMS's
condition-level requirement for successful lab participation in approved
proficiency testing corresponds to at least 19 CAP, 3 COLA, and 4 JCAHO
requirements.
Despite the difficulty of identifying CLIA equivalent condition-level
deficiencies, two of the three accrediting organizations we reviewed have
systems to identify labs they survey that have serious quality problems.
COLA estimated that about 9 percent of labs it surveyed in 2004 were
subject to closer scrutiny because of the seriousness of the problems
identified. According to JCAHO, about 5 percent of the labs it surveyed in
2004 were not in compliance with a significant number of requirements. The
third accrediting organization, CAP, has criteria for identifying labs
that warrant greater scrutiny, but CAP officials told us that identifying
such labs had to be accomplished on a case by case basis, rather than
through a database inquiry.14
CMS Use of Data for Monitoring Lab Quality Is Limited
CMS does not effectively use available data, such as the results of
surveys and proficiency testing data, to monitor and assess lab quality.
Although CMS tracks the most frequently cited deficiencies at labs in an
effort to improve quality, it does not routinely track the proportion of
labs, by state, in which state survey agencies identify condition-level
deficiencies-those that denote serious or systemic problems. As noted
earlier, variability in survey findings suggests inconsistencies in how
surveys are conducted. CMS also does not require exempt-state programs and
accrediting organizations to routinely submit data on serious deficiencies
identified at the labs they inspect, unless the deficiencies pose
immediate jeopardy to the public or an individual's health.
We also found that CMS does not effectively use proficiency testing data
to assess clinical lab quality. Proficiency testing is an important
indicator of lab quality because it is an objective assessment of a lab's
ability to produce accurate test results and is conducted more frequently
than surveys-three times a year versus once every 2 years. In the absence
of comparable survey data, proficiency testing results provide a uniform
way to assess the quality of lab testing across survey organizations.
Although CMS's analysis of proficiency testing data showed improvements
over time, our analysis of proficiency testing data for 1999 through 2003
suggests that there has been an increase in proficiency testing failures
for labs inspected by CAP and JCAHO, which generally inspect hospital
labs, and a decrease in such failures for labs surveyed by state survey
agencies and COLA, which tend to inspect physician office labs.
13However, JCAHO officials noted that in 2004, about 90 percent of the
labs it surveyed had a deficiency in at least one requirement. JCAHO
classifies all of its requirements as serious.
14As a result, CAP plans to spend in excess of $9 million during 2006 and
2007 to develop an integrated data system that pulls together multiple
factors-survey results, complaints, proficiency testing, findings of other
inspection bodies, and changes in lab directors-to enable it to readily
identify problem labs.
Importantly, CMS's decision to require proficiency testing for almost all
laboratory tests only three times a year is inconsistent with the
statutory requirement. CLIA requires that proficiency testing be conducted
"on a quarterly basis, except where the Secretary determines for technical
and scientific reasons that a particular examination or procedure may be
tested less frequently (but not less often than twice per year)."15 In
CMS's 1992 rule implementing CLIA, the agency provided a rationale for
reducing the frequency of proficiency testing, but did not provide a
technical and scientific basis for reducing the frequency for particular
procedures or tests.16 CMS told us that officials from CMS and the Centers
for Disease Control and Prevention had together determined that the
reduced frequency was based on technical and scientific grounds and
supplied a brief, undated narrative which it attributed to the Centers for
Disease Control and Prevention. However, the narrative focused on the
relative costs and benefits of proficiency testing at various intervals
and did not include an analysis of the technical and scientific
considerations with regard to particular tests that presented a basis for
reducing the frequency.
CLIA Program Oversight Is Inadequate
Oversight by CMS and survey organizations is not adequate to ensure that
labs meet CLIA requirements. For example, the goal of educating lab
workers during surveys takes precedence over the identification and
reporting of deficiencies, while the use of volunteer rather than staff
surveyors by one accrediting organization raises questions about
appropriate levels of training and the appearance of a conflict of
interest. The significant increase in complaints since CAP took steps to
help ensure that lab workers know how to file a complaint suggests that
some quality problems at labs inspected by some survey organizations may
not be reported. In addition, sanctions are not being used effectively as
an enforcement tool to promote labs' compliance with CLIA requirements, as
evidenced by the relatively few labs with repeat condition-level
deficiencies on consecutive surveys from 1998 through 2004 that had
sanctions imposed. Furthermore, CMS is not meeting its responsibility to
determine that accrediting organization and exempt-state program
requirements and processes continue to be at least equivalent to CLIA's.
Finally, ongoing CMS validation reviews do not provide an independent
assessment of the extent to which surveys identify all condition-level
deficiencies-primarily due to their timing.
15Pub. L. No. 100-578, S: 2, 102 Stat. 2903, 2907-08, 42 U.S.C. S:
263a(f)(3)(2000).
16In its rationale, CMS noted that experts were divided on the appropriate
frequency of proficiency testing and further justified the change by
explaining that fewer events of proficiency testing would give
laboratories more time to analyze the causes of test failures, thus
enhancing the value of proficiency testing as an educational tool.
Balance Between Educational and Regulatory Roles by CMS and Survey Organizations
Appears to Be Inappropriate
The goal of educating lab workers sometimes takes precedence over, or
precludes, the identification and reporting of deficiencies that affect
the quality of lab testing.17 For example, surveyors from one state survey
agency told us they do not cite condition-level deficiencies when lab
workers are new but prefer to educate the new staff. As a result, data on
the quality of lab testing and trends in quality over time may be
misleading. CMS also appears to be inappropriately stressing education
over regulation. For instance, in its 2005 implementation of proficiency
testing for lab technicians who interpret Pap smears, a test for cervical
cancer, CMS instructed state surveyors to refrain from citing deficiencies
at labs whose staff fail the tests in 2005 or 2006. According to CMS, this
educational focus allows labs and their staff to become familiar with the
proficiency testing program; however, it is important to note that there
was about a 13-year time lag between the 1992 regulations that implemented
CLIA and the 2005 implementation of Pap smear proficiency testing.18 In
addition, CMS noted that it was concerned about some of the high initial
Pap smear proficiency testing failure rates. An inappropriate balance
between the educational and regulatory roles is also evident in some
accrediting organization practices. For instance, for COLA, the process of
educating labs begins even prior to a survey, when labs are encouraged to
complete a self-assessment to identify COLA requirements with which they
are not in compliance. A CAP surveyor we interviewed with over 30 years of
lab experience estimated that the majority of pathologists-individuals who
generally serve as CAP survey team leaders-view surveys as educational,
rather than as assessments of compliance with lab requirements.
17Although CLIA neither requires nor precludes an educational role for
surveyors, the preamble to CMS's implementing regulation noted that
surveys are intended, in part, to provide an opportunity for on-site
education regarding accepted laboratory procedures.
18Because of lab testing errors that led to women's deaths, Congress
required a specific type of proficiency testing for individuals who
interpret the results of Pap smear tests, which requires examining glass
slides under a microscope. Although CLIA was enacted in 1988, CMS told us
that cost, the inability to find a national testing provider, and other
technical issues delayed establishing a Pap smear proficiency testing
program until 2005.
Use of Volunteer Surveyors by CAP Raises Concerns
The use of volunteer inspectors by CAP raises concerns about appropriate
levels of training and the appearance of a conflict of interest. Although
state survey agencies, exempt-state programs, COLA, and JCAHO employ
dedicated staff surveyors, CAP relies primarily on volunteer teams
consisting of lab workers from other CAP-inspected labs to conduct
surveys.19 In contrast to the mandatory training and continuing education
programs in place for the staff surveyors of other survey organizations,
training for CAP's volunteer surveyors is currently optional.20 According
to data provided by CAP, two-thirds of volunteer surveyors who had
recently participated in a survey had no formal training in the 3 to 5
years preceding the survey. While full-time surveyors employed by other
survey organizations conduct from 30 to about 200 surveys per year, CAP
volunteer surveyors have much less experience conducting surveys because
they only survey about one lab each year. CAP officials told us they plan
to establish a mandatory training program for survey team leaders
beginning in mid-2006.21 However, the required training will take only 1
or 2 days. In contrast, state survey agency inspectors must complete 5
days of basic training, while COLA staff inspectors participate in a
5-week orientation program and an annual 20 hours of continuing education.
CAP's method for staffing survey teams also raises concerns about the
appearance of a conflict of interest. Typically, inspection team leaders
are pathologists who direct other labs in the community, and the
inspection team is comprised of several employees from the team leader's
lab. In the event of differing opinions about survey findings, team
members who are subordinates to the team leader may feel that they have no
other recourse than to follow the team leader's instructions-such as
downgrading the record of an inspection finding to a less serious
category. Recognizing that team members' objectivity may be compromised in
this situation, CAP's revised conflict of interest policy instructs all
parties to be cautious to retain objectivity in fact finding throughout
the inspection process.
19As of November 2005, CAP also employed 11 full-time surveyors.
20Currently, CAP volunteer surveyors are encouraged to participate in
surveyor training at least once every 3 years.
21Mandatory training for survey team members is targeted to begin in 2007.
Lab Workers Who File Complaints About Quality Problems in Lab Testing Not
Afforded Whistle-blower Protections
Some lab workers may not be filing complaints about quality problems at
their labs because of anonymity concerns or because they may not be
familiar with filing procedures. Based on OSCAR data and data obtained
from exempt-state programs and accrediting organizations for 2002 through
2004, few complaints were received about lab testing relative to the
number of labs-significantly less than one complaint per lab per year.22
We found that lab workers may not know how to file a complaint. CAP
experienced a significant increase in the number of complaints it received
since October 2004 when it began requiring CAP-inspected labs to display
posters on how to file complaints. Specifically, from October through
December 2004, CAP received an average of 22 complaints per month,
compared to an average of 11 complaints per month in the 9 months
preceding the poster requirement.23
Because of the difficulty of protecting the anonymity of lab workers who
file complaints, whistle-blower protections for such individuals are
particularly important. Two of the three accrediting organizations we
interviewed have whistle-blower protections-CAP and JCAHO.24 While
officials from New York and Washington's exempt-state programs told us
that whistle-blower laws in their states provide some protection for lab
workers who file complaints, officials in most of the other 10 states we
interviewed told us that they did not have any whistle-blower protections
or were unable to identify specific protections that applied to lab
workers in their state. Although there are no federal whistle-blower
protections specifically for workers in labs covered by CLIA, legislation
was introduced in 2005 to provide such protections.25
22Information about complaints is from OSCAR data and data obtained from
exempt-state programs and accrediting organizations for 2002 through 2004.
The modifications to OSCAR did not affect data on the number of
complaints. The complaint information in OSCAR excludes complaints that do
not require an on-site survey.
23In September 2005, COLA also began requiring labs to display a
complaints poster similar to CAP's. Neither CMS nor JCAHO plan to require
a similar complaints poster. Effective July 2005, JCAHO required labs to
educate staff on how to report concerns about lab quality to the Joint
Commission but does not specify use of a poster to do so.
24COLA does not have a formal whistle-blower policy. COLA officials told
us that they promptly investigate all complaints, many of them from former
lab employees, and keep the identity of the complainants anonymous.
Lab Sanctions Are Rarely Imposed
Few labs were sanctioned by CMS from 1998 through 2004-even those with the
same condition-level deficiencies on consecutive surveys-because many
proposed sanctions are never imposed. Our analysis of CMS enforcement data
from 1998 through 2004 found that while over 9,000 labs had sanctions
proposed during these years, only 501 labs were sanctioned.26 This equates
to less than 3 percent of the approximately 19,700 labs inspected by state
survey agencies. Before sanctions go into effect, labs are given a grace
period to correct condition-level deficiencies, unless the deficiencies
involve an imminent and serious threat to patient health and a significant
hazard to public health. Most labs correct the deficiencies within the
grace period. CMS officials told us that it was appropriate to give labs
an opportunity to correct such deficiencies within a prescribed time frame
and thus avoid sanctions.
However, the number of labs with the same repeat condition-level
deficiencies from one survey to the next also raises questions about the
overall effectiveness of the CLIA enforcement process. From 1998 through
2004, 274 labs surveyed by state survey agencies had the same
condition-level deficiency cited on consecutive surveys and 24 of these
labs had the same condition-level deficiency cited on more than two
surveys.27 This analysis may understate the percentage of labs with repeat
condition-level deficiencies because OSCAR data prior to 2004 no longer
reflect about two-thirds of condition-level requirements and associated
deficiencies at the time of those surveys. We found that only 30 of the
274 labs with repeat condition-level deficiencies had sanctions
imposed-either principal, alternative, or both. With respect to accredited
labs, from 1998 through 2004, less than 1 percent of accredited labs (81)
lost their accreditation; few of these labs were subsequently sanctioned
by CMS and many still participate in the CLIA program. Moreover, CMS did
not sanction 3 labs that COLA concluded had cheated on proficiency testing
by referring the samples to another lab to be tested.28 By statute, the
intentional referral of samples to another lab for proficiency testing is
a serious deficiency that should result in automatic revocation of a lab's
CLIA certificate for at least 1 year.29 Based on our interviews, we found
that the 3 labs were allowed to continue testing because they had
initiated corrective actions; in effect, these labs were given an
opportunity to correct a deficiency that appears to have required a loss
of their CLIA certificate for at least 1 year.
25H.R. 686, 109th Cong. (2005).
26Since CMS data list only the number of labs with proposed sanctions by
year, this number may double-count labs that had proposed sanctions in
multiple years.
27Thirty-three states and the District of Columbia had at least one lab
with the same repeat condition-level deficiency.
CMS Is Late in Ensuring CLIA Equivalency of Exempt States' and Accrediting
Organizations' Inspection Requirements and Processes
We found that CMS has been late in determining that exempt states' and
accrediting organizations' inspection requirements and processes are at
least equivalent to CLIA's. Because CMS has not completed its equivalency
reviews within required time frames, accrediting organizations and exempt
state programs have continued to operate without proper approval.30
Equivalency reviews for CAP, COLA, JCAHO, and Washington due to be
completed between November 1, 1997, and April 30, 2001, were an average of
about 40 months late. In August 1995, CMS determined that New York's next
equivalency review should be completed by June 30, 2001, but was over 4
years past due as of December 2005. Similarly, COLA's equivalency review
was about 3 years past due. Furthermore, although federal regulations
require CMS to review equivalency when an accrediting organization or
exempt-state program adopts new requirements, CMS has not reviewed changes
in the inspection requirements prior to use by these entities.31 As a
result, such survey organizations may introduce changes that are
inconsistent with CLIA requirements. For example, JCAHO made a significant
change to its inspection requirements in January 2004; CMS did not begin
an in-depth review of JCAHO's revised requirements until early 2005-over a
year after they were implemented by JCAHO. According to CMS, its review
has identified several critical areas where JCAHO standards are less
stringent than those of CLIA. JCAHO acknowledged the need to make some
adjustments to its revised requirements.
28A fourth lab was ultimately sanctioned for proficiency testing cheating
by CMS but was allowed to continue testing for almost 2 years after having
its accreditation revoked.
29Pub. L. No. 100-578, S: 2, 102 Stat. at 2911, codified at 42 U.S.C. S:
263a(i)(4)(2000).
30CMS must verify the equivalency of accrediting organizations and
exempt-state programs, and by regulation, CMS requires such survey
organizations to seek reapproval at least once every 6 years, or more
frequently if deemed necessary. CMS establishes the time frames for when
the next reapproval should occur, which have ranged from about 15 months
to about 6 years.
31See 42 C.F.R. S: 493.573(a)(3)(2005).
CMS officials attributed delays in making equivalency determinations and
reviewing interim changes to having too few staff. The CLIA program,
located in CMS's Center for Medicaid and State Operations (CMSO),
currently has approximately 21 full-time-equivalent positions compared to
a peak of 29 such positions several years ago. As required by statute, the
CLIA program is funded by lab fees and since its inception the program's
fees have exceeded expenses. As of September 30, 2005, the CLIA program
had a carryover balance of about $70 million-far more than required to
hire an additional six to seven staff members. However, CMS officials told
us that because the CLIA program staff are part of CMSO, they are subject
to the personnel limits established for CMSO, regardless of whether or not
the program has sufficient funds to hire more staff.
CMS Validation Reviews Skip Some State Survey Agencies and Many Lack
Independence
CMS validation reviews that are intended to evaluate lab surveys conducted
by both states and accrediting organizations do not provide CMS with an
independent assessment of the extent to which surveys identify all
serious-that is, condition-level or condition-level
equivalent-deficiencies. CMS requires its regional offices to conduct
validation reviews of 1 percent of labs inspected by state survey agencies
in a year.32 However, CMS does not specifically require that validations
occur in each state. As a result, from 1999 through 2003, there were 11
states in which no validation reviews were conducted in multiple years.
Without validating at least some surveys in each state, CMS is unable to
determine if the states are appropriately identifying deficiencies.
Many validation reviews occur at the same time a survey organization
conducts its inspection and, in our view, the collaboration among the two
teams during these simultaneous surveys prevents an independent
evaluation. Seventy-five percent of validations of state lab surveys were
conducted simultaneously from fiscal years 1999 through 2003.33 According
to CMS officials, the large proportion of simultaneous validation reviews
provides an opportunity for federal surveyors to share information with
state surveyors, monitor their conformance with CLIA inspection
requirements, and identify training and technical assistance needs.
However, we found that such reviews do not provide an accurate assessment
of state surveyors' ability to identify condition-level deficiencies. Of
the 13 validation reviews that identified missed condition-level
deficiencies, only 1 was a simultaneous review. Regarding validation
reviews of accrediting organization's survey of labs, CMS officials were
unable to tell us how many of the roughly 275 validation reviews conducted
each year from fiscal year 1999 through fiscal year 2003 were
simultaneous.34 However, JCAHO estimated that 33 percent of its validation
reviews were conducted simultaneously. CMS officials told us that the
agency's intent in instituting simultaneous reviews was for state and
accrediting organization surveyors to share best practices, to promote
understanding of each other's programs, and to foster accrediting
organization improvement. In contrast, most of the state survey agency
officials we interviewed told us that simultaneous validation reviews do
not provide a realistic evaluation of the adequacy of accrediting
organizations' inspection processes.
32In contrast, validation reviews of 5 percent of labs inspected by
accrediting organizations during a year are conducted by state survey
agency personnel.
33These validation reviews include both exempt-state and state survey
agency lab surveys.
Concluding Observations
Clinical labs play a pivotal role in the nation's health care system by
diagnosing many diseases, including potentially life-threatening diseases,
so that individuals receive appropriate medical care. Given this important
role, lab tests must be accurate and reliable. Our work demonstrated that
the oversight of clinical labs needs to be strengthened in several areas.
Without standardized survey findings across all survey organizations, CMS
cannot tell whether the quality of lab testing has improved or worsened
over time or whether deficiencies are being appropriately identified.
Using data to analyze activities across survey organizations can be a
powerful tool in improving CMS oversight of the CLIA program, yet CMS has
not taken the lead in ensuring the availability and use of data from
survey organizations to help it monitor their performance. Furthermore,
the agency is not requiring that labs participate in proficiency testing
on a quarterly basis, as required by CLIA. More broadly, CMS and survey
organization oversight of the lab survey process is not adequate to
enforce CLIA requirements. Educating labs to ensure high-quality testing
should complement but not replace the enforcement of CLIA inspection
requirements. Labs with the same serious deficiencies on consecutive
surveys often escape sanctions, even though Congress authorized
alternative sanctions to give CMS more flexibility to achieve lab
compliance. Without the threat of real consequences, labs may not be
sufficiently motivated to comply with CLIA inspection requirements. By
allowing validation reviews to occur simultaneously with surveys and
permitting some states to go without validation reviews over a period of
several years, CMS is not making full use of this oversight tool.
Moreover, independent validation reviews of accrediting organization
surveys are critical because CMS has not conducted equivalency reviews
within the time frames it established. The recommendations we have made
would help CMS to consistently identify and address lab quality problems.
34CMS did not begin tracking this information until August 2003.
Mr. Chairman, this concludes my prepared remarks. I would be happy to
answer any questions that you or other Members of the Subcommittee may
have.
Contact and Acknowledgments
For further information regarding this statement, please contact Leslie G.
Aronovitz at (312) 220-7600 or [email protected]. Contact points for our
Offices of Congressional Relations and Public Affairs may be found in the
last page of this statement. Walter Ochinko, Assistant Director; Jenny
Grover; Kevin Milne; and Michelle Rosenberg contributed to this statement.
(290548)
This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed in
its entirety without further permission from GAO. However, because this
work may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this material
separately.
GAO's Mission
The Government Accountability Office, the audit, evaluation and
investigative arm of Congress, exists to support Congress in meeting its
constitutional responsibilities and to help improve the performance and
accountability of the federal government for the American people. GAO
examines the use of public funds; evaluates federal programs and policies;
and provides analyses, recommendations, and other assistance to help
Congress make informed oversight, policy, and funding decisions. GAO's
commitment to good government is reflected in its core values of
accountability, integrity, and reliability.
Obtaining Copies of GAO Reports and Testimony
The fastest and easiest way to obtain copies of GAO documents at no cost
is through GAO's Web site ( www.gao.gov ). Each weekday, GAO posts newly
released reports, testimony, and correspondence on its Web site. To have
GAO e-mail you a list of newly posted products every afternoon, go to
www.gao.gov and select "Subscribe to Updates."
Order by Mail or Phone
The first copy of each printed report is free. Additional copies are $2
each. A check or money order should be made out to the Superintendent of
Documents. GAO also accepts VISA and Mastercard. Orders for 100 or more
copies mailed to a single address are discounted 25 percent. Orders should
be sent to:
U.S. Government Accountability Office 441 G Street NW, Room LM Washington,
D.C. 20548
To order by Phone: Voice: (202) 512-6000 TDD: (202) 512-2537 Fax: (202)
512-6061
To Report Fraud, Waste, and Abuse in Federal Programs
Contact:
Web site: www.gao.gov/fraudnet/fraudnet.htm E-mail: [email protected]
Automated answering system: (800) 424-5454 or (202) 512-7470
Congressional Relations
Gloria Jarmon, Managing Director, [email protected] (202) 512-4400 U.S.
Government Accountability Office, 441 G Street NW, Room 7125 Washington,
D.C. 20548
Public Affairs
Paul Anderson, Managing Director, [email protected] (202) 512-4800 U.S.
Government Accountability Office, 441 G Street NW, Room 7149 Washington,
D.C. 20548
www.gao.gov/cgi-bin/getrpt? GAO-06-879T .
To view the full product, including the scope
and methodology, click on the link above.
For more information, contact Leslie G. Aronovitz at (312) 220-7600 or
[email protected].
Highlights of GAO-06-879T , a testimony before the Subcommittee on
Criminal Justice, Drug Policy and Human Resources, Committee on Government
Reform, House of Representatives
June 27, 2006
CLINICAL LABS
CMS and Survey Organization Oversight Is Not Sufficient To Ensure Lab
Quality
Today's hearing focuses on oversight of clinical labs. The Clinical
Laboratory Improvement Amendments of 1988 (CLIA) strengthened quality
requirements for labs that perform tests to diagnose or treat disease.
About 36,000 labs that perform certain complex tests must be surveyed
biennially by a state survey agency, a state CLIA-exempt program, or a
private accrediting organization. CMS oversees implementation of CLIA
requirements, which includes determining the CLIA equivalency of the
inspection requirements used by exempt states and accrediting
organizations. GAO was asked to discuss (1) the quality of lab testing and
(2) the adequacy of CLIA oversight. To examine these issues, GAO analyzed
data on lab performance and reviewed the procedures used by CMS and survey
organizations to implement CLIA and oversee lab performance. This
testimony is based on the GAO report, Clinical Lab Quality: CMS and Survey
Organization Oversight Should Be Strengthened, GAO-06-416 (June 16, 2006).
What GAO Recommends
In a report released today, GAO made numerous recommendations to the CMS
Administrator that would strengthen program oversight. CMS noted that the
report provided insights into areas where it can improve oversight and
said that it would implement 11 of GAO's 13 recommendations.
In summary, insufficient data exist to identify the extent of serious
quality problems at labs. When CMS implemented revised CLIA survey
requirements in 2004, it modified historical state survey agency findings
and, as a result, data prior to 2004 no longer reflect key survey
requirements in effect at the time of those surveys. The limited data
available suggest that state survey agency inspections do not identify all
serious deficiencies. In addition, the lack of a straightforward method to
link similar requirements across survey organizations makes it virtually
impossible to assess lab quality in a standardized manner. Furthermore,
CMS does not effectively use available data, such as the proportion of
labs with serious deficiencies or proficiency testing results, to monitor
lab quality. Proficiency testing is an objective measurement of a lab's
ability to consistently produce accurate test results. GAO's analysis of
proficiency testing data suggests that lab quality may not have improved
at hospital labs in recent years.
Oversight of clinical lab quality is not adequate to ensure that labs are
meeting CLIA requirements. Weaknesses in five areas mask real and
potential quality problems at labs. First, the balance struck between the
CLIA program's educational and regulatory goals is sometimes
inappropriately skewed toward education, which may result in
understatement of survey findings. For example, even though the initial
test failure rates were high, CMS instructed state survey agencies not to
cite deficiencies during the first two years of required Pap smear
proficiency testing, to allow labs and their staff to become familiar with
the program. Second, the manner in which one accrediting organization
structures its survey teams raised concerns about appropriate levels of
training and the appearance of a conflict of interest that could undermine
the integrity of the survey process. Third, concerns about anonymity and
lab workers' lack of familiarity with how to file a complaint suggests
that some quality problems are not being reported. Fourth, based on the
large number of labs with proposed sanctions from 1998 through 2004 that
were never imposed-even for labs with the same serious deficiencies on
consecutive surveys-it is unclear how effective CMS's enforcement process
is at motivating labs to consistently comply with CLIA requirements.
Finally, CMS is not meeting its requirement to determine in a timely
manner the continued equivalency of accrediting organization and
exempt-state program inspection requirements and processes, nor has the
agency reviewed changes to accrediting organization and exempt-state
program inspection requirements before implementation.
*** End of document. ***