Medicare: CMS Needs Additional Authority to Adequately Oversee	 
Patient Safety in Hospitals (20-JUL-04, GAO-04-850).		 
                                                                 
Hospitals accredited by the Joint Commission on Accreditation of 
Healthcare Organizations (JCAHO) are considered in compliance	 
with Medicare participation requirements. GAO examined the extent
to which JCAHO's pre-2004 hospital accreditation process	 
identified hospitals not complying with Medicare requirements,	 
the potential of JCAHO's new process for improving the detection 
of deficiencies in Medicare requirements, and the effectiveness  
of CMS's oversight of JCAHO's hospital accreditation program. GAO
analyzed CMS data on hospitals state surveyors found to have	 
deficiencies in Medicare requirements that JCAHO surveyors did	 
not detect, analyzed CMS's measure of JCAHO's ability to detect  
noncompliance with Medicare requirements, and interviewed JCAHO  
officials.							 
-------------------------Indexing Terms------------------------- 
REPORTNUM:   GAO-04-850 					        
    ACCNO:   A11027						        
  TITLE:     Medicare: CMS Needs Additional Authority to Adequately   
Oversee Patient Safety in Hospitals				 
     DATE:   07/20/2004 
  SUBJECT:   Health care programs				 
	     Health surveys					 
	     Hospitals						 
	     Institution accreditation				 
	     Program evaluation 				 
	     Safety regulation					 
	     Safety standards					 
	     Noncompliance					 
	     Health insurance					 
	     Health policy					 
	     Policies and procedures				 
	     Standards (health care)				 
	     Medicare Program					 

******************************************************************
** This file contains an ASCII representation of the text of a  **
** GAO Product.                                                 **
**                                                              **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced.  Tables are included, but    **
** may not resemble those in the printed version.               **
**                                                              **
** Please see the PDF (Portable Document Format) file, when     **
** available, for a complete electronic file of the printed     **
** document's contents.                                         **
**                                                              **
******************************************************************
GAO-04-850

United States Government Accountability Office

GAO

                       Report to Congressional Requesters

July 2004

MEDICARE

CMS Needs Additional Authority to Adequately Oversee Patient Safety in Hospitals

GAO-04-850

Highlights of GAO-04-850, a report to congressional requesters

Hospitals accredited by the Joint Commission on Accreditation of
Healthcare Organizations (JCAHO) are considered in compliance with
Medicare participation requirements. GAO examined the extent to which
JCAHO's pre-2004 hospital accreditation process identified hospitals not
complying with Medicare requirements, the potential of JCAHO's new process
for improving the detection of deficiencies in Medicare requirements, and
the effectiveness of CMS's oversight of JCAHO's hospital accreditation
program. GAO analyzed CMS data on hospitals state surveyors found to have
deficiencies in Medicare requirements that JCAHO surveyors did not detect,
analyzed CMS's measure of JCAHO's ability to detect noncompliance with
Medicare requirements, and interviewed JCAHO officials.

GAO believes that Congress should consider giving CMS the authority over
JCAHO's hospital accreditation program that it has over other
accreditation programs and recommends that CMS modify its methods for
assessing JCAHO's performance. CMS agreed with GAO's recommendations.
JCAHO stated that GAO's methodology was incomplete and did not
comprehensively assess its overall performance. GAO emphasized that its
engagement was limited to one aspect of deficiency detection and was not
intended to reflect JCAHO's overall performance.

www.gao.gov/cgi-bin/getrpt?GAO-04-850.

To view the full product, including the scope and methodology, click on
the link above. For more information, contact Janet Heinrich at (202)
512-7119.

July 2004

MEDICARE

CMS Needs Additional Authority to Adequately Oversee Patient Safety in Hospitals

JCAHO's pre-2004 hospital accreditation process did not identify most of
the hospitals found by state survey agencies in CMS's annual validation
survey sample to have deficiencies in Medicare requirements. In comparing
the results of the two surveys, CMS considered whether it was reasonable
to conclude that the deficiencies found by state survey agencies existed
at the time JCAHO surveyed the hospital. In a sample of 500
JCAHO-accredited hospitals, state agency validation surveys conducted in
fiscal years 2000 through 2002 identified 31 percent (157 hospitals) with
deficiencies in Medicare requirements. Of these 157 hospitals, JCAHO did
not identify 78 percent (123 hospitals) as having deficiencies in Medicare
requirements. For the same validation survey sample, JCAHO also did not
identify the majorityabout 69 percentof deficiencies in
Medicare requirements found by state agencies. Importantly, the number of
deficiencies found by validation surveys represents 2 percent of the
11,000 Medicare requirements surveyed by state agencies in the sample
during this time period. At the same time, a single deficiency in a
Medicare requirement can limit the hospital's capability to provide
adequate care and ensure patient safety and health. Inadequacies in
nursing practices or deficiencies in a hospital's physical environment,
which includes fire safety, are examples of deficiencies in Medicare
requirements that could endanger multiple patients.

The potential of JCAHO's new hospital accreditation process to improve the
detection of deficiencies in Medicare requirements is unknown because the
process was just implemented in January 2004. JCAHO plans to move from
using announced to unannounced surveys in 2006, which would afford JCAHO
the opportunity to observe hospitals' operations when the hospitals have
not prepared in advance to be surveyed. In addition, the pilot test of the
new accreditation process was of limited value in predicting whether it
will be an improvement over the pre-2004 process in detecting
deficiencies. Limitations in the pilot test included that hospitals were
not randomly selected to participate; that observers from JCAHO
accompanied each surveyor, thus possibly affecting surveyors' actions; and
that JCAHO evaluated the results instead of an independent entity.

CMS has limited oversight authority over JCAHO's hospital accreditation
program because the program's unique legal status effectively prevents CMS
from taking actions that it has the authority to take with other health
care accreditation programs to ensure satisfactory performance. For
example, requiring JCAHO's hospital accreditation program to submit to a
direct review process or placing the program on probation while monitoring
its performance. Further, CMS relies on a measure to evaluate how well
JCAHO's hospital accreditation program detects deficiencies in Medicare
requirements that provides limited information and can mask problems with
program performance, uses statistical methods that are insufficient to
assess JCAHO's performance, and has reduced the number of validation
surveys it conducts.

Contents

  Letter 1

Results in Brief 4
Background 6
JCAHO's Pre-2004 Hospital Accreditation Process Often Did Not

Detect Serious Deficiencies Found by State Survey Agencies 10
Potential of JCAHO's New Hospital Accreditation Process Is
Unknown, and Testing Was Limited 16
CMS Oversight Authority of JCAHO's Hospital Accreditation

Program Is Limited and Needs Improvement 20
Conclusions 28
Matter for Congressional Consideration 28
Recommendations for Executive Action 29
Agency and Other External Comments and Our Evaluation 29

Appendix I Scope and Methodology

Appendix II Medicare Conditions of Participation

Appendix III Features of JCAHO's New Accreditation Process

Appendix IV 	Comments from the Centers for Medicare & Medicaid Services

Appendix V 	Comments from the Joint Commission on Accreditation of
Healthcare Organizations

Appendix VI GAO Contact and Staff Acknowledgments 50

GAO Contact 50
Acknowledgments 50

Related GAO Products

51

Tables

Table 1: Hospitals in CMS's Validation Survey Sample with Serious
Deficiencies that State Survey Agencies Identified but
JCAHO Surveyors Did Not, Fiscal Years 2000-2002 11

Table 2: Percentage of Serious Deficiencies Identified by State
Survey Agencies but Not by JCAHO Surveyors in CMS's
Validation Survey Sample, Fiscal Years 2000-2002 12

Table 3: Number of Serious Deficiencies, by COP, Identified by
State Survey Agencies but Not by JCAHO Surveyors in
CMS's Validation Survey Sample, Fiscal Years 2000-2002 14

Table 4: Accreditation Decisions for Hospitals Surveyed Under
JCAHO's New Survey Process Pilot Test as Compared to
Results from JCAHO's Pre-2004 Survey Process 19

Table 5: Hypothetical Examples of the Effect on the Rate of
Disparity of a Decrease in the Number of Hospitals with
Serious Deficiencies in a Sample of 200 Hospitals 25

Table 6: Number of Hospitals Targeted for Validation Surveys
Compared with Usable Traditional Validation Surveys
Completed 26

Table 7: Medicare Conditions of Participation 36
Table 8: JCAHO's Description of Features of Its New Hospital
Accreditation Process 38

Abbreviations

AOA American Osteopathic Association
CMS Centers for Medicare & Medicaid Services
COP condition of participation
HHS Department of Health and Human Services
JCAHO Joint Commission on Accreditation of Healthcare

Organizations OIG Office of Inspector General PFP priority focus process
PPR periodic performance review

This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed in
its entirety without further permission from GAO. However, because this
work may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this material
separately.

United States Government Accountability Office Washington, DC 20548

July 20, 2004

The Honorable Charles E. Grassley
Chairman
Committee on Finance
United States Senate

The Honorable Pete Stark
Ranking Minority Member
Subcommittee on Health
Committee on Ways and Means
House of Representatives

In fiscal year 2002, nearly 7.4 million Medicare beneficiaries received
inpatient health care at hospitals that participated in Medicare. Federal
law establishes criteria for hospitals for purposes of Medicare. The
Centers for Medicare and Medicaid Services (CMS), the agency
responsible for administering Medicare, has established quality and
patient
safety requirements called conditions of participation (COP) that
hospitals
must meet in order to be eligible for Medicare payment. Hospitals that are
accredited by the Joint Commission on Accreditation of Healthcare
Organizations (JCAHO) are generally deemed under federal law to be
compliant with Medicare requirements for patient safety and health and
become eligible for payments from Medicare.1,2 No other health care
accreditation program has this same statutory authority.

JCAHO is a private, not-for-profit organization that accredits most of the
hospitals that participate in Medicare. JCAHO sets standards that
accredited hospitals must meet and reports that these standards are more
comprehensive than the Medicare COPs.3 In January 2004, JCAHO
implemented a new hospital accreditation process with goals that included
further enhancing health care quality and safety.

1See 42 U.S.C. S: 1395bb(a) (2000).

2JCAHO is referred to in statute under its former name, the Joint
Commission on Accreditation of Hospitals.

3JCAHO develops its standards with a committee of experts and
stakeholders, such as the government, hospitals, and consumers.

CMS oversight of JCAHO's hospital accreditation program is limited because
it cannot restrict or remove JCAHO's accreditation authority if the agency
detects problems. To oversee the program, CMS conducts on-site validation
surveys of a sample of JCAHO-accredited hospitals and reports annually to
Congress on the results of these surveys. The validation surveys, which
are performed by agencies that CMS has agreements with in each state, help
CMS determine whether Medicare quality and safety requirements are being
met. CMS compares the results of these state surveys against survey
results obtained through JCAHO's hospital accreditation program. CMS uses
a measure called the rate of disparity that summarizes the extent to which
an accreditation program has failed to cite deficiencies identified by
state agency validation surveys. We are using the term serious deficiency
in this report to indicate a deficiency in one or more Medicare COPs.
Examples of serious deficiencies include a hospital's inability to provide
adequate nursing services or failure to implement and enforce infection
control policies. According to CMS, serious deficiencies substantially
limit a hospital's capability to render adequate care and adversely affect
the safety and health of patients.

Questions have been raised by the Department of Health and Human Services'
(HHS) Office of Inspector General (OIG) and others regarding whether
accreditation by JCAHO ensures that hospitals provide adequate care.
Specifically, experts have questioned how well JCAHO's hospital
accreditation process identifies deficiencies in hospitals that could
jeopardize patient safety and health. A comprehensive study by the HHS OIG
found that JCAHO's surveys were not likely to identify patterns of
deficient care.4

You asked that we examine the effectiveness of JCAHO's hospital
accreditation process in ensuring that hospitals comply with Medicare COPs
to ensure the safety and health of Medicare beneficiaries. Specifically,
we (1) examined the extent to which JCAHO's pre-2004 hospital
accreditation process identified deficiencies in Medicare COPs that were
identified by state survey agencies, (2) determined whether JCAHO's new
hospital accreditation process has potential for improving the detection
of deficiencies in Medicare COPs and whether the process was adequately
tested, and (3) examined the effectiveness of CMS's oversight of JCAHO's
hospital accreditation program.

4HHS OIG, The External Review of Hospital Quality: A Call for Greater
Accountability, OEI-01-97-00050 (Washington, D.C.: July 1999).

To determine the extent to which JCAHO's pre-2004 hospital accreditation
process identified deficiencies in Medicare COPs that were identified by
state survey agencies, we used data from a CMS comparison of state
validation survey findings with findings of JCAHO's hospital accreditation
surveys, which indicated whether JCAHO found deficiencies in its
standards. Of the four possible outcomes to this comparison of survey
findings-(1) JCAHO and state agencies both identify no deficiencies, (2)
JCAHO identifies deficiencies not found by state agencies, (3) JCAHO and
state agencies both identify the same deficiencies, and (4) state agencies
identify deficiencies that JCAHO does not-we focused on the fourth because
it highlights the need for CMS oversight of the hospital accreditation
program. For the second outcome, there could be two reasons for the
disparity between JCAHO's and state survey agencies' findings: hospitals
corrected deficiencies identified by JCAHO prior to the state agency
survey or the state survey agency did not identify a deficiency that
existed. In addition, not all JCAHO findings are equivalent to
noncompliance with a Medicare COP. To determine whether JCAHO's findings
on deficiencies in its standards were comparable to the state agencies'
findings, CMS staff compared the two surveys and considered whether it was
reasonable to conclude that the deficiencies found by state survey
agencies existed at the time JCAHO surveyed the hospital. For deficiencies
that CMS determined that JCAHO failed to identify, CMS met with JCAHO to
address disputed findings and consider additional evidence on
comparability offered by JCAHO. CMS provided results for a sample of 500
JCAHO-accredited hospitals from fiscal years 2000 through 2002. We
determined that the data CMS provided on serious deficiencies were
adequate for addressing the issues in this report. On the basis of this
sample of 500 JCAHO-accredited hospitals, we determined, using CMS's data,
both the percentage of serious deficiencies and the percentage of
hospitals with serious deficiencies identified by the state survey
agencies where JCAHO surveyors did not find comparable deficiencies. The
analysis we performed on the results of the validation surveys was limited
to the hospitals included in the validation survey sample and cannot be
generalized to all JCAHO-accredited hospitals.

To determine whether JCAHO's new hospital accreditation process has
potential for improving the detection of serious deficiencies, we
identified changes in the accreditation process and analyzed significant
new features. To determine whether JCAHO's new hospital accreditation
process was adequately tested, we reviewed the testing procedures and
results that JCAHO used to determine the effectiveness of its new survey
process in identifying quality and safety deficiencies. Because the new
accreditation process was implemented recently, we did not have

Results in Brief

information to compare JCAHO survey performance in detecting serious
deficiencies with state agency survey performance.

To determine the adequacy of CMS's oversight of JCAHO's hospital
accreditation program, we reviewed relevant statutory and regulatory
provisions regarding oversight of health care accreditation programs and
how CMS had implemented this authority in order to provide oversight. To
supplement our review, we conducted interviews with officials from CMS,
state survey agencies, and JCAHO; representatives from other organizations
active in health care accreditation and the hospital industry; and experts
in quality of care. We conducted our work from June 2003 through July 2004
in accordance with generally accepted government auditing standards. (For
a complete description of our scope and methodology, see app. I.)

JCAHO's pre-2004 hospital accreditation process did not identify most of
the hospitals found by state survey agencies in CMS's annual validation
survey sample to have serious deficiencies in Medicare COPs. In a sample
of 500 JCAHO-accredited hospitals, state agency validation surveys
conducted in fiscal years 2000 through 2002 identified 31 percent (157
hospitals) with serious deficiencies; of these, JCAHO did not identify 78
percent (123 hospitals) as having serious deficiencies. For the same
validation survey sample, JCAHO also did not identify the
majorityabout 69 percentof serious deficiencies found by
state agencies. Importantly, the number of deficiencies found by
validation surveys represents 2 percent of the 11,000 Medicare COPs
surveyed by state agencies in the sample during this time period. At the
same time, a single serious deficiency can limit a hospital's capability
to provide adequate care and ensure patient safety and health.
Inadequacies in nursing practices or deficiencies in a hospital's physical
environment, which includes fire safety, are examples of serious
deficiencies that could endanger multiple patients.

The potential of JCAHO's new hospital accreditation process to improve the
detection of serious deficiencies over the pre-2004 process is unknown
because the process was just implemented in January 2004. JCAHO plans to
move from announced to unannounced surveys in 2006, which would afford
JCAHO the opportunity to observe hospitals' operations when the hospitals
have not prepared in advance to be surveyed. In addition, the pilot test
of the new accreditation process was of limited value in predicting
whether it will be an improvement over the pre-2004 process in detecting
deficiencies. Limitations in the pilot test included that hospitals

participating in the pilot were not randomly selected and that JCAHO
evaluated the results instead of an independent entity.

CMS has limited oversight authority over JCAHO's hospital accreditation
program because the program's unique legal status effectively prevents CMS
from taking actions, such as requiring JCAHO's hospital accreditation
program to submit to a direct review process or placing the program on
probation while monitoring its performance, that it has the authority to
take with other health care accreditation programs to ensure satisfactory
performance. Furthermore, CMS's existing oversight of JCAHO's hospital
accreditation program needs improvement. Although CMS officials said that
validation surveys are conducted to assure Congress that JCAHO's
accreditation process provides a reasonable assurance that hospitals
comply with Medicare requirements, there are limitations to the agency's
validation survey program. CMS has no formal written protocol for
selecting the hospitals to include in the state agency validation survey
sample; relies on a measure-the rate of disparity-that provides limited
information and could mask problems with an accreditation program's
performance in detecting serious deficiencies; uses statistical methods
that are insufficient to accurately portray JCAHO's performance; and has
reduced the percentage of validation surveys from 5 percent to
approximately 1 percent of JCAHO-accredited hospitals, which provides less
reliable information on the performance of JCAHO's hospital accreditation
program.

We suggest that Congress consider giving CMS the same oversight authority
over JCAHO's hospital accreditation program that CMS has for all other
health care accreditation programs. To improve CMS's assessment of JCAHO's
hospital accreditation process, we recommend that CMS modify the measure
it uses to indicate how well an accreditation program detects serious
deficiencies in Medicare COPs; maximize the extent to which validation
survey findings can be generalized to all JCAHO-accredited hospitals and
include its survey protocol in its annual reports to Congress; and
annually conduct validation surveys on a sample of JCAHO-accredited
hospitals that is equal to at least 5 percent of all JCAHO-accredited
hospitals.

CMS and JCAHO commented on a draft of this report. In its comments, CMS
concurred with our findings and recommendations. JCAHO stated that it did
not object to our matter for congressional consideration that CMS be given
the same oversight authority over JCAHO's hospital accreditation program
that it has over other health care accreditation programs. JCAHO took
issue with our methodology, which it said was

incomplete and did not comprehensively assess the performance of JCAHO's
hospital accreditation program. Our review was not intended to be a
comprehensive evaluation of JCAHO's hospital accreditation program.
Rather, we focused on the ability of JCAHO's hospital accreditation
program to ensure that hospitals that accept Medicare patients comply with
Medicare COPs. In the same vein, JCAHO stated that the report does not
sufficiently recognize JCAHO's identification of deficiencies in its
surveys that may be corrected before state surveyors arrive. We added
language to the report to emphasize that our focus was on the serious
deficiencies state survey agencies found that JCAHO did not because these
serious deficiencies demonstrate the importance of CMS oversight of the
hospital accreditation process. JCAHO also stated that we misrepresented
the potential of its new accreditation process to detect deficiencies in
Medicare COPs and provided new data for the first quarter of 2004 that
indicate that 2004 JCAHO surveys may have detected a greater percentage of
deficiencies related to patient care compared with the pre-2004
accreditation process. However, we maintain that until CMS validation
surveys for 2004 are completed, there is no basis on which to determine
whether the new process improves the detection of noncompliance with
Medicare COPs. CMS and JCAHO also provided technical comments on the
report, which we incorporated as appropriate.

To participate in Medicare, hospitals must maintain standards of patient
safety and health that comply with Medicare COPs. For example, the COP
related to nursing services includes such requirements for hospitals as
providing a 24-hour nursing service that is supervised or furnished by a
registered nurse. There are currently 23 Medicare COPs.5 (See app. II for
a description of the 23 Medicare COPs.) CMS proposed revisions to all of
the COPs in 1997, but it did not finalize them. Since then, CMS has
revised several of the COPs, including those concerning the life safety
code; quality assessment and performance improvement; organ, tissue, and
eye donations; and nurse anesthetist supervision.

Health care accreditation programs other than JCAHO's hospital
accreditation program may generally adopt their own requirements if CMS
determines that an accreditation program's requirements are at least

5One of the 23 COPs cannot be deemed by an accreditation organization. CMS
relies on organizations other than the accreditation organizations to
certify that hospitals comply with the COP that requires hospitals to
establish a utilization review plan for services provided to Medicare
beneficiaries.

Background

equivalent to Medicare COPs.6 If CMS also determines, among other things,
that the accreditation program's survey process is likely to identify any
serious deficiencies in COPs, it must generally grant "deeming authority"
to the accreditation program and treat entities accredited by these
organizations as meeting Medicare COPs. CMS has the authority to review
these programs, and it can impose a probationary period while monitoring
performance and remove deeming authority if warranted.

JCAHO

Most hospitals demonstrate compliance with standards equivalent to
Medicare COPs through accreditation by JCAHO.7 In 2002, JCAHO accredited
4,211, or 82 percent, of Medicare-participating hospitals.8 Hospitals
accredited by JCAHO received payments for Medicare-covered inpatient
services of approximately $98 billion, or 90 percent, of the $109 billion
that was spent on hospital care in 2002. JCAHO, as part of its
accreditation-related activities, also develops survey procedures, trains
its surveyors, and formulates performance measures. JCAHO is governed by a
29-member board of commissioners and has a staff of over 1,000.9

JCAHO's deeming authority for hospitals is established in statute and
therefore can only be changed by Congress. As a result of this unique
statutory authority, hospitals accredited by JCAHObecause they
meet

6Specifically, the agency's regulations require the accreditation
organization's standards to be at least as stringent as the Medicare COPs,
when taken as a whole. See 42 C.F.R. S: 488.6(a) (2003).

7Forty-nine states allow JCAHO hospital accreditation as a full or partial
substitute for meeting health care quality standards and other
requirements for state licensure.

8The remaining 18 percent of hospitals choose to be accredited by the
American Osteopathic Association (AOA) or to be certified by state survey
and certification agencies.

9The board includes seven members chosen by the American Hospital
Association, seven chosen by the American Medical Association, three
chosen by the American College of Physicians-American Society of Internal
Medicine, three chosen by the American College of Surgeons, and one chosen
by the American Dental Association. In addition, the board consists of a
nurse-at-large and six public members. The president of JCAHO is an ex
officio member of the board.

JCAHO standardsare deemed to meet Medicare COPs as well.10 In
contrast, the American Osteopathic Association (AOA)-a private,
not-forprofit professional organization that offers accreditation services
for hospitals and other health care organizations-holds deeming authority
that is subject to CMS's direct review and approval.11 While hospital
accreditation is its largest program, JCAHO also has accreditation
authority under Medicare for certain other health care providers,
including clinical laboratories, hospices, ambulatory surgical centers,
and home health care agencies. All of these other JCAHO accreditation
programs are subject to CMS's direct review and approval.

To be accredited by JCAHO, a hospital must meet eligibility requirements,
satisfactorily complete a triennial on-site survey process, and continue
to maintain JCAHO's standards between surveys. The accreditation surveys
that JCAHO conducts every 3 years are particularly important. For most
hospitals, the triennial survey is the only time that JCAHO conducts an
onsite review of the hospital's compliance with all quality standards and
issues decisions on how well the hospital has complied with JCAHO's
standards. In 2004, JCAHO implemented a new hospital accreditation survey
process, which, according to JCAHO, is intended to reduce the cost of
accreditation to health care organizations and JCAHO, enhance public
confidence that health care organizations are in continuous compliance
with standards, increase the real and perceived value of accredited
organizations, meet the requirements of deeming authorities and
purchasers, and improve satisfaction for hospitals participating in the
accreditation program.

10When Congress first established JCAHO's deeming authority in 1965, it
prohibited federal authorities from issuing standards on patient health
and safety for hospitals higher than comparable requirements for hospital
accreditation by JCAHO in deference to the expertise of professional
accreditation organizations sponsored by medical and hospital
associations. See Pub. L. No. 89-97, S: 102(a), 79 Stat. 286, 315 (1965).
Subsequent legislation removed the prohibition and required JCAHO to
demonstrate that its standards were at least equivalent to any such higher
standards issued by the Secretary in order to have deeming authority in
that area. See Pub. L. No. 92-603, S: 244(c), 86 Stat. 1329, 1423 (1972).

11AOA solely accredits approximately 2 percent of hospitals and JCAHO and
AOA jointly accredit less than 1 percent of hospitals. While JCAHO and AOA
are currently the only hospital accrediting organizations, federal law
permits CMS to approve any other national accreditation body that
demonstrates that Medicare requirements will be met by hospitals it
accredits.

CMS Oversight of JCAHO

CMS exercises oversight of JCAHO's hospital accreditation program
primarily through its validation surveys and annual reports to Congress.
Under federal law, CMS must continually study the operation and
administration of Medicare, including validating the JCAHO hospital
accreditation process, and submit annual reports to Congress.12 CMS has
agreements with state agencies to conduct validation surveys. There are
different kinds of validation surveys, including traditional validation
surveyssurveys conducted on a sample of hospitals within 60 days
of their triennial JCAHO survey. 13 Traditional validation surveys provide
the basis for assessing the effectiveness of JCAHO's hospital
accreditation process in detecting deficiencies in Medicare COPs, which
JCAHOaccredited hospitals are treated as meeting. Validation surveys also
include 18-month surveys, which monitor how well JCAHO-accredited
hospitals are complying with Medicare COPs midway between their 3-year
JCAHO surveys, and allegation surveys, which are triggered by complaints
or other reports of situations that pose potential threats to patient
health and safety in JCAHO-accredited hospitals. CMS has the authority to
remove the deemed status of a JCAHO-accredited hospital where a state
agency's validation survey results in a finding that the hospital is out
of compliance with one or more Medicare COP.

CMS uses a rate of disparity measure to summarize the extent to which an
accreditation program, such as JCAHO's hospital accreditation program, has
not found serious deficiencies identified by CMS through state agency
validation surveys. For a hospital accreditation program, using the
results from validation surveys, the rate of disparity for hospitals
surveyed by the state survey agencies is calculated as the difference
between the number of hospitals found with serious deficiencies by state
agencies and the number of hospitals found with comparable deficiencies by
the accreditation program, divided by the number of hospitals sampled. CMS
regulations provide that if the validation survey results for an
accreditation organization with deeming authority indicate a rate of
disparity that reaches the threshold level of 20 percent disparity or
greater, CMS will notify the organization that its deeming authority may
be in jeopardy and that the agency is initiating a deeming authority
review.14 With respect to JCAHO, CMS includes the rate of disparity in its
annual

12See 42 U.S.C. S: 1395ll(b).
13For this report, we will refer to traditional validation surveys as
validation surveys.
1442 C.F.R. 488.8(e).

reports to Congress in which it reports the results of its validation
program for JCAHO's hospital accreditation program.

JCAHO's Pre-2004 Hospital Accreditation Process Often Did Not Detect
Serious Deficiencies Found by State Survey Agencies

JCAHO's pre-2004 hospital accreditation process often did not identify
either hospitals with serious deficiencies or the individual serious
deficiencies found by state survey agencies through CMS's validation
program. In a sample of 500 JCAHO-accredited hospitals, state agency
validation surveys conducted in fiscal years 2000 through 2002 identified
31 percent (157 hospitals) with serious deficiencies; of these, JCAHO did
not identify 78 percent (123 hospitals) as having serious deficiencies.
For the same validation survey sample, the majority of the serious
deficiencies state survey agencies identified but JCAHO did not were in
the physical environment COP category, which covers fire safety and
prevention.

JCAHO Did Not Identify From fiscal years 2000 through 2002, JCAHO did not
identify 123 of the 157 Three-Quarters of the hospitals (78 percent) with
serious deficiencies that CMS's validation Hospitals That State program
identified out of a sample of 500 JCAHO-accredited hospitals.

Table 1 shows the hospitals with serious deficiencies that state
surveySurvey Agencies Found to agencies identified and JCAHO did not
during fiscal years 2000 throughHave Serious Deficiencies 2002. In 343 of
the 500 hospital validation surveys, state agency surveyors

did not find serious deficiencies. Both state agency surveyors and JCAHO

surveyors identified 34 hospitals as having a serious deficiency.

Table 1: Hospitals in CMS's Validation Survey Sample with Serious
Deficiencies that State Survey Agencies Identified but JCAHO Surveyors Did
Not, Fiscal Years 2000-2002

  Hospitals state survey agencies found to have serious deficiencies Hospitals
with serious deficiencies identified by state survey agencies but not identified
                                   by JCAHOa

Fiscal year

                                      Number of hospitals in CMS's validation

sample Number Percent Number Percent

                     2000      184           61          33           49      
                     2001      204           61          30           49      
                     2002      112           35          31           25      
                    Total      500          157          31               123 

Source: GAO analysis of CMS data.

Note: Hospitals with serious deficiencies are defined as those not meeting
one or more of the Medicare COPs. From fiscal year 2000 through 2002,
JCAHO surveyed 4,666 hospitals for accreditation.

aDetermined by CMS through its matching of deficient COPs found by state
agency surveyors to JCAHO surveyors' findings of JCAHO standards out of
compliance.

According to JCAHO, disparity between state agency and JCAHO findings in
the 123 hospitals in part may be attributed to the timing of the two
surveys, JCAHO's phasing in of new requirements, different interpretations
of the COPs by state surveyors, and inherent surveyor bias. However, in
its comparison to determine disparity between the two surveys, CMS does
consider whether it is reasonable to conclude that the deficiencies found
by state survey agencies existed at the time JCAHO surveyed the hospital.

JCAHO Did Not Detect Two-Thirds of the Serious Deficiencies Identified by
State Survey Agencies

From fiscal year 2000 through 2002, JCAHO did not detect 167 of the 241
serious deficiencies (69 percent) identified through CMS's validation
program from a sample of 500 JCAHO-accredited hospitals. The number of
serious deficiencies found by CMS's validation program represents 2
percent of the 11,000 Medicare COPs surveyed by state agencies in the
sample and were found in 157 hospitals. However, one serious deficiency in
any one of these hospitals could limit its ability to provide adequate
care to its patients. For example, a serious deficiency in the nursing
services COP at a hospital in Texas found by a state agency but missed by
JCAHO in 2000 included such problems as failure to prepare and administer
drugs in accordance with federal and state laws, inadequate supervision
and

evaluation of the clinical activities of nonemployee nursing personnel,
and nursing care and procedures provided to patients that were not within
the scope of accepted standards of practice. Among hospitals with serious
deficiencies identified by CMS's validation program but not by JCAHO,
there were on average 1.1 serious deficiencies per hospital, with a range
from 1 to 6. Table 2 shows the percentage of serious deficiencies
identified by CMS's validation program but not by JCAHO for fiscal years
2000 through 2002.

Table 2: Percentage of Serious Deficiencies Identified by State Survey
Agencies but Not by JCAHO Surveyors in CMS's Validation Survey Sample,
Fiscal Years 20002002

Serious deficiencies identified by state survey agencies but not by JCAHOa

Fiscal year

           Number of serious deficiencies identified by state survey agencies

Number of serious deficiencies identified by JCAHO Number Percent

                        2000        82              12                     70 
                        2001       103              40                     63 
                        2002        56              22                     34 
                       Total       241              74                    167 

Source: GAO analysis of CMS data.

Note: Hospitals with serious deficiencies are defined as those not meeting
one or more of the Medicare COPs.

aDetermined by CMS through its matching of deficient COPs found by state
agency surveyors to JCAHO surveyors' findings of JCAHO standards out of
compliance.

Of the 167 serious deficiencies identified by CMS's validation program
from fiscal year 2000 through 2002 but not detected by JCAHO, 87 were
related to a hospital's physical environment, which includes life safety
code standards on fire prevention and safety.15 For these 3 years, JCAHO
did not detect 81 percent of the serious physical environment deficiencies
identified by state agency surveyors. Table 3 shows the number of serious
deficiencies, by category, identified by state survey agencies in CMS's

15Between fiscal years 2000 and 2002, JCAHO used more recent life safety
code standards than state survey agencies performing validation surveys.
CMS stated that these differences could account for some of the disparate
findings between JCAHO's surveys and state agency validation surveys.
However, CMS considered these different standards in determining whether
JCAHO had not detected serious deficiencies in the life safety code.

validation program but missed by JCAHO surveyors. The larger number of
deficiencies in physical environment may be related to the difference in
how state agencies generally survey separately a hospital's compliance
with the life safety code portion of the physical environment COP. JCAHO
surveys assess compliance with the life safety code using a combination of
the hospital's self-assessment, a hospital building tour, and observations
made by all surveyors during the survey process. Examples of deficiencies
in physical environment that JCAHO did not identify but CMS's validation
program found in a hospital in Alabama in 2000 included the following:
several exterior exits lacked emergency exit lighting; several exterior
exits were illuminated only by single light bulbs; fire alarm system and
fire extinguishers had not been inspected annually as required; and an
automatic sprinkler system had not been inspected annually and maintained
by certified personnel as required. Serious deficiencies in the COP on
physical environment compromise patient safety and health.

Table 3: Number of Serious Deficiencies, by COP, Identified by State
Survey Agencies but Not by JCAHO Surveyors in CMS's Validation Survey
Sample, Fiscal Years 2000-2002

                                                            Number of serious
                                    Number of serious deficiencies identified
                                      deficiencies identified by state survey
                                          by state survey agencies but not by
                                                          COP agencies JCAHOa

Physical environment 107

                                Quality of care

Anesthesia services 3

Discharge planning 2

Emergency services 2

Food and dietetic services 5

Governing body 16

Infection control 15

Laboratory services 1

Medical record services 7

Medical staff 10

Nursing services 17

Organ, tissue, and eye procurement 5

Outpatient services 1

Patients' rights 10

Pharmaceutical services 14

Quality assurance 18

               Radiologic services                 1                    0 
             Rehabilitation services               1                    1 
            Respiratory care services              1                    1 
                Surgical services                  5                    4 
            Total quality-of-care COPs            134                  80 

Source: GAO analysis of CMS data.

Note: Neither state survey agencies nor JCAHO identified serious
deficiencies in two of the categoriescompliance with laws and
nuclear medicine serviceswhich are not included in this table.

aDetermined by CMS through its matching of deficient COPs found by state
agency surveyors to JCAHO surveyors' findings of JCAHO standards out of
compliance.

The total number of deficiencies not identified by JCAHO in the
quality-ofcare COP categories-those COPs that involve the oversight and
delivery of patient care-is similar to the number not identified by JCAHO
in the

physical environment COP. While the number of serious deficiencies not
found by JCAHO in individual quality-of-care COP categories is smaller
than the number not found in physical environment, when these
quality-ofcare COPs are combined, the proportion of serious deficiencies
JCAHO missed is almost 60 percent of the total number of serious
deficiencies identified by state survey agencies. The following are
examples of hospitals found to be out of compliance with multiple
quality-of-care COPs:

o  	In 2000, CMS removed the deemed status as a Medicare provider of a
JCAHO-accredited hospital in California for failure to comply with two
COPs, one of which was infection control. The hospital failed to provide a
sanitary environment to avoid sources and transmission of infections and
communicable diseases and failed to develop a system for ensuring the
sterilization of medical instruments.

o  	Also in 2000, CMS notified a hospital in Texas that if it did not
implement a plan of correction the hospital's participation in the
Medicare program would be terminated. Serious deficiencies at this
hospital included lack of compliance with the pharmaceutical services and
nursing services COPs because medications were administered without
physician orders and a double dose of narcotics was given in the emergency
room, with no explanation for the excessive dosage, to a patient who later
died.

State surveyors in CMS's validation program also may miss serious
deficiencies. In related work on skilled nursing facilities and home
health agencies, we found that the number of serious deficiencies found by
state agencies was highly variable among states and may be understated.16
State agencies' detection of serious deficiencies in hospitals also varied
widely among states for the 3 years we reviewed. For example, state survey
agencies in California, Illinois, and Ohio found serious deficiencies in
over 45 percent of the surveys they conducted between fiscal years 2000
through 2002. In contrast, Florida and New York found serious deficiencies
in less than 10 percent of the surveys they conducted, and Louisiana did
not find serious deficiencies in any of the surveys it conducted.17

16U.S. General Accounting Office, Medicare Home Health Agencies:
Weaknesses in Federal and State Oversight Mask Potential Quality Issues,
GAO-02-382 (Washington, D.C.: July 19, 2002) and U.S. General Accounting
Office, Nursing Home Quality: Prevalence of Serious Problems, While
Declining, Reinforces Importance of Enhanced Oversight, GAO-03-561
(Washington, D.C.: July 15, 2003).

17All six states conducted at least 15 validation surveys from fiscal year
2000 through 2002.

Potential of JCAHO's New Hospital Accreditation Process Is Unknown, and
Testing Was Limited

The potential of JCAHO's new hospital accreditation process to improve the
identification of serious deficiencies is unknown because it is too soon
after its January 2004 implementation for a meaningful evaluation; in
addition, JCAHO's testing of the new process was limited. CMS has not had
the opportunity to complete its validation program for 2004 to determine
whether JCAHO surveyors using the new process are missing serious
deficiencies later identified by state agency validation surveys. While
unannounced surveys, which are planned for implementation in 2006, have
the potential to improve the detection of serious deficiencies, other
features of the new process that JCAHO did not test before implementation
may have limitations that could affect the potential of the new process to
identify problems with patient care. JCAHO's pilot test of the new process
had limitations, including using a sample of hospitals that volunteered
for the pilot instead of using a random sample and selfevaluating the
results instead of using an independent entity.

Potential of New Process Is Unknown

Periodic Performance Review

Because JCAHO's new accreditation process was implemented in January 2004,
it is too soon to know whether the new process is better at detecting
serious deficiencies in Medicare COPs than the pre-2004 accreditation
process. A JCAHO official told us the new process will aid in the
detection of deficiencies, but we found that some of the features may have
shortcomings that could limit their effectiveness. New features of the
accreditation process include the hospital's self-assessment of compliance
with accreditation standards midway through the accreditation cycle,
surveyor review of the care provided to specific patients to determine the
adequacy of the hospital's health care delivery system, and performance of
all accreditation surveys on an unannounced basis beginning in 2006. (See
app. III for a description of selected new features of JCAHO's new
hospital accreditation process.)

Periodic performance reviews assess hospital compliance with applicable
standards and are performed at the 18-month midpoint between 3-year onsite
accreditation surveys. According to JCAHO, the periodic performance review
will have several benefits. These include providing hospitals with a
process to assess their ongoing compliance and requiring them to correct
or plan to correct all deficiencies identified. Periodic performance
reviews must be conducted either by the hospital as a self-assessment or,
if the hospital chooses, by JCAHO through an on-site review.

However, periodic performance reviews may not necessarily improve the
detection of deficiencies. JCAHO did not pilot test these reviews for the
potential to detect deficiencies and did not test whether hospitals that

             Priority Focus Process and Patient Tracer Methodology

conducted reviews do a better job of continuing to comply with standards.
In addition, for hospitals performing self-assessments, JCAHO will not
check these self-assessments to determine whether hospitals fully and
accurately identified quality problems and developed adequate corrective
action plans to address the problems identified.

According to JCAHO, the priority focus process and patient tracer
methodology together have the potential to enhance the ability of surveys
to detect deficiencies by directing the attention of surveyors to key
patient care areas. The priority focus process uses a data-based formula
to identify a limited number of areas in each hospital that are
particularly important to patient health and safety. Priority focus areas
might include infection control, medication management, or patient safety.
Surveyors use the priority focus process combined with the patient tracer
methodology to focus their surveys to specific areas for review. The
patient tracer methodology guides their choice of current patients to
"trace" through the experience of care within an organization. For
example, if the hospital's priority focus process data suggest that a
patient with an orthopedic-related diagnosis such as a hip fracture should
be traced, the JCAHO surveyor would review the patient's medical record,
noting where the patient had entered into the hospital and any services
and transfers that occurred. Then the surveyor would retrace the steps in
the patient's care process by observing and talking to staff in some of
the areas in which the patient received care. If the patient entered
through the emergency department, was transferred to a medical/surgical
unit, and then went to the operating room, the surveyor would go to these
areas to interview staff about the care given to this specific patient.
With information from patient tracers, the surveyor will assess whether
any compliance issues exist with JCAHO standards. If the surveyor
identifies a compliance issue while tracing one patient, the surveyor may
review the records of similar patients to determine whether the problem is
isolated or represents a pattern of care.

However, JCAHO did not test the extent to which the priority focus process
and the patient tracer methodology could help surveyors detect
deficiencies. A JCAHO official told us these new features of the
accreditation process were intended to help surveyors trace patients in a
consistent way and not necessarily to improve the detection of
deficiencies.

Unannounced Surveys

JCAHO plans to conduct all hospital accreditation surveys on an
unannounced basis beginning in 2006.18 JCAHO stated that unannounced
surveys will ensure that hospital performance is based on the observation
of hospitals' routine operations rather than on how they operate after
they have the opportunity to prepare to be surveyed. A JCAHO official also
indicated that unannounced surveys will be more likely to detect
deficiencies. The OIG and other organizations share JCAHO's position on
the value of unannounced surveys of hospitals and other health care
organizations. The value of unannounced surveys also has been recognized
for nursing homes, which state agencies survey on an unannounced basis.

JCAHO's Pilot Test of New Process Was Limited

JCAHO's pilot test of its new hospital accreditation process was limited
and therefore unable to help determine the potential of the new process to
detect deficiencies in Medicare COPs. According to JCAHO, the pilot test
suggests that the new process was more likely than the former process to
find quality problems. However, the pilot test sample included hospitals
that volunteered or were selected by JCAHO and were not randomly selected,
pilot test surveyors were accompanied by observers from JCAHO's central
office, and pilot test results were not independently evaluated. In
addition, CMS has not completed its fiscal year 2004 validation program,
which will include hospitals surveyed by JCAHO using the new process and
thus does not yet have sufficient data on which to base a meaningful
evaluation.

According to JCAHO's analysis of the pilot test, the new hospital
accreditation process is more likely to identify quality problems since
proportionately more hospitals under the new process received unfavorable
accreditation decisions. JCAHO based its conclusion on a comparison of
survey outcomes, called accreditation decisions, between 18 hospitals in
the pilot test conducted in 2002 and 2003 and the 1,524 hospitals that had
been surveyed under the pre-2004 accreditation process during 2003. Table
4 presents the data JCAHO used to make the comparison. As shown,
proportionately fewer hospitals under the new process were accredited
without having to make corrections. Although JCAHO provided the
accreditation decision outcomes for these 18 pilot

18In 2004 and 2005, JCAHO will continue to conduct its accreditation
surveys on an announced basis.

tests, it stated it preferred to use the number of "requirements for
improvement" as the basis for analysis.

Table 4: Accreditation Decisions for Hospitals Surveyed Under JCAHO's New
Survey Process Pilot Test as Compared to Results from JCAHO's Pre-2004
Survey Process

Pilot test of new survey process Pre-2004 survey process

                              Number of Percentage of Number of Percentage of 
                Accreditation hospitals   hospitals       hospitals hospitals 
                     decision  surveyed   surveyed          surveyed surveyed 
                Accreditation         0             0           320           
              Survey findings                         
            with requirements                         
             for improvementa        13            72          1,191          
                  Conditional                         
                accreditation         3            17           13            
           Preliminary denial                         
             of accreditation         2            11            0            
                        Total        18           100              1,524b 100 

Source: JCAHO.

Note: JCAHO reported that it conducted pilot tests of the new
accreditation process in an additional 12 hospitals in 2001. However,
JCAHO was unable to provide the accreditation decisions for these 12 pilot
site hospitals.

aHospitals in the pilot test with deficiencies were accredited contingent
upon evidence of correcting deficiencies. The hospitals in the comparison
group with deficiencies received accreditation with requirements for
improvement.

bThese 1,524 hospitals represent all those surveyed for accreditation by
JCAHO during 2003.

However, JCAHO's pilot test analysis was limited in three respects, which
may have accounted for the smaller number of favorable accreditation
decisions hospitals received under the new process.

o  	The hospitals participating in the pilot test were not randomly
selected by JCAHO. As a result, these hospitals may not be representative
of all JCAHO-accredited hospitals and therefore results cannot be
generalized.

o  	During the pilot test, an observer from JCAHO's central office
accompanied each surveyor, and the knowledge that they were being observed
may have influenced the surveyors' actions.19 Under the pre-2004

process, observers only rarely accompanied JCAHO surveyors.

o  JCAHO conducted its own evaluation of pilot test results. Evaluation of

CMS Oversight Authority of JCAHO's Hospital Accreditation Program Is
Limited and Needs Improvement

the pilot test by an entity independent of either JCAHO or the hospitals
tested could help to ensure that survey outcomes were impartially
interpreted. For example, CMS used an independent group to evaluate its
redesign of the nursing home survey process.

CMS has limited oversight authority over JCAHO's hospital accreditation
program, and its existing oversight activities need improvement. The
unique status of JCAHO's hospital accreditation program, which is
specified in statute, does not permit CMS to take corrective action, such
as restricting or removing its deeming authority. Additionally, CMS uses a
measure that provides limited information to evaluate the performance of
JCAHO's hospital accreditation program, has significantly reduced the
number of surveys conducted as part of CMS's validation program, and does
not use measures that are based on sound statistical methods to assess the
performance of JCAHO's hospital accreditation program.

CMS Oversight Authority of JCAHO Is Limited

Because of JCAHO's unique legal status, CMS's oversight of JCAHO's
hospital accreditation program is limited in two major ways: Unlike other
accreditation programs with deeming authority, JCAHO does not have to
reapply to CMS to reauthorize its deeming authority, and CMS cannot take
action to address performance problems with JCAHO's hospital accreditation
program.

JCAHO's hospital accreditation program is the only Medicare accreditation
program for which CMS does not have to conduct an evaluation of the
accreditation standards and the processes used to conduct surveys. Without
this evaluation, CMS is deprived of key oversight tools it is authorized
to use with other accreditation programs: detailed information

19For example, we found in our nursing home survey work in 1999 that state
surveyors may perform their tasks more attentively when they are being
observed by federal surveyors than they would if performing their surveys
unobserved, thus masking a state surveyor's typical performance. U.S.
General Accounting Office, Nursing Home Care: Enhanced HCFA Oversight of
State Programs Would Better Ensure Quality, GAO/HEHS-00-6 (Washington,
D.C.: Nov. 4, 1999).

about any proposed changes to the accreditation process and public input.
CMS cannot require JCAHO to provide information about proposed changes to
its accreditation requirements and hospital survey processes. Also,
because it is not required to reapply to CMS for deeming authority, JCAHO
does not have to provide CMS information that other accreditation programs
must provide, such as a detailed description of its survey processes, a
comparison of its standards to Medicare requirements, and the
qualifications of its surveyors, which CMS reviews to ensure that the
programs comply with Medicare requirements. For example, when JCAHO's
hospice accreditation program applied for deeming status in 1999, CMS
required changes to JCAHO's hospice accreditation process, including
requiring JCAHO to make unannounced surveys of Medicarecertified hospices.
According to a CMS official, JCAHO's hospital accreditation program has
provided much of the information required of other accreditation
organizations; however, CMS has no authority to require JCAHO to make
changes to the hospital accreditation program as it does with other health
care accreditation programs. Statutory provisions regarding public notice
and comment do not apply to JCAHO's hospital accreditation program as they
do to other accreditation programs. The reapplication process for other
accreditation programs requires affording the public an opportunity to
provide input to CMS on an accreditation program's request for deeming
authority. Because JCAHO does not have to reapply for deeming authority,
the public does not have the opportunity to review and comment on JCAHO's
hospital accreditation program.20

A second limitation is CMS's inability to address any performance issues
with JCAHO's hospital accreditation program. Although the rate of
disparity for JCAHO's hospital accreditation program exceeded 20 percent
in fiscal years 2000, 2001, and 2002 a rate that would have
triggered a deeming authority review for any other Medicare accreditation
programCMS was unable to take enforcement action to address
JCAHO's performance. When other Medicare accreditation programs have a
rate of disparity of 20 percent or more, CMS can take steps such as
imposing a year-long probationary period and removing deeming authority at
the end of the probationary period if the rate of disparity remains at 20
percent or more. For JCAHO, however, CMS's actions toward correcting the
program's deficiencies are limited to including recommendations for

20Whenever CMS considers, approves or removes an accreditation
organization's deeming authority, the agency is required to publish
detailed notices in the Federal Register, and consider public comment. See
42 U.S.C. S: 1395bb(b)(3); 42 C.F.R. S: 488.8(b) and (f)(7).

improvement in its annual reports to Congress and negotiating with JCAHO
to voluntarily adopt CMS's recommendations.

In its annual report to Congress, CMS made recommendations in fiscal year
2002 aimed at improving JCAHO's ability to detect serious deficiencies in
the life safety code, part of the COP on physical environment. CMS noted
that JCAHO permits hospitals to self-assess compliance with life safety
code requirements.21 While CMS stated that it did not object to the
concept of hospital self-assessment of life safety code requirements, it
made five recommendations to JCAHO for improving implementation:

1. 	Require hospitals to use qualified personnel, such as fire marshals
and architects, to conduct self-assessments of compliance with the life
safety code requirements.

2. 	Set minimum standards for identifying and improving life safety code
deficiencies identified by hospital self-assessments.

3. 	Require hospitals to submit their self-assessments on life safety code
issues prior to JCAHO conducting accreditation surveys to provide
surveyors and personnel in JCAHO's central office time to review the
material prior to the accreditation surveys.

4. 	Increase the use of JCAHO experts in the life safety code requirements
in its central office.

5. 	Address the issue of hospitals that do not make improvement within
self-determined time frames.

JCAHO did not adopt all of these recommendations. It disagreed with the
first recommendation. Its response indicated that its requirement to use
qualified personnel to complete the self-assessment, while more general,
was sufficient. It further indicated that policies were in place for CMS's
second and fifth recommendations. CMS later agreed that JCAHO's policies
do satisfactorily address the fifth recommendation. JCAHO planned to
examine ways to adopt CMS's third and fourth recommendations. CMS however,
had no authority to compel JCAHO to

21Beginning in 1995, JCAHO-accredited hospitals have assessed their own
compliance with the life safety code and developed correction plans, which
JCAHO must approve. If hospitals are in compliance with their correction
plans, JCAHO's surveyors do not record outstanding life safety code
deficiencies.

comply with the remaining recommendations. According to CMS, it continues
to discuss implementation of its recommendations with JCAHO. JCAHO stated
that while its initial response to CMS's recommendations in 2003 reflected
then current JCAHO policies, subsequent policy evolutions are addressing
CMS's recommendations. Specifically, JCAHO is working with the American
Society of Hospital Engineers to develop a process for review by experts
of hospital self-assessments on life safety code issues prior to JCAHO's
conducting on-site accreditation surveys and to identify those hospitals
for which engineering expertise should be added to on-site surveys.

CMS's Validation Program Needs Improvement

Rate of Disparity

CMS states that the goal of its validation program is to provide
reasonable assurance to Congress that the JCAHO accreditation process
ensures hospital compliance with Medicare COPs. However, the measure CMS
uses to evaluate the performance of JCAHO's hospital accreditation program
provides limited information and could mask problems with an accreditation
program's performance in detecting serious deficiencies, and it is based
on a target sample size of 1 percent of JCAHO-accredited hospitals. In
addition, CMS does not report the extent to which its sample reflects the
performance of the larger population of JCAHO-accredited hospitals.

The rate of disparity between JCAHO's hospital accreditation survey
findings and state survey agency findings, as currently calculated by CMS,
does not fully explain the performance of JCAHO's hospital accreditation
program in detecting serious deficiencies. CMS uses this measure in its
reports to Congress to assess JCAHO's hospital accreditation program and
as the basis for making recommendations for improvement. CMS calculates
the rate of disparity as the difference between the number of hospitals
found with serious deficiencies by state survey agencies and the number of
hospitals found with serious deficiencies by the accreditation survey,
divided by the number of hospitals in the sample. For example, if state
survey agencies conducted 200 surveys as part of CMS's validation program
and found 60 hospitals out of compliance with at least one COP, but
JCAHO's survey found that only 22 of the hospitals were out of compliance,
the rate of disparity would be 19 percent ((60 -22)/200).

CMS has established in regulation a rate of disparity of 20 percent or
greater as the threshold for taking action against an accreditation
program. According to a CMS official, the use of 20 percent as the
threshold is not based on empirical evidence but rather on what CMS
believed Congress would find acceptable. Consequently, the threshold

may not be appropriately placed to indicate unacceptable performance by a
hospital accreditation program. For example, if JCAHO failed to identify
serious deficiencies in all 14 hospitals that the state agencies
identified with serious deficiencies from a sample of 79 hospitals, the
rate of disparity would be a satisfactory 18 percent ((14-0)/79).22

CMS's rate of disparity measure used in isolation does not consistently
reflect an accreditation program's ability to detect serious deficiencies.
As the number of hospitals with serious deficiencies detected by state
survey agencies decreases, regardless of JCAHO's performance in detecting
them, it is more likely that the rate of disparity will be less than CMS's
20 percent threshold. As a result, the performance of JCAHO's hospital
accreditation program is difficult to judge based on this measure alone.
For example, if state survey agencies performed 200 validation surveys and
found 100 hospitals or 50 percent with serious deficiencies and JCAHO
found 30 hospitals or 30 percent of the hospitals found by state agencies,
the rate of disparity would be 35 percent ((100-30)/200). However, if the
state agencies found 50 hospitals, or 25 percent, of the 200 hospitals
with serious deficiencies and JCAHO found 15 hospitals, or 30 percent of
the hospitals that the state agencies identified, the rate of disparity
would be almost 18 percent ((50-15)/200). The percentage of serious
deficiencies found by state survey agencies and also by JCAHO remained the
same in both examples, but the rate of disparity was improved
significantly by the larger number of hospitals without serious
deficiencies in the second example. This indicates that the rate of
disparity does not consistently measure the accreditation program's
ability to detect serious deficiencies found by state survey agencies.
(See table 5.) In addition to the rate of disparity, other components,
such as the proportion of hospitals with serious deficiencies and the
total number of serious deficiencies found by state agencies but missed by
the accreditation program, are important indicators of an accreditation
program's overall performance.

22The example is based on the analysis of the rate of disparity in
American Institutes of Research, Measurement and Evaluation of Revised
Accredited Hospital Validation and Oversight (Washington, D.C.: Nov. 6,
2002).

Table 5: Hypothetical Examples of the Effect on the Rate of Disparity of a
Decrease in the Number of Hospitals with Serious Deficiencies in a Sample of 200
                                   Hospitals

Example 1 Example 2 State agencies JCAHO State agencies JCAHO

Number of hospitals
with serious
deficiencies 100 30 50

Percentage of
hospitals state
agencies found with
serious deficiencies
that were also found
by JCAHO 30% 30%

Percentage of hospitals
without serious
deficiencies identified
by state agencies 50% 75%

             Rate of disparity 35% ((100-30)/200) 18% ((50-15)/200)

               Performance level Above threshold Below threshold

Statistical Analysis of Validation Survey Sample

Source: GAO.

Note: CMS's rate of disparity threshold is 20 percent.

CMS does not analyze the statistical results of its validation survey
samples in ways that would allow it to better assess JCAHO's ability to
detect serious deficiencies. CMS has not documented the methods it uses to
select hospitals for validation surveys and did not supply us with clear
technical justification for the methods used. Further, CMS's validation
sample includes hospitals that, because of its sampling method, have
varying chances of selection, but it does not take this into account when
calculating statistics based on the sample. According to CMS's sampling
method, the selection of hospitals is influenced by factors such as the
month in the fiscal year that JCAHO performed the accreditation survey and
how many hospitals were targeted for completion that year in the state in
which the hospital was located. Thus, some hospitals have a greater chance
of selection than others. CMS also does not take these different chances
of selection into account when calculating statistics for its annual
reports to Congress, which prevents CMS from accurately assessing JCAHO's
performance. Moreover, CMS does not measure and report in its annual
reports the extent to which its estimates based on the

validation survey sample are likely to reflect how well JCAHO detects
deficiencies in the larger population of hospitals it accredits.23

In addition, the number of usable traditional validation surveys completed
is smaller than the number of hospitals CMS samples for validation
surveys. This difference may affect the accuracy of the data that CMS
presents to Congress if the hospitals where the traditional surveys were
completed produce different results than those where surveys are not
completed or are not usable. During its sampling process, CMS selects a
sample size close to the targeted number of hospitals each year. Some
hospitals from this sample may be excluded because CMS chose to perform
another type of survey for them that cannot be used to validate a JCAHO
accreditation survey. In addition, state agencies are not always able to
complete the requested traditional validation surveys within 60 days from
the JCAHO accreditation survey, as required, or a hospital may be excluded
because it lost its deemed status or closed. The size of the difference
between the number of hospitals sampled and the number of usable
traditional validation surveys completed therefore varies, as it did
during the 3-year review period (see table 6).

Table 6: Number of Hospitals Targeted for Validation Surveys Compared with
Usable Traditional Validation Surveys Completed

                           Hospitals                       Usable traditional 
                            targeted                       
                      for validation Hospitals sampled for validation surveys 
          Fiscal Year       surveysa validation surveys b          completedc 
                 2000            236                   236                184 
                 2001            227                   217                204 
                 2002            227                   235                112 

Source: CMS.

aThe targeted number is set at the beginning of the fiscal year and is
used for planning and resource allocation by CMS and the state survey
agencies.

bThe sampled hospitals are the hospitals selected for validation surveys
during the year.

cUsable surveys exclude those not completed, those completed after the
required 60-day time frame, and other types of surveys that can not be
used to validate a JCAHO accreditation survey.

23For example, CMS does not measure and report the precision of the
estimates from the sample of validation surveys through the use of
confidence intervals or margins of error, which define the range of
estimates that sample results would yield given different random samples
for a specified level of certainty.

Annual Number of Validation Surveys

CMS reduced the number of validation surveys conducted by state agencies
from a target of approximately 5 percent of the total number of hospitals
that JCAHO accredits to a target of approximately 1 percent, with at least
one survey in each state. Reducing the target of validation surveys from 5
percent to 1 percent results in the number of validation surveys being
reduced from 227 in fiscal year 2002 to a target of 75 validation surveys
in fiscal year 2003 and 72 in fiscal year 2004.

Reducing the targeted number of validation surveys to 1 percent provides
less reliable information on how well JCAHO's hospital accreditation
program ensures compliance with Medicare COPs. For example, for a 5percent
target, the estimate of the proportion of JCAHO-accredited hospitals with
a particular deficiency that is derived from the validation survey could
be as much as 6.0 percentage points higher or lower, for a range of 12.0
percentage points. If the 5-percent target produced an estimate that 50
percent of JCAHO-accredited hospitals had a particular deficiency, the
percentage of JCAHO-accredited hospitals not complying could range from
44.0 to 56.0 percent. However, for a 1-percent target the estimate could
be 11.4 percentage points higher or lower, for a range of about 22.8
percentage points. For example, if the 1-percent target produced an
estimate that 50 percent of JCAHO-accredited hospitals had a particular
deficiency, the percentage of JCAHO-accredited hospitals not complying
with a Medicare COP could range from 38.6 to 61.4 percent.24

This reduction in the number of validation surveys is of additional
concern because it coincides with the implementation of JCAHO's new
accreditation process, which has an unproven capacity to detect
deficiencies. CMS's target sample size for traditional validation surveys
for fiscal year 2004 will be further reduced because the sample also
includes 18-month validation surveys. In 2004, CMS is planning to conduct
17 of these 18-month surveys as part of its overall validation survey
target of 72. Thus, CMS could be using as few as 55 validation surveys to
determine JCAHO's performance.

24These estimates were developed assuming that the validation surveys are
conducted on a simple random sample of JCAHO-accredited hospitals and a 95
percent confidence level.

Conclusions

Matter for Congressional Consideration

For 3 consecutive years, JCAHO's hospital accreditation program, which
accredits most of the hospitals participating in Medicare, exceeded CMS's
threshold for unacceptable performance. CMS validation surveys during that
time period confirmed that JCAHO missed the majority of serious
deficiencies found by state survey agencies. Yet, CMS was unable to take
action against JCAHO's hospital accreditation program as it can with other
accreditation programs because it lacked the authority to do so. Although
CMS has recommended in its annual reports to Congress that JCAHO make
changes in its hospital accreditation program to improve its ability to
detect serious deficiencies, some of these recommendations have not been
implemented. Thus, it is vital for patient safety that JCAHO hospital
accreditation surveys detect existing serious deficiencies and deny
accreditation to hospitals that do not comply with Medicare COPs.

CMS is unable to present to Congress an adequate assessment of JCAHO's
performance because of limitations in its process for selecting hospitals
for validation surveys and analysis of the survey results. CMS does not
consistently portray the extent to which serious deficiencies are missed
and does not identify the limitations in reporting the estimates it makes
from its survey sample. CMS cannot assure Congress that JCAHOaccredited
hospitals meet Medicare COPs because the measure for the rate of
disparity, which determines poor performance, allows JCAHO to miss the
majority of serious deficiencies and still be in an acceptable range of
performance. Further, CMS's reduction in the number of validation surveys
it uses to determine the performance of JCAHO's hospital accreditation
program will provide less reliable information at a time when JCAHO is
implementing a new hospital accreditation process that is unproven in its
ability to detect serious deficiencies. In light of these limitations in
CMS's validation of JCAHO's hospital accreditation program, we believe
that CMS must improve its oversight so it can provide Congress with more
accurate information regarding JCAHO's performance.

Given the serious limitations in JCAHO's hospital accreditation program
and that efforts to improve this program through informal action by CMS
have not led to necessary improvements, Congress should consider giving
CMS the same kind of authority over JCAHO's hospital accreditation program
that it has over all other Medicare accreditation programs.

Recommendations for 	To strengthen the ability of CMS to identify and
report to Congress on JCAHO's ability to ensure that the hospitals it
accredits protect the safety

Executive Action 	and health of patients through compliance with the
Medicare COPs, we recommend that the Administrator of CMS take the
following three actions:

o  	modify the method used to measure the rate of disparity between
validation survey findings and accreditation program findings to provide a
reasonable assurance that Medicare COPs are being met and consider whether
additional measures are needed to accurately reflect an accreditation
program's ability to detect deficiencies in Medicare COPs;

o  	provide in the annual report to Congress an estimate, based on the
validation survey sample, of the performance of all JCAHO-accredited
hospitals, including the limitations and protocols for these estimates
based on generally accepted sampling and statistical methodologies; and
develop a written protocol for these calculations; and

o  annually conduct traditional validation surveys on a sample of JCAHO-

Agency and Other External Comments and Our Evaluation

accredited hospitals that is equal to at least 5 percent of all
JCAHOaccredited hospitals.

CMS and JCAHO commented on a draft of this report. In its comments, CMS
concurred with our recommendations. JCAHO stated it had no objection to
our suggestion that Congress give CMS the same authority over its hospital
accreditation program as it does over other Medicare accreditation
programs. However, JCAHO took issue with the methodology we used for
evaluating the performance of its hospital accreditation program. CMS's
and JCAHO's specific comments and our response follow. CMS's comments are
reprinted in appendix IV and JCAHO's comments are reprinted in appendix V.
CMS and JCAHO also provided technical comments, which we incorporated as
appropriate.

CMS stated that it has begun to examine the need for additional or
alternative measures for the rate of disparity calculation. CMS also
stated it will seek additional resources to further develop and implement
new sampling and statistical methodologies that may allow results to be
projected to all JCAHO-accredited hospitals, and to increase the
validation sample size. CMS specifically noted that it considers
life-safety code compliance, on the part of all provider types, to be
critically important. In the past 8 years, in its annual reports to
Congress and its dialogues with JCAHO regarding its hospital accreditation
program, it has identified physical environment as an important area where
JCAHO needs to focus attention, and CMS noted that 68 percent of
facilities that had a deficiency

finding not identified by JCAHO had them in the physical environment area.

JCAHO stated that our methodology for evaluating the performance of its
hospital accreditation program was incomplete and did not provide a
comprehensive assessment of its program's performance. We did not intend
to do a comprehensive evaluation of JCAHO's overall hospital accreditation
program. Rather, we focused our evaluation on how well JCAHO's hospital
accreditation program ensures hospitals' compliance with Medicare
participation requirements. There are four possible outcomes to a
comparison between JCAHO's accreditation survey and a state validation
survey: (1) both JCAHO and state agencies identify no deficiencies, (2)
JCAHO identifies deficiencies not found by state agencies, (3) both JCAHO
and state agencies identify the same deficiencies, and (4) state agencies
identify deficiencies that JCAHO does not. We limited our evaluation to
the fourth outcome because it illustrates the need for CMS oversight of
the hospital accreditation process. We have clarified the scope of our
evaluation to emphasize our focus on this outcome.

JCAHO raised a concern that our characterization of JCAHO's missed
deficiencies that state survey agencies found misleads readers to believe
that JCAHO misses hospitals with deficiencies 78 percent of the time. We
have revised language in the report to further emphasize that the missed
deficiency rate applies to hospitals in the validation survey sample in
which the state survey agencies found deficiencies and cannot be
generalized to all JCAHO-accredited hospitals. JCAHO further stated that
our report does not take into account that JCAHO's hospital accreditation
program detects deficiencies in hospitals that CMS does not find. However,
it is to be expected that state survey agencies will not find all
deficiencies found by JCAHO because hospitals may have corrected the
deficiencies prior to the state agency surveys.

JCAHO stated that we misrepresented the potential of the new accreditation
process in detecting deficiencies in Medicare COPs and provided new data
regarding its first quarter 2004 performance that indicate that JCAHO
surveys may have detected a greater percentage of deficiencies related to
patient care compared with the pre-2004 accreditation process. However, we
maintain that until CMS validation surveys for 2004 are completed, there
is no basis on which to determine whether the new process improves the
detection of deficiencies in Medicare COPs. In addition, JCAHO stated and
we agree that evaluating and improving the quality of care in hospitals is
not about counting deficiencies, it is about finding those deficiencies
that, if not fixed, will

generate poor results for patients and making sure that these deficiencies
are remedied in a timely fashion.

JCAHO stated that we mischaracterized its response to the five
recommendations that CMS made in 2002 to improve JCAHO's ability to detect
deficiencies in the life safety code and that it is involved in frequent
and ongoing dialogue with CMS regarding the recommendations and other life
safety code issues. We have clarified language in the report regarding
JCAHO's response to CMS's recommendations.

As agreed with your offices, unless you publicly announce its contents
earlier, we plan no further distribution of this report until 30 days
after its
date. We will then send copies of this report to the Secretary of Health
and
Human Services and other interested parties. We will also make copies
available to others upon request. In addition, the report will be
available at
no charge at the GAO Web site at http://www.gao.gov.

If you or your staffs have any questions about this report, please call me
at
(202) 512- 7119. Another contact and key contributors are listed in
appendix VI.

Janet Heinrich
Director, Health CarePublic Health Issues

                       Appendix I: Scope and Methodology

We examined the extent to which JCAHO's pre-2004 survey process identified
hospitals with deficiencies and individual deficiencies in Medicare COPs
that were identified by state survey agencies. We chose these measures
because they reflect performance in detecting and correcting serious
deficiencies, which according to CMS, substantially limit a hospital's
capability to render adequate care and adversely affect the health and
safety of patients. We reviewed data, provided by CMS, on 500 traditional
validation surveys conducted by state survey agencies during fiscal years
2000 through 2002. In these validation surveys, state survey agencies
documented whether they found serious deficiencies in Medicare COPs. CMS
compared state survey agency findings with JCAHO's accreditation surveys
that identified deficiencies in JCAHO's standards. CMS then determined
whether the state survey agencies' findings on serious deficiencies in the
22 Medicare COPs that can be deemed were comparable to JCAHO's findings on
deficiencies in JCAHO's standards in the following way. Two CMS experts
such as nurses reviewed the comparability of serious deficiencies in the
quality-of-care conditions identified in validation surveys to
deficiencies in JCAHO's accreditation standards identified in JCAHO's
hospital accreditation surveys. Two experts, such as building engineers,
reviewed the comparability of serious deficiencies identified in the
validation surveys on the condition on physical environment. Where there
was disagreement, the two experts met to resolve their differences. CMS
does not have written protocols for determining comparability. Experts are
expected to use their best professional judgment. CMS experts also had to
consider whether it is reasonable to conclude that the deficiencies
existed at the time that JCAHO surveyed the hospital. For those
deficiencies that CMS determines that JCAHO has failed to identify, it met
with JCAHO to address disputed findings and to consider additional
evidence on comparability offered by JCAHO. There are four possible
outcomes to this comparison of survey findings-(1) JCAHO and state
agencies both identify no deficiencies, (2) JCAHO identifies deficiencies
not found by state agencies, (3) JCAHO and state agencies both identify
the same deficiencies, and (4) state agencies identify deficiencies that
JCAHO does not-we focused on the fourth because it highlights the need for
CMS oversight of the hospital accreditation program. For the second
outcome, there could be two reasons for the disparity between JCAHO's and
state survey agencies' findings: hospitals corrected deficiencies
identified by JCAHO prior to the state agency survey or the state survey
agency did not identify a deficiency that existed. In addition, not all
JCAHO findings are equivalent to noncompliance with a Medicare COP.

Appendix I: Scope and Methodology

From these 500 surveys, we determined the number of hospitals with serious
deficiencies and the total number of serious deficiencies identified by
state agencies but that CMS determined were not identified by JCAHO. These
data include 123 hospitals in which state survey agencies identified one
or more serious deficiencies and JCAHO did not make comparable findings
according to CMS. These data also include 167 serious deficiencies
identified by state agencies but that CMS determined comparable findings
were not identified by JCAHO.

For fiscal years 2001 and 2002, we obtained from CMS a comparison between
the validation surveys conducted by the state survey agencies and the
accreditation surveys conducted by JCAHO, which identified serious
deficiencies identified by the state agencies but not by JCAHO as
determined by CMS. For fiscal year 2000, CMS did not supply its
determinations of the comparability of findings in validation and
accreditation surveys for 31 of 82 serious deficiencies. We followed a
protocol similar to the one used by CMS to determine the comparability of
the remaining 31 serious deficiencies, which included 29 quality-of-care
serious deficiencies and 2 physical environment serious deficiencies. Two
analysts with nursing backgrounds compared the findings and made
determinations on their comparability based on their professional
judgment. In cases of disagreement, a third analyst with a background in
nursing made the determination.

We did not include 1998 and 1999 data in our analysis because CMS used a
method that undercounted the number of deficiencies identified by state
survey agencies but not identified by JCAHO. CMS did not count as
deficient those cases in which state survey agencies determined that a
hospital was not meeting the COP on physical environment but JCAHO
determined that the hospital was in compliance because the hospital was
following correction plans approved by JCAHO.

To determine the potential of JCAHO's new accreditation process in
improving the detection of deficiencies in Medicare COPs, we reviewed
material supplied by JCAHO on development and testing of its new process
and interviewed JCAHO officials about the steps taken to test the new
process and to analyze results. We also examined the features of the new
accreditation process by reviewing descriptive material obtained from
JCAHO and interviewing experts in health care quality. Because the new
accreditation process was implemented in January 2004, we were limited in
our ability to determine the effectiveness of the new accreditation
process because we were not able to perform a comparative

Appendix I: Scope and Methodology

analysis of validation survey and JCAHO survey results under the new
process.

To examine the effectiveness of CMS's oversight of JCAHO's accreditation
process, we analyzed the laws and regulations that define CMS's authority
and JCAHO's authority. We reviewed the annual reports submitted to
Congress on JCAHO's performance in identifying serious deficiencies and
reviewed correspondence between CMS and JCAHO and interviewed officials in
both organizations. We analyzed the rate of disparity that CMS uses to
determine the performance of JCAHO's hospital accreditation process in
identifying deficiencies in Medicare COPs.

To evaluate CMS's statistical methodology for the validation surveys, we
interviewed CMS officials about the sampling and statistical methods. In
the absence of written methodological documentation, we relied on
information provided by CMS officials to evaluate the methodology. They
gave us the following information about their sampling method. At the
beginning of each year, CMS determines a target for the number of
hospitals that will be sampled for validation surveys in each state. Each
month, CMS receives a list of hospitals scheduled for a JCAHO
accreditation survey in that month. Prior to sampling, CMS removes from
the list those hospitals that have received a validation survey in the
last 3year accreditation cycle and hospitals that do not participate in
Medicare. In the first month of the year, CMS selects a random sample of
hospitals to be surveyed from JCAHO's list. In subsequent months, CMS
removes hospitals in states in which the state target has been met and
then selects a random sample of hospitals. Prior to sending the list to
state survey agencies, CMS determines which hospitals will receive
traditional validation surveys and which will receive other types of
surveys that cannot be used to assess the performance of JCAHO's hospital
accreditation program. State survey agencies must then complete
traditional validation surveys within 60 days of the completion of JCAHO's
accreditation survey for the results to be used by CMS to measure the
performance of JCAHO's hospital accreditation program. According to CMS
officials, the sampling procedures CMS uses are necessary because they are
not informed more than 1 month in advance which hospitals JCAHO will
survey for accreditation.

Appendix I: Scope and Methodology

In reviewing the sampling procedures they described, we determined that
CMS initially selects a probability sample of hospitals for its state
agency validation surveys.1 However, hospitals have varying chances of
selection in the sample depending on the month in the fiscal year that
JCAHO performs the accreditation survey and the number of hospitals
targeted for completion that year in the state in which the hospital was
located. Additionally, the way that CMS determines which type of survey
the sampled hospital receives is not random. Therefore, the analysis we
performed is limited to those hospitals included in the validation survey
sample and cannot be projected to all JCAHO-accredited hospitals.

1In a probability sample, each eligible hospital accredited in a given
year would have to have a known, nonzero chance for selection in the
sample.

Appendix II: Medicare Conditions of Participation

To participate in Medicare, hospitals must maintain standards of patient
safety and health that comply with Medicare requirements. There are
currently 23 Medicare COPs. Table 7 provides a description of each
Medicare COP.

Table 7: Medicare Conditions of Participation

                            Medicare COP Description

Anesthesia servicesa	Anesthesia services must be well organized and
directed by a qualified doctor of medicine or osteopathy. The service is
responsible for all anesthesia administered.

Compliance with federal, state, and local laws	A hospital must comply with
applicable federal laws on patient health and safety and state and local
laws on hospital and personnel licensing.

Discharge planning 	A hospital must have a discharge planning process
applicable to all patients. Policies and procedures must be in writing.

Emergency servicesa	If emergency services are provided they must be
organized under the direction of a qualified member of the medical staff
and have adequate medical and nursing personnel qualified in emergency
care to meet the needs anticipated by the facility.

Food and dietetic services 	Dietary services must be organized, directed,
and staffed by qualified personnel. Contracted services must meet certain
requirements.

Governing body	A hospital must have a legally responsible governing body
or persons charged with the responsibilities of a governing body.

Infection control 	A hospital's sanitary environment must avoid sources
and transmission of infections and communicable diseases. It must have an
active program to prevent, control, and investigate infections and
communicable diseases.

Laboratory services The hospital must maintain, or have available,
adequate laboratory services.

Medical record services 	A hospital must have a medical record service
that has administrative responsibility for medical records.

Medical staff 	A hospital must have an organized medical staff that abides
by bylaws approved by the governing body and is responsible for the
quality of patient medical care.

Nuclear medicine servicesa	Nuclear medicine services must meet the needs
of the patients in accordance with acceptable standards of practice.

Nursing services 	An organized nursing service must provide 24-hour
nursing services that are supervised or furnished by registered nurses.

Organ, tissue, and eye procurement 	The hospital must have and implement
written protocols on procurement and have adequate organ transplant
policies.

Outpatient servicesa	Outpatient services must meet patient needs
consistent with acceptable standards of practice.

Patients' rights A hospital must protect and promote patients' rights.

Pharmaceutical services The hospital must have pharmaceutical services
that meet patient needs.

Physical environment 	Hospital construction, arrangements, and maintenance
must ensure patient safety and provide diagnostic and treatment facilities
and special hospital services appropriate to community needs.

Quality assessment and performance improvement A hospital must have an
effective, hospitalwide quality assurance program.

Appendix II: Medicare Conditions of Participation

                            Medicare COP Description

Radiologic services 	The hospital must maintain, or have available,
diagnostic radiologic services. Therapeutic services provided must meet
professionally approved standards for safety and personnel qualifications.

Rehabilitation servicesa	Rehabilitation, physical therapy, occupational
therapy, audiology, or speech pathology services must be organized and
staffed to ensure the health and safety of patients.

Respiratory servicesa	Respiratory services must meet patient needs in
accordance with acceptable standards of practice.

a

Surgical services	Surgical services must be well organized and provided in
accordance with acceptable standards of practice. Outpatient services must
be consistent with inpatient care quality in accordance with the
complexity of services offered.

Utilization review	Utilization review plans must provide for review of the
services that a hospital and its medical staff provide to Medicare and
Medicaid patients.

Source: GAO summary of Medicare COPs.

                  aOptional services not required by Medicare.

Appendix III: Features of JCAHO's New Accreditation Process

In January 2004, JCAHO introduced a new hospital accreditation process
that includes several new features. Table 8 includes a description of
selected new features of JCAHO's hospital accreditation process.

Table 8: JCAHO's Description of Features of Its New Hospital Accreditation
                                    Process

Feature of the new
accreditation process Description

Periodic performance review 	The periodic performance review (PPR) is a
new form of evaluation that is conducted by the organization and focuses
on patient safety and quality of care issues. The organization
self-evaluates its compliance with all standards that are applicable to
the services that the organization provides, and develops a plan of action
for all areas of performance identified as needing improvement. JCAHO will
work with the organization to refine its plan of action to assure that its
corrective efforts are on target. The organization will also identify
measures of success for validating resolution of the identified problem
areas when the organization undergoes its complete on-site survey 18
months later.

Three options to the full PPR are available to organizations. The options
and their requirements are:

Option 1

The organization performs the mid-cycle self-assessment, develops the plan
of action and measures of success but does not submit PPR data to JCAHO.
The organization attests that it has completed the foregoing activities
but has, for substantive reasons, been advised not to submit its
selfassessment or plan of action to JCAHO.

The organization may discuss standards-related issues with JCAHO staff
without identifying its specific levels of standards compliance.

At the time of the complete on-site survey, the organization provides its
measures of success to JCAHO for assessment.

Option 2

The organization need not conduct a mid-cycle self-assessment or develop a
plan of action.

The organization undergoes an on-site survey at the mid-point of its
accreditation cycle. The survey will be approximately one-third the length
of a typical full on-site survey and the organization will be charged a
fee to cover survey costs.

The organization develops and submits to JCAHO a plan of action to address
any areas of noncompliance found during the on-site survey. JCAHO will
work with the organization to refine its plan of action. At the time of
the complete on-site survey, the organization provides its measures of
success to JCAHO for assessment.

Option 3

The mid-cycle survey would be performed, as in Option 2, but, if the
organization chooses, no written documentation or report of the survey
would be left with the organization. Findings would be conveyed orally.
This would eliminate the availability of a survey report for possible
discovery from the organization, and would permit the organization, as is
the case with Option 1, to control the language and documentation of the
mid-cycle assessment activity. At the subsequent full survey, surveyors
would not discuss with the organization, unless asked to do so, the fact
that any particular standard had been found out of compliance at the
mid-cycle assessment. Rather, they would focus on compliance with those
standards at the time of the full survey.

If the plan of action is approved, the organization's accreditation
decision will remain the same. However, if the plan of action is not
approved, the organization's accreditation decision will be changed to
reflect the appropriate status. At the triennial on-site survey,
implementation of the plan of action will be validated

Appendix III: Features of JCAHO's New Accreditation Process

Feature of the new
accreditation process Description

Priority focus process 	The priority focus process (PFP) is a data-driven
tool that focuses survey activity on issues most relevant to patient
safety and quality of care at the specific health care organization being
surveyed. The PFP uses automation to gather pre-survey data from multiple
sources including JCAHO, the hospital and other public sources. The PFP
then applies rules to 1) identify relevant standards and appropriate
survey activities, and 2) guide the selection of patient tracers. As part
of the priority focus process, surveyors will track patients through their
experience of care within an organization, assessing the quality and
safety of care provided. The PFP does not imply that priority areas are
out of compliance or deficient in any way. Rather, it lends consistency to
the surveyor's on-site sampling process. The PFP also helps to focus the
surveyor's assessment on quality and safety issues specific to an
individual health care organization.

The output of the PFP process will include: the top four to five priority
focus areasthe processes, systems, or structures within a health
care organization known to significantly impact the safety and quality of
care specific to the health care organization being surveyed.

Tracer methodology 	An evaluation method in which surveyors select a
patient and use that individual's record as a roadmap or "tracer" to
assess and evaluate an organization's compliance with selected standards
and the organization's systems of providing care and services. Using
tracers, JCAHO surveyors will look at the care provided by each department
within an organization, and how departments work together. Surveyors
retrace the specific care processes that the individual experienced by
observing and talking to staff in areas that the individual received care.
As the individual's case is examined, the surveyor may identify
performance issues in one or more steps of the processor the
interfaces between stepsthat affect the care of the patient.
Surveyors will look for commonalities that might point to potential
system-level issues in the organization. The tracer activity also provides
several opportunities for surveyors to provide education to organization
staff and leaders, as well as to share best practices from other similar
health care organizations.

Tracer patients will primarily be selected from an active patient list.
Typically, individuals selected for the tracer activity are those who have
received multiple or complex services

                                 Source: JCAHO.

Appendix IV: Comments from the Centers for Medicare & Medicaid Services

Appendix IV: Comments from the Centers for Medicare & Medicaid Services

Appendix IV: Comments from the Centers for Medicare & Medicaid Services

Appendix IV: Comments from the Centers for Medicare & Medicaid Services

Appendix IV: Comments from the Centers for Medicare & Medicaid Services

Appendix IV: Comments from the Centers for Medicare & Medicaid Services

            Page 46 GAO-04-850 Medicare Patient Safety in Hospitals

            Page 47 GAO-04-850 Medicare Patient Safety in Hospitals

            Page 48 GAO-04-850 Medicare Patient Safety in Hospitals

            Page 49 GAO-04-850 Medicare Patient Safety in Hospitals

Appendix VI: GAO Contact and Staff Acknowledgments

GAO Contact Marcia A. Mann, (202) 512-9526

Acknowledgments 	In addition to the contact named above, Elaine Swift,
Linda Kohn, Behn Kelly, Elizabeth T. Morrison, Roseanne Price, and Marie
Stetser made key contributions to this report.

Related GAO Products

Medicare Home Health Agencies: Weaknesses in Federal and State Oversight
Mask Potential Quality Issues. GAO-02-382. Washington, D.C.: July 19,
2002.

Medicare: HCFA's Approval and Oversight of Private Accreditation
Organizations. GAO/ HEHS-99-197R. Washington, D.C.: September 30, 1999.

Home Health Care: HCFA Properly Evaluated JCAHO's Ability to Survey Home
Health Agencies. GAO/HRD-93-33. Washington, D.C.: October 26, 1992.

Health Care: Criteria Used to Evaluate Hospital Accreditation Process Need
Reevaluation. GAO/HRD-90-89. Washington, D.C.: June 11, 1990.

GAO's Mission	The Government Accountability Office, the audit, evaluation
and investigative arm of Congress, exists to support Congress in meeting
its constitutional responsibilities and to help improve the performance
and accountability of the federal government for the American people. GAO
examines the use of public funds; evaluates federal programs and policies;
and provides analyses, recommendations, and other assistance to help
Congress make informed oversight, policy, and funding decisions. GAO's
commitment to good government is reflected in its core values of
accountability, integrity, and reliability.

Obtaining Copies of The fastest and easiest way to obtain copies of GAO
documents at no cost

is through GAO's Web site (www.gao.gov). Each weekday, GAO postsGAO
Reports and newly released reports, testimony, and correspondence on its
Web site. To Testimony have GAO e-mail you a list of newly posted products
every afternoon, go to

www.gao.gov and select "Subscribe to Updates."

Order by Mail or Phone	The first copy of each printed report is free.
Additional copies are $2 each. A check or money order should be made out
to the Superintendent of Documents. GAO also accepts VISA and Mastercard.
Orders for 100 or more copies mailed to a single address are discounted 25
percent. Orders should be sent to:

U.S. Government Accountability Office 441 G Street NW, Room LM Washington,
D.C. 20548

To order by Phone: 	Voice: (202) 512-6000 TDD: (202) 512-2537 Fax: (202)
512-6061

To Report Fraud, Contact:
Waste, and Abuse in Web site: www.gao.gov/fraudnet/fraudnet.htm

E-mail: [email protected] Programs Automated answering system: (800)
424-5454 or (202) 512-7470

Congressional 	Gloria Jarmon, Managing Director, [email protected] (202)
512-4400 U.S. Government Accountability Office, 441 G Street NW, Room 7125

Relations Washington, D.C. 20548

Public Affairs Jeff Nelligan, Managing Director, [email protected] (202)
512-4800 U.S. Government Accountability Office, 441 G Street NW, Room 7149
Washington, D.C. 20548

                               Presorted Standard
                              Postage & Fees Paid
                                      GAO
                                Permit No. GI00

United States
Government Accountability Office
Washington, D.C. 20548-0001

Official Business
Penalty for Private Use $300

Address Service Requested
*** End of document. ***