Health Care: Employers and Individual Consumers Want Additional
Information on Quality (Letter Report, 09/29/95, GAO/HEHS-95-201).
Pursuant to a congressional request, GAO provided information on health
care quality issues, focusing on: (1) how consumers use health care
performance reports that contain comparative data on the quality of
health care providers; and (2) what information consumers consider most
important.
GAO found that: (1) employers and individuals use information that
measures and compares the quality of health care furnished by providers
and health plans when making purchasing decisions; (2) consumers want
performance reporting efforts to continue and are requesting that more
data be made publicly available; (3) consumers want standardized and
comparable health care information to assess health care providers' or
health plans' performance; (4) many employers get health care
performance data through business coalitions, consultants, and their own
data collection efforts; and (5) although employers have begun
cooperating with one another to enhance their purchasing decisions, few
employers make health care data available to their employees.
--------------------------- Indexing Terms -----------------------------
REPORTNUM: HEHS-95-201
TITLE: Health Care: Employers and Individual Consumers Want
Additional Information on Quality
DATE: 09/29/95
SUBJECT: Health maintenance organizations
Health resources utilization
Data integrity
Information dissemination operations
Health insurance
Health care services
Medical information systems
Quality assurance
Employee medical benefits
Surveys
IDENTIFIER: Medicare Program
Medicaid Program
Health Plan Employer Data and Information Set
**************************************************************************
* This file contains an ASCII representation of the text of a GAO *
* report. Delineations within the text indicating chapter titles, *
* headings, and bullets are preserved. Major divisions and subdivisions *
* of the text, such as Chapters, Sections, and Appendixes, are *
* identified by double and single lines. The numbers on the right end *
* of these lines indicate the position of each of the subsections in the *
* document outline. These numbers do NOT correspond with the page *
* numbers of the printed product. *
* *
* No attempt has been made to display graphic images, although figure *
* captions are reproduced. Tables are included, but may not resemble *
* those in the printed version. *
* *
* A printed copy of this report may be obtained from the GAO Document *
* Distribution Facility by calling (202) 512-6000, by faxing your *
* request to (301) 258-4066, or by writing to P.O. Box 6015, *
* Gaithersburg, MD 20884-6015. We are unable to accept electronic orders *
* for printed documents at this time. *
**************************************************************************
Cover
================================================================ COVER
Report to the Ranking Minority Member, Committee on Labor and Human
Resources, U. S. Senate
September 1995
HEALTH CARE - EMPLOYERS AND
INDIVIDUAL CONSUMERS WANT
ADDITIONAL INFORMATION ON QUALITY
GAO/HEHS-95-201
Health Care Report Cards
(101456)
Abbreviations
=============================================================== ABBREV
CABG - coronary artery bypass graft
CalPERS - California Public Employees' Retirement System
ERISA - Employee Retirement Income Security Act of 1974
FAcct - Foundation for Accountability
HCFA - Health Care Financing Administration
HEDIS - Health Plan Employer Data and Information Set
HMO - health maintenance organization
NCQA - National Committee for Quality Assurance
Letter
=============================================================== LETTER
B-257441
September 29, 1995
The Honorable Edward M. Kennedy
Ranking Minority Member
Committee on Labor and Human Resources
United States Senate
Dear Senator Kennedy:
Employers and individual consumers are no longer concerned only about
the escalating cost of health care; they are increasingly concerned
that efforts to reduce health care costs may now also be reducing its
quality. Some cost-control efforts might unduly encourage providers
to withhold care. Other cost-control efforts restrict or eliminate
individuals' choice of provider. As employers negotiate for lower
premiums or limit employees' access to providers, they want to ensure
that their employees still receive quality care. Individual
consumers want to be assured that they have access to quality
providers and that they make the right health care decisions. As a
result, both employers who purchase health care and individual
consumers have demanded more information about quality.
In response to these demands, some states, large employers, and
health plans have been publishing performance reports describing the
quality of health care providers. These "report cards" include
information such as the frequency with which preventive services are
provided and the degree of success in treating certain diseases. The
federal government, as the nation's largest health care purchaser,
has also become increasingly involved in the movement to develop and
publish performance reports, especially for Medicare and Medicaid
beneficiaries.
Little has been known, however, about how useful published reports
have been or how they could be made more helpful. As a result, you
asked us to study (1) how consumers use available published
comparative data and (2) what information consumers want.\1
To obtain the views of individual consumers, we attempted to contact
over 1,000 persons who had requested a report card published by a
state agency or health plan. We were not able to obtain usable
information from most of these persons--some did not respond to our
attempts, some were unavailable, some did not remember receiving the
report card, and some had used the information for work-related
research rather than to select a provider or plan. We conducted
telephone interviews with the remaining 153 people who had requested
the report card for use in making health care purchasing decisions.
We also conducted 7 group interviews with a total of 64 employees at
7 locations.\2 These consumers may not have had previous experience
with report cards.
To obtain the views of employers, we interviewed representatives of
65 businesses around the country with health coverage for fewer than
5 to over 100,000 employees. Because the employers and individual
consumers are not nationally representative, their experiences and
opinions cannot be generalized to all employers and consumers.
We conducted our review from November 1994 to June 1995 in accordance
with generally accepted government auditing standards. (See app. I
for a more detailed discussion of our scope and methodology.)
--------------------
\1 Employers are consumers of health plan services to the extent that
they use administrative services, including data. Recognizing that,
some plans include both employers and individual consumers in their
customer satisfaction surveys.
\2 Four of these worksites were federal agencies. Many private
sector employers we contacted refused our request to discuss health
plan purchasing decisions with their employees.
RESULTS IN BRIEF
------------------------------------------------------------ Letter :1
Many employers and individual consumers we interviewed are using
information that measures and compares the quality of health care
furnished by providers and health plans when making their purchasing
decisions. For example, employers are using report cards to select
and monitor the performance of providers and plans furnishing
services to their employees, negotiate with insurance carriers, and
market managed care plans to employees. As one employer remarked,
"We'd like to get some kind of value-based decision for purchasing
health care." Individuals are using report cards to choose providers
or plans, to enhance their knowledge of providers or plans, and to
reassure themselves of their own or their employers' provider
choices.
Employers and individual consumers we interviewed wanted performance
reporting efforts to continue. In fact, they are requesting more
data than are publicly available. However, they believed the
information would be more useful if their concerns about the
reliability and validity of the data were addressed. For example,
some individual consumers used terms such as "self-serving,"
"one-sided," and "nontrustworthy" to describe the reports they
received from health plans.
Employers and individual consumers we interviewed also reported that
the most useful information would measure health care outcomes. One
comment was, "The number one thing people ask . . . is not . . .
. `Am I going to get that mammogram?' it's . . . . `Am I going
to die?'" However, they acknowledged that it is very difficult to
attribute outcomes to quality of care rather than to factors such as
the patient's health or lifestyle choices. They also said they want
standardized and comparable health care information to assess health
care providers' or health plans' performance equally.
Many of the employers we interviewed are getting some of the data
they want through business coalitions, consultants, and their own
data collection efforts. But these sources are not available to
individual consumers, and few employers were sharing these data with
their employees. One employer said, "I don't know if the data we'd
be giving them would be the complete picture." Although some
employers stated that their employees did not want or would not
understand data comparing quality, their employees told us such
information would be helpful.
BACKGROUND
------------------------------------------------------------ Letter :2
Employers have been the driving force behind the growing move to
compare health care providers and plans on the basis of their
performance. These employers have worked both individually and
collaboratively with providers, health plans, and government to
produce information that will allow them to assess the quality of the
care they purchase. Health plans have been publishing reports
comparing their performance to their peers or to a national standard.
State governments have published comparative information, often
focused on specific procedures performed in hospitals. Although the
federal government was responsible for the first widespread public
disclosure of hospital performance data in 1987, it discontinued this
practice in 1993. As a payer of health care services on behalf of
Medicare and Medicaid beneficiaries, the Health Care Financing
Administration (HCFA) lags behind others in making performance data
public.
Report cards can include a variety of performance indicators, either
structural, process, or outcome based. Structural indicators measure
the resources and organizational arrangements in place to deliver
care, such as the ratio of nurses to inpatient beds. Process
indicators measure the physician and other provider activities
carried out to deliver the care, such as the rates of childhood
immunization. Outcome indicators measure the results of the
physician and other provider activities, such as mortality,
morbidity, and customer satisfaction.
EMPLOYERS TOOK THE
INITIATIVE IN REPORT CARD
DEVELOPMENT
---------------------------------------------------------- Letter :2.1
In 1989, a group of employers initiated one of the most significant
efforts to identify uniform and standardized performance indicators.
This effort resulted in the creation of a performance measurement
system known as the Health Plan Employer Data and Information Set
(HEDIS). Several business coalitions and health care organizations
used the first HEDIS measures in 1991. The nonprofit National
Committee for Quality Assurance (NCQA) has led the effort to revise
the measures, issuing HEDIS 2.0 in 1993 and HEDIS 2.5 in 1995.
Current HEDIS measures focus on process indicators. (See table 1 for
a list of some key HEDIS measures.)
Table 1
Selected HEDIS Performance Measures
Quality/ Physician Membership/
access Utilization network finance
------------- ------------- ------------- -------------
Childhood Coronary Physician Membership
immunization bypass rate turnover disenrollment
Cholesterol Angioplasty Board Medical loss
screening rate certification ratio
Mammography Cardiac Administrativ
catheterizati e loss ratio
Pap smear on rate
Revenue
Prenatal care Hysterectomy requirements
visit rate per member
per month
Diabetic Prostatectomy
retinal exam rate Tier rates
Major Laminectomy
affective rate
disorder
follow-up Cesarean
section rate
Members
visiting Obstetrical
provider hospital
stay
Asthma
admission Readmission
rate for chemical
dependency
Low
birthweight Hospital
rate days/1,000
enrollees
----------------------------------------------------------
Source: NCQA Report Card Pilot Project Technical Report (Washington,
D.C.: NCQA, Feb. 1995).
Using HEDIS as a base, some employers have begun to distribute to
their employees educational materials that include outcome measures.
For example, the California Public Employees' Retirement System
(CalPERS) recently distributed to its employees a performance report
about the health plans it offers. Although it had furnished some
comparative information to its employees in previous years, the
information generally featured cost and benefits. CalPERS' May 1995
Health Plan Quality/Performance Report is its first effort at
distributing comprehensive information that includes both specific
quality performance indicators and member satisfaction survey
results. The quality performance data are based on HEDIS indicators
measuring health maintenance organizations' (HMO) success with
providing childhood immunizations, cholesterol screening, prenatal
care, cervical and breast cancer screening results, and diabetic eye
exams. Employee survey results include employee satisfaction with
physician care, hospital care, and the overall plan, and the results
of a question asking whether members would recommend the plan to a
fellow employee or friend.
Some employers are using third-party health care accrediting
organizations to measure health plan performance using structural
indicators. These employers are requiring the health plans they
contract with to be accredited by organizations such as NCQA and the
Joint Commission on Accreditation of Healthcare Organizations.
Furthermore, some accrediting agencies publicize their accreditation
decisions, which allows employers and individual consumers to
consider accreditation status in their health care purchasing
decisions. For example, a consortium of employers has elected to
exclude a Florida HMO from new business with its employer-sponsored
health plans because of the HMO's failure to obtain accreditation.
HEALTH PLANS MAKE
PERFORMANCE DATA PUBLIC
---------------------------------------------------------- Letter :2.2
Health plans have published comparative information intended to
assist individual consumers in their health care choices and health
care providers in their quality improvements. For example, in 1993,
Kaiser Permanente Northern California Region released a report on 102
performance measures divided into the following categories:
childhood health, maternal care, cardiovascular disease, cancer,
common surgical procedures, other adult health, and mental
health/substance abuse. (See fig. 1.) Although Kaiser was one of
the first health plans to publish this kind of information, an
increasing number of health plans are now providing similar
information.
Figure 1: Example of Health
Plan Presentation of
Comparative Health Care Data
(See figure in printed
edition.)
Source: Kaiser Permanente, Northern California Region, 1993 Quality
Report Card (Oakland, Calif.: Kaiser Permanente, 1993).
Health plans have been exploring new ways to make information readily
available and understandable to individual consumers. For example,
on September 15, 1995, HealthPartners, Inc., will initiate a
consumer-oriented program using touch-screen computers.\3 Initially,
at least 50 computers will be installed permanently at 50 employer
sites, and at least 100 computers will be rotated among other
employers. This will allow employees to obtain details about any one
of the plans' primary care sites, such as its physicians'
credentials, on-site services offered, and specialists to which its
physicians refer. Because health plan members are expected to enroll
in a specific care delivery system--a set of primary care sites with
affiliated specialists--HealthPartners will furnish data about each
care system to help plan members make a decision about which one to
join. Currently these data include preventive screening rates and
patient satisfaction measures. HealthPartners anticipates expanding
the availability of touch-screen computers to more public spaces,
such as shopping malls, after physician concerns about data
confidentiality and other matters are resolved.
--------------------
\3 HealthPartners, Inc., is the parent company of health care
organizations that include group and staff model HMOs located in
Minnesota's Twin Cities. They provide health care services and
coverage to more than 650,000 members.
STATE LEGISLATURES MANDATE
PUBLIC DISSEMINATION OF DATA
---------------------------------------------------------- Letter :2.3
The states have also been active in providing information about
provider performance to the public. Forty states have mandated the
collection, analysis, and public distribution of health care data,
such as hospital use, charges or cost of care, effectiveness of
health care, and performance of hospitals.\4 \5 For example,
Pennsylvania has released four report cards on the hospitals and
physicians in the state performing coronary artery bypass graft
surgery (CABG) since 1992. Providing both costs and mortality rates,
the reports are publicized through the local media and are available
free to consumers. (See fig. 2.)
Figure 2: Example of
Pennsylvania State Comparison
of Health Care Providers
(See figure in printed
edition.)
Source: Pennsylvania Health Care Cost Containment Council, A
Consumer Guide to Coronary Artery Bypass Graft Surgery, Vol. IV
(Harrisburg, PA: June 1995). Statistics were based on 1993 data.
--------------------
\4 The 10 states that have not mandated these activities are Alabama,
Alaska, Hawaii, Idaho, Louisiana, Michigan, Mississippi, Montana,
Nebraska, and Wyoming. Colorado's legislature eliminated funding for
its state data commission as of July 1995, and North Carolina's
program will cease in October 1995.
\5 A self-insured company that administers its own health plan may
not be under any obligation to report its performance. In
Employer-based Health Plans: Issues, Trends, and Challenges Posed by
ERISA (GAO/HEHS-95-167, July 25, 1995), we reported that the National
Governors' Association believed that the Employee Retirement Income
Security Act of 1974 (ERISA) prohibited their states from developing
standard data collection systems applicable to all health plans.
FEDERAL GOVERNMENT IS MOVING
SLOWLY
---------------------------------------------------------- Letter :2.4
In 1987, HCFA initially publically released hospital mortality
information, but did so only in response to a request under the
Freedom of Information Act (5 U.S.C. 552). The published
information, collected as part of HCFA's oversight efforts, included
the observed and expected mortality rates for Medicare beneficiaries
in each hospital that performed CABG surgery. HCFA published the
information annually until 1993, when the HCFA Administrator
discontinued the reports. He cited problems with the reliability of
HCFA's methods to adjust the data to account for the influence of
patient characteristics on the outcomes. HCFA has not published any
other information about the performance of Medicare providers.
HCFA's responsibility to Medicare beneficiaries in the selection and
oversight of Medicare contract HMOs is similar to that of employers
to their employees in selecting health plans. However, HCFA does not
routinely provide beneficiaries the results of its monitoring reviews
or other performance-related information such as HMO disenrollment
rates. In August 1995, we recommended that HCFA publish (1)
comparative performance data it collects on HMOs such as complaint
rates, disenrollment rates, and rates and outcomes of appeals and (2)
the results of its investigations or any findings of noncompliance by
HMOs.\6
Our recommendation that HCFA publish performance data was consistent
with the views of experts we interviewed about the federal
government's role in ensuring that Medicare beneficiaries receive
quality care. These experts cited the need for gathering health plan
information such as (1) performance measures, (2) patient
satisfaction, and (3) assurances that basic organizational standards
have been met. Furthermore, they believed that when the information
is obtained, it should be shared with beneficiaries to assist them in
their health care purchasing decisions.\7
Although HCFA has not been publishing data on Medicare providers, it
is collaborating with others to publish performance information about
Medicaid providers. HCFA has been participating with NCQA and the
American Public Welfare Association on behalf of the State Medicaid
Agencies Directors Group to tailor HEDIS to the particular needs of
state Medicaid agencies, health plans that serve Medicaid recipients,
and the recipients themselves. In July 1995 the work group released
the first draft of Medicaid HEDIS\8 and is expected to release a
final version of the document in Fall 1995 after considering comments
received.
--------------------
\6 Medicare: Increased HMO Oversight Could Improve Quality and
Access to Care (GAO/HEHS-95-155, Aug. 3, 1995).
\7 Medicare: Enhancing Health Care Quality Assurance
(GAO/T-HEHS-95-224, July 27, 1995).
\8 Draft Medicaid HEDIS: An Adaptation of the Health Plan Employer
Data and Information Set 2.0/2.4, NCQA (Washington, D.C.: NCQA, July
1995).
PUBLIC/PRIVATE PARTNERSHIPS
FORM TO PRODUCE AND
DISSEMINATE OUTCOME DATA
---------------------------------------------------------- Letter :2.5
Like HEDIS, many of the most recent initiatives to provide data
involve a partnership between private and public players. For
example, a more recent public/private initiative that includes some
of the major employers involved in developing HEDIS is the Foundation
for Accountability (FAcct), created in June 1995. At a meeting of
the Jackson Hole Group, some of the nation's largest employers and
HCFA, together representing more than 80 million people, or almost a
third of the U.S. population, agreed to combine their expertise and
purchasing power. This action grew out of employer frustration with
current performance data that focus on plan and provider structure
and process rather than outcomes of care. FAcct intends to recommend
measures of health care quality that can be easily understood by the
general public so that people can make informed decisions when
choosing a health plan. FAcct also hopes to encourage the common
adoption of these standards to establish uniformity and minimize
health plan reporting burdens as well as develop a means of educating
diverse audiences about the significance and applications of health
plan accountability.
LITTLE IS KNOWN ABOUT WHAT
INFORMATION IS NEEDED OR
WANTED
---------------------------------------------------------- Letter :2.6
Experts have noted that studies performed to determine how consumers
make decisions when no comparative information on quality has been
available may not be helpful in determining what information
consumers would actually use. Adding to the conclusions of numerous
researchers that individual consumers give more weight to information
from acquaintances than to expert opinion, researchers at Brandeis
University reported in 1994 that Massachusetts state employees they
surveyed valued information about quality but did not value report
card information. From this apparent contradiction, the researchers
concluded that survey respondents view quality as something other
than what is described in report cards.\9
In 1995, NCQA reported that almost all consumers participating in
focus groups NCQA sponsored stated that they would use better
evaluative information if it were available to them.\10 In addition,
when NCQA provided participants with sample report cards, NCQA noted
that in every group, participants were able to critically evaluate
the information, raising the same questions about the validity of the
data that experts debate.
In 1994, we reported that while performance measures or report cards
could be a useful tool to educate consumers about the health care
that plans provide, the report cards being developed may not reflect
the needs of some users.\11
Employers have been the primary users of information comparing
quality of care; little is known about the extent to which this
information is meeting individual consumers' needs.
The sections that follow discuss in more detail the results of our
efforts to determine, from the consumers' perspectives, the extent to
which they use quality of care information in making health care
choices and the types of information consumers find useful in
arriving at decisions.
--------------------
\9 Brandeis University: The Heller School, Consumer Information:
Decisive Factors in Health Plan Choice (Waltham, Mass.: Aug. 15,
1994).
\10 NCQA Consumer Information Project Focus Group Report: Executive
Summary (Washington, D.C.: NCQA, Spring 1995).
\11 Health Care Reform: "Report Cards" Are Useful But Significant
Issues Need to Be Addressed (GAO/HEHS-94-219, Sept. 29, 1994).
CONSUMERS WE INTERVIEWED USE
INFORMATION COMPARING QUALITY
IN HEALTH CARE DECISION-MAKING
------------------------------------------------------------ Letter :3
Many of the employers and individual consumers of health care we
talked with are increasingly using information that compares the
quality of care furnished by health care providers or health plans to
make purchasing decisions and to encourage providers and plans to
improve the quality of their care. However, some of those we
interviewed told us they are not using the information because they
are unaware that it exists, they have not been able to find it in
some markets, they believe the available information does not meet
their needs, or they lack the resources or time to find and use the
information. Further, they stated that the information would be more
useful if their concerns about the reliability and validity of the
information were addressed.
USE OF QUALITY INFORMATION
VARIES
---------------------------------------------------------- Letter :3.1
Most of the employers we spoke with were either actively seeking or
using information on quality or stated that they would use it if it
were available to help choose health plans or individual providers
for their employees. Many employers told us that because they
limited the employee's choice of provider by using an approach that
restricted or encouraged the use of specific providers, they felt a
greater need to ensure that they provided access to quality
providers. For example, the human resource manager of a midwestern
manufacturing firm told us that
"we'd like to get some kind of value-based decision for
purchasing health care. The pure pricing arrangements, the
deals . . . have not really been a complete answer for us.
Those arrangements don't address quality, and we're coming to
believe that that's got to be the cornerstone of your health
care plan."
Employers' use of the data varied considerably. Some of the larger
self-insured employers were using data to select individual providers
to include in their own network. Other self-insured employers
preferred to leave the selection of providers and quality assurance
functions up to either the HMO or the plan administrator with whom
they contracted. In the words of the benefits manager for a major
Northeast financial services employer,
"I think that they [HMOs] should be in the business of comparing
hospitals, picking out the high-quality, cost-effective
providers; that's what I'm paying them to do. I just want to
make sure they're doing it, and feel comfortable that they're
doing it."
These employers used the comparative data as a "red flag," signaling
a possible decline in quality. For example, one large southeastern
self-insured employer stated that he watched for trends in
performance measures that might serve as a warning that a problem was
developing.
Some smaller employers reported that they had neither the resources
nor the time to find or use report cards but wanted the information
to be available to the insurance agents or purchasing alliance staff
they relied on to make health insurance recommendations.
Employers told us that they are also using the data as a tool to
market a specific plan to their employees or to negotiate contract
terms with the insurance carriers. Numerous employers told us that
providing employees with data comparing quality of care was
particularly helpful in convincing their workers that managed care
plans do not compromise the quality of care provided. Employers
stated that they use the data to influence providers and plans to
improve quality. For example, one employer told us that during
contract negotiations, data were used comparing hospitals on specific
procedures, such as hysterectomies, to encourage hospitals to reduce
unnecessary surgeries.
The individual consumers we talked with in Pennsylvania, California,
and Minnesota who had requested and received specific report cards
generally used the information and found it to be very helpful in
making health care purchasing decisions. These consumers received
either (1) information about patient outcomes for physicians and
hospitals performing specific procedures or (2) information on a
specific plan.
More specifically, individual consumers in Pennsylvania and
California reported using the procedure-specific reports to
select the best surgeon or hospital because they or someone in
their family anticipated having the surgery described in the
report,
select the best surgeon or hospital for procedures other than those
described in the report,
review the ranking of the surgeon who had performed their surgery
before they had obtained the report,
ask more informed questions of their doctors,
increase their general knowledge,
provide advice to others, or
satisfy their curiosity.
Individual consumers using a plan-specific report card told us that
they used the information to select a health plan or to increase
their knowledge about the health plan chosen by their employer, such
as the services provided or the financial health of the plan.
Consumers using either a plan- or procedure-specific report card who
had no choice of provider reported that the information gave them
reassurance.
Although most individual consumers we interviewed found the report
card helpful, some did not. Some consumers reported that they did
not use the information because it focused on one procedure or health
plan, or because it was limited to a specific state or area. Other
consumers told us that they were unaware the information existed
until after they or a family member no longer needed it. For
example, a Pennsylvania woman stated that she wished she had known
about this information before her mother died after heart surgery,
because it might have helped her select a provider.
CONSUMERS' CONCERNS ABOUT
COMPARATIVE DATA LIMIT
DATA'S USEFULNESS
---------------------------------------------------------- Letter :3.2
Both employers and individual consumers echoed many of the same
concerns expressed by health care experts and previously reported by
us that comparative information may not be measuring what it is
intended to measure. Experts have varying beliefs about what
information should be included in a report card because of
acknowledged difficulties with the reliability and validity of data
sources and systems designed to measure quality. Areas of concern
for purchasers we interviewed focused on risk adjustment, age of
data, subjectivity, and bias.
More specifically, consumers, both corporate and individual,
questioned whether procedure-specific data were properly adjusted to
account for differences in patient characteristics that might
contribute to adverse outcomes. They were skeptical about whether
factors such as age, severity of condition, and functional status\12
could be accounted for to ensure that outcomes were an accurate
reflection of provider quality. We have reported previously that
severity-adjusted performance measurement systems are in a relatively
early stage of development and may not provide information for
accurately comparing hospitals' performance. We concluded that
additional information and methodological improvements are needed to
provide more useful data on which to base purchasing decisions.\13
Numerous individual consumers commented that the report card data
they had received were too old to accurately reflect current provider
or plan performance. For example, a consumer using plan-specific
information stated that the information was not helpful because it
was already 2 years old when published. Another commented that even
in a short time, cost data can become outdated. A consumer using
health-plan specific information told us that
"they [the report cards] are to reassure the public, but they
can't be used to make health care decisions because they are too
general and outdated from the time the data was gathered until
the decision is made."
Some consumers stated that selecting a health care provider is a
subjective decision that is difficult to quantify. In the words of a
Pennsylvania consumer, while the report card was a good publication,
"it is limited by trying to objectify something that will always be
subjective." For example, consumers differ in what they want from a
provider. Some consumers mentioned that it is more important for
some patients to feel at ease with their doctor than it is for
others. Although many consumers stated that they wanted information
on customer satisfaction, others felt it was of limited value because
"just because you're happy with your doctor doesn't mean I would be
happy with him or her." Another individual consumer questioned a
patient's ability to assess a doctor's medical knowledge, technical
skills, and ability.
Some employers explained that the subjective nature of the health
care purchasing decision results in their reluctance to use
quantitative data to select providers for their employees. As
expressed by a representative of a major East Coast manufacturer:
"Quality is in the eyes of the beholder . . . . It is not
appropriate for the employer to place a value on one outcome
over another. It is up to the patient to place that value. Is
it more important that I be alive but it's okay if I'm hurt or
I'd rather die than be [incapacitated]?"
A representative of a medium-sized manufacturing firm stated that
"you know I think that a big part of the problem, and we're
guilty of it too, is imposing our own tastes or beliefs on other
people . . . . In health care we do a lot of deciding of
what's good for people on the basis of our own beliefs, and the
issues that [concern] a $9 an hour person are not the same ones
that I'm contending with . . . . The highly paid person may
not have any problem in going out of network--may be able to
afford to go to Mayo's [the Mayo clinic] and decide, `hey,
that's where I'm going with this problem. I'm not going to
stick around [city deleted].' Whereas somebody on the shop floor
has got to stay in [city deleted]."
Individual consumers questioned the objectivity of the health care
data produced and distributed by the provider or plan. Many
consumers stated they would be less likely to believe the information
if it is gathered and reported by the provider or plan rather than an
independent third party. For example, one individual stated that "an
unscrupulous provider could make sure they hit home runs on all of
these particular items [the quality measurements] . . . ."
Individual consumers who requested and received report card
information from health plans used terms such as "self-serving,"
"one-sided," and "nontrustworthy" to describe the report. These
respondents saw the purpose of the reports as a provider's public
relations effort to "blow its own horn" or use the report as a
"marketing tool" rather than to provide information to the consumer.
--------------------
\12 "Functional status" is the extent to which people are able to
perform activities of daily living and their basic social roles.
\13 Health Care: Employers Urge Hospitals to Battle Costs Using
Performance Data Systems (GAO/HEHS-95-1, Oct. 3, 1994).
CONSUMERS WANT MORE INFORMATION
------------------------------------------------------------ Letter :4
Consumers we interviewed want more information than they currently
have. Both employers and individual consumers want information that
emphasizes outcomes rather than process or structure measures of
quality. They want standardized information that allows them to
compare providers and plans. Few employers we interviewed are
sharing unpublished data with employees, and they differ from one
another on whether or not they believe their employees would use it
to make decisions. Individual consumers generally stated that they
wanted reports on quality to make decisions, but many emphasized that
such reports would never be the sole source of information; they
would only augment the advice of others.
CONSUMERS WANT OUTCOMES MORE
THAN OTHER KINDS OF MEASURES
---------------------------------------------------------- Letter :4.1
When emphasizing that they want information on the outcome of health
care provided, consumers are asking for a measure that allows them to
select providers who will improve their health status or that of
their employees. For example, in describing the need for outcome
data, one employer stated that rather than just knowing how many
women received mammography screening for breast cancer, he wanted to
know if the number of women who died or were incapacitated from
breast cancer was being reduced.
A major northeastern food manufacturer used outcomes to relate
quality assurance in health care to its manufacturing quality
assurance program to explain that "outcome data . . . is the only
way to measure quality . . . . Once you have the outcome, you can
go back and look at the processes themselves."
A large West Coast employer stated that what the company really wants
is information on health status.
"What we'd like as a measure is we'd like to know that the plan
has improved the health status of the population served . . .
. That might be different for some subpopulations. So, [we
would like to see reports] moving much more to population-based
approaches."
A medium-sized manufacturer stated that
"in general, you're looking for quality and you're looking for
value, so maybe [we need] more of a functional analysis. There
is some subjective information that needs to be obtained along
with the length of stay and cost of stay and some of these other
factors that we're just not getting yet . . . . You need to
do a kind of functional analysis as well, to say 30 days after
that angioplasty was that patient back at work, and were they
working 40 hours per week, and were they doing their job . . .
. How's your quality of life after you've had this?"
A large northeastern financial services firm said
"the number one thing people ask . . . when they're
considering an HMO . . . is not like gee, `Am I going to get
that mammogram,' it's `What if I get sick, am I going to die,
are they going to take care of me?'"
Both employers and individual consumers stated that although data
reflecting the outcomes may be the best measure of provider quality,
it is very difficult to know whether outcomes result from quality of
care or factors such as the patient's condition or lifestyle
choices.\14
--------------------
\14 Some experts estimate that it will take 10 to 15 years and
millions of dollars to develop a technically sophisticated
measurement system that is able to accurately attribute outcomes to
care.
DATA MUST BE STANDARDIZED
AND EASILY COMPARABLE
---------------------------------------------------------- Letter :4.2
Both employers and individual consumers told us that they want
standardized data that could be used to compare health care
providers' and health plans' performances. Though noting that
efforts such as HEDIS exist, employers told us that these measurement
systems are still in the developmental phase. They also said that
without standardization, such as in definitions of disease or
methodology for analyzing data, this information is not comparable
regionally or nationally. Numerous employers were participating in
standardization efforts such as the Midwest Business Group on
Health's efforts to standardize customer satisfaction surveys. Many
of the larger employers we interviewed hire consulting firms that
provide them with some level of standardization and comparability.
An official at a medium-sized manufacturing firm stated that
"the government should prescribe some standards and force
providers to adhere to these standards in the publishing of
information. The government should say, `You're going to code
this disease this way, and you do it consistently and uniformly
. . . .'"
Individual consumers stated that the way the information was
presented was very important to them. For example, some wanted to
have providers or plans compared side-by-side on one or two pages.
Consumers using the procedure-specific reports uniformly praised the
table format that provided this kind of direct comparison. (See fig.
2 for an example of a table providing a comparison of providers from
a procedure-specific report.)
Some individual consumers wanted the information to cover a wider
geographic area, and others emphasized the need for
community-specific data. For example, some Pennsylvania consumers
stated that the report card for that state pertained only to
providers in Pennsylvania. Consumers living in Philadelphia would
like to have had this type of information for surrounding states
because their providers, while close to their homes, were located in
other states. The same concern came up in a midwestern city that
bordered two states.
EMPLOYERS HAVE SOURCES OF
DATA NOT AVAILABLE TO
EMPLOYEES
---------------------------------------------------------- Letter :4.3
Many employers are getting some of the quality of care data they want
through business coalitions, consultants, and their own data
collection efforts. This information is generally unavailable to the
individual consumer, and few employers we interviewed were sharing
these data with their employees. Employers differed in their opinion
of whether or not their employees would use these data in making
health care choices. Some employers stated that they do, or would,
share information on quality with their employees because such
information would help their employees make informed decisions. For
example, a midwestern service employer stated that it would be
important to
"hav[e] some report card concepts that the employees could
understand the information, user friendly . . . consistent .
. . . I want to have a tool for the employees to make that
decision. If the employees are making that decision, they are
going to change the marketplace. They are going to improve the
quality of the system because the doctors and the hospitals are
going to have to alter their practices because of the
information that has been gathered and is presented and
understood by employees. They are then making intelligent
decisions as far as where to get their health care . . . .
Empower the people to make the decision."
A major employer located in the Northeast stated that
"if we are going to have value-based purchasing which would
drive a competitive marketplace in health care, we have to
involve consumers who make the ultimate choice. Therefore the
information has to be relevant for them."
Other employers believed that their employees did not want or would
not understand data comparing quality. A large East Coast
manufacturer stated that
"I think it's not speaking to how they make decisions. I think
we'd overwhelm them . . . . Also, I don't know if the data
we'd be giving them would be the complete picture."
Another large East Coast manufacturer stated that
"I've been in health care benefits for 15 years. I don't know
how to make the choice. What happens to poor Harry the Huffer
working on the shop floor when you give him . . . the
morbidity in this hospital is here, and you know the readmission
rate is this, and the reinfection rate is this, and the guy
says, `I don't know what I should do.' Because what they do to
our counselors is say, `I don't want to make choices.'"
Nevertheless, employees we interviewed disagreed with those employers
who said that employees would not use the information. Employees'
concerns included issues of validity and reliability such as risk
assessment and accuracy rather than their ability to understand the
data. Most of the individual consumers who had requested the
published reports found them to be easy to understand, using terms
such as "clear," "concise," and "well organized." They found the
charts and tables particularly useful. For those who had some
problems understanding the reports, additional assistance was useful.
For example, a Pennsylvania consumer who had been unable to fully
understand the published report on her own had no trouble after it
was explained by the state agency that had produced the report.
Another Pennsylvania consumer stated that the first report card she
received was difficult to understand but that by the third report she
received, she found it very useful.
Many individual consumers emphasized that published information would
never be the sole source of data for their health care decisions but
would be used in addition to other information such as personal
consultation with their physician, friends, family members, or
coworkers.
CONCLUSION
------------------------------------------------------------ Letter :5
Data comparing health care plans and providers helped the consumers
we interviewed make their health care purchasing decisions. However,
performance reports have not yet achieved their fullest potential.
Consumers said they needed more reliable and valid data, more readily
available and standardized information, and a greater emphasis on
outcome measures.
Meeting the information needs of individual consumers continues to
lag behind meeting the employer needs. Attention must be paid to
ensuring that individual consumers have access to health care data.
While employers themselves have initiated efforts to cooperate with
one another, few we interviewed are making complete health care data
available to assist individual consumers in making purchasing
decisions. Relevant stakeholders have not yet addressed the issues
of disseminating performance data to individual consumers so that
they can make responsive, informed decisions about their health care
coverage.
---------------------------------------------------------- Letter :5.1
We are sending copies of this report to interested congressional
committees and other interested parties. We will make copies
available to others on request.
This report was prepared under the direction of Carlotta C. Joyner,
Associate Director. Other major contributors to this report include
Sandra K. Isaacson, Assistant Director; Susan Lawes; Lise Levie;
Lesia Mandzia; and Janice Raynor. Please call me on (202) 512-6806,
or Dr. Joyner on (202) 512-7002, if you have any questions.
Sincerely yours,
Janet L. Shikles
Assistant Comptroller General
SCOPE AND METHODOLOGY
=========================================================== Appendix I
To obtain information on how consumers use data comparing the quality
of health care providers or health plans and what information they
want when making health care purchasing decisions, we interviewed
both employers and individual consumers. To obtain the view of
employers, we interviewed officials at over 60 businesses. The size
of these businesses ranged from under 5 employees to over 100,000
employees. These employers were selected on the basis of the
following criteria: (1) size of workforce (small, medium, and
large); (2) geographic variability; and (3) variation in whether or
not they used published report cards. "Small" employers were defined
as those with fewer than 50 employees, "medium" as having 50 to 499,
and "large" as 500 or more employees. Because the businesses were
not randomly selected, their experiences and opinions cannot be
generalized to all employers. We also interviewed a major private
sector management consulting firm that supplies comparative health
care data to employers.
To obtain the views of individual consumers who had received a report
card, we conducted telephone interviews during January, February, and
March 1995 with 153 consumers who had requested and received
published report cards to determine how they used the information
(see table I.1). The report cards they received were published by
either California or Pennsylvania state agencies or by health
maintenance organizations in California and Minnesota. These report
cards were selected because they were the most recently available in
which the issuing entity had a record of requesters and the state or
HMO was willing to assist in the study. The consumers we talked with
had received this information sometime during 1993 and 1994. The
reports published by the state agencies contained only
procedure-specific indicators, while the health plan reports focused
on various plan and procedure quality indicators related to the
individual health plans.
Table I.1
Summary of Telephone Survey With
Individual Consumers
Number of Number of
individuals who individual
requested report consumers we
State or health plan card information interviewed\a
---------------------- ---------------- ----------------
Pennsylvania 633 120
California 43 6
Medica 338 22
Kaiser Permanente 73 5
(HMO)
==========================================================
Total 1,087 153
----------------------------------------------------------
\a These do not include those consumers who requested the report
solely for their work, those who did not receive or could not recall
the details of the report, or those who chose not to or were unable
to participate.
Although we attempted to contact all 1,087 individuals who had
requested the report cards issued by those states or health plans,
many of these individuals did not choose to participate, could not
recall receiving the information, or had requested the information
for reasons other than making health care purchasing decisions, such
as for school or work. Because we spoke with only a small number of
individuals who had requested information for consumer-related
purposes and they were not chosen at random, their experiences and
opinions cannot be generalized to the entire consumer population that
requested report card information.
We also conducted interviews with seven groups of employees around
the country who may not have had previous experience with such
reports. We conducted these interviews with employees from four
federal government agencies and three private
corporations--manufacturing, sales, and service. These employees
were selected because their employers offered them more than one
health insurance plan to choose from when making their health care
insurance purchasing decisions. The number of participants in each
group ranged from 8 to 10 and included employees with varying
marital, family, and age status as well as employees enrolled in both
indemnity and managed care plans.
RELATED GAO PRODUCTS
============================================================ Chapter 0
Medicare: Increased HMO Oversight Could Improve Quality and Access
to Care (GAO/HEHS-95-155, Aug. 3, 1995).
Employer-Based Health Plans: Issues, Trends, and Challenges Posed by
ERISA (GAO/HEHS-95-167, July 25, 1995).
Medicare: Enhancing Health Care Quality Assurance
(GAO/T-HEHS-95-224, July 27, 1995).
Health Care: Employers Urge Hospitals to Battle Costs Using
Performance Data Systems (GAO/HEHS-95-1, Oct. 3, 1994).
Health Care Reform: "Report Cards" Are Useful but Significant Issues
Need to Be Addressed (GAO/HEHS-94-219, Sept. 29, 1994).
Access to Health Insurance: Public and Private Employers' Experience
With Purchasing Cooperatives (GAO/HEHS-94-142, May 31, 1994).
Managed Health Care: Effect on Employers' Cost Difficult to Measure
(GAO/HRD-94-3, Oct. 19, 1993).