Physician Performance: Report Cards Under Development but Challenges
Remain (Letter Report, 09/30/1999, GAO/HEHS-99-178).

Pursuant to a congressional request, GAO reviewed efforts to provide the
public with information on physician performance, focusing on: (1) the
issues involved in measuring and reporting on physician and physician
group performance; (2) efforts to develop physician report cards; and
(3) initiatives under way that may address impediments to measuring
physician and physician group performance.

GAO noted that: (1) measuring performance in health care is challenging
in terms of identifying measures that truly reflect the quality of care
individuals receive; (2) it is also difficult to make comparisons across
plans or providers that account for differences in the patients whom
they treat that can affect health care outcomes; (3) measuring the
performance of physician groups and individual physicians is even more
difficult; (4) individual physicians or groups perform a wide variety of
services and typically perform any individual service for a small number
of patients; (5) only a fraction of these services can be clearly linked
to a measurable outcome; (6) to make meaningful comparisons among
physicians, analysts must adjust any measure selected to take into
consideration the extent to which a characteristic like the severity of
a medical condition affects the outcomes from care; (7) to avoid these
difficulties, approaches to performance measurement generally focus on
physician groups instead of individual physicians, and they measure
processes such as whether services are provided in accordance with
agreed upon norms rather than outcomes of care; (8) adding to the
challenges, however, are concerns that consumers have regarding the
privacy of their personal medical information and that physicians have
regarding the accuracy of performance measurement data; (9) even though
the data and measures that are available are limited, several different
private and public organizations have developed physician and physician
group report cards using these data and measures; (10) in New York and
Pennsylvania, state agencies that have reported on the performance of
individual cardiac surgeons since the early 1990s have reported improved
performance scores since they began publishing them; (11) while
significant, these efforts at physician report cards are in their early
stages or are limited in scope, and difficulties remain; (12) some
organizations are collaborating to develop more comprehensive,
standardized performance measures and to facilitate the exchange of
clinical and administrative data between physicians, plans, and
purchasers; and (13) at the federal level, the Department of Health and
Human Services is working on a performance measurement system for its
Medicare fee-for-service program and has been supporting research and
working with other organizations to develop physician performance
measures.

--------------------------- Indexing Terms -----------------------------

 REPORTNUM:  HEHS-99-178
     TITLE:  Physician Performance: Report Cards Under Development but
	     Challenges Remain
      DATE:  09/30/1999
   SUBJECT:  Health care programs
	     Physicians
	     Health care services
	     Consumer education
	     Health maintenance organizations
	     Surveys
	     Performance measures
IDENTIFIER:  American Medical Accreditation Program
	     Medicare Fee-for-Service Program

******************************************************************
** This file contains an ASCII representation of the text of a  **
** GAO report.  Delineations within the text indicating chapter **
** titles, headings, and bullets are preserved.  Major          **
** divisions and subdivisions of the text, such as Chapters,    **
** Sections, and Appendixes, are identified by double and       **
** single lines.  The numbers on the right end of these lines   **
** indicate the position of each of the subsections in the      **
** document outline.  These numbers do NOT correspond with the  **
** page numbers of the printed product.                         **
**                                                              **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced.  Tables are included, but    **
** may not resemble those in the printed version.               **
**                                                              **
** Please see the PDF (Portable Document Format) file, when     **
** available, for a complete electronic file of the printed     **
** document's contents.                                         **
**                                                              **
** A printed copy of this report may be obtained from the GAO   **
** Document Distribution Center.  For further details, please   **
** send an e-mail message to:                                   **
**                                                              **
**                                            **
**                                                              **
** with the message 'info' in the body.                         **
******************************************************************

Cover
================================================================ COVER

Report to Congressional Requesters

September 1999

PHYSICIAN PERFORMANCE - REPORT
CARDS UNDER DEVELOPMENT BUT
CHALLENGES REMAIN

GAO/HEHS-99-178

Physician Report Cards

(108386)

Abbreviations
=============================================================== ABBREV

  AHCPR - Agency for Health Care Policy and Research
  BBA - Balanced Budget Act of 1997
  BHCAG - Buyers Health Care Action Group
  CABG - coronary artery bypass graft
  HCFA - Health Care Financing Administration
  HEDIS - Health Plan Employer Data and Information Set
  HHS - Department of Health and Human Services
  HIPAA - Health Insurance Portability and Accountability Act of 1996
  HMO - health maintenance organization
  IPA - independent practice association
  NCQA - National Committee on Quality Assurance
  PBGH - Pacific Business Group on Health
  PPO - preferred provider organization

Letter
=============================================================== LETTER

B-281938

September 30, 1999

The Honorable James M.  Jeffords
Chairman
Committee on Health, Education, Labor, and Pensions
United States Senate

The Honorable Joseph I.  Lieberman
United States Senate

While information has become available comparing health plan
benefits, costs, customer satisfaction, and quality, it is not clear
that this information is meeting consumers' needs or expectations. 
With plans having overlapping panels of physicians and hospitals, it
has become increasingly difficult for consumers to differentiate
plans.  Moreover, few consumers receive information on which to base
one of their biggest health care decisionstheir choice of doctors. 
Recognizing this, some organizations are attempting to measure and
report on the performance of physicians and physician groupswith the
hope that the results can be used to compare the quality of their
care and services.  How well physician and physician group
performance measures assist consumers to make choices and how well
they drive improvements in the health care market is unknown. 
Because of the growing interest in promoting informed health care
decisions through public dissemination of performance information,
you asked us to examine (1) the issues involved in measuring and
reporting on physician and physician group performance, (2) current
efforts to develop physician report cards, and (3) initiatives under
way that may address impediments to measuring physician and physician
group performance. 

To meet your request, we interviewed officials of purchasing groups,
health plans, accreditation agencies, and federal programs; experts
in health care performance measurement; and representatives from
organizations that have formed to advance performance measurement. 
We visited three large health care purchasers--the Pacific Business
Group on Health (PBGH), the Buyers Health Care Action Group (BHCAG),
and the Health Care Financing Administration (HCFA)--and we
interviewed two private health plans that publish physician group
report cards--PacifiCare and Health Net--to discuss their efforts to
measure and report on physician performance.  We also reviewed report
cards on cardiac surgeons issued by New York and Pennsylvania state
agencies.  We based our selection of purchaser groups and state
initiatives on their reputations as innovators in the area of
consumer health care information.  The health plans we selected have
publicly reported the results of their comparisons of physicians in
their networks.  We performed our work from January 1999 through
August 1999 in accordance with generally accepted government auditing
standards. 

   RESULTS IN BRIEF
------------------------------------------------------------ Letter :1

Measuring performance in health care is challenging in terms of
identifying measures that truly reflect the quality of care
individuals receive.  It is also difficult to make comparisons across
plans or providers that account for differences in the patients whom
they treat that can affect health care outcomes.  Measuring the
performance of physician groups and individual physicians is even
more difficult.  Individual physicians or groups perform a wide
variety of services and typically perform any individual service for
a small number of patients.  Only a fraction of these services can be
clearly linked to a measurable outcome.  To make meaningful
comparisons among physicians, analysts must adjust any measure
selected to take into consideration the extent to which a
characteristic like the severity of a medical condition affects the
outcomes from care.  To avoid these difficulties, current approaches
to performance measurement generally focus on physician groups
instead of individual physicians, and they measure processes such as
whether services are provided in accordance with agreed-upon norms
rather than outcomes of care.  Adding to the challenges, however, are
concerns that consumers have regarding the privacy of their personal
medical information and that physicians have regarding the accuracy
of performance measurement data. 

Even though the data and measures that are currently available are
limited, several different private and public organizations have
developed physician and physician group report cards using these data
and measures.  For example, two purchasing groups and two California
health plans are avoiding some problems associated with measuring the
performance of individual physicians (such as small sample sizes) by
reporting on the performance of physician groups.  In addition, in
New York and Pennsylvania, state agencies that have reported on the
performance of individual cardiac surgeons since the early 1990s have
reported improved performance scores since they began publishing
them.  While significant, these efforts at physician report cards are
in their early stages or are limited in scope, and difficulties
remain.  For example, medical group report cards provide information
that is closer to the level of the individual physician than health
plan report cards do but, depending on the size of the medical group,
may not be very helpful for making an informed choice of a physician. 
In addition, questions about the accuracy and completeness of the
data and the adequacy of the risk adjustment methodology limit
consumer and physician confidence in the report cards. 

Some organizations are collaborating to develop more comprehensive,
standardized performance measures and to facilitate the exchange of
clinical and administrative data between physicians, plans, and
purchasers.  For example, several national accreditation
organizations have formed a council to develop common performance
measures.  At the federal level, the Department of Health and Human
Services (HHS) is working on a performance measurement system for its
Medicare fee-for-service program and has been supporting research and
working with other organizations to develop physician performance
measures.  In addition, HHS is establishing standards for
administrative claims and encounter data as well as unique
identifiers for individuals, plans, and providers--efforts that
should help HHS and others in their performance measurement efforts. 

   BACKGROUND
------------------------------------------------------------ Letter :2

To date, most performance information has provided only data on
health plans as a whole.  Changes in the health care
market--particularly the growth in the size of plans, the shifting of
greater financial risk to physicians or physician groups, and the
requirement in some cases that beneficiaries receive all their care
from selected physicians within a plan--have made plan comparisons
less useful for many consumers.  Many consumers do not get to choose
their health plan and even for consumers who can choose among plans,
an individual physician's performance may deviate greatly from the
health plan's average.  These and other factors have prompted calls
for physician report cards that can help consumers select physicians
from those available within their health plan. 

Report cards are generally publicly released reports on the quality
of care that provide comparative information on plan characteristics
and performance.  One widespread report card for health plans is
prepared by the National Committee on Quality Assurance (NCQA).  NCQA
uses its Health Plan Employer Data and Information Set (HEDIS) to
report on plan performance.  HEDIS includes more than 60 performance
indicators covering quality, access to and satisfaction with care,
membership and use of services, finance, and management. 

When the development of health plan report cards began, plans were
expected to differ significantly in their provider networks,
organizational structure, and philosophical orientation, and the
differences were expected to be reflected in the overall quality of
the plans.  But the marketplace has not evolved this way.  Instead,
to attract members and gain market share, health plans began building
larger, often overlapping networks that offer consumers more provider
choices than previously available.  With the same providers
represented in two or more competing plans, it has become
increasingly difficult to differentiate between plans.  Thus, even
for consumers who have a choice of plans, the comparative plan
information currently available may not demonstrate differences in
plan performance. 

Physician practices are also undergoing significant change; more
physicians are joining medical groups, and these medical groups are
contracting with many health plans.  The proportion of physicians in
group practices rose from approximately 11 percent in 1965 to 34
percent in 1995.\1 In addition, according to the Medical Group
Management Association, its members contract with an average of 21
health maintenance organizations (HMO) and preferred provider
organizations (PPO).\2 As physician groups contract with more plans,
individual plans may have less influence over physician practice
patterns and the quality of services they provide, because any one
plan may account for only a small percentage of a medical group's
total volume of patients or income. 

In addition, a growing number of physicians receive capitated
paymenta fixed monthly payment per patientunder which they accept
financial risk for providing a portion of or all patient care
services.  As plans shift more financial risk to physician groups, a
group's economic incentive is to minimize expensive services for sick
patients.  In a 1996-97 survey, more than half of physicians (54
percent) reported that their practices received capitation for some
of their patients.\3 In locations such as Seattle, Washington, and
Orange County, California, nearly three-fourths of physicians
reported receiving capitation for some of their patients.  To the
extent that physician assumption of financial risk affects quality of
care, this trend further shifts the focus on quality from plans to
physicians. 

For most employed Americans, their employer determines the number and
type of health insurance plans available to them.  For workers whose
employers do not offer a choice of plans, report cards that compare
plans have no utility.  A 1997 survey conducted by the Research
Triangle Institute found that less than one in five--17 percent--of
private employers that offered insurance to their employees provided
a choice among plans.\4

Another study reported that of employers that offer health insurance,
92 percent of small firms and 44 percent of larger firms (those with
more than 200 employees) offered only one plan in 1998.\5 Counting
employees rather than employers, less than half--only 41 percent--of
employees who are offered health insurance can choose from two or
more health plans.\6

Health plan report cards may also be of little use to more than 14
million of the country's 40 million Medicare beneficiariesthose who
did not have a choice of managed care plans in 1998.\7

--------------------
\1 Henry J.  Kaiser Family Foundation, Trends and Indicators in the
Changing Health Care Market Place (Menlo Park, Calif.:  Aug.  1998). 

\2 The Medical Group Management Association is a national
professional and trade association.  It represents administrators of
7,491 medical group practices that included 181,974 physicians in
1997.  PPOs are similar to fee-for-service plans but provide
enrollees a financial incentive--lower cost sharing--to receive care
from a network of providers that are normally reimbursed at a
discounted fee-for-service rate. 

\3 Center for Studying Health System Change, Data Bulletin:  Results
from the Community Tracking Study (Washington, D.C.:  Fall 1997). 

\4 The Robert Wood Johnson Foundation, 1997 Employer Health Insurance
Survey (Princeton, N.J.:  1997). 

\5 Henry J.  Kaiser Family Foundation, Health Benefits of Small
Employers in 1998 (Menlo Park, Calif.:  Feb.  1999), p.  18. 

\6 Robert Wood Johnson Foundation. 

\7 In 1998, 4 million Medicare beneficiaries had only one managed
care plan available in their county and 10.6 million beneficiaries
lived in counties with no plan at all.  See Medicare Managed Care
Plans:  Many Factors Contribute to Recent Withdrawals; Plan Interest
Continues (GAO/HEHS-99-91, Apr.  27, 1999). 

   DEVELOPING PHYSICIAN REPORT
   CARDS IS CHALLENGING
------------------------------------------------------------ Letter :3

The heterogeneity of health care makes performance measurement
challenging in terms of identifying measures that truly reflect the
quality of care that individuals receive.  Making valid comparisons
across plans or providers that ultimately account for patient
differences that affect outcomes is also difficult.  These challenges
are magnified in attempts to measure the performance of physician
groups and individual physicians.  For example, unlike plans that
have a large number of enrollees, individual groups or physicians
generally see a small number of patients with specific conditions. 
These attempts are further complicated by a concern that consumers
and physicians have regarding the use of performance measurement data
in the first place. 

      SELECTING APPROPRIATE
      MEASURES IS DIFFICULT
---------------------------------------------------------- Letter :3.1

Medicine involves a wide variety of services, only a portion of which
can be clearly linked to health outcomes.  Health outcomes are also
influenced by factors such as a patient's age, medical history, and
heredity.  In addition, most individual physicians perform any
specific service a relatively few times in a given year, making it
more difficult to adjust for differences in patients and creating
other statistical problems.  Efforts to create report cards on health
plans and providers are also complicated by the different needs of
consumers.  Current attempts to resolve the difficulties of physician
performance measurement include aggregating physicians into groups;
focusing on certain types of physicians, such as cardiac surgeons,
who perform highly specialized services; and substituting measures of
process for measures of outcome. 

To make meaningful comparisons among physicians, analysts must select
measures while taking into account factors that can affect the
outcomes of care, such as a patient's medical history.  For example,
whether the patient is treated for a first or second heart attack
affects the likelihood of a successful outcome from cardiac surgery. 
Measures that did not account for differences in such factors would
penalize physicians who treat the sickest patients.  Without proper
adjustment in the measures, physicians might choose to avoid
high-risk patients in order to maintain higher performance scores.\8
Today, such adjustments are limited, and it may never be possible to
account for every characteristic of patients that could affect their
health outcomes.\9

Another challenge in performance measurement is that different
consumers have different information needs that are not likely to be
adequately met all in a single physician report card.  Consumers
prefer performance information that matches their own medical
conditions and situations.  For the majority of consumers who are
generally in good health, clinical quality indicators may not be as
relevant as service quality indicators, such as the waiting times for
an appointment.  But for individuals with chronic ill health, those
who use physician services the most, clinical quality measures may be
more critical.  It may not be possible to measure and report on
physicians at the level of detail that is meaningful and useful to
all consumers. 

Current performance measurement practices make an effort to detect
physician and physician group practice differences, report on a
variety of indicators, and generally avoid the need to identify
detailed patient characteristics.  One approach to solving the
measurement problems associated with individual physicians' seeing
only a small number of patients with a given condition is to focus on
physicians organized into groups, so that the number of patients with
a given medical condition is high enough to provide meaningful data. 

Most quality indicators in use today focus on process measures, such
as the percentage of women older than 50 in a plan who receive
screening mammograms.  Many of the measures NCQA uses for health
plans in HEDIS are process measures.  Measures of medical care
process are popular in part because the data required are obtained
relatively easily from administrative databases.  Furthermore,
process measures, particularly for preventive services, avoid the
difficulties inherent in trying to adjust the results for differences
in patient characteristics.  But process measures have many
shortcomings:  They capture a very limited range of medical services,
they tend to measure whether a service was provided when it was
called for but not how well it was done, and they focus heavily on
preventive care services because the universe of patients who should
be receiving them is most easily identified.  Outcome measures, those
that indicate whether a patient's health improved after care,
generally remain elusive. 

--------------------
\8 The results of a recent study on physician profiles for patients
with diabetes suggests that physicians might refuse to care for sick
patients, those who have failed therapy or those who do not adhere to
treatment plans, in order to improve their profile scores.  See
Timothy P.  Hofer and others, The Unreliability of Individual
Physician Report Cards' for Assessing the Cost and Quality of Care
of a Chronic Disease, Journal of the American Medical Association,
Vol.  281, No.  22 (1999), pp.  2098, 2104, and 2105. 

\9 President's Advisory Commission on Consumer Protection and Quality
in the Health Care Industry, Quality First:  Better Health Care for
All Americans, Final Report to the President of the United States
(Washington, D.C.:  1998). 

      ASSEMBLING PERFORMANCE DATA
      REQUIRES COOPERATION
---------------------------------------------------------- Letter :3.2

Creating a physician performance measurement system involves
collecting and verifying medical care data.  In order to collect the
data, the concerns of consumers and physicians regarding how the data
will be used must be addressed.  To ensure that measures of
performance are accurate, the data going into the measures must be
verified and free from manipulation. 

Physicians are concerned about the potential that inaccurate
performance scores will unfairly affect their practice.  Physicians
we interviewed told us that issues of data quality and the
appropriate attribution of performance scores to individual
physicians must be addressed before performance measurement data are
made public.  Although administrative records, such as claims for
payment, are readily available for the fee-for-service sector, they
often do not include all the information that performance measurement
requires, such as a patient's condition or the results of services
rendered, and for some indicators they are not collected because they
were created for billing purposes and not for performance
measurement.  Medical records provide much more complete information,
but their analysis is expensive because few records are automated. 
Gathering information through surveys falls somewhere between
administrative data and medical record review in terms of ease and
expense.  One limitation of ratings of consumer satisfaction is that
consumers cannot always tell if the care they received was
appropriate or technically good; research has not shown a consistent
relationship between consumer satisfaction and the technical quality
of care. 

Another concern for physicians is the method of determining which
physicians should be held accountable for specific actions or
outcomes.  Some physicians argue that it is difficult to fairly
attribute a change in health status to a particular physician because
many other factors come into play.  For example, some patients may
see a variety of physicians over the course of a year, each
potentially recommending or performing a needed service.  If a
patient has not received a particular service, which physician should
be held accountable for the omission?  Or a physician may have
recommended a very effective treatment to a patient, but that
patient's condition did not improve because he or she did not comply
with the physician's recommendation.  The question of attribution
becomes even more difficult as systems of health care become more
integrated and a team of providers rather than one physician is
responsible for patients' health care. 

Because physicians control the majority of the data necessary to
measure their performance, these concerns must be addressed if
measurement efforts are to be successful.  Physicians are responsible
for coding administrative data, whether they are for reimbursement
for claims or other data used by managed care plans.  Physicians also
maintain medical records for individual patients.  To automate these
records in order to make performance measurement data better would be
expensive, and it is unlikely that physicians or physician groups
will provide the resources for automation. 

Ensuring that the data that are collected are accurate is another
challenge.  In any measurement system, participants may manipulate
the data to improve their performance scores.  For example,
physicians could exaggerate the severity of patients' conditions to
ensure a more favorable rating.  Or they could simply avoid taking on
difficult cases in order to improve their success rates.  Preventing
this sort of manipulation requires activities such as auditing the
data by comparing them to medical records. 

Consumers are concerned about the privacy of their personal medical
information, and this concern may lead to rules that restrict efforts
to provide objective information on physician performance.  One
survey found that no more than about one-third of adults in the
United States trust health plans and government programs to maintain
confidentiality all or most of the time.\10 Meanwhile, consumers want
an unbiased, expert source of information about health care
quality.\11 State laws vary significantly, but in some states efforts
to protect the privacy of medical records could affect efforts to
ensure that reported measures are comparable and that the data are
not manipulated.  For example, in Minnesota, any release of a
patient's health records for research purposes requires, among other
things, that the provider attempt to acquire the patient's consent
and determine that individually identifiable records are necessary,
that the researcher's safeguards are adequate, and that the
researcher will not use the records for purposes other than those in
the original request without the patient's consent.  According to a
BHCAG official, Minnesota's state privacy laws forced the group to
abandon its attempts to collect HEDIS data from care systems and have
hampered efforts to obtain survey data regarding quality of care for
people with chronic conditions.  Finding the appropriate balance
between allowing access to medical records to ensure reliable,
unbiased information on health care quality and maintaining privacy
concerns is subject to considerable debate. 

--------------------
\10 California Healthcare Foundation, Medical Privacy and
Confidentiality Survey (Oakland, Calif.:  Jan.  28, 1999). 

\11 Consumers do not necessarily trust currently available
performance information.  Three of 10 Americans surveyed said that
information on quality of care from groups of doctors or state
medical societies had little credibility; 4 of 10 had little faith in
information from government agencies, and nearly 5 of 10 said that
information from newspapers, television, and other media was not very
believable.  See Agency for Health Care Policy and Research and Henry
J.  Kaiser Family Foundation, Americans as Health Care Consumers: 
The Role of Quality Information, Highlights of a National Survey
(Washington, D.C.:  1996). 

   REPORT CARDS ON PHYSICIANS AND
   PHYSICIAN GROUPS INDICATE
   PROGRESS BUT THEIR USEFULNESS
   REMAINS LIMITED
------------------------------------------------------------ Letter :4

Several private and public organizations are involved in a variety of
activities to measure and report on physician performance.  A
purchaser and two health plans in California and a purchaser in
Minnesota have worked on moving performance measurement down to the
level of the medical group or independent practice association (IPA)
or, in Minnesota, a care system.\12 In New York and Pennsylvania,
state agencies have published risk-adjusted mortality rates for
specific procedures performed by cardiac surgeons.  While the current
report cards demonstrate that some reporting is possible,
shortcomings in these physician and group report cards include the
doubtful value to consumers of scores for large medical groups,
questions about the quality and the expense of collecting the data on
which reporting is based, and inconsistencies among the report cards. 

--------------------
\12 A medical group is two or more doctors who work together to
provide medical services to patients.  Typically, doctors who work in
a medical group--both primary care doctors and specialists--share a
single office or several offices if the group is very large.  An
independent practice association is a network of individual
physicians who practice medicine by themselves or in small groups
(often composed of one type of doctor, such as pediatricians) and who
join together as an association to provide a range of primary and
specialty care services to patients.  Care systems began in 1997,
when the employer members of BHCAG began contracting with health care
providers directly rather than with health plans.  The providers
organized themselves into care systems, with some care systems
resembling multispecialty medical groups and others looking more like
independent practice associations. 

      PURCHASER AND HEALTH PLAN
      REPORT CARDS COMPARE
      PHYSICIAN GROUPS
---------------------------------------------------------- Letter :4.1

Several organizations have developed report cards for physician
groups.  PBGH, two California health plans--PacifiCare and Health
Net--and BHCAG in Minnesota have moved a step closer to reporting
physician performance by publishing report cards on medical groups
and IPAs. 

         PACIFIC BUSINESS GROUP ON
         HEALTH
-------------------------------------------------------- Letter :4.1.1

PBGH is a business coalition of 33 public and private purchasers of
health care representing more than 3 million employees, retirees, and
dependents.  As physician networks overlapped more and
differentiation in California health plans blurred, PBGH started
partnering with medical groups and IPAs on quality improvement
initiatives.  Together, they developed a publicly reportable
measurement tool called the Physician Value Check Survey.  In 1996,
the survey covered 49 California medical groups (and 9 from the
Pacific Northwest) that ranged in size from approximately 15,000
patients to more than 1 million patients.  Responses were obtained
from 31,000 patients.\13

PBGH chose a survey to collect data because it did not have the
resources to mount a full-scale medical record review, and the survey
was a less costly means of evaluating physician group performance and
obtaining information on the consumers' perspectives.  So that PBGH
could generalize the results to all patients seen by a medical group,
PBGH and its partners drew patient samples from each medical group's
entire patient population rather than just from PBGH members.  PBGH
used the survey results to publicly compare medical groups in several
areas:  a summary report card with measures such as overall
satisfaction, a preventive care services report card, and specific
report cards on care for high blood pressure and high cholesterol. 
For each report card, it classified the groups into three categories: 
above average, average, and below average.  (See the appendix for
details from the PBGH report card.)

--------------------
\13 According to PBGH's Director of Research, these 31,000
respondents represent more than 8 million enrollees in managed care
plans.  In 1996, the Physician Value Check Survey was sent to 1,000
patients between the ages of 18 and 70 randomly sampled in each
medical group.  The overall survey response rate was about 55
percent.  PBGH administered the Physician Value Check Survey again in
1998; however, the results from this survey were not available at the
time of our review.  PBGH officials said they plan to release the
1998 Physician Value Check Survey results on September 23, 1999, with
scores on the changes between 1996 and 1998that is, to see whether
for the same group of patients, the physician groups' performance
improved, worsened, or stayed the same over time. 

         PACIFICARE
-------------------------------------------------------- Letter :4.1.2

PacifiCare of California is an HMO that has since 1998 produced a
medical group report card called the Quality Index.  The publicly
reported Quality Index uses measures selected from PacifiCare's
internal provider profiles, which contain data on more than 60
clinical and service performance measures for its medical groups and
IPAs.\14 PacifiCare selected 14 of these measures for inclusion in
the Quality Index.\15 It based its selection on the preferences of
focus groups of consumers and the extent to which the physicians
could take actions that affected the measured activities. 

The information in the Quality Index is compiled from the health
plan's administrative databases, customer service department records,
and enrollee satisfaction surveys.  Thus, the Quality Index reflects
the health care experiences and opinions of only PacifiCare
enrollees.  The Quality Index includes measures of clinical
performance, service performance, enrollee satisfaction, and
administrative data submission.  In 1999, the Quality Index included
process measures such as eye examinations for people with diabetes
that were not included in the 1998 report.  The reported Quality
Index scores are percentile ranks for medical groups or IPAs compared
with all other groups.  Groups ranking in the top 10 percentile of a
measure are considered best practice groups for that measure. 
According to PacifiCare's Medical Director, improvement has occurred
in several areas, such as mammography screening rates and retinal
examinations for diabetics.  (See the appendix for details from the
PacifiCare Quality Index.)

--------------------
\14 PacifiCare profiles medical groups and IPAs that have about 500
or more PacifiCare enrollees.  To be included in the Quality Index
report, groups must have at least 1,000 PacifiCare commercial
enrollees and 500 Secure Horizons enrollees (its Medicare managed
care program).  Using this methodology, PacifiCare is able to report
on physician organizations that provide care to the majority of its
enrollees. 

\15 The March 1999 Quality Index included 28 measures14 for
PacifiCare commercial enrollees and 14 for Secure Horizons enrollees. 

         HEALTH NET
-------------------------------------------------------- Letter :4.1.3

Health Net is another managed care plan in California with more than
2.2 million enrollees.  Its Participating Physician Group Report Card
includes information for 131 medical groups in California, all of
which are under performance-based contracts.  In 1999, a percentage
of each medical group's payment is contingent on the quality of care
it provides to enrollees, as measured by both their satisfaction
ratings and other process measures.\16

Thus, Health Net provides (1) information to the plan's enrollees to
encourage them to vote with their feet by migrating to the top
performing groups and (2) a direct financial incentive for the
medical groups that is associated with their performance.  Health
Net's report card is derived from a satisfaction survey of the plan's
enrollees.\17 It includes numerical scores representing the
percentage of respondents who reported that they were satisfied
regarding each of 17 measures.  Health Net divides the medical groups
into three categories for comparison:  excellent, very good, and
good.  (See the appendix for details from the Health Net
Participating Physician Report Card.)

Health Net is also developing report cards on care for certain
chronic conditions.  For these, Health Net uses administrative data
to identify enrollees with a given condition and sends them a
standardized survey that measures such things as the number of work
days lost to illness or injury.  It also measures compliance with
national guidelines on management of the condition or disease. 
Health Net published a report card on asthma care in December 1998
and is currently working on report cards for diabetes and congestive
heart failure.\18 (See the appendix for details from the asthma
report card.)

--------------------
\16 Health Net has three reward components to its performance-based
contracts:  (1) pay for excellence, given to the top 25 groups; (2)
pay for performance, given to groups that exceed fixed performance
targets; and (3) pay for improvement, given to groups that improve,
even if they are ranked relatively low. 

\17 Health Net sent its 1998 enrollee satisfaction survey to more
than 500,000 enrollee households in California.  According to the
President for Health Benchmarks, Inc.  (the organization responsible
for producing Health Net's physician group report cards), the overall
survey response rate was about 30 to 40 percent.  Only physician
groups with 75 or more plan enrollees responding to Health Net's
satisfaction survey were included in the report card. 

\18 Health Net's Asthma Care Report Card ratings were based on a 1996
survey administered to more than 32,000 Health Net enrollees who
suffered from asthma.  Because only California physician groups with
35 or more plan enrollees responding to the survey were included in
the report, the Asthma Care Report Card included information for 47
medical groups. 

         BUYERS HEALTH CARE ACTION
         GROUP
-------------------------------------------------------- Letter :4.1.4

In 1997, the employer members of BHCAG began a program of contracting
with health care providers directly rather than with health plans. 
The providers organized themselves into care systems, with some care
systems resembling multispecialty medical groups and others looking
more like IPAs.\19 BHCAG set out to adopt HEDIS health plan measures
for each care system in the program.  However, the purchasing group
decided to abandon its effort to use the HEDIS measures because the
number of patients within each care system who met the criteria for a
particular measure was too small for valid, comparative analysis. 
BHCAG was unable to identify more than 100 plan enrollees for any of
the measures in more than one or two care systems.  According to a
BHCAG official, BHCAG also decided not to develop the HEDIS data base
for the care systems' patient population for two reasons.  First,
Minnesota's medical record confidentiality law prevented BHCAG from
auditing the data to ensure their accuracy.  Second, because patients
who are not associated with BHCAG member firms are not necessarily
obligated to seek primary care from a single care system, it would be
difficult to establish the base for many HEDIS measures.  Instead,
BHCAG developed and distributed a satisfaction survey to members'
employees and reported the results to its enrollees and the general
public. 

In 1996-98, BHCAG reported data on 12 measures from the survey,
focusing on such issues as access to services and interactions with
physicians.  Beginning in 1999, BHCAG adopted a modified version of
the Consumer Assessment of Health Plans survey developed by the
Agency for Health Care Policy and Research (AHCPR).  While BHCAG has
approximately 150,000 individuals enrolled in care systems, they are
unevenly distributed:  About 75 percent are enrolled in three larger
care systems.  To increase its sample sizes for care outside the
Minneapolis St.  Paul metropolitan area, BHCAG conducted the survey
with Minnesota state employees, which increased the total potential
survey population from about 150,000 to about 300,000. 

--------------------
\19 A characteristic of these care systems that sets them apart from
many health plans is that primary care physicians can belong to only
one care system. 

         STATE REPORT CARDS ON
         CARDIAC SURGEONS
-------------------------------------------------------- Letter :4.1.5

Since the early 1990s, the New York Department of Health and the
Pennsylvania Health Care Cost Containment Council have published
physician-specific mortality rates for patients undergoing coronary
artery bypass graft (CABG) surgery.\20 Because patients'
characteristics such as age and other health problems play a large
role in the rates of complications and deaths associated with CABG
surgery, efforts have been made to adjust the performance measures
for differences in patients' conditions.  For example, the New York
risk-adjustment process incorporates approximately 40 risk factors
for each patient.  The New York Department of Health also seeks to
verify the data through activities such as cross-matching cardiac
surgery with other Department databases and reviewing medical records
for a sample of cases. 

The New York Department of Health reported that the state's CABG
surgery mortality rate dropped by more than 30 percent following the
publication of the report card, from 3.52 percent in 1989 to 2.44
percent in 1996.  Similarly, the Pennsylvania Health Care Cost
Containment Council reported that inhospital mortality was 22-percent
lower in 1995 than it was in 1991 (3.8 percent compared with 4.9
percent).  However, these results and the effect of the report cards
have not been without controversy. 

Some researchers assert that performance reporting has played a
significant role in the decline in the CABG surgery death rate.  They
point to evaluations and improvements in CABG surgery processes,
changes in referral patternssuch as concentrating the most difficult
cases with top-performing physiciansand a reduction in the number of
surgeons who perform these procedures only a few times each year.\21
Critics of the New York program contend that performance reporting is
not responsible for a decline in the mortality rate.  They claim
other factors such as surgeons' electing not to operate on critically
ill patients and possibly referring high-risk cases to out-of-state
practitioners.  They also question how much the mortality rate has
declined, suggesting that an apparently spurious increase in the risk
factors may have accounted for most of the total reduction in the
statewide risk-adjusted mortality rate.\22

--------------------
\20 New York is expanding its project to include balloon angioplasty. 

\21 Edward Hannan and others, Improving the Outcomes of Coronary
Artery Bypass Surgery in New York State, Journal of the American
Medical Association, Vol.  271, No.  10 (1994), pp.  761 and 766. 

\22 Jesse Green and Neil Wintfeld, Report Cards on Cardiac Surgeons: 
Assessing New York State's Approach, New England Journal of
Medicine, Vol.  332, No.  18 (1995), pp.  1229 and 1232. 

      THE USEFULNESS OF PHYSICIAN
      AND PHYSICIAN GROUP REPORT
      CARDS REMAINS LIMITED
---------------------------------------------------------- Letter :4.2

The early experience with physician report cards indicates that
organizations are able to address some of the methodological
challenges in performance measurement and provide some comparative
information.  However, their ability to accurately report on broad
measures of physician performance in a useful manner remains limited. 

First, while the current report cards measure the performance of
physician groups that are smaller than health plans, the medical
groups may still be too large to make the cards useful to consumers. 
One medical group that appears in all three California report cards
includes nearly 700 physicians.  A consumer faced with the task of
selecting a physician could question whether having a set of summary
statistics covering so many physicians is really any more helpful
than having planwide performance measures for thousands of
physicians. 

Next, the quality of the data and the expense of collecting them are
also issues.  PBGH, BHCAG, and Health Net used surveys to gather
data.  As with all survey data, they reflect only the views of
patients who chose to respond and then record their recollection or
perception of the care they received, which may or may not be
accurate.  To address these issues, steps must be taken to see if
there is bias among respondents compared with nonrespondents and to
limit questions to those that patients are likely to answer
accurately.  And while surveys generally cost less than medical
record reviews, they are still expensive to conduct.  According to
one PBGH official, the Physician Value Check Survey costs
approximately $15,000 per medical group. 

PacifiCare's Quality Index relies more heavily on gathering and
analyzing administrative encounter data for its performance scores--a
process that is generally less costly than using surveys but that has
other limitations.  Administrative data reflect how physicians report
the services and procedures they provided rather than the patients'
recollection.  However, in some cases, the administrative data are
not complete.  According to PacifiCare's Medical Director, the plan
receives data from physicians on only about 70 percent of their
encounters with patients.  He added that publishing the Quality Index
has dramatically increased the volume of encounter data submissions. 
Before the Quality Index was published, PacifiCare received
information on about 2 million encounters each month; 2 months after
its publication, the plan was receiving data on about 5 million
encounters per month.  Despite the increase in the volume of
encounter data that medical groups provide, some groups question the
completeness and the quality of the raw data. 

Third, current report cards do not provide consistent results.  For
example, the PacifiCare and Health Net report cards demonstrate some
of the difficulties when different organizations measure groups in
different ways.  The two plans use different methods and different
data sources:  PacifiCare uses administrative and other internal data
sources, and Health Net uses enrollee satisfaction survey responses. 
The two plans also base their performance measures on a subset of a
patient population--plan enrollees rather than all patients in a
medical group.  Enrollee satisfaction scores from the two plans were
significantly different in some cases.  From the enrollees'
responses, Health Net classified one medical group as excellent"--a
classification it gave to only about one-fourth of the medical
groups--while PacifiCare classified the same group in the bottom
third.  A consumer looking at Health Net's report card might be more
inclined to select that medical group than a consumer looking at
PacifiCare's report card, while a consumer who read both report cards
would be confused as to how a single group could get such disparate
ratings from two plans' enrollees.  According to experts we
interviewed, such divergent scores on similar measures lead to
skepticism among physicians and the general public about the
usefulness of report cards. 

Finally, the differences in the specifications of the measures and
presentation issues may cause additional confusion.  While the report
cards measure some of the same aspects of care, their measures of
clinical quality can be defined and reported differently.  For
example, one plan may report the percentage of enrollees who were
satisfied with a service, while another might report only the
percentage who indicated that they were very or extremely satisfied. 
In addition, reporting issues such as the relative scale a plan uses
can accentuate narrow differences among medical groups.  For example,
under PacifiCare's scale, if all plans in the comparison fall between
85 and 95 percent on a particular measure, the group performing the
service 85 percent of the time could show up in the bottom 10th
percentile, while the group performing the service 95 percent of the
time could be listed as a best practices group.  Conversely, if all
the groups perform a recommended service less than half the time,
some of them will still be ranked as best practices groups.  Such
complexity in interpreting the results can make consumers wary of
report cards. 

   NEW COLLABORATIVE EFFORTS AND
   DATA STANDARDIZATION MAY HELP
   MEET SOME CHALLENGES
------------------------------------------------------------ Letter :5

While the work of purchasers, plans, and state agencies represents
progress toward resolving the difficulties with measuring health care
quality, further development is needed.  Several national groups have
been organized to cooperatively develop standardized approaches to
measurement-related issues.  In addition, HHS is taking some steps to
facilitate better performance measurement.  These efforts are in
their infancy, and it will take time to see what, if any, effect they
have on measuring physician performance. 

      PROFESSIONAL ORGANIZATIONS
      ARE BEGINNING TO COLLABORATE
      ON IMPROVEMENTS TO PHYSICIAN
      REPORT CARDS
---------------------------------------------------------- Letter :5.1

Developing a commonly accepted, standardized set of performance
measures is a critical step in creating a system of performance
measurement that will allow apples to apples comparisons in health
care.  Some of the organizations we talked to have recently joined
together to address participant concerns about performance
information and the factors in the marketplace that impede the flow
of data.  It remains to be seen whether these coalitions can forge
agreement on critical issues that must be addressed in the long term. 
For these organizations to be successful, disparate groups will have
to reach consensus on a number of issues and that will take time. 

The California Information Exchange is a partnership of purchasers,
providers, and other organizations established to promote and protect
the exchange of data among health care partners such as health plans,
purchasers, and providers.\23 According to one Exchange official, the
group was formed, in part, to overcome political impediments to the
exchange of health care information.  The Exchange has formed working
groups to develop agreements to be used to define the content, proper
use, and format for enrollment data; provider and provider group
identifiers; laboratory and encounter records; individual patient
identifiers; member identification cards; eligibility data; and
pharmacy records.  To date, the Exchange's work groups have adopted
agreements or rules of exchange for enrollment data, encounter data,
eligibility data, member identification cards, and pharmacy records. 
The Exchange plans to test these agreements in a series of pilot
projects. 

The Performance Measurement Coordinating Council is sponsored by
three health accreditation agencies:  the American Medical
Association's American Medical Accreditation Program, the Joint
Commission on Accreditation of Health Care Organizations, and NCQA. 
Comprising 15 members chosen by the founding organizations, it was
created in May 1998 to develop efficient and consistent performance
measures for different levels in the health care system.  The Council
brings together organizations working on quality measurement in
different areas of the health care industry with different points of
view on attribution, the public reporting of performance data, and
the like.  For example, the American Medical Accreditation Program
comes from an organization dedicated to representing the interests of
physicians and is most cautious about attributing performance data to
individual physicians and reporting on performance to the public.  At
the same time, NCQA, which is a health plan accreditation
organization, is a strong advocate for the public reporting of data. 
The progress of the Council illustrates the time it can take to work
on performance measures.  For example, it took the Council 8 months
to progress from its formation to the announcement that it intended
to develop a common measurement agenda and to address a range of
performance measurement issues.  As of May 1999, one year after its
formation, the council had identified and started work on diabetes
care measures--the first of its measurement sets. 

The National Forum for Health Care Quality Measurement and Reporting
is a private, nonprofit entity whose purpose is to develop a
comprehensive quality measurement and public reporting strategy.  The
Forum followed from the recommendations of the President's Advisory
Commission on Consumer Protection and Quality in the Health Care
Industry.\24 Goals for the Forum include allowing meaningful quality
comparisons of health care providers and plans and promoting
competition in the quality of health care services.  In March 1999,
the Forum planning committee approved the initial members of its
board of directors.  Representation on the Forum's board is broad,
including academic researchers and representatives from AHCPR; HCFA;
representatives from consumer, public, and private purchasers;
providers and plans; and research and quality improvement councils. 
As with the Performance Measurement Coordinating Council, the Forum's
efforts are taking time.  The Forum took approximately 9 months to
select its board of directors and does not expect individual work
groups to begin work until early 2000. 

--------------------
\23 Exchange partners include the American Medical Group Association,
the California Association of Health Plans, the California Healthcare
Association, the California Medical Association, the National IPA
Coalition, and the Pacific Business Group on Health. 

\24 Advisory Commission on Consumer Protection and Quality in the
Health Care Industry, Quality First:  Better Health Care for All
Americans, Report to the President of the United States (Washington
D.C.:  1998). 

      HHS IS TAKING STEPS TO
      REPORT ON MEDICARE
      PHYSICIANS' PERFORMANCE AND
      TO STANDARDIZE HEALTH DATA
---------------------------------------------------------- Letter :5.2

HHS has been involved in performance measurement through its
administration of the Medicare program, efforts to standardize health
data, and support of research.  In addition to meeting the
information needs of Medicare beneficiaries, these efforts can have a
substantial effect on report cards generated by private purchasers
and plans.  Because most of HHS' initiatives are still in progress,
this effect has yet to be determined. 

The Balanced Budget Act of 1997 (BBA) requires that HHS provide
comparative data to Medicare beneficiaries including information
about the benefits, quality, and performance (to the extent
available) of health care options in their area to assist them in
making informed choices under the Medicare+Choice Program.\25 To
provide better quality and performance comparisons, HCFA contracted
with Health Economics Research Inc.  in September 1997 for assistance
in developing a performance measurement system.  The contractor is
studying the feasibility of using HEDIS measures for fee-for-service
Medicare at the group practice, local, and national levels.  As part
of this 3-1/2-year contract, five clinical measures relevant to the
Medicare population--retinal eye examinations for diabetics,
follow-up care after mental health hospitalization, breast cancer
screening with mammography, beta blocker treatment after a heart
attack, and the Health of Seniors survey results--are being examined
at four large group practices. 

Although the study on HEDIS measures for group practices is not
expected to be completed until 2001, some difficulties, such as those
associated with small sample sizes, have already been identified.\26
For example, while the four group practices each had between 4,000
and 40,000 Medicare fee-for-service beneficiaries, sample sizes for
each measure fell considerably once population subsets of gender,
age, or condition were identified.  HEDIS specifies 411 patients as a
sufficient sample size, but this was obtained only for two of the
three claims-based measures--breast cancer screening and retinal eye
examinations.  For these two measures, the sample size was large
enough only for the three group practices with more than 20,000
Medicare fee-for-service beneficiaries. 

HHS' efforts to establish standards for information transactions and
data elements, including unique identifiers for individuals, plans,
and providers, may also have an effect on performance measurement
systems.  Under the Health Insurance Portability and Accountability
Act of 1996 (HIPAA) (P.L.  104-191), HHS is required to adopt
standards to support the exchange of information on administrative
and financial health care transactions.\27 The standards are to
include data elements and code sets for the electronic exchange of
information; unique health identifiers for health care providers,
health plans, employers, and individuals for use in the health care
system; and security protections against the unauthorized disclosure
and use of health information.  The standards are to apply to all
health plans, health care clearinghouses, and health care providers
that transmit health information electronically.\28

Some standards under development, such as the unique identifier for
individuals, have been contentious.  In response to concerns about
privacy, three bills were introduced in the 105\th and 106\th
Congresses to repeal the requirement for HHS to adopt a standard
unique health identifier.  While the two bills from the 105\th
Congress expired and one is pending before the current Congress, the
Omnibus Consolidated and Emergency Supplemental Appropriations Act
for 1999 (P.L.  105-277) provided that no funds available under the
act be used to adopt a final standard for individual unique health
identifiers until legislation is enacted specifically approving the
standard.  While the implementation of the HIPAA standards has the
potential to significantly improve the usability of the health data
available for performance measurement, it will not address all the
data challenges, such as those related to data accuracy. 

An additional HHS effort under way to further physician performance
measurement is the development of a consumer satisfaction survey for
fee-for-service Medicare beneficiaries.  AHCPR officials said that
they are interested in adapting the CAHPS survey--an instrument for
measuring consumer satisfaction and experience with health plans--to
the provider level.  AHCPR is studying the use of the CAHPS survey
with smaller units, such as group practices or individual physicians. 
In addition, AHCPR is sponsoring research on performance measurement
and is working with others to develop a framework for measuring
health care performance. 

--------------------
\25 Created by the BBA, the Medicare+Choice program is designed to
allow beneficiaries to choose health care from Medicare's traditional
fee-for-service program and a range of health plans, such as health
maintenance organizations and provider-sponsored organizations,
participating in Medicare. 

\26 In addition to the challenge of developing comparable performance
information for group practices under fee-for-service Medicare, HCFA
will have to ensure that the information provided to beneficiaries is
clear, sufficient, and helpful to their decisionmaking or it will not
be used.  For example, we previously reported on problems with HCFA's
efforts to provide comparative information on health plans--HCFA had
not provided information that was easy for beneficiaries to
understand.  See Medicare:  HCFA Should Release Data to Aid
Consumers, Prompt Better HMO Performance (GAO/HEHS-97-23, Oct.  22,
1996) and Medicare:  Progress to Date in Implementing Certain Major
Balanced Budget Act Reforms (GAO/T-HEHS-99-87, Mar.  17, 1999). 
Physician-level information runs the risk of having similar problems. 

\27 Transactions include health claims or equivalent encounter
information, enrollment and disenrollment in a health plan, health
care payment and remittance advice, health plan premium payments,
first report of injury, health claims status, and referral
certification and authorization. 

\28 Under HIPAA, standards were required to be enacted by August 21,
1999, regarding the privacy of individually identifiable health
information that is electronically exchanged.  Because this deadline
was not met, HIPAA now requires the Secretary of HHS to establish
standards by regulation no later than February 21, 2000. 

   CONCLUSIONS
------------------------------------------------------------ Letter :6

Consumers could use more information on the quality of health care
providers to help them make informed choices about where to seek
care.  Comparative information on physicians is important to all
consumers, whether they enroll in traditional Medicare or in a
private health plan or face a choice of primary care physicians when
they join a managed care plan.  Yet the field of physician
performance measurement is still in its infancy.  Challenges to
developing physician report cards include selecting performance
measures that satisfy the information needs of various audiences and
gaining the cooperation of physicians and consumers required to
assemble consistent and credible performance data.  The experience of
several organizations in producing medical group or specialty care
report cards indicates that steps can be taken to better inform
consumers, but the challenges that remain limit the report cards'
usefulness.  Given sufficient time, public and private efforts to
develop a consensus on standardized data collection and comparable
quality measurement may lead to more useful measures for consumers
through a more efficient system for providers and plans nationwide. 

   AGENCY COMMENTS
------------------------------------------------------------ Letter :7

We obtained informal comments on a draft of this report from the
Senior Clinical Adviser in HCFA's Office of Clinical Standards and
Quality.  She agreed with the report's general findings.  She
suggested that under the fee-for-service payment system, we highlight
the problem of determining which physician is accountable for
managing a patient's care.  The logistics of establishing the
linkages by means of existing medical records is another area of
concern that she recommended we stress in the report. 

We also obtained comments on the draft from an expert in quality
measurement who suggested that we include more information on the
methodological challenges of assessing physician performance.  He
felt that a stronger critique of the validity of currently available
measures would be helpful in the analysis of physician report cards. 
He noted that it is impossible to differentiate among providers with
current physician report cards and warned against the dangers of
misinformation.  He encouraged us to place more emphasis on the need
for research efforts to develop better measures that provide valid
information and to improve our understanding of preferred clinical
strategies.  He also emphasized the need for developing electronic
medical records for access to clinically relevant data. 

---------------------------------------------------------- Letter :7.1

As agreed with your office, unless you publicly announce its contents
earlier, we plan no further distribution of this report until 30 days
from the date of this letter.  We will then send copies of this
report to the Secretary of HHS and others who are interested.  We
will also make copies available to others on request. 

The information contained in this report was developed by Rosamond
Katz, Assistant Director, Mark Ulanowicz, and Patricia K.  Yamane. 
Please contact me at (202) 512-7114 or Rosamond Katz at (202)
512-7148 if you or your staff have any questions. 

Sincerely yours,

Janet Heinrich
Associate Director, Health Financing
 and Public Health Issues

REPORT CARDS COMPARING MEDICAL
GROUPS
==================================================== Appendix Appendix

In California, a purchasing group, the Pacific Business Group on
Health (PBGH), and two health plans, PacifiCare and Health Net, have
moved a step closer to reporting on physician performance by
publishing report cards on medical groups and independent practice
associations (IPA).  In addition, another purchasing group, the
Buyers Health Care Action Group (BHCAG) in Minnesota, has published
report cards on care systems, which can be similar to large medical
groups.  Below, we illustrate the information generated on 3 of the
11 medical groups common to all three California report cards and one
Minnesota care system.  In California, medical group A includes more
than 100 physicians, medical group B has 675 physicians, and medical
group C has more than 300 physicians.  In Minnesota, the care system
includes 550 physicians. 

   PACIFIC BUSINESS GROUP ON
   HEALTH
-------------------------------------------------- Appendix Appendix:1

PBGH publishes a report card on medical groups that is based on its
Physician Value Check Survey.  Figure I.1 shows the scores for three
southern California medical groups as reported on the PBGH Internet
site.  The numerical scores are divided into three categories that
indicate relative measures of performance:  above average, below
average, and average. 

   Figure I.1:  1996 PBGH
   Physician Value Check Survey
   Scores for Three Southern
   California Medical Groups

   (See figure in printed
   edition.)

Note:  The numerical scores represent scores based on survey
responses for the medical groups.  Scores above and below average
indicate a relative measure of the groups. 

Source:  Pacific Business Group on Health, California Consumer
Healthscope, at http://www.healthscope.org. 

A consumer reading this report card would learn that medical group B
received high overall scores for satisfaction and cholesterol
screening.  However, a person with high blood pressure would notice
that while groups B and C scored below average in prescribing
medicine for high blood pressure, group C had better success reducing
its patients' blood pressure.  People with diabetes would not find
any information specific to the treatment of their condition. 

   PACIFICARE
-------------------------------------------------- Appendix Appendix:2

PacifiCare's Quality Index report card reflects the health care
experiences and opinions of members of the PacifiCare health plan
only.  Figure I.2 shows the 1998 Quality Index scores for the same
medical groups highlighted in figure I.1.  The numerical scores
represent a percentile rank for a medical group compared with that of
all other groups.  PacifiCare identifies groups as best practice
groups for a particular measure if they are in the top 10 percent
relative to other groups.  These are denoted by a diamond next to the
number in the table. 

   Figure I.2:  1998 PacifiCare
   Quality Index Scores for Its
   Commercial Members at Three
   Medical Groups

   (See figure in printed
   edition.)

\a Data below threshold:  the medical group did not have enough
PacifiCare enrollees with congestive heart failure to allow for
statistically valid measurement. 

\b Includes responses from both commercial and Secure Horizons
members. 

Source:  PacifiCare. 

The comparative performance information in figure I.2 is limited and
selective and may not be adequate for choosing a medical group.  A
consumer reading this report card would learn that PacifiCare members
using medical group B were not happy with their access to care
relative to the other groups' patients--it scored in the bottom 15
percent for access-related complaints--but were very satisfied with
the group's primary care physicians--rating them in the top 10
percent in satisfaction.  The report card also indicates that medical
group C was in the bottom third of medical groups for cervical cancer
screening but in the top 10 percent for benefits appeals to
PacifiCare. 

   HEALTH NET
-------------------------------------------------- Appendix Appendix:3

Health Net's Participating Physician Group Report Card is derived
from a satisfaction survey of the plan's members.  Figure I.3 shows
selected Health Net report card scores for the same three southern
California medical groups as shown in figures I.1 and I.2.  The
numerical scores represent the percentage of respondents who reported
that they were satisfied regarding each measure, and the groups are
classified as excellent, very good, or good. 

   Figure I.3:  1998 Health Net
   Participating Physician Group
   Report Care

   (See figure in printed
   edition.)

Source:  Health Net Participating Physician Group Report Card, Sept. 
1998. 

A consumer reading this report card would find that the three medical
groups were largely undifferentiated.  They all were rated either
very good or excellent, both overall and within the three broad
categories of quality of care, access to care, and medical group
satisfaction.  Unlike the PacifiCare Quality Index report, the Health
Net participating physician group report card provides information
only on members' satisfaction with each issue--it does not provide
information on the extent to which particular services, such as
mammograms or cervical cancer screenings, were provided. 

Health Net is also developing report cards on care provided by
medical groups for certain chronic conditions.  Figure I.4 shows the
asthma report card scores for medical groups A and B.  Medical group
C did not have enough asthma patients responding to the survey to be
included in the comparison.  The four stars denote very good.

   Figure I.4:  1998 Health Net
   Asthma Report Card

   (See figure in printed
   edition.)

Source:  Health Net Participating Physician Group Asthma Care Report
Card, Dec.  1998. 

For patients with asthma trying to choose a medical group, the Health
Net asthma report card provides a considerable amount of information
on clinical quality, including information on outcomes of care.  It
indicates that even though both groups were rated very good for
asthma care, no patients with asthma in group A and fewer than 1 in
10 in group B reported using a peak-flow meter daily, even though
daily use is recommended in national clinical guidelines.  In
addition, the report tells consumers that a higher share of the
survey respondents from medical group A reported no asthma-related
absences from work or school in the past month than respondents from
medical group B. 

   MINNESOTA'S BUYERS HEALTH CARE
   ACTION GROUP
-------------------------------------------------- Appendix Appendix:4

BHCAG is currently reporting on 12 measures focusing on such issues
as access to services and interactions with physicians.  Figure I.5
shows the results of the BHCAG survey for one care system, a
550-physician multispecialty medical group.  The comparison rating
shows whether the survey ratings for the care system are better than,
similar to, or below the average rating.  The numerical scores are
the statistics for each measure. 

   Figure I.5:  1998 BHCAG Report
   Card on One Care System

   (See figure in printed
   edition.)

Source:  Choice Plus 1999 Consumer Satisfaction Survey Results. 

A consumer reading this report card would learn that this care system
was scored average by its patients in terms of overall satisfaction
and quality of care but was scored below average in the areas related
to interaction with physicians, such as a physician's explaining
medical procedures and tests. 

*** End of document. ***