    Physician Performance: Report Cards Under Development but Challenges
    Remain (Letter Report, 09/30/1999, GAO/HEHS-99-178).

    Pursuant to a congressional request, GAO reviewed efforts to provide the
    public with information on physician performance, focusing on: (1) the
    issues involved in measuring and reporting on physician and physician
    group performance; (2) efforts to develop physician report cards; and
    (3) initiatives under way that may address impediments to measuring
    physician and physician group performance.

    GAO noted that: (1) measuring performance in health care is challenging
    in terms of identifying measures that truly reflect the quality of care
    individuals receive; (2) it is also difficult to make comparisons across
    plans or providers that account for differences in the patients whom
    they treat that can affect health care outcomes; (3) measuring the
    performance of physician groups and individual physicians is even more
    difficult; (4) individual physicians or groups perform a wide variety of
    services and typically perform any individual service for a small number
    of patients; (5) only a fraction of these services can be clearly linked
    to a measurable outcome; (6) to make meaningful comparisons among
    physicians, analysts must adjust any measure selected to take into
    consideration the extent to which a characteristic like the severity of
    a medical condition affects the outcomes from care; (7) to avoid these
    difficulties, approaches to performance measurement generally focus on
    physician groups instead of individual physicians, and they measure
    processes such as whether services are provided in accordance with
    agreed upon norms rather than outcomes of care; (8) adding to the
    challenges, however, are concerns that consumers have regarding the
    privacy of their personal medical information and that physicians have
    regarding the accuracy of performance measurement data; (9) even though
    the data and measures that are available are limited, several different
    private and public organizations have developed physician and physician
    group report cards using these data and measures; (10) in New York and
    Pennsylvania, state agencies that have reported on the performance of
    individual cardiac surgeons since the early 1990s have reported improved
    performance scores since they began publishing them; (11) while
    significant, these efforts at physician report cards are in their early
    stages or are limited in scope, and difficulties remain; (12) some
    organizations are collaborating to develop more comprehensive,
    standardized performance measures and to facilitate the exchange of
    clinical and administrative data between physicians, plans, and
    purchasers; and (13) at the federal level, the Department of Health and
    Human Services is working on a performance measurement system for its
    Medicare fee-for-service program and has been supporting research and
    working with other organizations to develop physician performance
    measures.

    --------------------------- Indexing Terms -----------------------------

     REPORTNUM:  HEHS-99-178
         TITLE:  Physician Performance: Report Cards Under Development but
    	     Challenges Remain
          DATE:  09/30/1999
       SUBJECT:  Health care programs
    	     Physicians
    	     Health care services
    	     Consumer education
    	     Health maintenance organizations
    	     Surveys
    	     Performance measures
    IDENTIFIER:  American Medical Accreditation Program
    	     Medicare Fee-for-Service Program

    ******************************************************************
    ** This file contains an ASCII representation of the text of a  **
    ** GAO report.  Delineations within the text indicating chapter **
    ** titles, headings, and bullets are preserved.  Major          **
    ** divisions and subdivisions of the text, such as Chapters,    **
    ** Sections, and Appendixes, are identified by double and       **
    ** single lines.  The numbers on the right end of these lines   **
    ** indicate the position of each of the subsections in the      **
    ** document outline.  These numbers do NOT correspond with the  **
    ** page numbers of the printed product.                         **
    **                                                              **
    ** No attempt has been made to display graphic images, although **
    ** figure captions are reproduced.  Tables are included, but    **
    ** may not resemble those in the printed version.               **
    **                                                              **
    ** Please see the PDF (Portable Document Format) file, when     **
    ** available, for a complete electronic file of the printed     **
    ** document's contents.                                         **
    **                                                              **
    ** A printed copy of this report may be obtained from the GAO   **
    ** Document Distribution Center.  For further details, please   **
    ** send an e-mail message to:                                   **
    **                                                              **
    **                                            **
    **                                                              **
    ** with the message 'info' in the body.                         **
    ******************************************************************

    Cover
    ================================================================ COVER

    Report to Congressional Requesters

    September 1999

    PHYSICIAN PERFORMANCE - REPORT
    CARDS UNDER DEVELOPMENT BUT
    CHALLENGES REMAIN

    GAO/HEHS-99-178

    Physician Report Cards

    (108386)

    Abbreviations
    =============================================================== ABBREV

      AHCPR - Agency for Health Care Policy and Research
      BBA - Balanced Budget Act of 1997
      BHCAG - Buyers Health Care Action Group
      CABG - coronary artery bypass graft
      HCFA - Health Care Financing Administration
      HEDIS - Health Plan Employer Data and Information Set
      HHS - Department of Health and Human Services
      HIPAA - Health Insurance Portability and Accountability Act of 1996
      HMO - health maintenance organization
      IPA - independent practice association
      NCQA - National Committee on Quality Assurance
      PBGH - Pacific Business Group on Health
      PPO - preferred provider organization

    Letter
    =============================================================== LETTER

    B-281938

    September 30, 1999

    The Honorable James M.  Jeffords
    Chairman
    Committee on Health, Education, Labor, and Pensions
    United States Senate

    The Honorable Joseph I.  Lieberman
    United States Senate

    While information has become available comparing health plan
    benefits, costs, customer satisfaction, and quality, it is not clear
    that this information is meeting consumers' needs or expectations.
    With plans having overlapping panels of physicians and hospitals, it
    has become increasingly difficult for consumers to differentiate
    plans.  Moreover, few consumers receive information on which to base
    one of their biggest health care decisionstheir choice of doctors.
    Recognizing this, some organizations are attempting to measure and
    report on the performance of physicians and physician groupswith the
    hope that the results can be used to compare the quality of their
    care and services.  How well physician and physician group
    performance measures assist consumers to make choices and how well
    they drive improvements in the health care market is unknown.
    Because of the growing interest in promoting informed health care
    decisions through public dissemination of performance information,
    you asked us to examine (1) the issues involved in measuring and
    reporting on physician and physician group performance, (2) current
    efforts to develop physician report cards, and (3) initiatives under
    way that may address impediments to measuring physician and physician
    group performance.

    To meet your request, we interviewed officials of purchasing groups,
    health plans, accreditation agencies, and federal programs; experts
    in health care performance measurement; and representatives from
    organizations that have formed to advance performance measurement.
    We visited three large health care purchasers--the Pacific Business
    Group on Health (PBGH), the Buyers Health Care Action Group (BHCAG),
    and the Health Care Financing Administration (HCFA)--and we
    interviewed two private health plans that publish physician group
    report cards--PacifiCare and Health Net--to discuss their efforts to
    measure and report on physician performance.  We also reviewed report
    cards on cardiac surgeons issued by New York and Pennsylvania state
    agencies.  We based our selection of purchaser groups and state
    initiatives on their reputations as innovators in the area of
    consumer health care information.  The health plans we selected have
    publicly reported the results of their comparisons of physicians in
    their networks.  We performed our work from January 1999 through
    August 1999 in accordance with generally accepted government auditing
    standards.

       RESULTS IN BRIEF
    ------------------------------------------------------------ Letter :1

    Measuring performance in health care is challenging in terms of
    identifying measures that truly reflect the quality of care
    individuals receive.  It is also difficult to make comparisons across
    plans or providers that account for differences in the patients whom
    they treat that can affect health care outcomes.  Measuring the
    performance of physician groups and individual physicians is even
    more difficult.  Individual physicians or groups perform a wide
    variety of services and typically perform any individual service for
    a small number of patients.  Only a fraction of these services can be
    clearly linked to a measurable outcome.  To make meaningful
    comparisons among physicians, analysts must adjust any measure
    selected to take into consideration the extent to which a
    characteristic like the severity of a medical condition affects the
    outcomes from care.  To avoid these difficulties, current approaches
    to performance measurement generally focus on physician groups
    instead of individual physicians, and they measure processes such as
    whether services are provided in accordance with agreed-upon norms
    rather than outcomes of care.  Adding to the challenges, however, are
    concerns that consumers have regarding the privacy of their personal
    medical information and that physicians have regarding the accuracy
    of performance measurement data.

    Even though the data and measures that are currently available are
    limited, several different private and public organizations have
    developed physician and physician group report cards using these data
    and measures.  For example, two purchasing groups and two California
    health plans are avoiding some problems associated with measuring the
    performance of individual physicians (such as small sample sizes) by
    reporting on the performance of physician groups.  In addition, in
    New York and Pennsylvania, state agencies that have reported on the
    performance of individual cardiac surgeons since the early 1990s have
    reported improved performance scores since they began publishing
    them.  While significant, these efforts at physician report cards are
    in their early stages or are limited in scope, and difficulties
    remain.  For example, medical group report cards provide information
    that is closer to the level of the individual physician than health
    plan report cards do but, depending on the size of the medical group,
    may not be very helpful for making an informed choice of a physician.
    In addition, questions about the accuracy and completeness of the
    data and the adequacy of the risk adjustment methodology limit
    consumer and physician confidence in the report cards.

    Some organizations are collaborating to develop more comprehensive,
    standardized performance measures and to facilitate the exchange of
    clinical and administrative data between physicians, plans, and
    purchasers.  For example, several national accreditation
    organizations have formed a council to develop common performance
    measures.  At the federal level, the Department of Health and Human
    Services (HHS) is working on a performance measurement system for its
    Medicare fee-for-service program and has been supporting research and
    working with other organizations to develop physician performance
    measures.  In addition, HHS is establishing standards for
    administrative claims and encounter data as well as unique
    identifiers for individuals, plans, and providers--efforts that
    should help HHS and others in their performance measurement efforts.

       BACKGROUND
    ------------------------------------------------------------ Letter :2

    To date, most performance information has provided only data on
    health plans as a whole.  Changes in the health care
    market--particularly the growth in the size of plans, the shifting of
    greater financial risk to physicians or physician groups, and the
    requirement in some cases that beneficiaries receive all their care
    from selected physicians within a plan--have made plan comparisons
    less useful for many consumers.  Many consumers do not get to choose
    their health plan and even for consumers who can choose among plans,
    an individual physician's performance may deviate greatly from the
    health plan's average.  These and other factors have prompted calls
    for physician report cards that can help consumers select physicians
    from those available within their health plan.

    Report cards are generally publicly released reports on the quality
    of care that provide comparative information on plan characteristics
    and performance.  One widespread report card for health plans is
    prepared by the National Committee on Quality Assurance (NCQA).  NCQA
    uses its Health Plan Employer Data and Information Set (HEDIS) to
    report on plan performance.  HEDIS includes more than 60 performance
    indicators covering quality, access to and satisfaction with care,
    membership and use of services, finance, and management.

    When the development of health plan report cards began, plans were
    expected to differ significantly in their provider networks,
    organizational structure, and philosophical orientation, and the
    differences were expected to be reflected in the overall quality of
    the plans.  But the marketplace has not evolved this way.  Instead,
    to attract members and gain market share, health plans began building
    larger, often overlapping networks that offer consumers more provider
    choices than previously available.  With the same providers
    represented in two or more competing plans, it has become
    increasingly difficult to differentiate between plans.  Thus, even
    for consumers who have a choice of plans, the comparative plan
    information currently available may not demonstrate differences in
    plan performance.

    Physician practices are also undergoing significant change; more
    physicians are joining medical groups, and these medical groups are
    contracting with many health plans.  The proportion of physicians in
    group practices rose from approximately 11 percent in 1965 to 34
    percent in 1995.\1 In addition, according to the Medical Group
    Management Association, its members contract with an average of 21
    health maintenance organizations (HMO) and preferred provider
    organizations (PPO).\2 As physician groups contract with more plans,
    individual plans may have less influence over physician practice
    patterns and the quality of services they provide, because any one
    plan may account for only a small percentage of a medical group's
    total volume of patients or income.

    In addition, a growing number of physicians receive capitated
    paymenta fixed monthly payment per patientunder which they accept
    financial risk for providing a portion of or all patient care
    services.  As plans shift more financial risk to physician groups, a
    group's economic incentive is to minimize expensive services for sick
    patients.  In a 1996-97 survey, more than half of physicians (54
    percent) reported that their practices received capitation for some
    of their patients.\3 In locations such as Seattle, Washington, and
    Orange County, California, nearly three-fourths of physicians
    reported receiving capitation for some of their patients.  To the
    extent that physician assumption of financial risk affects quality of
    care, this trend further shifts the focus on quality from plans to
    physicians.

    For most employed Americans, their employer determines the number and
    type of health insurance plans available to them.  For workers whose
    employers do not offer a choice of plans, report cards that compare
    plans have no utility.  A 1997 survey conducted by the Research
    Triangle Institute found that less than one in five--17 percent--of
    private employers that offered insurance to their employees provided
    a choice among plans.\4

    Another study reported that of employers that offer health insurance,
    92 percent of small firms and 44 percent of larger firms (those with
    more than 200 employees) offered only one plan in 1998.\5 Counting
    employees rather than employers, less than half--only 41 percent--of
    employees who are offered health insurance can choose from two or
    more health plans.\6

    Health plan report cards may also be of little use to more than 14
    million of the country's 40 million Medicare beneficiariesthose who
    did not have a choice of managed care plans in 1998.\7

    --------------------
    \1 Henry J.  Kaiser Family Foundation, Trends and Indicators in the
    Changing Health Care Market Place (Menlo Park, Calif.:  Aug.  1998).

    \2 The Medical Group Management Association is a national
    professional and trade association.  It represents administrators of
    7,491 medical group practices that included 181,974 physicians in
    1997.  PPOs are similar to fee-for-service plans but provide
    enrollees a financial incentive--lower cost sharing--to receive care
    from a network of providers that are normally reimbursed at a
    discounted fee-for-service rate.

    \3 Center for Studying Health System Change, Data Bulletin:  Results
    from the Community Tracking Study (Washington, D.C.:  Fall 1997).

    \4 The Robert Wood Johnson Foundation, 1997 Employer Health Insurance
    Survey (Princeton, N.J.:  1997).

    \5 Henry J.  Kaiser Family Foundation, Health Benefits of Small
    Employers in 1998 (Menlo Park, Calif.:  Feb.  1999), p.  18.

    \6 Robert Wood Johnson Foundation.

    \7 In 1998, 4 million Medicare beneficiaries had only one managed
    care plan available in their county and 10.6 million beneficiaries
    lived in counties with no plan at all.  See Medicare Managed Care
    Plans:  Many Factors Contribute to Recent Withdrawals; Plan Interest
    Continues (GAO/HEHS-99-91, Apr.  27, 1999).

       DEVELOPING PHYSICIAN REPORT
       CARDS IS CHALLENGING
    ------------------------------------------------------------ Letter :3

    The heterogeneity of health care makes performance measurement
    challenging in terms of identifying measures that truly reflect the
    quality of care that individuals receive.  Making valid comparisons
    across plans or providers that ultimately account for patient
    differences that affect outcomes is also difficult.  These challenges
    are magnified in attempts to measure the performance of physician
    groups and individual physicians.  For example, unlike plans that
    have a large number of enrollees, individual groups or physicians
    generally see a small number of patients with specific conditions.
    These attempts are further complicated by a concern that consumers
    and physicians have regarding the use of performance measurement data
    in the first place.

          SELECTING APPROPRIATE
          MEASURES IS DIFFICULT
    ---------------------------------------------------------- Letter :3.1

    Medicine involves a wide variety of services, only a portion of which
    can be clearly linked to health outcomes.  Health outcomes are also
    influenced by factors such as a patient's age, medical history, and
    heredity.  In addition, most individual physicians perform any
    specific service a relatively few times in a given year, making it
    more difficult to adjust for differences in patients and creating
    other statistical problems.  Efforts to create report cards on health
    plans and providers are also complicated by the different needs of
    consumers.  Current attempts to resolve the difficulties of physician
    performance measurement include aggregating physicians into groups;
    focusing on certain types of physicians, such as cardiac surgeons,
    who perform highly specialized services; and substituting measures of
    process for measures of outcome.

    To make meaningful comparisons among physicians, analysts must select
    measures while taking into account factors that can affect the
    outcomes of care, such as a patient's medical history.  For example,
    whether the patient is treated for a first or second heart attack
    affects the likelihood of a successful outcome from cardiac surgery.
    Measures that did not account for differences in such factors would
    penalize physicians who treat the sickest patients.  Without proper
    adjustment in the measures, physicians might choose to avoid
    high-risk patients in order to maintain higher performance scores.\8
    Today, such adjustments are limited, and it may never be possible to
    account for every characteristic of patients that could affect their
    health outcomes.\9

    Another challenge in performance measurement is that different
    consumers have different information needs that are not likely to be
    adequately met all in a single physician report card.  Consumers
    prefer performance information that matches their own medical
    conditions and situations.  For the majority of consumers who are
    generally in good health, clinical quality indicators may not be as
    relevant as service quality indicators, such as the waiting times for
    an appointment.  But for individuals with chronic ill health, those
    who use physician services the most, clinical quality measures may be
    more critical.  It may not be possible to measure and report on
    physicians at the level of detail that is meaningful and useful to
    all consumers.

    Current performance measurement practices make an effort to detect
    physician and physician group practice differences, report on a
    variety of indicators, and generally avoid the need to identify
    detailed patient characteristics.  One approach to solving the
    measurement problems associated with individual physicians' seeing
    only a small number of patients with a given condition is to focus on
    physicians organized into groups, so that the number of patients with
    a given medical condition is high enough to provide meaningful data.

    Most quality indicators in use today focus on process measures, such
    as the percentage of women older than 50 in a plan who receive
    screening mammograms.  Many of the measures NCQA uses for health
    plans in HEDIS are process measures.  Measures of medical care
    process are popular in part because the data required are obtained
    relatively easily from administrative databases.  Furthermore,
    process measures, particularly for preventive services, avoid the
    difficulties inherent in trying to adjust the results for differences
    in patient characteristics.  But process measures have many
    shortcomings:  They capture a very limited range of medical services,
    they tend to measure whether a service was provided when it was
    called for but not how well it was done, and they focus heavily on
    preventive care services because the universe of patients who should
    be receiving them is most easily identified.  Outcome measures, those
    that indicate whether a patient's health improved after care,
    generally remain elusive.

    --------------------
    \8 The results of a recent study on physician profiles for patients
    with diabetes suggests that physicians might refuse to care for sick
    patients, those who have failed therapy or those who do not adhere to
    treatment plans, in order to improve their profile scores.  See
    Timothy P.  Hofer and others, The Unreliability of Individual
    Physician Report Cards' for Assessing the Cost and Quality of Care
    of a Chronic Disease, Journal of the American Medical Association,
    Vol.  281, No.  22 (1999), pp.  2098, 2104, and 2105.

    \9 President's Advisory Commission on Consumer Protection and Quality
    in the Health Care Industry, Quality First:  Better Health Care for
    All Americans, Final Report to the President of the United States
    (Washington, D.C.:  1998).

          ASSEMBLING PERFORMANCE DATA
          REQUIRES COOPERATION
    ---------------------------------------------------------- Letter :3.2

    Creating a physician performance measurement system involves
    collecting and verifying medical care data.  In order to collect the
    data, the concerns of consumers and physicians regarding how the data
    will be used must be addressed.  To ensure that measures of
    performance are accurate, the data going into the measures must be
    verified and free from manipulation.

    Physicians are concerned about the potential that inaccurate
    performance scores will unfairly affect their practice.  Physicians
    we interviewed told us that issues of data quality and the
    appropriate attribution of performance scores to individual
    physicians must be addressed before performance measurement data are
    made public.  Although administrative records, such as claims for
    payment, are readily available for the fee-for-service sector, they
    often do not include all the information that performance measurement
    requires, such as a patient's condition or the results of services
    rendered, and for some indicators they are not collected because they
    were created for billing purposes and not for performance
    measurement.  Medical records provide much more complete information,
    but their analysis is expensive because few records are automated.
    Gathering information through surveys falls somewhere between
    administrative data and medical record review in terms of ease and
    expense.  One limitation of ratings of consumer satisfaction is that
    consumers cannot always tell if the care they received was
    appropriate or technically good; research has not shown a consistent
    relationship between consumer satisfaction and the technical quality
    of care.

    Another concern for physicians is the method of determining which
    physicians should be held accountable for specific actions or
    outcomes.  Some physicians argue that it is difficult to fairly
    attribute a change in health status to a particular physician because
    many other factors come into play.  For example, some patients may
    see a variety of physicians over the course of a year, each
    potentially recommending or performing a needed service.  If a
    patient has not received a particular service, which physician should
    be held accountable for the omission?  Or a physician may have
    recommended a very effective treatment to a patient, but that
    patient's condition did not improve because he or she did not comply
    with the physician's recommendation.  The question of attribution
    becomes even more difficult as systems of health care become more
    integrated and a team of providers rather than one physician is
    responsible for patients' health care.

    Because physicians control the majority of the data necessary to
    measure their performance, these concerns must be addressed if
    measurement efforts are to be successful.  Physicians are responsible
    for coding administrative data, whether they are for reimbursement
    for claims or other data used by managed care plans.  Physicians also
    maintain medical records for individual patients.  To automate these
    records in order to make performance measurement data better would be
    expensive, and it is unlikely that physicians or physician groups
    will provide the resources for automation.

    Ensuring that the data that are collected are accurate is another
    challenge.  In any measurement system, participants may manipulate
    the data to improve their performance scores.  For example,
    physicians could exaggerate the severity of patients' conditions to
    ensure a more favorable rating.  Or they could simply avoid taking on
    difficult cases in order to improve their success rates.  Preventing
    this sort of manipulation requires activities such as auditing the
    data by comparing them to medical records.

    Consumers are concerned about the privacy of their personal medical
    information, and this concern may lead to rules that restrict efforts
    to provide objective information on physician performance.  One
    survey found that no more than about one-third of adults in the
    United States trust health plans and government programs to maintain
    confidentiality all or most of the time.\10 Meanwhile, consumers want
    an unbiased, expert source of information about health care
    quality.\11 State laws vary significantly, but in some states efforts
    to protect the privacy of medical records could affect efforts to
    ensure that reported measures are comparable and that the data are
    not manipulated.  For example, in Minnesota, any release of a
    patient's health records for research purposes requires, among other
    things, that the provider attempt to acquire the patient's consent
    and determine that individually identifiable records are necessary,
    that the researcher's safeguards are adequate, and that the
    researcher will not use the records for purposes other than those in
    the original request without the patient's consent.  According to a
    BHCAG official, Minnesota's state privacy laws forced the group to
    abandon its attempts to collect HEDIS data from care systems and have
    hampered efforts to obtain survey data regarding quality of care for
    people with chronic conditions.  Finding the appropriate balance
    between allowing access to medical records to ensure reliable,
    unbiased information on health care quality and maintaining privacy
    concerns is subject to considerable debate.

    --------------------
    \10 California Healthcare Foundation, Medical Privacy and
    Confidentiality Survey (Oakland, Calif.:  Jan.  28, 1999).

    \11 Consumers do not necessarily trust currently available
    performance information.  Three of 10 Americans surveyed said that
    information on quality of care from groups of doctors or state
    medical societies had little credibility; 4 of 10 had little faith in
    information from government agencies, and nearly 5 of 10 said that
    information from newspapers, television, and other media was not very
    believable.  See Agency for Health Care Policy and Research and Henry
    J.  Kaiser Family Foundation, Americans as Health Care Consumers:
    The Role of Quality Information, Highlights of a National Survey
    (Washington, D.C.:  1996).

       REPORT CARDS ON PHYSICIANS AND
       PHYSICIAN GROUPS INDICATE
       PROGRESS BUT THEIR USEFULNESS
       REMAINS LIMITED
    ------------------------------------------------------------ Letter :4

    Several private and public organizations are involved in a variety of
    activities to measure and report on physician performance.  A
    purchaser and two health plans in California and a purchaser in
    Minnesota have worked on moving performance measurement down to the
    level of the medical group or independent practice association (IPA)
    or, in Minnesota, a care system.\12 In New York and Pennsylvania,
    state agencies have published risk-adjusted mortality rates for
    specific procedures performed by cardiac surgeons.  While the current
    report cards demonstrate that some reporting is possible,
    shortcomings in these physician and group report cards include the
    doubtful value to consumers of scores for large medical groups,
    questions about the quality and the expense of collecting the data on
    which reporting is based, and inconsistencies among the report cards.

    --------------------
    \12 A medical group is two or more doctors who work together to
    provide medical services to patients.  Typically, doctors who work in
    a medical group--both primary care doctors and specialists--share a
    single office or several offices if the group is very large.  An
    independent practice association is a network of individual
    physicians who practice medicine by themselves or in small groups
    (often composed of one type of doctor, such as pediatricians) and who
    join together as an association to provide a range of primary and
    specialty care services to patients.  Care systems began in 1997,
    when the employer members of BHCAG began contracting with health care
    providers directly rather than with health plans.  The providers
    organized themselves into care systems, with some care systems
    resembling multispecialty medical groups and others looking more like
    independent practice associations.

          PURCHASER AND HEALTH PLAN
          REPORT CARDS COMPARE
          PHYSICIAN GROUPS
    ---------------------------------------------------------- Letter :4.1

    Several organizations have developed report cards for physician
    groups.  PBGH, two California health plans--PacifiCare and Health
    Net--and BHCAG in Minnesota have moved a step closer to reporting
    physician performance by publishing report cards on medical groups
    and IPAs.

             PACIFIC BUSINESS GROUP ON
             HEALTH
    -------------------------------------------------------- Letter :4.1.1

    PBGH is a business coalition of 33 public and private purchasers of
    health care representing more than 3 million employees, retirees, and
    dependents.  As physician networks overlapped more and
    differentiation in California health plans blurred, PBGH started
    partnering with medical groups and IPAs on quality improvement
    initiatives.  Together, they developed a publicly reportable
    measurement tool called the Physician Value Check Survey.  In 1996,
    the survey covered 49 California medical groups (and 9 from the
    Pacific Northwest) that ranged in size from approximately 15,000
    patients to more than 1 million patients.  Responses were obtained
    from 31,000 patients.\13

    PBGH chose a survey to collect data because it did not have the
    resources to mount a full-scale medical record review, and the survey
    was a less costly means of evaluating physician group performance and
    obtaining information on the consumers' perspectives.  So that PBGH
    could generalize the results to all patients seen by a medical group,
    PBGH and its partners drew patient samples from each medical group's
    entire patient population rather than just from PBGH members.  PBGH
    used the survey results to publicly compare medical groups in several
    areas:  a summary report card with measures such as overall
    satisfaction, a preventive care services report card, and specific
    report cards on care for high blood pressure and high cholesterol.
    For each report card, it classified the groups into three categories:
    above average, average, and below average.  (See the appendix for
    details from the PBGH report card.)

    --------------------
    \13 According to PBGH's Director of Research, these 31,000
    respondents represent more than 8 million enrollees in managed care
    plans.  In 1996, the Physician Value Check Survey was sent to 1,000
    patients between the ages of 18 and 70 randomly sampled in each
    medical group.  The overall survey response rate was about 55
    percent.  PBGH administered the Physician Value Check Survey again in
    1998; however, the results from this survey were not available at the
    time of our review.  PBGH officials said they plan to release the
    1998 Physician Value Check Survey results on September 23, 1999, with
    scores on the changes between 1996 and 1998that is, to see whether
    for the same group of patients, the physician groups' performance
    improved, worsened, or stayed the same over time.

             PACIFICARE
    -------------------------------------------------------- Letter :4.1.2

    PacifiCare of California is an HMO that has since 1998 produced a
    medical group report card called the Quality Index.  The publicly
    reported Quality Index uses measures selected from PacifiCare's
    internal provider profiles, which contain data on more than 60
    clinical and service performance measures for its medical groups and
    IPAs.\14 PacifiCare selected 14 of these measures for inclusion in
    the Quality Index.\15 It based its selection on the preferences of
    focus groups of consumers and the extent to which the physicians
    could take actions that affected the measured activities.

    The information in the Quality Index is compiled from the health
    plan's administrative databases, customer service department records,
    and enrollee satisfaction surveys.  Thus, the Quality Index reflects
    the health care experiences and opinions of only PacifiCare
    enrollees.  The Quality Index includes measures of clinical
    performance, service performance, enrollee satisfaction, and
    administrative data submission.  In 1999, the Quality Index included
    process measures such as eye examinations for people with diabetes
    that were not included in the 1998 report.  The reported Quality
    Index scores are percentile ranks for medical groups or IPAs compared
    with all other groups.  Groups ranking in the top 10 percentile of a
    measure are considered best practice groups for that measure.
    According to PacifiCare's Medical Director, improvement has occurred
    in several areas, such as mammography screening rates and retinal
    examinations for diabetics.  (See the appendix for details from the
    PacifiCare Quality Index.)

    --------------------
    \14 PacifiCare profiles medical groups and IPAs that have about 500
    or more PacifiCare enrollees.  To be included in the Quality Index
    report, groups must have at least 1,000 PacifiCare commercial
    enrollees and 500 Secure Horizons enrollees (its Medicare managed
    care program).  Using this methodology, PacifiCare is able to report
    on physician organizations that provide care to the majority of its
    enrollees.

    \15 The March 1999 Quality Index included 28 measures14 for
    PacifiCare commercial enrollees and 14 for Secure Horizons enrollees.

             HEALTH NET
    -------------------------------------------------------- Letter :4.1.3

    Health Net is another managed care plan in California with more than
    2.2 million enrollees.  Its Participating Physician Group Report Card
    includes information for 131 medical groups in California, all of
    which are under performance-based contracts.  In 1999, a percentage
    of each medical group's payment is contingent on the quality of care
    it provides to enrollees, as measured by both their satisfaction
    ratings and other process measures.\16

    Thus, Health Net provides (1) information to the plan's enrollees to
    encourage them to vote with their feet by migrating to the top
    performing groups and (2) a direct financial incentive for the
    medical groups that is associated with their performance.  Health
    Net's report card is derived from a satisfaction survey of the plan's
    enrollees.\17 It includes numerical scores representing the
    percentage of respondents who reported that they were satisfied
    regarding each of 17 measures.  Health Net divides the medical groups
    into three categories for comparison:  excellent, very good, and
    good.  (See the appendix for details from the Health Net
    Participating Physician Report Card.)

    Health Net is also developing report cards on care for certain
    chronic conditions.  For these, Health Net uses administrative data
    to identify enrollees with a given condition and sends them a
    standardized survey that measures such things as the number of work
    days lost to illness or injury.  It also measures compliance with
    national guidelines on management of the condition or disease.
    Health Net published a report card on asthma care in December 1998
    and is currently working on report cards for diabetes and congestive
    heart failure.\18 (See the appendix for details from the asthma
    report card.)

    --------------------
    \16 Health Net has three reward components to its performance-based
    contracts:  (1) pay for excellence, given to the top 25 groups; (2)
    pay for performance, given to groups that exceed fixed performance
    targets; and (3) pay for improvement, given to groups that improve,
    even if they are ranked relatively low.

    \17 Health Net sent its 1998 enrollee satisfaction survey to more
    than 500,000 enrollee households in California.  According to the
    President for Health Benchmarks, Inc.  (the organization responsible
    for producing Health Net's physician group report cards), the overall
    survey response rate was about 30 to 40 percent.  Only physician
    groups with 75 or more plan enrollees responding to Health Net's
    satisfaction survey were included in the report card.

    \18 Health Net's Asthma Care Report Card ratings were based on a 1996
    survey administered to more than 32,000 Health Net enrollees who
    suffered from asthma.  Because only California physician groups with
    35 or more plan enrollees responding to the survey were included in
    the report, the Asthma Care Report Card included information for 47
    medical groups.

             BUYERS HEALTH CARE ACTION
             GROUP
    -------------------------------------------------------- Letter :4.1.4

    In 1997, the employer members of BHCAG began a program of contracting
    with health care providers directly rather than with health plans.
    The providers organized themselves into care systems, with some care
    systems resembling multispecialty medical groups and others looking
    more like IPAs.\19 BHCAG set out to adopt HEDIS health plan measures
    for each care system in the program.  However, the purchasing group
    decided to abandon its effort to use the HEDIS measures because the
    number of patients within each care system who met the criteria for a
    particular measure was too small for valid, comparative analysis.
    BHCAG was unable to identify more than 100 plan enrollees for any of
    the measures in more than one or two care systems.  According to a
    BHCAG official, BHCAG also decided not to develop the HEDIS data base
    for the care systems' patient population for two reasons.  First,
    Minnesota's medical record confidentiality law prevented BHCAG from
    auditing the data to ensure their accuracy.  Second, because patients
    who are not associated with BHCAG member firms are not necessarily
    obligated to seek primary care from a single care system, it would be
    difficult to establish the base for many HEDIS measures.  Instead,
    BHCAG developed and distributed a satisfaction survey to members'
    employees and reported the results to its enrollees and the general
    public.

    In 1996-98, BHCAG reported data on 12 measures from the survey,
    focusing on such issues as access to services and interactions with
    physicians.  Beginning in 1999, BHCAG adopted a modified version of
    the Consumer Assessment of Health Plans survey developed by the
    Agency for Health Care Policy and Research (AHCPR).  While BHCAG has
    approximately 150,000 individuals enrolled in care systems, they are
    unevenly distributed:  About 75 percent are enrolled in three larger
    care systems.  To increase its sample sizes for care outside the
    Minneapolis St.  Paul metropolitan area, BHCAG conducted the survey
    with Minnesota state employees, which increased the total potential
    survey population from about 150,000 to about 300,000.

    --------------------
    \19 A characteristic of these care systems that sets them apart from
    many health plans is that primary care physicians can belong to only
    one care system.

             STATE REPORT CARDS ON
             CARDIAC SURGEONS
    -------------------------------------------------------- Letter :4.1.5

    Since the early 1990s, the New York Department of Health and the
    Pennsylvania Health Care Cost Containment Council have published
    physician-specific mortality rates for patients undergoing coronary
    artery bypass graft (CABG) surgery.\20 Because patients'
    characteristics such as age and other health problems play a large
    role in the rates of complications and deaths associated with CABG
    surgery, efforts have been made to adjust the performance measures
    for differences in patients' conditions.  For example, the New York
    risk-adjustment process incorporates approximately 40 risk factors
    for each patient.  The New York Department of Health also seeks to
    verify the data through activities such as cross-matching cardiac
    surgery with other Department databases and reviewing medical records
    for a sample of cases.

    The New York Department of Health reported that the state's CABG
    surgery mortality rate dropped by more than 30 percent following the
    publication of the report card, from 3.52 percent in 1989 to 2.44
    percent in 1996.  Similarly, the Pennsylvania Health Care Cost
    Containment Council reported that inhospital mortality was 22-percent
    lower in 1995 than it was in 1991 (3.8 percent compared with 4.9
    percent).  However, these results and the effect of the report cards
    have not been without controversy.

    Some researchers assert that performance reporting has played a
    significant role in the decline in the CABG surgery death rate.  They
    point to evaluations and improvements in CABG surgery processes,
    changes in referral patternssuch as concentrating the most difficult
    cases with top-performing physiciansand a reduction in the number of
    surgeons who perform these procedures only a few times each year.\21
    Critics of the New York program contend that performance reporting is
    not responsible for a decline in the mortality rate.  They claim
    other factors such as surgeons' electing not to operate on critically
    ill patients and possibly referring high-risk cases to out-of-state
    practitioners.  They also question how much the mortality rate has
    declined, suggesting that an apparently spurious increase in the risk
    factors may have accounted for most of the total reduction in the
    statewide risk-adjusted mortality rate.\22

    --------------------
    \20 New York is expanding its project to include balloon angioplasty.

    \21 Edward Hannan and others, Improving the Outcomes of Coronary
    Artery Bypass Surgery in New York State, Journal of the American
    Medical Association, Vol.  271, No.  10 (1994), pp.  761 and 766.

    \22 Jesse Green and Neil Wintfeld, Report Cards on Cardiac Surgeons:
    Assessing New York State's Approach, New England Journal of
    Medicine, Vol.  332, No.  18 (1995), pp.  1229 and 1232.

          THE USEFULNESS OF PHYSICIAN
          AND PHYSICIAN GROUP REPORT
          CARDS REMAINS LIMITED
    ---------------------------------------------------------- Letter :4.2

    The early experience with physician report cards indicates that
    organizations are able to address some of the methodological
    challenges in performance measurement and provide some comparative
    information.  However, their ability to accurately report on broad
    measures of physician performance in a useful manner remains limited.

    First, while the current report cards measure the performance of
    physician groups that are smaller than health plans, the medical
    groups may still be too large to make the cards useful to consumers.
    One medical group that appears in all three California report cards
    includes nearly 700 physicians.  A consumer faced with the task of
    selecting a physician could question whether having a set of summary
    statistics covering so many physicians is really any more helpful
    than having planwide performance measures for thousands of
    physicians.

    Next, the quality of the data and the expense of collecting them are
    also issues.  PBGH, BHCAG, and Health Net used surveys to gather
    data.  As with all survey data, they reflect only the views of
    patients who chose to respond and then record their recollection or
    perception of the care they received, which may or may not be
    accurate.  To address these issues, steps must be taken to see if
    there is bias among respondents compared with nonrespondents and to
    limit questions to those that patients are likely to answer
    accurately.  And while surveys generally cost less than medical
    record reviews, they are still expensive to conduct.  According to
    one PBGH official, the Physician Value Check Survey costs
    approximately $15,000 per medical group.

    PacifiCare's Quality Index relies more heavily on gathering and
    analyzing administrative encounter data for its performance scores--a
    process that is generally less costly than using surveys but that has
    other limitations.  Administrative data reflect how physicians report
    the services and procedures they provided rather than the patients'
    recollection.  However, in some cases, the administrative data are
    not complete.  According to PacifiCare's Medical Director, the plan
    receives data from physicians on only about 70 percent of their
    encounters with patients.  He added that publishing the Quality Index
    has dramatically increased the volume of encounter data submissions.
    Before the Quality Index was published, PacifiCare received
    information on about 2 million encounters each month; 2 months after
    its publication, the plan was receiving data on about 5 million
    encounters per month.  Despite the increase in the volume of
    encounter data that medical groups provide, some groups question the
    completeness and the quality of the raw data.

    Third, current report cards do not provide consistent results.  For
    example, the PacifiCare and Health Net report cards demonstrate some
    of the difficulties when different organizations measure groups in
    different ways.  The two plans use different methods and different
    data sources:  PacifiCare uses administrative and other internal data
    sources, and Health Net uses enrollee satisfaction survey responses.
    The two plans also base their performance measures on a subset of a
    patient population--plan enrollees rather than all patients in a
    medical group.  Enrollee satisfaction scores from the two plans were
    significantly different in some cases.  From the enrollees'
    responses, Health Net classified one medical group as excellent"--a
    classification it gave to only about one-fourth of the medical
    groups--while PacifiCare classified the same group in the bottom
    third.  A consumer looking at Health Net's report card might be more
    inclined to select that medical group than a consumer looking at
    PacifiCare's report card, while a consumer who read both report cards
    would be confused as to how a single group could get such disparate
    ratings from two plans' enrollees.  According to experts we
    interviewed, such divergent scores on similar measures lead to
    skepticism among physicians and the general public about the
    usefulness of report cards.

    Finally, the differences in the specifications of the measures and
    presentation issues may cause additional confusion.  While the report
    cards measure some of the same aspects of care, their measures of
    clinical quality can be defined and reported differently.  For
    example, one plan may report the percentage of enrollees who were
    satisfied with a service, while another might report only the
    percentage who indicated that they were very or extremely satisfied.
    In addition, reporting issues such as the relative scale a plan uses
    can accentuate narrow differences among medical groups.  For example,
    under PacifiCare's scale, if all plans in the comparison fall between
    85 and 95 percent on a particular measure, the group performing the
    service 85 percent of the time could show up in the bottom 10th
    percentile, while the group performing the service 95 percent of the
    time could be listed as a best practices group.  Conversely, if all
    the groups perform a recommended service less than half the time,
    some of them will still be ranked as best practices groups.  Such
    complexity in interpreting the results can make consumers wary of
    report cards.

       NEW COLLABORATIVE EFFORTS AND
       DATA STANDARDIZATION MAY HELP
       MEET SOME CHALLENGES
    ------------------------------------------------------------ Letter :5

    While the work of purchasers, plans, and state agencies represents
    progress toward resolving the difficulties with measuring health care
    quality, further development is needed.  Several national groups have
    been organized to cooperatively develop standardized approaches to
    measurement-related issues.  In addition, HHS is taking some steps to
    facilitate better performance measurement.  These efforts are in
    their infancy, and it will take time to see what, if any, effect they
    have on measuring physician performance.

          PROFESSIONAL ORGANIZATIONS
          ARE BEGINNING TO COLLABORATE
          ON IMPROVEMENTS TO PHYSICIAN
          REPORT CARDS
    ---------------------------------------------------------- Letter :5.1

    Developing a commonly accepted, standardized set of performance
    measures is a critical step in creating a system of performance
    measurement that will allow apples to apples comparisons in health
    care.  Some of the organizations we talked to have recently joined
    together to address participant concerns about performance
    information and the factors in the marketplace that impede the flow
    of data.  It remains to be seen whether these coalitions can forge
    agreement on critical issues that must be addressed in the long term.
    For these organizations to be successful, disparate groups will have
    to reach consensus on a number of issues and that will take time.

    The California Information Exchange is a partnership of purchasers,
    providers, and other organizations established to promote and protect
    the exchange of data among health care partners such as health plans,
    purchasers, and providers.\23 According to one Exchange official, the
    group was formed, in part, to overcome political impediments to the
    exchange of health care information.  The Exchange has formed working
    groups to develop agreements to be used to define the content, proper
    use, and format for enrollment data; provider and provider group
    identifiers; laboratory and encounter records; individual patient
    identifiers; member identification cards; eligibility data; and
    pharmacy records.  To date, the Exchange's work groups have adopted
    agreements or rules of exchange for enrollment data, encounter data,
    eligibility data, member identification cards, and pharmacy records.
    The Exchange plans to test these agreements in a series of pilot
    projects.

    The Performance Measurement Coordinating Council is sponsored by
    three health accreditation agencies:  the American Medical
    Association's American Medical Accreditation Program, the Joint
    Commission on Accreditation of Health Care Organizations, and NCQA.
    Comprising 15 members chosen by the founding organizations, it was
    created in May 1998 to develop efficient and consistent performance
    measures for different levels in the health care system.  The Council
    brings together organizations working on quality measurement in
    different areas of the health care industry with different points of
    view on attribution, the public reporting of performance data, and
    the like.  For example, the American Medical Accreditation Program
    comes from an organization dedicated to representing the interests of
    physicians and is most cautious about attributing performance data to
    individual physicians and reporting on performance to the public.  At
    the same time, NCQA, which is a health plan accreditation
    organization, is a strong advocate for the public reporting of data.
    The progress of the Council illustrates the time it can take to work
    on performance measures.  For example, it took the Council 8 months
    to progress from its formation to the announcement that it intended
    to develop a common measurement agenda and to address a range of
    performance measurement issues.  As of May 1999, one year after its
    formation, the council had identified and started work on diabetes
    care measures--the first of its measurement sets.

    The National Forum for Health Care Quality Measurement and Reporting
    is a private, nonprofit entity whose purpose is to develop a
    comprehensive quality measurement and public reporting strategy.  The
    Forum followed from the recommendations of the President's Advisory
    Commission on Consumer Protection and Quality in the Health Care
    Industry.\24 Goals for the Forum include allowing meaningful quality
    comparisons of health care providers and plans and promoting
    competition in the quality of health care services.  In March 1999,
    the Forum planning committee approved the initial members of its
    board of directors.  Representation on the Forum's board is broad,
    including academic researchers and representatives from AHCPR; HCFA;
    representatives from consumer, public, and private purchasers;
    providers and plans; and research and quality improvement councils.
    As with the Performance Measurement Coordinating Council, the Forum's
    efforts are taking time.  The Forum took approximately 9 months to
    select its board of directors and does not expect individual work
    groups to begin work until early 2000.

    --------------------
    \23 Exchange partners include the American Medical Group Association,
    the California Association of Health Plans, the California Healthcare
    Association, the California Medical Association, the National IPA
    Coalition, and the Pacific Business Group on Health.

    \24 Advisory Commission on Consumer Protection and Quality in the
    Health Care Industry, Quality First:  Better Health Care for All
    Americans, Report to the President of the United States (Washington
    D.C.:  1998).

          HHS IS TAKING STEPS TO
          REPORT ON MEDICARE
          PHYSICIANS' PERFORMANCE AND
          TO STANDARDIZE HEALTH DATA
    ---------------------------------------------------------- Letter :5.2

    HHS has been involved in performance measurement through its
    administration of the Medicare program, efforts to standardize health
    data, and support of research.  In addition to meeting the
    information needs of Medicare beneficiaries, these efforts can have a
    substantial effect on report cards generated by private purchasers
    and plans.  Because most of HHS' initiatives are still in progress,
    this effect has yet to be determined.

    The Balanced Budget Act of 1997 (BBA) requires that HHS provide
    comparative data to Medicare beneficiaries including information
    about the benefits, quality, and performance (to the extent
    available) of health care options in their area to assist them in
    making informed choices under the Medicare+Choice Program.\25 To
    provide better quality and performance comparisons, HCFA contracted
    with Health Economics Research Inc.  in September 1997 for assistance
    in developing a performance measurement system.  The contractor is
    studying the feasibility of using HEDIS measures for fee-for-service
    Medicare at the group practice, local, and national levels.  As part
    of this 3-1/2-year contract, five clinical measures relevant to the
    Medicare population--retinal eye examinations for diabetics,
    follow-up care after mental health hospitalization, breast cancer
    screening with mammography, beta blocker treatment after a heart
    attack, and the Health of Seniors survey results--are being examined
    at four large group practices.

    Although the study on HEDIS measures for group practices is not
    expected to be completed until 2001, some difficulties, such as those
    associated with small sample sizes, have already been identified.\26
    For example, while the four group practices each had between 4,000
    and 40,000 Medicare fee-for-service beneficiaries, sample sizes for
    each measure fell considerably once population subsets of gender,
    age, or condition were identified.  HEDIS specifies 411 patients as a
    sufficient sample size, but this was obtained only for two of the
    three claims-based measures--breast cancer screening and retinal eye
    examinations.  For these two measures, the sample size was large
    enough only for the three group practices with more than 20,000
    Medicare fee-for-service beneficiaries.

    HHS' efforts to establish standards for information transactions and
    data elements, including unique identifiers for individuals, plans,
    and providers, may also have an effect on performance measurement
    systems.  Under the Health Insurance Portability and Accountability
    Act of 1996 (HIPAA) (P.L.  104-191), HHS is required to adopt
    standards to support the exchange of information on administrative
    and financial health care transactions.\27 The standards are to
    include data elements and code sets for the electronic exchange of
    information; unique health identifiers for health care providers,
    health plans, employers, and individuals for use in the health care
    system; and security protections against the unauthorized disclosure
    and use of health information.  The standards are to apply to all
    health plans, health care clearinghouses, and health care providers
    that transmit health information electronically.\28

    Some standards under development, such as the unique identifier for
    individuals, have been contentious.  In response to concerns about
    privacy, three bills were introduced in the 105\th and 106\th
    Congresses to repeal the requirement for HHS to adopt a standard
    unique health identifier.  While the two bills from the 105\th
    Congress expired and one is pending before the current Congress, the
    Omnibus Consolidated and Emergency Supplemental Appropriations Act
    for 1999 (P.L.  105-277) provided that no funds available under the
    act be used to adopt a final standard for individual unique health
    identifiers until legislation is enacted specifically approving the
    standard.  While the implementation of the HIPAA standards has the
    potential to significantly improve the usability of the health data
    available for performance measurement, it will not address all the
    data challenges, such as those related to data accuracy.

    An additional HHS effort under way to further physician performance
    measurement is the development of a consumer satisfaction survey for
    fee-for-service Medicare beneficiaries.  AHCPR officials said that
    they are interested in adapting the CAHPS survey--an instrument for
    measuring consumer satisfaction and experience with health plans--to
    the provider level.  AHCPR is studying the use of the CAHPS survey
    with smaller units, such as group practices or individual physicians.
    In addition, AHCPR is sponsoring research on performance measurement
    and is working with others to develop a framework for measuring
    health care performance.

    --------------------
    \25 Created by the BBA, the Medicare+Choice program is designed to
    allow beneficiaries to choose health care from Medicare's traditional
    fee-for-service program and a range of health plans, such as health
    maintenance organizations and provider-sponsored organizations,
    participating in Medicare.

    \26 In addition to the challenge of developing comparable performance
    information for group practices under fee-for-service Medicare, HCFA
    will have to ensure that the information provided to beneficiaries is
    clear, sufficient, and helpful to their decisionmaking or it will not
    be used.  For example, we previously reported on problems with HCFA's
    efforts to provide comparative information on health plans--HCFA had
    not provided information that was easy for beneficiaries to
    understand.  See Medicare:  HCFA Should Release Data to Aid
    Consumers, Prompt Better HMO Performance (GAO/HEHS-97-23, Oct.  22,
    1996) and Medicare:  Progress to Date in Implementing Certain Major
    Balanced Budget Act Reforms (GAO/T-HEHS-99-87, Mar.  17, 1999).
    Physician-level information runs the risk of having similar problems.

    \27 Transactions include health claims or equivalent encounter
    information, enrollment and disenrollment in a health plan, health
    care payment and remittance advice, health plan premium payments,
    first report of injury, health claims status, and referral
    certification and authorization.

    \28 Under HIPAA, standards were required to be enacted by August 21,
    1999, regarding the privacy of individually identifiable health
    information that is electronically exchanged.  Because this deadline
    was not met, HIPAA now requires the Secretary of HHS to establish
    standards by regulation no later than February 21, 2000.

       CONCLUSIONS
    ------------------------------------------------------------ Letter :6

    Consumers could use more information on the quality of health care
    providers to help them make informed choices about where to seek
    care.  Comparative information on physicians is important to all
    consumers, whether they enroll in traditional Medicare or in a
    private health plan or face a choice of primary care physicians when
    they join a managed care plan.  Yet the field of physician
    performance measurement is still in its infancy.  Challenges to
    developing physician report cards include selecting performance
    measures that satisfy the information needs of various audiences and
    gaining the cooperation of physicians and consumers required to
    assemble consistent and credible performance data.  The experience of
    several organizations in producing medical group or specialty care
    report cards indicates that steps can be taken to better inform
    consumers, but the challenges that remain limit the report cards'
    usefulness.  Given sufficient time, public and private efforts to
    develop a consensus on standardized data collection and comparable
    quality measurement may lead to more useful measures for consumers
    through a more efficient system for providers and plans nationwide.

       AGENCY COMMENTS
    ------------------------------------------------------------ Letter :7

    We obtained informal comments on a draft of this report from the
    Senior Clinical Adviser in HCFA's Office of Clinical Standards and
    Quality.  She agreed with the report's general findings.  She
    suggested that under the fee-for-service payment system, we highlight
    the problem of determining which physician is accountable for
    managing a patient's care.  The logistics of establishing the
    linkages by means of existing medical records is another area of
    concern that she recommended we stress in the report.

    We also obtained comments on the draft from an expert in quality
    measurement who suggested that we include more information on the
    methodological challenges of assessing physician performance.  He
    felt that a stronger critique of the validity of currently available
    measures would be helpful in the analysis of physician report cards.
    He noted that it is impossible to differentiate among providers with
    current physician report cards and warned against the dangers of
    misinformation.  He encouraged us to place more emphasis on the need
    for research efforts to develop better measures that provide valid
    information and to improve our understanding of preferred clinical
    strategies.  He also emphasized the need for developing electronic
    medical records for access to clinically relevant data.

    ---------------------------------------------------------- Letter :7.1

    As agreed with your office, unless you publicly announce its contents
    earlier, we plan no further distribution of this report until 30 days
    from the date of this letter.  We will then send copies of this
    report to the Secretary of HHS and others who are interested.  We
    will also make copies available to others on request.

    The information contained in this report was developed by Rosamond
    Katz, Assistant Director, Mark Ulanowicz, and Patricia K.  Yamane.
    Please contact me at (202) 512-7114 or Rosamond Katz at (202)
    512-7148 if you or your staff have any questions.

    Sincerely yours,

    Janet Heinrich
    Associate Director, Health Financing
     and Public Health Issues

    REPORT CARDS COMPARING MEDICAL
    GROUPS
    ==================================================== Appendix Appendix

    In California, a purchasing group, the Pacific Business Group on
    Health (PBGH), and two health plans, PacifiCare and Health Net, have
    moved a step closer to reporting on physician performance by
    publishing report cards on medical groups and independent practice
    associations (IPA).  In addition, another purchasing group, the
    Buyers Health Care Action Group (BHCAG) in Minnesota, has published
    report cards on care systems, which can be similar to large medical
    groups.  Below, we illustrate the information generated on 3 of the
    11 medical groups common to all three California report cards and one
    Minnesota care system.  In California, medical group A includes more
    than 100 physicians, medical group B has 675 physicians, and medical
    group C has more than 300 physicians.  In Minnesota, the care system
    includes 550 physicians.

       PACIFIC BUSINESS GROUP ON
       HEALTH
    -------------------------------------------------- Appendix Appendix:1

    PBGH publishes a report card on medical groups that is based on its
    Physician Value Check Survey.  Figure I.1 shows the scores for three
    southern California medical groups as reported on the PBGH Internet
    site.  The numerical scores are divided into three categories that
    indicate relative measures of performance:  above average, below
    average, and average.

       Figure I.1:  1996 PBGH
       Physician Value Check Survey
       Scores for Three Southern
       California Medical Groups

       (See figure in printed
       edition.)

    Note:  The numerical scores represent scores based on survey
    responses for the medical groups.  Scores above and below average
    indicate a relative measure of the groups.

    Source:  Pacific Business Group on Health, California Consumer
    Healthscope, at http://www.healthscope.org.

    A consumer reading this report card would learn that medical group B
    received high overall scores for satisfaction and cholesterol
    screening.  However, a person with high blood pressure would notice
    that while groups B and C scored below average in prescribing
    medicine for high blood pressure, group C had better success reducing
    its patients' blood pressure.  People with diabetes would not find
    any information specific to the treatment of their condition.

       PACIFICARE
    -------------------------------------------------- Appendix Appendix:2

    PacifiCare's Quality Index report card reflects the health care
    experiences and opinions of members of the PacifiCare health plan
    only.  Figure I.2 shows the 1998 Quality Index scores for the same
    medical groups highlighted in figure I.1.  The numerical scores
    represent a percentile rank for a medical group compared with that of
    all other groups.  PacifiCare identifies groups as best practice
    groups for a particular measure if they are in the top 10 percent
    relative to other groups.  These are denoted by a diamond next to the
    number in the table.

       Figure I.2:  1998 PacifiCare
       Quality Index Scores for Its
       Commercial Members at Three
       Medical Groups

       (See figure in printed
       edition.)

    \a Data below threshold:  the medical group did not have enough
    PacifiCare enrollees with congestive heart failure to allow for
    statistically valid measurement.

    \b Includes responses from both commercial and Secure Horizons
    members.

    Source:  PacifiCare.

    The comparative performance information in figure I.2 is limited and
    selective and may not be adequate for choosing a medical group.  A
    consumer reading this report card would learn that PacifiCare members
    using medical group B were not happy with their access to care
    relative to the other groups' patients--it scored in the bottom 15
    percent for access-related complaints--but were very satisfied with
    the group's primary care physicians--rating them in the top 10
    percent in satisfaction.  The report card also indicates that medical
    group C was in the bottom third of medical groups for cervical cancer
    screening but in the top 10 percent for benefits appeals to
    PacifiCare.

       HEALTH NET
    -------------------------------------------------- Appendix Appendix:3

    Health Net's Participating Physician Group Report Card is derived
    from a satisfaction survey of the plan's members.  Figure I.3 shows
    selected Health Net report card scores for the same three southern
    California medical groups as shown in figures I.1 and I.2.  The
    numerical scores represent the percentage of respondents who reported
    that they were satisfied regarding each measure, and the groups are
    classified as excellent, very good, or good.

       Figure I.3:  1998 Health Net
       Participating Physician Group
       Report Care

       (See figure in printed
       edition.)

    Source:  Health Net Participating Physician Group Report Card, Sept.
    1998.

    A consumer reading this report card would find that the three medical
    groups were largely undifferentiated.  They all were rated either
    very good or excellent, both overall and within the three broad
    categories of quality of care, access to care, and medical group
    satisfaction.  Unlike the PacifiCare Quality Index report, the Health
    Net participating physician group report card provides information
    only on members' satisfaction with each issue--it does not provide
    information on the extent to which particular services, such as
    mammograms or cervical cancer screenings, were provided.

    Health Net is also developing report cards on care provided by
    medical groups for certain chronic conditions.  Figure I.4 shows the
    asthma report card scores for medical groups A and B.  Medical group
    C did not have enough asthma patients responding to the survey to be
    included in the comparison.  The four stars denote very good.

       Figure I.4:  1998 Health Net
       Asthma Report Card

       (See figure in printed
       edition.)

    Source:  Health Net Participating Physician Group Asthma Care Report
    Card, Dec.  1998.

    For patients with asthma trying to choose a medical group, the Health
    Net asthma report card provides a considerable amount of information
    on clinical quality, including information on outcomes of care.  It
    indicates that even though both groups were rated very good for
    asthma care, no patients with asthma in group A and fewer than 1 in
    10 in group B reported using a peak-flow meter daily, even though
    daily use is recommended in national clinical guidelines.  In
    addition, the report tells consumers that a higher share of the
    survey respondents from medical group A reported no asthma-related
    absences from work or school in the past month than respondents from
    medical group B.

       MINNESOTA'S BUYERS HEALTH CARE
       ACTION GROUP
    -------------------------------------------------- Appendix Appendix:4

    BHCAG is currently reporting on 12 measures focusing on such issues
    as access to services and interactions with physicians.  Figure I.5
    shows the results of the BHCAG survey for one care system, a
    550-physician multispecialty medical group.  The comparison rating
    shows whether the survey ratings for the care system are better than,
    similar to, or below the average rating.  The numerical scores are
    the statistics for each measure.

       Figure I.5:  1998 BHCAG Report
       Card on One Care System

       (See figure in printed
       edition.)

    Source:  Choice Plus 1999 Consumer Satisfaction Survey Results.

    A consumer reading this report card would learn that this care system
    was scored average by its patients in terms of overall satisfaction
    and quality of care but was scored below average in the areas related
    to interaction with physicians, such as a physician's explaining
    medical procedures and tests.

    *** End of document. ***