Defense Health Care: DOD Could Improve Its Beneficiary Feedback
Approaches (Letter Report, 02/06/98, GAO/HEHS-98-51).
Pursuant to a congressional request, GAO reviewed: (1) whether the
Department of Defense (DOD) solicits feedback from beneficiaries of its
managed health care program, TRICARE, and, if so, how this is done and
what the data show; (2) what other means are available to beneficiaries
to provide feedback and what such beneficiary-initiated feedback could
reveal about TRICARE's success; and (3) how DOD's approaches to
obtaining feedback compare with the private sector's and whether
opportunities exist to improve DOD's beneficiary feedback tracking and
reporting.
GAO noted that: (1) DOD obtains and uses TRICARE beneficiary feedback in
several ways across the military health system (MHS): (2) DOD conducts a
broad annual beneficiary questionnaire survey and a monthly survey of
patients' perceptions of military treatment facilities (MTF) outpatient
visits--both of which are based on private-sector models--to measure
levels of satisfaction with TRICARE; (3) DOD reports the survey results
throughout the MHS; (4) DOD does not conduct such surveys of MTF
inpatient users or civilian network care users, though DOD officials
told GAO that they are now planning to develop an MTF inpatient survey;
(5) as TRICARE continues to be phased in across the MHS, DOD's annual
surveys are indicating fairly levels of overall beneficiary satisfaction
with the program, but lower satisfaction levels with aspects of military
care; (6) DOD also tracks and reports beneficiary-initiated
feedback--complaints and other comments--in ways that vary throughout
the MHS; (7) a wide range exists in how much feedback information is
tracked and in how the different levels of units that compose
TRICARE--and other DOD offices--do the tracking; (8)
beneficiary-initiated feedback reporting throughout the MHS varies as
well; (9) because of the variability of DOD's recording of these data,
reliably depicting the range, magnitude, or frequency of beneficiary
feedback about TRICARE is not possible; (10) private health care
managers rely extensively on beneficiary feedback; (11) surveys, which
provide data about whole customer populations, and customer-initiated
complaints, which show where specific problems have occurred, are used
together as key tools to measure plan performance and identify systemic
problems; (12) while no direct private-sector parallel to MHS exists,
DOD's feedback efforts are somewhat similar to the private sector's,
although adopting certain private practices might improve DOD's feedback
systems; (13) more reliable beneficiary feedback data would also help
DOD to make customer satisfaction an outcome measure in the next round
of TRICARE contracts, which DOD is trying to base more on outcomes and
less on process; and (14) but, to improve its beneficiary feedback
approaches, DOD will need to consider a number of cost-benefit issues,
the varying sophistication levels of beneficiary feedback management
throughout MHS, and other matters.
--------------------------- Indexing Terms -----------------------------
REPORTNUM: HEHS-98-51
TITLE: Defense Health Care: DOD Could Improve Its Beneficiary
Feedback Approaches
DATE: 02/06/98
SUBJECT: Comparative analysis
Managed health care
Military personnel
Health centers
Customer service
Surveys
Health care services
Beneficiaries
IDENTIFIER: DOD TRICARE Program
NCQA Health Plan Employer Data and Information Set
DOD TRICARE Standard Program
DOD TRICARE Prime Program
******************************************************************
** This file contains an ASCII representation of the text of a **
** GAO report. Delineations within the text indicating chapter **
** titles, headings, and bullets are preserved. Major **
** divisions and subdivisions of the text, such as Chapters, **
** Sections, and Appendixes, are identified by double and **
** single lines. The numbers on the right end of these lines **
** indicate the position of each of the subsections in the **
** document outline. These numbers do NOT correspond with the **
** page numbers of the printed product. **
** **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced. Tables are included, but **
** may not resemble those in the printed version. **
** **
** Please see the PDF (Portable Document Format) file, when **
** available, for a complete electronic file of the printed **
** document's contents. **
** **
** A printed copy of this report may be obtained from the GAO **
** Document Distribution Center. For further details, please **
** send an e-mail message to: **
** **
** **
** **
** with the message 'info' in the body. **
******************************************************************
Cover
================================================================ COVER
Report to the Subcommittee on Military Personnel, Committee on
National Security, House of Representatives
February 1998
DEFENSE HEALTH CARE - DOD COULD
IMPROVE ITS BENEFICIARY FEEDBACK
APPROACHES
GAO/HEHS-98-51
TRICARE Beneficiary Feedback
(101600)
Abbreviations
=============================================================== ABBREV
COTR - contracting officer's technical representative
DOD - Department of Defense
HEDIS - Health Plan Employer Data and Information Set
HMO - health maintenance organization
JCAHO - Joint Commission on the Accreditation of Healthcare
Organizations
MHS - Military Health System
MTF - military treatment facility
NCQA - National Committee for Quality Assurance
PCM - primary care manager
TSO - TRICARE Support Office
Letter
=============================================================== LETTER
B-278623
February 6, 1998
The Honorable Stephen E. Buyer
Chairman
The Honorable Gene Taylor
Ranking Minority Member
Subcommittee on Military Personnel
Committee on National Security
House of Representatives
Medical care is of great importance to the 8 million people eligible
to receive health care services through the $15 billion-per-year
Military Health System (MHS). In 1993, the Department of Defense
(DOD) began implementing a major change in its health care system:
conversion to a managed care program known as TRICARE. Just as in
the private sector, where customer feedback is used as a key
management tool, an important measure of TRICARE's success should be
whether beneficiaries are satisfied with TRICARE and what their views
are of DOD's new health care system.
In light of the importance of TRICARE, you asked that we review (1)
whether DOD solicits TRICARE beneficiaries' feedback and, if so, how
this is done (such as through surveys) and what the data show; (2)
what other means are available to beneficiaries to provide feedback
and what such beneficiary-initiated feedback could reveal about
TRICARE's success; and (3) how DOD's approaches to obtaining feedback
compare with the private sector's and whether opportunities exist to
improve DOD's beneficiary feedback tracking and reporting.
In doing this work, we interviewed and obtained documents from MHS
officials, including officials of the Office of the Assistant
Secretary of Defense (Health Affairs), the Army's Office of the
Surgeon General, the Navy's Bureau of Medicine and Surgery, and the
Air Force's Office of the Surgeon General. We visited seven military
treatment facilities (MTF) in three TRICARE regions: Southeast,
Southern California, and Southwest. Also, we interviewed and
obtained documents from headquarters and field office representatives
of two TRICARE managed care support contractors: Foundation Health
Federal Services and Humana Military Healthcare Services.\1 To
compare DOD's beneficiary feedback approaches with private
approaches, we interviewed representatives from the Joint Commission
on the Accreditation of Healthcare Organizations (JCAHO); the
National Committee for Quality Assurance (NCQA); Kaiser Permanente, a
major commercial health maintenance organization (HMO); and Inova
Health System, a Northern Virginia hospital chain. Finally, we
discussed TRICARE beneficiaries' perceptions of the program with
representatives of one beneficiary group, the National Military
Family Association. Although we identified the varying ways that DOD
and its contractors gather and process beneficiary-initiated comments
and obtained numerous sample comments, we did not review whether or
how individual complaints were resolved or whether they were valid.
Details of our scope and methodology appear in appendix I.
--------------------
\1 DOD contracts with managed care support contractors, which are
private sector health care organizations, to carry out such tasks as
developing networks of civilian providers to supplement the services
of MTFs.
RESULTS IN BRIEF
------------------------------------------------------------ Letter :1
DOD obtains and uses TRICARE beneficiary feedback in several ways
across the MHS. DOD conducts a broad annual beneficiary
questionnaire survey and a monthly survey of patients' perceptions of
MTF outpatient visits--both of which are based on private sector
models--to measure levels of satisfaction with TRICARE. DOD reports
the survey results throughout the MHS. DOD does not conduct such
surveys of MTF inpatient users or civilian network care users, though
DOD officials told us they are now planning to develop an MTF
inpatient survey. As TRICARE continues to be phased in across the
MHS, DOD's annual surveys\2 are indicating fairly high levels of
overall beneficiary satisfaction with the program, but lower
satisfaction levels with particular aspects of military care. The
MTF outpatient surveys show satisfaction levels that, on average,
exceed those of civilian HMO beneficiaries. DOD officials cautioned,
however, that TRICARE is too new for these results to be used as an
overall program success measure.
DOD also tracks and reports beneficiary-initiated
feedback--complaints and other comments--in ways that vary throughout
the MHS. A wide range exists both in how much feedback information
is tracked and in how the different levels of units that compose
TRICARE--and other DOD offices--do the tracking.
Beneficiary-initiated feedback reporting throughout the MHS varies as
well; in the regions we visited, reporting was mostly ad hoc. And,
because of the variability of DOD's recording of these data, reliably
depicting the range, magnitude, or frequency of beneficiary feedback
about TRICARE is not possible. Nevertheless, our review did identify
examples of complaints about access to care, quality of care and
administrative services, and care-related cost issues. We also found
examples of positive comments about TRICARE.
Private health care managers rely extensively on beneficiary
feedback. Surveys, which provide data about whole customer
populations, and customer-initiated complaints, which show where
specific problems have occurred, are used together as key tools to
measure plan performance and identify systemic problems. While no
direct private sector parallel to MHS exists, DOD's feedback efforts
are somewhat similar to the private sector's, although adopting
certain private practices might improve DOD's feedback systems. For
example, if DOD implemented the MTF inpatient survey it is planning
and conducted a survey of network care users (surveys that are
similar to those used by private sector plans) it would have more
complete information about TRICARE. Also, if DOD consistently
recorded and aggregated complaint data across the system--which NCQA
believes to be a prudent approach to customer feedback management,
and which private sector plan managers routinely do--DOD could
identify trends and target core process problems needing attention
across the MHS.
More reliable beneficiary feedback data would also help DOD to make
customer satisfaction an outcome measure in the next round of TRICARE
contracts, which DOD is trying to base more on outcomes and less on
process.\3
But, to improve its beneficiary feedback approaches, DOD will need to
consider a number of cost-benefit issues, the varying sophistication
levels of beneficiary feedback management throughout MHS, and other
matters.
--------------------
\2 The most recent annual survey results available are for 1996, when
TRICARE health care delivery had begun in only the Golden Gate,
Northwest, Pacific, Southern California, and Southwest regions.
\3 See Defense Health Care: Despite TRICARE Procurement
Improvements, Problems Remain (GAO/HEHS-95-142, Aug. 3, 1995).
BACKGROUND
------------------------------------------------------------ Letter :2
DOD's primary military medical mission is to maintain the health of
1.6 million active duty service personnel\4 and be prepared to
deliver health care during wartime. Also, as an employer, DOD offers
health services to 6.6 million additional military-related
beneficiaries, including active duty members' dependents and military
retirees and their dependents. Most care is provided in 115
hospitals and 471 clinics--called military treatment
facilities--operated by the Army, Navy, and Air Force worldwide.
This direct delivery health system is supplemented by DOD-funded care
provided in civilian facilities. In fiscal year 1997, DOD spent
about $12 billion for direct care and about $3.5 billion for civilian
facility care.
In the late 1980s, in response to increasing health care costs and
uneven access to care, DOD initiated, with congressional authority, a
series of demonstrations to evaluate alternative health care delivery
approaches. On the basis of this experience, DOD designed TRICARE as
its managed health care program. TRICARE is intended to ensure a
high-quality, consistent health care benefit; preserve choice of
health care providers for beneficiaries; improve access to care; and
contain health care costs. TRICARE is designed to give beneficiaries
a choice among three approaches to health care: TRICARE Prime, an
HMO-like option; TRICARE Extra, which is similar to a preferred
provider option; and TRICARE Standard, a fee-for-service-type option.
The TRICARE program uses regional managed care support contracts to
augment its MTFs. The contractors' responsibilities include
developing civilian provider networks, performing utilization
management functions,\5 processing claims, and providing such support
functions as beneficiary education and enrollment. The 11 TRICARE
regions in the United States are covered by seven managed care
support contracts, and health care delivery has commenced under five
of the contracts (see fig. 1).
Figure 1: TRICARE Regions and
Their Health Care Delivery
Start Dates
(See figure in printed
edition.)
Note: TRICARE Europe and TRICARE Latin America are not shown.
\a Projected implementation date.
The Office of the Assistant Secretary of Defense (Health Affairs)
(hereafter referred to as Health Affairs) sets TRICARE policy and has
overall responsibility for the program. The managed care support
contractors are overseen by the TRICARE Support Office (TSO), a part
of Health Affairs. The Army, Navy, and Air Force Surgeons General
have authority over the MTFs in their respective services. To
coordinate MTF and contractor services, each region is headed by a
"lead agent," which is led by a designated MTF commander and
supported by a joint-service staff. The lead agent responds to
direction from Health Affairs, but the services retain authority and
control over their medical facilities and personnel. Therefore, lead
agents seek to affect operations by working cooperatively with the
MTFs in their region and the regional managed care support
contractor.
--------------------
\4 TRICARE also covers members of the Coast Guard, the Commissioned
Corps of the Public Health Service, and the National Oceanic and
Atmospheric Administration who are eligible for military health care.
\5 Utilization management involves using such techniques as
preadmission hospital certification, concurrent and retrospective
reviews, and case management to determine the appropriateness,
timeliness, and medical necessity of an individual's treatment.
DOD USES SURVEYS TO SOLICIT
SOME BENEFICIARY FEEDBACK
------------------------------------------------------------ Letter :3
DOD conducts beneficiary satisfaction surveys--a common private
sector health care practice--to measure TRICARE's performance and
reports the results throughout the MHS. Health Affairs currently
conducts two such ongoing surveys: an annual systemwide survey of
all eligible beneficiaries and a monthly survey of patients'
perceptions of outpatient visits at MTFs.\6 Both surveys are based on
widely used private sector survey instruments. Health Affairs'
TRICARE Marketing Office also conducted a survey of TRICARE Prime
enrollees' satisfaction in 1996. Health Affairs officials told us
that a systemwide survey targeted to MTF inpatient care is currently
being planned, and a survey targeted to civilian TRICARE network care
is under discussion. DOD policy requires most other beneficiary
surveys--whether proposed by the services, MTFs, or managed care
support contractors--to first be approved by Health Affairs. The
annual surveys have indicated generally high overall satisfaction
levels, with mixed results for satisfaction with particular aspects
of military health care. The MTF outpatient surveys have shown
satisfaction levels higher than civilian HMOs', and the TRICARE Prime
enrollee survey showed satisfaction levels somewhat lower than those
of the private sector. However, officials also told us that it is
too soon to use DOD's survey results as a measure of TRICARE's
overall success. Detailed descriptions of the surveys are provided
in appendix II.
--------------------
\6 DOD conducts three additional beneficiary surveys not focused on
beneficiary satisfaction: a survey of health-related behaviors among
military personnel, a health assessment of TRICARE Prime enrollees,
and a survey of MHS beneficiaries' health care sources (see app.
II).
ANNUAL SURVEYS
---------------------------------------------------------- Letter :3.1
Public Law 102-484 requires DOD to conduct an annual beneficiary
survey. The survey's purpose is to provide a comprehensive look at
how beneficiaries view their health care--including their health
status, the availability of health services, and related matters.
The questions and scales used in the annual survey were based on
private sector surveys that had been extensively tested for
reliability and validity. DOD uses the survey responses to represent
all eligible beneficiaries' views and reports results for each MTF
catchment area.\7
DOD's 1996 annual survey results show that active duty family
members' satisfaction generally increased when compared with 1994-95
results, while satisfaction decreased for retirees and their family
members. But retirees' satisfaction generally remained higher than
that of active duty family members in both surveys. Moreover, active
duty family members' satisfaction was slightly higher in regions in
which TRICARE had been implemented than in the other regions. In the
1994-95 survey, retirees and their family members in TRICARE regions
reported higher satisfaction than their counterparts in the other
regions, but in 1996 the two results were about the same, as shown in
figure 2.
Figure 2: Overall Beneficiary
Satisfaction With TRICARE,
1994-95 and 1996 Annual Surveys
(See figure in printed
edition.)
Notes: DOD defined the Golden Gate, Northwest, Pacific-Hawaii,
Southern California, and Southwest regions as TRICARE regions because
it considered the program to have been in place in these regions for
a sufficient period at the time of the 1996 survey.
"Overall satisfaction" is based on responses to two statements: "I
am satisfied with the health care I receive" and "I would recommend
this type of health care to my family or friends."
While overall satisfaction levels were fairly high, satisfaction with
certain aspects of military health care was somewhat lower, according
to the 1996 annual survey (see fig. 3).
Figure 3: Comparison of
Beneficiaries' Satisfaction
With Specific Aspects of
TRICARE, 1996 Annual Survey
(See figure in printed
edition.)
Results for beneficiaries not enrolled in TRICARE Prime are for only
those who had the option of enrolling and therefore do not include
regions without TRICARE or any beneficiaries aged 65 or over.
"Quality of care" focuses on individuals' satisfaction with skill,
thoroughness, and outcomes of health care. "Interpersonal concern"
looks at attention, courtesy, and concern shown by physicians and
other medical personnel. "Access to appointments" addresses
convenience of arranging appointments. "Choice" focuses on
individuals' ability to choose a provider and to see their provider
of choice.
DOD survey officials told us it was too soon to use these annual
survey results to assess TRICARE because the program is new and not
yet implemented nationwide. Also, they said two surveys constitute
an insufficient basis from which to identify trends, and several more
annual surveys are needed of the fully implemented program before the
results can be used as an overall system performance measure.
Nonetheless, the lead official for DOD's survey efforts told us of
uses already being made of the annual survey's results. For example,
the 1994-95 results showed that beneficiaries were more satisfied
with civilian care than with military care, which led Health Affairs
and the service Surgeon General offices to design a survey targeting
MTF outpatients' perceptions of the care they received. (This survey
will be discussed further below.) Also, in implementing its new
Enrollment Based Capitation financing approach,\8 DOD is using the
annual survey's health status measures and results to adjust the
various MTF enrollee populations for their projected health care
needs. DOD is risk-adjusting the enrollee populations on the basis
of such demographic factors as age, sex, beneficiary category, and
military service, which correlate with differing health care service
need levels.
--------------------
\7 An MTF's catchment area is the area within a radius of
approximately 40 miles of the facility.
\8 Enrollment Based Capitation is a new MTF financing approach
introduced by Health Affairs on a pilot basis in Oct. 1997. Rather
than allocating MTF funding on the basis of estimated care users, as
is done now, the new approach allocates funds primarily on the basis
of the number of TRICARE Prime enrollees at each MTF, adjusted for
the projected use and provision of MTF care.
OUTPATIENT SURVEYS
---------------------------------------------------------- Letter :3.2
Health Affairs also conducts a monthly MTF survey of patients'
perceptions of outpatient visits. The survey provides detailed
information on specific visits to individual clinics at all MTFs in
the 50 states. Health Affairs officials told us that because the
1994-95 annual survey results showed that beneficiaries were more
satisfied with civilian care than with military care, this survey was
designed to more closely examine MTF care. The MTF outpatient survey
was also based on survey questions developed, tested, and used by the
private sector, which has facilitated comparisons of MTF and civilian
care satisfaction levels.\9 Health Affairs provides detailed survey
results reports to MTFs and summary reports to lead agents and
service commands.
DOD provided us with April, May, and June 1997\10 MTF outpatient
survey results for each service and region. The results measure
satisfaction on a 5-point scale in three areas: (1) access to care
for a single visit, (2) quality of care during that visit, and (3)
staff interaction with the survey respondent during the visit. The
reports also include private sector survey results that show how
civilian HMO users rate their satisfaction in the same areas. Figure
4 shows results for the entire MHS, each service, and the civilian
managed care industry. Satisfaction among the three services' MTFs
is similar, and averages for all three are somewhat higher than
national civilian benchmarks. Results by region are also consistent
across the MHS, and all of the region averages exceed civilian HMO
benchmarks. See appendix II for each region's results and
comparative civilian HMO scores in corresponding geographic areas.
Figure 4: Beneficiary
Satisfaction With MTF
Outpatient Care Visits Compared
With Civilian HMO Benchmarks,
April-June 1997
(See figure in printed
edition.)
Notes: Satisfaction is measured on a 5-point scale, with 1 equaling
"poor" and 5 equaling "excellent."
"Satisfaction with access" focuses on individuals' satisfaction with
referral for specialty care, access to medical care, office wait
time, time to return phone calls, ease of making phone appointments,
and appointment wait time. "Satisfaction with quality" addresses
overall quality of care received, how well care met needs,
thoroughness of treatment, how much individual was helped, and
explanations of procedures and tests. "Satisfaction with staff
interaction" focuses on personal interest in patient, advice on ways
to avoid illness or stay healthy, amount of time with doctor and
staff, attention to what patients said, and friendliness and courtesy
of staff.
--------------------
\9 DOD compared its survey results with those found in the National
Research Corporation's Health Care Market Guide, which reports
results from interviews with more than 130,000 HMO enrollees.
\10 April, May, and June 1997 are the first 3 consecutive months for
which survey results were available.
TRICARE PRIME SURVEY
---------------------------------------------------------- Letter :3.3
In 1996, to help direct TRICARE marketing and beneficiary education
efforts, Health Affairs' TRICARE Marketing Office conducted a
telephone survey of beneficiaries enrolled in TRICARE Prime. The
survey addressed enrollees' understanding of the Prime program,
satisfaction with program aspects, perceptions about access and
quality changes after Prime's implementation, and intentions
regarding reenrolling in TRICARE Prime. Health Affairs compared the
survey results with civilian managed care programs' satisfaction
levels.
DOD's survey report describes high overall satisfaction levels, with
about two-thirds of Prime enrollees either satisfied or very
satisfied with TRICARE, and slightly higher ratings from non-active
duty TRICARE Prime enrollees.\11 Only 7 percent of respondents said
they were unlikely to reenroll in TRICARE Prime, while 88 percent
said they were likely or very likely to do so. DOD reported,
however, that overall satisfaction levels with TRICARE Prime trailed
the civilian sector average by about 16 percentage points. The
report notes, though, that the results may be skewed by response
format differences between DOD's questionnaire and the civilian
instrument. Table 1 shows the survey results for overall
satisfaction.
Table 1
Overall Satisfaction With TRICARE Prime,
1996 Beneficiary Satisfaction Survey
(Percentage)
Active
duty
Active personnel Retiree
duty family family
personnel members Retirees members
------------ ---------- ---------- -------- ----------
Very 19 25 29 27
satisfied
Satisfied 45 48 41 44
Neither 23 15 11 13
satisfied
or
dissatisfie
d
Dissatisfied 9 9 11 10
Very 4 3 8 6
dissatisfied
----------------------------------------------------------
--------------------
\11 Active duty personnel are automatically enrolled in TRICARE Prime
and are not allowed to obtain care under TRICARE Extra or Standard.
EFFORTS ARE UNDER WAY TO
SURVEY INPATIENT AND
CONTRACTOR NETWORK CARE
---------------------------------------------------------- Letter :3.4
DOD does not currently conduct systemwide surveys targeted to MTF
inpatient or network user satisfaction. However, Health Affairs
officials told us that a working group of representatives from the
Army, Navy, and Air Force Surgeons' General Offices is planning to
develop a means of surveying beneficiaries about their MTF inpatient
care. The group has begun by reviewing inpatient surveys currently
used by MTFs and the civilian health care industry.
Also, DOD recently eliminated a contract requirement that each
managed care support contractor conduct its own annual beneficiary
survey. Health Affairs officials told us they concluded that
contractor-conducted surveys might lack the appearance of
independence and were somewhat at odds with Health Affairs' interest
in standardizing surveys and reducing the survey burden on
beneficiaries. Officials of Health Affairs, the services, and
managed care support contractors are now discussing how best to
obtain beneficiary views on network care using such a targeted
survey.
DOD OBTAINS AND PROCESSES
BENEFICIARY-INITIATED COMMENTS
IN VARYING WAYS ACROSS THE MHS
------------------------------------------------------------ Letter :4
DOD documents, analyzes for trends, and reports on TRICARE
beneficiaries' complaints and compliments in differing ways
throughout the MHS.\12 All MHS levels, from DOD headquarters offices
to TSO to MTFs and managed care support contractors, receive
beneficiary-initiated feedback through such means as phone calls,
letters, and personal visits. Like the private sector, DOD officials
told us they use this information to identify problems and gauge
performance of various MTF services. We obtained many examples of
beneficiary-initiated complaints and other comments covering a host
of issues. However, because beneficiary comments were not
consistently documented, the examples we obtained cannot be viewed as
representative of all TRICARE beneficiary-initiated feedback.
Nevertheless, the examples do illustrate the types of issues military
health care beneficiaries choose to raise. Detailed descriptions of
feedback-related processes are provided in appendix III.
Because neither DOD nor the services require MTFs to follow a
standard procedure for tracking and reporting beneficiary comments,
MTFs are free to establish their own feedback systems.\13 As a
result, the MTFs we visited have differing ways of obtaining,
documenting, and analyzing beneficiary-initiated feedback. The MTFs
also have different ways of reporting their feedback to MTF
management and others within the facility. We also found, with few
exceptions, that most reporting of feedback to entities outside MTFs
is not done systematically.
Lead agents also capture information on beneficiary-initiated
concerns in varying ways. Each of the three lead agents we visited
has systems according to which its MTFs and the regional managed care
support contractor report certain TRICARE-related issues to the lead
agent, including issues emanating from beneficiary comments. All
three lead agents also track in some way the beneficiary
feedback-related issues that they learn of. Lead agent officials
told us that they consider the complaints they receive to be a
valuable source of information about possible problems in their
regions. None of the offices provide formal feedback-related issues
reports to Health Affairs or the Surgeons General, although all have
a variety of informal ways of reporting issues to them.
Health Affairs, the Surgeons General, and TSO also receive
beneficiary feedback and have their own procedures for handling it.
These offices maintain tracking systems for the beneficiary feedback
they receive, but these systems primarily track who is responsible
for handling the case and response timeliness, not the specific
categories the beneficiary comments fall into. Staff in these
offices told us that they use the complaints they receive as
indicators of possible TRICARE problems.
Representatives of both of the managed care support contractors we
contacted told us that they extensively track complaints and use them
to identify system problems, and that their TRICARE tracking systems
mirror the systems they use for their commercial health plans. While
the managed care support contracts require periodic reports that
include beneficiary feedback volume and response timeliness, DOD does
not require the contractors to report their complaint tracking
results to the government. Yet, managed care support contractor
officials told us that they consider systematically tracked
beneficiary feedback and rigorous analysis of the root causes of
members' complaints to be hallmarks of a customer service-oriented
managed care plan.
Officials at the various MHS organizations we visited told us how
their complaint tracking procedures have led to problem
identification and elimination. For example, one MTF's deputy
commander told us that he saw an increase in "staff attitude"
complaints from patients at his facility. In response, he required
all facility staff to take customer service training. In another
case, lead agent officials told us how their tracking of complaints
indicated that TRICARE Prime enrollees were being required to drive
more than an hour for an MTF's specialty care, though this exceeded
the TRICARE requirement. The lead agent staff found that driving
time to the MTF routinely exceeded 1 hour because of heavy traffic in
parts of the MTF's catchment area. As a result, the staff arranged
for beneficiaries in those areas to go instead to closer network
providers.
Further, one contractor learned through complaints that civilian
providers were referring beneficiaries to collection agencies because
of unpaid bills. The contractor identified a number of problems
caused by beneficiary and provider mistakes, which led to improved
beneficiary and provider education efforts. This investigation also
identified a DOD policy that was causing claims to be inappropriately
denied. When a beneficiary needs medical care that cannot be
provided at an MTF, the facility can complete a "nonavailability
statement" certifying that the facility does not have the required
resources to provide the care needed and authorizing the beneficiary
to receive the care from a civilian provider. The contractor's
investigation found that when the computer record erroneously showed
that a nonavailability statement had not been issued, DOD's policy
was to not accept a paper copy of such a statement. The contractor
called this problem to the attention of DOD officials, and the policy
was changed.
--------------------
\12 DOD also has processes in place under TRICARE for beneficiaries
to appeal health care decisions with which they disagree.
\13 DOD hospitals are JCAHO accredited. The hospital accreditation
standards require that "the hospital [have] a way of providing for .
. . the patient's right to voice complaints about his or her care,
and to have those complaints reviewed and, when possible, resolved."
All the hospitals we visited provided a means for patients to voice
complaints. The JCAHO standards do not address complaint tracking
specifically or reporting in general, although a number of JCAHO
standards concern handling, tracking, and reporting quality of care
issues, some of which are identified through patient complaints.
SAMPLE COMMENTS COVER A WIDE
RANGE OF ISSUES
---------------------------------------------------------- Letter :4.1
DOD officials at MTFs and other offices, contractor officials, and a
beneficiary organization's representatives provided us with more than
2,600 examples of military health care beneficiary complaints and
compliments. The comments covered a wide range of areas, including
health care and administrative service quality, cost issues, and
access to care.
Because of the sample comments' many forms, it is not possible to
generalize across the system or to draw conclusions about comment
frequency, the full range of categories that complaints or other
comments may fall into, the number of comments in any particular
category, how types of comments vary over time, or how complaints
were resolved. Nonetheless, the following sample comments illustrate
the types of concerns and favorable comments that DOD health care
beneficiaries have expressed.
Examples of complaints about MTF quality of care or services included
the following:
-- An MTF doctor unfamiliar with how to prescribe a drug gave a
patient incorrect instructions on how often to take the
medicine. The patient's mother caught the mistake and confirmed
it by calling the MTF pharmacy.
-- The daughter of a retired military member who was admitted to an
MTF for cancer treatment complained that her father was not well
cared for. In particular, she complained that his clothes were
soiled but no one had cleaned him. Upon inquiry, MTF staff told
family members where they could get supplies to clean him
themselves. The daughter also complained that she had found his
intravenous bag empty and blood in the tubing, and that the
staff had acted as if this were "no big deal."
Sample complaints about the quality of care or services provided by
managed care support contractors follow:
-- A patient with a previously abnormal mammogram was told by her
surgeon that a 6-month follow-up mammogram was necessary. She
complained that although she discussed the need for follow-up
with her network primary care manager (PCM), the PCM delayed
making a referral.\14 The patient later switched PCMs and got
the referral, although the test was set for 10 rather than the
prescribed 6 months after the first test.
-- A mother complained that the scale her network pediatrician used
to weigh her newborn daughter was faulty. This led to an
inadequate assessment of the infant's weight and, subsequently,
the need to hospitalize the child for severe dehydration.
Complaints about MTF access to care included the following:
-- A patient drove for 3 hours to a 1:00 p.m. MTF appointment for
a diagnostic procedure. Upon arriving, he was told his
appointment was scheduled for 3:00 p.m. but he would probably
not be seen until 4:00 p.m. The patient had not eaten anything
for 36 hours--as the procedure required--and now had to wait
another 3 hours. He said that his requests for an explanation
were not met and that the clinic staff were not attentive to his
complaint.
-- A managed care support contractor's letter to a lead agent
described two incidents in which patients complained to the
contractor about inappropriate MTF emergency care delays. In
the first case, a woman with a serious medical problem called an
MTF emergency room but was told to call the managed care support
contractor's health care information line. The information line
nurse, however, told her to go immediately to the emergency
room. In the second case, an active duty member who had gone
directly to an MTF emergency room was turned away because he had
not first called the health care information line. When he
called, the nurse said he should return to the emergency room
for treatment.
Following are complaints about access to care in contractors'
networks:
-- When enrolling in TRICARE Prime, a beneficiary chose a
gynecologist as her PCM only to find that the doctor,
misidentified in the network listing, was a pediatrician. She
reported that, as a result, she spent an entire day trying to
arrange an appointment with the wrong doctor. After several
phone calls and letters, she received a new TRICARE card that
still listed the pediatrician as her PCM.
-- A beneficiary tried in vain to find a TRICARE network provider
in her area to treat her swollen knee. On her first call to the
contractor's toll-free number, she was given four doctors'
numbers; two of the numbers had been disconnected, one belonged
to a doctor not accepting TRICARE Standard patients, and one was
for a hospital emergency room. The patient tried the toll-free
number again and got two more numbers, but neither doctor was
working that day (Friday). On her third try, she was given six
more doctors' names, but only two came with phone numbers. She
was told to look up the other four in the phone book, but none
were listed. Of the two phone numbers she received, one was
invalid and the other proved to be that of a pediatrician.
Thus, after 2-1/2 hours of unsuccessful attempts to find a
doctor, she called an MTF she previously had not been able to
get through to and was given an appointment that same day.
Examples of complaints related to TRICARE costs and other financial
issues follow:
-- A TRICARE Prime enrollee referred by her MTF to a civilian
specialist complained that the doctor told her the reimbursement
from the managed care support contractor was "not sufficient to
perform the surgery [or cover] the cost of supplies."
-- A TRICARE Prime enrollee referred by his civilian PCM to a
civilian specialist began to receive bills for the care. The
managed care support contractor told the enrollee that the
civilian doctor was using an incorrect identification number and
that the doctor should resubmit the claim. The enrollee then
received a second bill and was told that the visit was being
treated as a point-of-service claim (which would require the
patient to pay a large part of the bill), even though his PCM
had properly referred him. He was later told to disregard the
second bill.
Complaints concerning both access to care and quality of
administrative services included the following:
-- A father was to be contacted within 5 days by an MTF radiology
clinic with an appointment time for his child's procedure. When
he was not called, he went to the clinic and was told that
"things happen." He found this response and the lack of an
apology to be "rude and uncaring." Subsequently, when he and the
child arrived for the appointment, it had to be cancelled
because the child had eaten too recently, although they had not
been told of the need to fast before the procedure.
We also obtained the following examples of favorable comments about
both the direct care system and contractor functions:
-- One MTF kept a log of all patients' comments. The list included
compliments about the friendliness, compassion, professionalism,
and technical skill of specific staff members, as well as
general compliments about, for example, the speed of access to
care or the clinic staff in general.
-- A beneficiary had 6 months of claims processing problems that
she described as "a nightmare." She wrote to the managed care
support contractor thanking a specific contractor staff member
for resolving her problem.
-- In a letter to a managed care support contractor, an Air Force
chief master sergeant complimented staff at the local contractor
office. He wrote: "Their enthusiasm and sincerity is
definitely the right attitude needed to administer a program
that has had the military `rank and file' feeling a little
uncomfortable."
--------------------
\14 Under TRICARE Prime, the PCM is the doctor responsible for
meeting a variety of enrollees' primary health care needs and for
referring enrollees for specialty care when necessary.
DOD COULD IMPROVE ITS FEEDBACK
TRACKING AND REPORTING BY
FURTHER ADOPTING PRIVATE SECTOR
PRACTICES
------------------------------------------------------------ Letter :5
DOD's efforts to track beneficiary feedback resemble those of the
private sector, but opportunities for improvement exist. Private
health care managers make extensive use of customer feedback from
surveys and rigorous customer complaint tracking and reporting.
While DOD's current survey efforts and emphasis on addressing
beneficiary complaints at the local level are not unlike private
practices, additional targeted surveys and more consistent complaint
tracking and reporting would better inform DOD managers about
beneficiaries' experiences and more closely reflect private sector
approaches to managing such information. Enhancing its current
feedback efforts would also help DOD achieve its goal of bringing
about a more outcomes-oriented TRICARE health system. Yet, given
that the MHS differs in key ways from private sector health care
systems, DOD would need to consider several basic cost and
implementation issues to improve its beneficiary feedback.
PRIVATE SECTOR USES SURVEYS
EXTENSIVELY TO SOLICIT
BENEFICIARY FEEDBACK
---------------------------------------------------------- Letter :5.1
Customer surveys are a common private sector health care feature.
Health plan officials told us they survey plan members to gauge
overall satisfaction and conduct targeted user surveys to measure
performance in particular areas. One large managed care plan
conducts an overall member survey, a survey of members who have
recently received health care services under the plan, and surveys
targeted toward patients' perceptions of their doctors. Health care
providers also use customer satisfaction surveys. Officials at a
hospital system that we contacted told us that every patient is asked
to fill out a survey after receiving care in one of the system's
facilities.
Survey results are also reported extensively throughout the private
organizations we contacted. Officials told us that the results are
used to identify problem areas, measure overall performance, and
compare the performance of different parts of the organization.
Officials at one managed care organization told us they report survey
results to both senior managers and staff throughout the
organization. These officials also provide special reports on
results in particular areas when departments request them. The
hospital system we contacted reports all patient comments, including
patient questionnaire responses, to the head of hospital operations
on a daily basis, while managers across the organization receive
quarterly reports.
In another case, several employers that came together to purchase
health care as a group identified extensive beneficiary surveying as
a key measure of their system's performance. For example, the group
reports customer satisfaction information from surveys to inform
beneficiaries when they are choosing providers. The group also
contracts for targeted surveys of particular covered populations,
including surveys focused on the health status of children and
seniors.
Surveys are also central to the accreditation of managed care
organizations. Both NCQA and JCAHO have accreditation programs that
require managed care plans that are seeking accreditation to conduct
member surveys. Health care purchasers, regulators, and consumers
use the results of the accreditation process to assess all aspects of
a plan's delivery systems: physicians, hospitals, other providers,
and administrative services.
A survey is also a requirement of the latest version of the Health
Plan Employer Data and Information Set (HEDIS).\15 HEDIS is a set of
standardized performance measures of health care plans' performance.
HEDIS is designed to provide purchasers and consumers with the
information they need to reliably compare managed care plans'
performance. To become part of the HEDIS database, health plans must
use NCQA's Member Satisfaction Survey and be prepared to report the
full set of survey results. NCQA makes consolidated results
available to consumers for use in selecting among health plans.
--------------------
\15 HEDIS is sponsored by NCQA but is not directly linked to NCQA
accreditation. Health plans may choose to gather and report
information as outlined in HEDIS irrespective of the decision to seek
NCQA accreditation.
PRIVATE SECTOR USES TRACKED
COMPLAINTS AS A KEY
MANAGEMENT TOOL
---------------------------------------------------------- Letter :5.2
Private health care managers also extensively track customer
complaints and use them to make system improvements. A large HMO's
member services director, for example, told us that members'
complaints and other comments, whether received in person, over the
phone, or in writing, are tracked by computer. The purpose is to
resolve members' problems, identify root causes, and eliminate system
flaws. Patient feedback tracking system reports are generated
monthly and sent to staff throughout the system, including the
Quality Assurance/Quality Improvement Committee.
The hospital system we examined also uses a computer system to track
all complaints, whether received in person, over the phone, in
writing, or in response to a patient satisfaction survey. Complaints
are sent to the senior staff member of the hospital area that the
complaint concerns. All patients' complaints are reported daily to
the system's hospital operations' vice president, and every quarter
to system managers. A senior official told us that complaints are
useful for identifying both one-time and systemwide problems. He
explained, for example, how patients had complained about giving the
same information to different people during the admitting process,
which led to the elimination of this redundancy.
Representatives of one California hospital reported that analyzing
patient complaints has become the hospital's least expensive, most
accurate method for understanding patients' perspectives on what
needs improvement at the hospital.\16 When facility staff realized
that individual complaints had been addressed in the past, but with
little documentation or tracking, they designed a comprehensive
complaint process that included procedures for capturing all
complaints, responding to complaints quickly, measuring complaint
severity, analyzing trends to uncover root causes of customer
dissatisfaction, and identifying and implementing system changes to
prevent future recurrences. The officials also reported that
questionnaire surveys are not appropriate for capturing dissatisfied
patients' spontaneous complaints.
Employers who purchase managed care coverage for their employees also
see the value of tracking customer complaints. For example, the HMO
Performance Standards set by one large employer state that its
selected plan "shall track and report to [the company] the number and
types of plan aggregate written and verbal complaints received by the
HMO." The standards require an annual report that lists complaints by
categories "including but not limited to access, clinical services,
providers, pharmacy, mental health/substance abuse, claims, and
reception services."
To obtain accreditation by NCQA and JCAHO as a managed care
organization, managed care plans must obtain and use member feedback.
Plans are required to track, report, and use customer complaints to
identify and address one-time and systemic problems. NCQA standards
require that customer feedback analysis include aggregating results;
noting trends in results over time; and identifying reasons for the
results, such as the causes of dissatisfaction in particular areas.
The standards also discuss how managed care organizations should use
feedback analysis results to prioritize improvement areas on the
basis of their significance to members. NCQA officials told us that
no one system is prescribed for managing member complaints. Rather,
NCQA surveyors look at a sample of complaints, determine if a system
for handling them exists, and decide if the plan is following its own
system. Similarly, JCAHO network accreditation standards require
health plans seeking accreditation to have customer complaint receipt
and management systems.\17
The extensive use of customer feedback is not just a private sector
health care feature; it exists throughout the private sector. A
report of the Vice President's National Performance Review describes
extensive customer complaint and survey use by "best-in-business"
companies and the applicability of these practices to government.\18
It also refers to Executive Order 12862, which directs federal
agencies to perform customer surveys, make complaint systems easily
accessible, provide the means to address customer complaints, and
measure customer service against the best-in-business.\19 The report
also describes customer feedback strategies used by best-in-business
companies including
-- facilitating customer complaints through the extensive use of
centralized customer help lines, 1-800 numbers, point-of-service
complaint or comment cards, and easy-to-use customer appeal
processes;
-- encouraging quick responses to customer complaints;
-- using computers to centrally track complaints at the
headquarters level;
-- reporting tracking results widely, including to top management;
and
-- using the results to identify dissatisfaction trends and root
causes to target core processes that need improvement.
--------------------
\16 Sister Julie Hyer and Roger Hite, Ph.D., "Using Complaints to
Analyze and Address Customer Needs," Strategies for Healthcare
Excellence (Santa Barbara, Calif.: COR Healthcare Resources, Aug.
1996), pp. 9-12.
\17 Both NCQA and JCAHO standards also require plans to have
mechanisms in place for members to appeal health care decisions with
which they disagree.
\18 National Performance Review, Serving the American Public: Best
Practices in Resolving Customer Complaints, Federal Benchmarking
Study Report (Washington, D.C.: The Vice President's National
Performance Review, Mar. 1996).
\19 President Clinton signed Executive Order 12862, "Setting Customer
Service Standards," on Sept. 11, 1993.
SOME DOD EFFORTS ARE
COMPARABLE TO THOSE OF THE
PRIVATE SECTOR, BUT
ENHANCEMENTS ARE POSSIBLE
---------------------------------------------------------- Letter :5.3
Some DOD efforts to track and use beneficiary feedback compare
favorably with private sector efforts. For example, DOD's
beneficiary surveys are similar to private health plan and hospital
surveys. Also, MTFs and other DOD offices use complaints to help
identify problems, as is done in the private sector. But, in our
view, DOD could make its current efforts more complete and
systematic--and thus more effective.
DOD's current beneficiary surveys provide a view of beneficiaries'
satisfaction with their care generally and their MTF outpatient care
specifically. However, adding targeted surveys of beneficiaries'
satisfaction with MTF inpatient care and TRICARE civilian network
care would enhance the usefulness of DOD's survey data. By doing so,
DOD decisionmakers would have a more complete picture of TRICARE's
customer satisfaction.
DOD could also obtain more detailed information about beneficiary-
initiated complaints and other comments if it standardized the way it
tracks and reports this feedback across the system. Currently, no
systemwide approach to tracking and reporting exists. As a result, a
serious problem that is surfaced by a complaint in one region or at
one MTF, for example, can remain unnoticed in other locales if no one
there complains. Moreover, with a consistent approach to tracking
and reporting feedback, MHS and contractor personnel could put the
complaints they receive into a systemwide perspective, even if they
were tracking complaints locally. Further, with standardized
tracking and reporting, personnel throughout the MHS could identify
trends beyond those at their own location. They would also know the
overall complaint volume by type and would probably find that the
problems they were seeing had already surfaced and been addressed
elsewhere, potentially saving time and resources otherwise spent on
reinventing the solutions.
With regular access to systematically tracked and reported complaint
data, senior DOD officials could analyze complaint activity across
the system, spot trends, and identify possible problems using data
currently unavailable to them. Consistent complaint data would also
equip senior officials with another tool for evaluating individual
MTF performance and making cross-system comparisons.
Standardizing feedback tracking and reporting would also enable DOD
to better judge TRICARE's contractor performance. DOD officials are
now working to make future TRICARE contracts less prescriptive in
nature and more outcomes based.\20 Past contracts have offered
bidding contractors little or no opportunity to use their best
commercial practices to introduce innovation or reduce costs to
accomplish DOD's goals. For the new contracts, DOD proposes to set
forth its overall objectives, such as maintaining customer
satisfaction, and provide a mandatory requirements list. Deciding on
an approach to satisfy the objectives and other requirements will be
left to the bidders.
In addition, DOD currently plans to use its annual survey and monthly
MTF outpatient survey results as program success measures. By adding
the other two surveys, DOD decisionmakers could focus more closely on
MTF inpatient and civilian network performance and use the level of
consequent beneficiary satisfaction as a key performance indicator.
DOD officials could be confident that beneficiary complaints were
being systematically categorized and reported so that such data could
be used as a measure of the performance of managed care support
contractors, MTFs, and TRICARE overall.
--------------------
\20 A key focus of the new TRICARE contract effort is to revise the
contracting process for managed care support services according to
acquisition reform principles in use elsewhere in the federal
government and growing out of the Vice President's Reinventing
Government activities. The proposed TRICARE contracts would
represent a new philosophy about managed care support contracts that
favored flexibility and avoided strict bureaucratic specifications,
with a focus on continuous quality improvement. In contrast, in the
past the government has issued as part of the request for proposal a
Statement of Work that described virtually all that the contractor
was expected to do.
MILITARY HEALTH CARE DIFFERS
IN KEY WAYS FROM PRIVATE
SECTOR HEALTH CARE
---------------------------------------------------------- Letter :5.4
DOD's multifaceted MHS role, DOD's relationship with its managed care
support contractors, and the unique chains of authority involved in
the roles of the three services in delivering military health care
differ from the structure of private sector health care. These
differences mean that DOD's feedback tracking and reporting is more
involved than the private sector's and that civilian standards for
this activity are not necessarily easily applicable to the MHS,
though the principles driving them apply to all managed care
environments, including TRICARE.
Typically, private employers purchase health care coverage for their
employees (or individuals purchase it directly) from health plans,
which contract with doctors and hospitals to provide covered
beneficiaries' care. DOD operates differently. As the
beneficiaries' employer, it both administers TRICARE and directly
provides much of the MHS' health care through the hundreds of
hospitals and outpatient clinics that it operates. Because of DOD's
merged responsibilities, which are usually held by separate entities
in the private sector, the checks and balances that exist in civilian
business relationships do not exist. For example, a civilian
employer that receives numerous complaints about a hospital in the
health plan's network can insist that the plan either drop the
hospital or lose the employer's business. But, should an MTF receive
such complaints, DOD's options would be more limited.
Differences among civilian health care purchasers, plans, and
providers are, for the most part, clear cut. In DOD, however,
TRICARE is a single health plan operated by two separate
entities--the direct care system (MTFs) and the managed care support
contractors--each responsible for managing program parts and
providing, or arranging for, health care services. Also, the
contractors' role overlaps that of the direct care system, with some
patients getting their care directly from DOD, others using the
contractor networks, others using non-network civilian providers, and
still others using some combination of the sources. Both DOD's
hospitals and DOD's contractors send patients to each other for some
care, but neither has real financial or other authority to control
what the other does. Because of the shared care administration and
delivery responsibilities, beneficiary-reported problems can appear
to each party to be the other's responsibility.
The role of the three services also distinguishes military from
civilian health care. While Health Affairs is responsible for
running TRICARE, the MTFs are under the authority of the Army, Navy,
and Air Force Surgeons General. And the regional lead agents, which
also respond to direction from Health Affairs, cannot direct the
activity of the MTFs in their regions but, instead, must rely on the
MTFs' cooperation to implement such new programs as regionwide
complaint tracking and reporting. Moreover, neither Health Affairs
nor the services can make changes in areas beyond their authority,
including changes needed to address problems that surface through
beneficiary feedback.
Currently, NCQA requires that a managed care plan seeking
accreditation have a single entity that is responsible for the entire
plan. An NCQA official told us that because TRICARE uses various
sources of care and various entities are responsible for seeing that
care is properly delivered, TRICARE has no single accountable entity
to examine. Instead, multiple accountability lines exist and, with
them, the potential for beneficiary-raised issues to go unaddressed
by any responsible organization. Notwithstanding the beneficiary
feedback implications, the accountable entity issue could take on
greater importance should DOD seek managed care plan accreditation
for TRICARE in the future, as DOD officials have told us it may.
COST AND IMPLEMENTATION
ISSUES NEED TO BE WORKED
THROUGH
---------------------------------------------------------- Letter :5.5
Within its health care system's unique context, DOD would need to
explore several basic issues to improve beneficiary feedback. The
cost of adding surveys and developing a single approach to handling
beneficiary complaints would need to be weighed against the benefits
sought. Also, DOD would need to decide how reporting the results of
complaint tracking should work to ensure that information flowed to
the appropriate organization and levels.
Regarding a single complaint tracking system, DOD, private sector,
and managed care support contractor representatives told us that care
should be taken to ensure that such a system not become overly
cumbersome or bureaucratic. Managed care support contractor
representatives told us such a system should be collaboratively
developed with them, flexible and adaptable to decisionmakers'
changing needs, and not overly prescriptive. They also pointed out
that contract-prescribed items are difficult to change because of the
time- consuming contract change order process and asked, therefore,
that their contracts not prescribe how they should develop such a
system.\21 Also, they told us that such a system could be composed of
tracking systems that were regional in scope and designed to
encourage strong DOD/contractor partnerships.
DOD would also, in our view, need to weigh potential training and
other costs of adapting existing MTF and other DOD office beneficiary
feedback recording systems. The costs of changing local systems
would probably vary from place to place. Locations already capturing
a great deal of beneficiary-initiated feedback data would probably
find a standardized approach comparatively easier to adopt than those
beginning the process for the first time.
Also, DOD would need to consider how to report issues to address the
MHS' multiple lines of authority. Because the services control their
respective MTFs, their chains of command would be prospective report
recipients. In addition, reporting protocols could include the
contractors and DOD contracting officers residing at lead agents and
at TSO. Finally, because Health Affairs has overall TRICARE
responsibility, it would also logically receive summary feedback,
because such information is designed to point up systemwide problems.
--------------------
\21 See Defense Health Care: Actions Under Way to Address Many
TRICARE Contract Change Order Problems (GAO/HEHS-97-141, July 14,
1997).
CONCLUSIONS
------------------------------------------------------------ Letter :6
DOD is spending a great deal of money to improve its $15
billion-per-year health care program by implementing TRICARE. An
investment of this magnitude heightens the importance of current,
accurate, and complete information about how beneficiaries are
reacting to and coping with the change. The beneficiary feedback
currently available to DOD managers provides useful information about
aspects of TRICARE's performance and possible problem areas. If DOD
were to make its current survey efforts more complete and to
consistently record and aggregate complaint information across the
system, DOD managers would have more valuable information with which
to measure TRICARE's success and identify and eliminate recurring,
systemic problems. Enhanced feedback would also help DOD make the
outcomes-based assessments it seeks for future TRICARE contracts.
DOD could improve its beneficiary feedback information by conducting
a civilian network care survey comparable to its monthly MTF
outpatient visit survey, a possibility that is now under discussion.
Also, while DOD does not currently have an MTF inpatient care survey,
we support DOD's plans to develop and conduct such a survey. DOD
could also benefit by working with the TRICARE contractors to begin
restructuring its complaint tracking and reporting systems to more
closely parallel private sector managed care practices by
consistently recording and aggregating complaint data across the DOD
health care system.
RECOMMENDATIONS
------------------------------------------------------------ Letter :7
To position DOD to obtain and make better use of beneficiary
feedback, both now and in the future, the Secretary of Defense should
direct the Assistant Secretary of Defense (Health Affairs) to
-- follow through in weighing the costs and benefits associated
with civilian network and MTF inpatient care surveys that are
comparable to DOD's current monthly MTF outpatient survey and,
as appropriate, implement these surveys and
-- collaborate with the TRICARE contractors to identify options
for, and weight the costs and benefits of, achieving consistency
in recording beneficiary complaints, analyzing trends, and
reporting beneficiary complaints and, as appropriate, implement
the most practical, financially prudent approach.
AGENCY COMMENTS AND OUR
EVALUATION
------------------------------------------------------------ Letter :8
In its written comments on a draft of this report, DOD agreed with
our recommendations regarding MTF inpatient and civilian network care
surveys and a consistent beneficiary complaint tracking and reporting
process. DOD added that the Army, Navy, and Air Force are now in
various stages of reviewing their TRICARE customer relations
approaches and assessing their beneficiary complaint processes.
DOD also suggested that beneficiary complaint tracking is currently
done at the lead agent level. However, at the lead agents visited,
we found that beneficiary feedback systems varied markedly, as did
the amounts and types of complaint data routinely captured. Also, in
line with our suggestion that Health Affairs would be a logical
recipient of beneficiary feedback data designed to point up
systemwide problems, DOD stated it is exploring a centralized process
for tracking beneficiary complaints at the Health Affairs level.
DOD also suggested technical report changes, which we incorporated as
appropriate. The full text of DOD's comments is included as appendix
IV.
---------------------------------------------------------- Letter :8.1
We are sending copies of this report to the Secretary of Defense and
will make copies available to others upon request.
Please contact me at (202) 512-7101 or Dan Brier, Assistant Director,
at (202) 512-6803 if you or your staff have any questions concerning
this report. Other GAO staff who made contributions to this report
are David Lewis, Evaluator-in-Charge; Linda Lootens, Senior
Evaluator; and Paul Wright, Evaluator.
Stephen P. Backhus
Director, Veterans' Affairs and
Military Health Care Issues
SCOPE AND METHODOLOGY
=========================================================== Appendix I
DEPARTMENT OF DEFENSE SURVEY
EFFORTS
--------------------------------------------------------- Appendix I:1
To identify Department of Defense (DOD) efforts to solicit
beneficiary feedback through surveys, we interviewed officials of
Health Affairs. We also obtained and reviewed documentation,
including survey instruments, relating to Health Affairs surveys that
included elements of TRICARE beneficiary satisfaction, as well as
documents related to other Health Affairs surveys. Through
discussion with Health Affairs officials, we determined that three
DOD surveys fell within the scope of this review: the Health Care
Survey of DOD Beneficiaries (1994-95 and 1996) (the annual survey),
the Customer Satisfaction Survey (April/May/June 1997) (the Military
Treatment Facility [MTF] outpatient survey), and the TRICARE Prime
Enrollee Satisfaction Study (1996).
We obtained DOD reports of these three surveys' results but did not
independently assess the survey instruments' statistical validity or
reliability. In this regard, the DOD official responsible for the
Health Affairs survey efforts told us that DOD uses experienced
contractors to design and conduct its surveys and that survey
questions are based on standard survey questions extensively
pretested for validity and reliability by the private sector, and
widely used in their surveys. Further, he believes DOD's rigorous
methods for sampling survey populations and weighing survey responses
on the basis of numerous proven variables result in statistically
valid survey data. DOD survey yield rates\22 are similar to the
average 50-percent yield rate for private sector surveys. The annual
survey yield rate has been about 60 to 65 percent, and the MTF
outpatient survey yield rate has been about 45 percent; both rates
have been increasing over time.
--------------------
\22 The yield rate is the number of survey instruments completed and
returned, divided by the number mailed out.
BENEFICIARY-INITIATED FEEDBACK
PROCESSES
--------------------------------------------------------- Appendix I:2
We interviewed and obtained documents from DOD officials and
contractor representatives across the Military Health System (MHS)
regarding policies and procedures for documenting, determining trends
in, and reporting beneficiary-initiated complaints and compliments.
At the DOD headquarters level, we met with Health Affairs officials
to discuss tracking beneficiary feedback within Health Affairs. We
also reviewed TRICARE Support Office (TSO) requirements for how
managed care support contractors are to track and report feedback
from beneficiaries and interviewed TSO officials about how they use
the beneficiary comments that they receive. In addition, we
interviewed representatives of the Army, Navy, and Air Force Surgeon
General and Inspector General offices about how their organizations
receive and handle beneficiary feedback. We also discussed with all
of these officials the means by which they exchange information on
feedback-related issues with other MHS locations.
To gather information from DOD field-level offices, we made site
visits to lead agents in three TRICARE regions and seven MTFs within
these regions:
LEAD AGENTS
----------------------------------------------------- Appendix I:2.0.1
-- Southeast Region
-- Southern California Region
-- Southwest Region
ARMY MTFS
----------------------------------------------------- Appendix I:2.0.2
-- Brooke Army Medical Center, Fort Sam Houston, Texas
-- Eisenhower Army Medical Center, Fort Gordon, Georgia
NAVY MTFS
----------------------------------------------------- Appendix I:2.0.3
-- Naval Hospital Camp Pendleton, California
-- Naval Medical Center San Diego, California
AIR FORCE MTFS
----------------------------------------------------- Appendix I:2.0.4
-- Wilford Hall Medical Center, Lackland Air Force Base, Texas
-- 12th Medical Group Clinic, Randolph Air Force Base, Texas
-- 61st Medical Squadron Clinic, Los Angeles Air Force Base,
California
We interviewed lead agent and MTF officials at these locations about
how they track and report beneficiary comments and obtained documents
related to these feedback tracking processes, including comment
database formats and summary reports, comment tracking log sheets,
complaint/comment forms, and procedures governing beneficiary
feedback tracking and reporting.
We also interviewed representatives of two managed care support
contractors--Foundation Health Federal Services and Humana Military
Healthcare Services--in their headquarters and regional offices and
at local contractor offices located in or near the MTFs we visited.
We discussed contractors' feedback tracking and reporting processes,
both as they fulfilled DOD requirements and as they met the
contractors' own internal purposes. We also obtained documentation
of the contractors' beneficiary tracking and reporting systems.
Although DOD's contracts require the managed care support contractors
to have mechanisms in place for beneficiaries to appeal managed care
decisions, we did not examine the appeals process as part of this
review.
BENEFICIARY-INITIATED FEEDBACK
EXAMPLES
--------------------------------------------------------- Appendix I:3
We collected over 2,600 examples of beneficiary-initiated complaints
and compliments from lead agents, MTFs, and managed care support
contractor officials in the three TRICARE regions we visited as well
as from the National Military Family Association, a beneficiary
group. For this report, we judgmentally selected example comments to
identify the types of issues that beneficiaries raised. However,
because of the variability of DOD's recording of beneficiary
comments, we could not determine the range, magnitude, or frequency
of beneficiary comments, and we did not review the validity of
complaints or how complaints were resolved by the military or
contractor organizations that received them.
Because the methods by which beneficiary-initiated comments were
documented varied, the set of example complaints and compliments we
obtained is not representative of beneficiary comments from either
the locations we visited or the MHS as a whole. In some cases, the
documentation we reviewed provided only what the beneficiary said; in
other cases, particularly in the case of complaints, the
documentation also included information about how the complaint was
handled. In other cases, the documentation consisted only of brief
database entries made by staff of the organization that handled the
complaint. We were also told that some complaints and compliments
were not recorded in any way.
We did not assess the validity of the beneficiary concerns. However,
we noted that in some cases the complaint files included information
indicating that the MTF found the complaint to be invalid. For
example, a patient who wanted to see a specialist not in the
contractor's network disenrolled from TRICARE Prime in order to avoid
paying the substantial cost required of TRICARE Prime enrollees for
out-of-network care. But the patient received care from the
specialist before the effective date of disenrollment, so the patient
was billed the high fee. The patient complained about the bill, but
the documentation indicated that the mistake was the patient's, not
the MTF's or the contractor's. Another patient complained about
being denied care when she could not get an ultrasound test early in
her pregnancy. However, her doctor told the MTF staff researching
the complaint that the test she wanted was not medically necessary.
Although we did not review whether or how DOD resolved the
beneficiary concerns in the example complaints we obtained, we noted
that in some cases the available complaint documentation explained
DOD or contractor efforts to research and resolve the complaints.
There were cases, for example, in which documentation indicated that
MTF or contractor staff called the appointment telephone line to test
the quality of the service it provided after a beneficiary complained
about being left on hold or being given an appointment date weeks or
months in the future. According to the case files, the appointment
line employees were typically able to set up acceptable appointments
for the beneficiaries immediately. Documentation also showed that
complaints about inattentive staff in MTF inpatient settings
apparently led to special training on the importance of being
responsive to patient requests.
PRIVATE SECTOR FEEDBACK
APPROACHES
--------------------------------------------------------- Appendix I:4
To compare DOD approaches to beneficiary feedback with those of the
private sector, we interviewed representatives from two health care
industry accreditation bodies--the Joint Commission on the
Accreditation of Healthcare Organizations (JCAHO) and the National
Committee for Quality Assurance--and obtained and reviewed copies of
their accreditation standards regarding customer surveys and handling
customer comments. We also reviewed the Vice President's National
Performance Review report describing the use of customer complaints
by successful companies throughout the private sector and the
applicability of such practices to government agencies. Further, we
discussed customer surveys and comment tracking with representatives
of two private sector health care providers--Kaiser Permanente, a
large commercial health maintenance organization, and Inova Health
System, a Northern Virginia hospital chain--and obtained documents
describing the methods these companies use to track, categorize, and
report comments from their customers. Private sector health care
accreditation organizations require plans to have procedures for
handling appeals of health care decisions, though we did not examine
these appeals processes or compare them with those in place under
TRICARE.
DOD SURVEYS
========================================================== Appendix II
ANNUAL HEALTH CARE SURVEY
-------------------------------------------------------- Appendix II:1
The Health Care Survey of DOD Beneficiaries (referred to in this
report as the annual survey) has six sections:
-- Use and source of care. This section asks beneficiaries 22
questions about annual visits, nights spent in a hospital, care
sources, and insurance coverage.
-- Familiarity with benefits. This section contains 13 questions
about whether beneficiaries have a source of information for
various aspects of their health care benefit.
-- Health status. This section contains 36 questions, widely used
and validated in the private sector, that measure distinct
aspects of physical and emotional health.
-- Access to care. This section contains 25 questions that look at
how easily beneficiaries enter the health care system (process
measures) and whether they receive necessary care (outcome
measures).
-- Satisfaction with care. This section contains 54 questions
about overall satisfaction with care received at military and
civilian facilities, and satisfaction with specific aspects of
the care.
-- Demographic information. This section asks about age,
education, gender, ethnicity and race, beneficiary group, and
length of time in residence as well as other factors important
to explaining health-related behaviors and opinions.
The annual survey was designed by a working group composed of survey
experts from Health Affairs, each of the three services, and a
representative from the Defense Manpower Data Center. The questions
and scales used in the annual survey were developed on the basis of a
review of private sector surveys that had been extensively tested for
reliability and validity. The survey is mailed to a random sample of
beneficiaries selected from catchment areas in the United States,
overseas, and in noncatchment areas. The 1996 annual survey was
mailed to a sample population of 156,838 adult beneficiaries eligible
for MHS health care. The survey sample was composed of the following
beneficiary types: active duty, active duty family members, retirees
under age 65, retirees aged 65 or older, retiree family members under
age 65, and retiree family members aged 65 or older. Beneficiaries
were included in the sample regardless of whether they were users of
military health care--either MTF care or DOD-funded civilian care.
Health Affairs has conducted the annual survey three times, at about
16- to 18-month intervals. The first survey was conducted in late
1994 and early 1995. Because it was conducted just before TRICARE
started,\23 it established a baseline against which changes in
beneficiaries' ratings of their health care could be tracked
following TRICARE's implementation. Questions on TRICARE Prime were
added to the 1996 and the 1997 survey instruments to (1) gauge how
beneficiaries perceive the program and (2) compare responses of
beneficiaries enrolled in TRICARE Prime and those who are not.
Health Affairs sends out several reports of the annual survey
results. Each TRICARE region receives one report that contains that
region's results by catchment area and by beneficiary group. Health
Affairs sends each regional report to the lead agent, who is then
responsible for distributing the results to the MTFs in that region.
According to DOD officials, it is important to get the information to
the local level where local officials can use the information to make
improvements. Also, Health Affairs sends to each service Surgeon
General a summary-level report that includes results for each of that
service's MTFs.
Health Affairs uses annual survey results as measures, along with a
wide variety of other measures, in its MHS Performance Report Cards
and in its Annual Quality Management Reports. The report cards,
which provide MTF commanders with data on their facility's health
care delivery performance, measure five areas: access, quality,
utilization, health behaviors, and health status. Annual survey
results that appear in the report card include three measures of
beneficiary satisfaction: access to appointments, access to system
resources, and quality. According to DOD officials, the report card
is one way to convert certain annual survey results to a catchment
area score. Annual Quality Management Reports are assessments of
quality across the system and also use the annual survey results.
DOD's summary of its 1994-95 and 1996 annual survey results is broken
out by different beneficiary types. One set of results consists of
responses from active duty family members and a second, retirees and
their family members. DOD officials told us that the summary they
provided us does not include active duty personnel responses because
the summary's focus was on beneficiaries with a choice in where they
obtain health care services, a choice that active duty personnel do
not have. The summary data that DOD provided also distinguish
between regions with TRICARE and those without. Regions with TRICARE
are defined as those that had had TRICARE in place for a sufficiently
long period at the time of the 1996 survey.\24
--------------------
\23 The TRICARE contract for Region 11 was awarded in Sept. 1994,
and services began on Mar. 1, 1995.
\24 The Golden Gate, Northwest, Southern California, and Southwest
regions and the Hawaii portion of the Pacific Region all met this
criterion.
MONTHLY MILITARY HOSPITAL CARE
SURVEY
-------------------------------------------------------- Appendix II:2
The Customer Satisfaction Survey (referred to in this report as the
MTF outpatient survey) measures patient satisfaction with the
effectiveness and efficiency of a recent, specified MTF outpatient
visit. According to Health Affairs officials, this survey is
intended to provide MTF Commanders and headquarters levels with
quick, frequent, civilian-benchmarked feedback on the satisfaction of
beneficiaries with their visits to MTF outpatient clinics. The
survey asks about the patients' satisfaction with their experience
both in obtaining the appointment and during the appointment.
According to DOD officials, this systemwide survey will replace most
of the ad hoc surveys currently being done locally at MTFs. DOD
officials said that a mail survey of this type is more reliable than
surveys handed out to patients in the MTF clinics.
DOD contracted with a health services research organization to design
and conduct the MTF outpatient survey.\25 DOD's contractor mails out
surveys each month to patients who received outpatient care at
clinics that have more than 200 outpatient visits per month. Over
the course of each year, the survey will be mailed to 200 patients at
each of about 2,100 clinics. The actual number of surveys mailed for
April 1997 appointments was 52,642. Each month, MTFs forward patient
appointment data to the contractor, who prepares a random sample of
names and mails questionnaires directly to the patients, 30 to 50
days after the appointment. The questionnaire is customized to the
date, doctor, and clinic of the appointment; asks 17 multiple choice
questions about the visit; and allows for written comments. The
contractor sends these written comments directly to the MTF
Commander, without analysis by the contractor. Patients mail the
completed questionnaires directly to the contractor, who produces
reports of that month's results as well as each clinic's average
results for the past 3 months.
Health Affairs distributes a number of different reports of the
results of the monthly outpatient surveys. The contractor reports
survey results at both MTF and individual clinic levels to MTFs on a
monthly basis. These reports provide a "rolling" picture of the past
3 months' data. The clinic-level report compares each clinic with
itself during the previous reporting period as well as with other
clinics within the MTF, peer clinics at other MTFs, and civilian
HMOs. The MTF-level report compares each MTF with itself during the
previous reporting period as well as with other MTFs within the same
service, MHS-wide averages, and civilian HMOs. The contractor also
prepares quarterly summary--"roll-up"--reports for lead agents,
Surgeons General, other service command entities, and Health Affairs
within 45 to 60 days of the end of each quarter. All of these
reports are standardized and one page long; they report on customer
satisfaction with access, quality, and staff interaction.
Figures II.1, II.2, and II.3 show each region's results and
comparison scores for civilian HMOs in the same geographic areas.
April/May/June 1997 was the first 3-month period for which survey
results were available. During this period, the Central, Heartland,
Mid-Atlantic, Northeast, and Pacific-Alaska regions did not yet have
TRICARE.
Figure II.1: Monthly MTF
Outpatient Visit Survey Results
for Satisfaction With Access
Compared With Civilian HMO
Benchmarks
(See figure in printed
edition.)
Notes: Satisfaction is measured on a 5-point scale, with 1 equaling
"poor" and 5 equaling "excellent."
"Satisfaction with access" focuses on individuals' satisfaction with
referral for specialty care, access to medical care, office wait
time, time to return phone calls, ease of making phone appointments,
and appointment wait time.
Figure II.2: Monthly MTF
Outpatient Visit Survey Results
for Satisfaction With Quality
Compared With Civilian HMO
Benchmarks
(See figure in printed
edition.)
Notes: Satisfaction is measured on a 5-point scale, with 1 equaling
"poor" and 5 equaling "excellent."
"Satisfaction with quality" focuses on individuals' satisfaction with
overall quality of care received, how well care met needs,
thoroughness of treatment, how much the individual was helped, and
explanations of procedures and tests.
Figure II.3: Monthly MTF
Outpatient Visit Survey Results
for Satisfaction With Staff
Interaction Compared With
Civilian HMO Benchmarks
(See figure in printed
edition.)
Notes: Satisfaction is measured on a 5-point scale, with 1 equaling
"poor" and 5 equaling "excellent."
"Satisfaction with staff interaction" focuses on individuals'
satisfaction with personal interest in the patient, advice on ways to
avoid illness/stay healthy, amount of time with doctor and staff,
attention to what patients said, and friendliness and courtesy of
staff.
--------------------
\25 The survey questions and scale are based on the latest version of
the Health Plan Employer Data and Information Set (HEDIS).
TRICARE PRIME ENROLLEE
SATISFACTION SURVEY
-------------------------------------------------------- Appendix II:3
Health Affairs' TRICARE Marketing Office commissioned a telephone
survey of TRICARE Prime enrollees who were enrolled in the program on
September 30, 1996. The survey consisted of 7,728 interviews
conducted between October 18 and December 8, 1996, and covered five
TRICARE regions: Golden Gate, Northwest, Pacific, Southern
California, and Southwest. The survey addressed a number of issues
related to enrollees' understanding of TRICARE Prime, satisfaction,
and reenrollment intentions. TRICARE Prime-specific questions from
this survey have been incorporated into the ongoing annual surveys.
OTHER DOD EFFORTS TO SOLICIT
BENEFICIARY FEEDBACK
-------------------------------------------------------- Appendix II:4
Health Affairs also conducts other surveys to solicit beneficiary
feedback on various topics unrelated to satisfaction with health
care:
-- The DOD Survey of Health Related Behaviors Among Military
Personnel is carried out about every 3 years to collect
worldwide data from active duty personnel on drug and alcohol
abuse and other health-related behaviors.
-- The Health Enrollment Assessment Review, a questionnaire
completed by patients as they enroll in TRICARE Prime, is used
to identify high-volume care users and their chronic conditions,
assess the need for preventive services, and motivate behavioral
change.
-- The MHS User Survey is conducted twice each year to collect data
on the health care sources of DOD's U.S. beneficiaries for use
in developing capitation budgets.
DOD has also used focus groups to obtain beneficiary feedback on
TRICARE's success. From October to December 1995, DOD hosted a
series of focus groups in the Southwest and Northwest regions to test
beneficiaries' knowledge of TRICARE at the time it was introduced in
these regions and, thus, the success of its beneficiary education and
marketing efforts. DOD officials told us the results of these focus
groups helped establish a baseline of beneficiary perceptions of and
attitudes toward the program to help in designing future TRICARE
marketing efforts.
In November 1996, Health Affairs issued a policy designed to
standardize surveys across the MHS, ensure that all survey
information is generalizable, allow comparisons with civilian plans,
and minimize the time and paperwork burden on beneficiaries. In
instituting this policy, Health Affairs intended to avoid surveys
that produce invalid results and results that cannot be compared
across MHS or with those of civilian health care plan surveys.
According to the policy, entities under MHS authority--MTFs, offices
of service Surgeons General, and managed care support
contractors--must obtain approval from Health Affairs before
conducting their own surveys.\26 According to Health Affairs
officials, however, MTFs and other entities can continue to gather
information from beneficiaries as long as they use open-ended
questions and do not attempt to generalize the results. In fact,
Health Affairs officials told us that a feedback or complaint system
that allows people to describe their concerns in their own words is a
useful tool for MTFs to use to identify particular areas of concern
to beneficiaries.
--------------------
\26 The only exception is for surveys done by specific services that
sample only members of that particular service, though service-level
approval is required for this type of survey.
BENEFICIARY-INITIATED FEEDBACK
========================================================= Appendix III
Beneficiaries make complaints and give compliments directly to many
offices throughout the MHS, using several different methods.
Beneficiaries contact Health Affairs, TSO, the Surgeons General,
Inspectors General, lead agents, and MTFs. And the managed care
support contractors receive such feedback in their headquarters
offices, regional offices, and local contractor offices.
Beneficiaries also express concerns to associations representing
beneficiaries' interests.
Beneficiaries communicate their concerns in a variety of ways. For
example, beneficiaries communicate orally through phone calls and in
person, as well as in written form through letters, electronic mail
messages, faxes, and filling out comment forms at MTF clinics. One
special category of letters received within MHS is priority
correspondence--letters regarding beneficiary concerns referred from
the White House, the Congress, the Secretary of Defense, or the three
service Secretaries. DOD requires managed care support contractors
to have a toll-free phone line for beneficiaries, and much of the
feedback that the contractors receive comes in over these lines.
Officials throughout DOD told us that they consider it important that
complaints be resolved at as low a level as possible. They said that
people who register dissatisfaction should not be "given the
runaround" in the process of trying to find someone to listen to and
deal with their complaint. This emphasis is consistent with the
National Performance Review report on the importance of empowering
front-line employees to provide "on-the-spot, just-in-time resolution
of [customers'] problems."
MILITARY TREATMENT FACILITIES
------------------------------------------------------- Appendix III:1
Each MTF we visited had procedures in place enabling beneficiaries to
comment directly to MTF staff while at the facility. Much of this
feedback is in the form of oral comments made directly to MTF staff
members or through comment cards or forms beneficiaries fill out.
MTF officials told us that they also receive comments through phone
calls, letters, and electronic mail.
The MTFs differed in their approaches to handling beneficiary
comments. Some MTFs had designated personnel throughout the facility
who served as patient representatives or patient advocates. These
staff were tasked with receiving beneficiary comments about their own
clinic or department. MTFs with patient representatives at this
level also had a senior patient representative whose job was to be
available to any beneficiaries with comments, whether concerning a
particular facility area, the whole facility, or military health care
in general. Other facilities did not have formally designated
patient representatives at clinics or departments but, instead, had a
single patient representative office where beneficiaries could go to
make comments.
Procedures for documenting beneficiary feedback also varied among the
MTFs visited. For example, some MTFs entered everything the patient
representatives received into a central patient feedback database,
and some also kept hard copy documentation of the comments that came
in. Another MTF had a system that required oral comments to be
documented in writing. Staff kept hard copies of both those comments
and the ones that came in through comment cards but did not enter the
comments into a database. Another MTF did little or no documentation
of oral or written beneficiary feedback. The head patient
representative at that facility said that he did not have enough time
to both handle patient concerns and prepare documentation, so he
opted to spend time with patients instead of doing paperwork.
Also, wide differences existed in how much the MTFs analyzed
beneficiary feedback for trends. For example, some used the
categorized patient feedback in their central database to prepare
regular feedback trend reports. They analyzed how the number of
complaints per type changed over time and which hospital areas were
generating more complaints. Other MTFs did little or no formal trend
analysis of beneficiary comments, although staff members at these
facilities told us that they relied on their experience with feedback
at the facility over time to notice trends.
We also found variation in how the feedback tracking results were
reported to MTF management or to others in the facility. For
example, some MTFs distributed formal feedback reports on trends to
senior MTF management, as well as reports about department-level
feedback to supervisory staff in various areas of the facility. At
another facility, however, internal reporting of patient feedback
consisted of oral input from the head patient representative to a
senior management committee, with no supporting documentation.
MTF patient representatives told us that these systems constitute the
formal structure that is in place to receive feedback, but that other
avenues exist. For example, they said that beneficiaries can speak
to staff members throughout the MTF if they have concerns and that
many do. People can speak with their doctor or other staff members
in the various clinics, or they can go to different parts of the
MTF's administrative structure, such as the managed care office or
the MTF commander's office. Even at MTFs with extensive feedback
documentation and trend analysis systems, staff members noted that
some of the feedback that comes in to staff other than patient
representatives does not make it into the MTFs' systems. For
example, one officer in an MTF command section told us that he hears
beneficiary complaints and handles them but does not typically report
what he hears to the central MTF patient representative office that
maintains a database of patient complaints.
MTF officials told us that they do not systematically report most
beneficiary feedback to Health Affairs or the service Surgeons
General. Officials at MTFs and other MHS offices told us that MTF
staff are expected to resolve problems that arise, whether identified
through beneficiary complaints or not. Health Affairs and the
service Surgeons General expect to be brought in only to handle
issues that the MTF cannot. While such issues do get referred to the
higher levels, the officials told us that information about problems
solved locally normally do not. The exception was regular reporting
of contractor-related issues to lead agents by MTFs.
One exception to the lack of systematic reporting of beneficiary
feedback is found in the Southern California Region. MTFs in that
region are part of a program led by the lead agent to systematically
report to the lead agent certain types of beneficiary comments. Lead
agent officials told us that MTFs in the region have been asked to
send to the lead agent the beneficiary complaints made to the MTF
concerning the managed care support contractor. For example, if a
beneficiary tells the patient representative about an enrollment card
problem or a problem getting contractor network care, the MTF will
send a copy of the complaint to the lead agent, where it will be
centrally tracked, as well as notify the managed care support
contractor of the problem. The regional managed care support
contractor has also been asked to do the same for MTF-related
complaints made to it. Lead agent officials told us that they hope
to expand this project to include all regional complaints in the
future.
Further, MTFs have systems in place for documenting and reporting
clinical health care quality issues, some of which come to light
through patient complaints. To maintain JCAHO accreditation, MTFs
must have systems in place to track clinical care quality issues.
MTF officials told us that such complaints, along with other clinical
quality issues identified at the facility, are documented and
analyzed for trends and become the subject of detailed review by
special committees as well as by MTF risk management and legal office
staff.
LEAD AGENTS
------------------------------------------------------- Appendix III:2
Some beneficiary concerns come directly to the lead agents through
letters or phone calls, others come through oral or written reports
from regional MTF staffs and the contractors, and still others are
referred to the lead agent by other offices.
The three lead agents we visited had issues tracking systems that
tracked, among other things, concerns that came to light through
complaints from beneficiaries. The Southeast Region lead agent
maintained a central log of complaints that came directly to the lead
agent as well as complaints forwarded to the lead agent by other DOD
offices (including priority correspondence complaints); complaints
about MTFs forwarded by the region's managed care support contractor;
and certain complaints received by MTFs in the region. Southeast
Region officials told us that beneficiary complaints about managed
care support contractor functions were frequently the subject of
discussion during regular telephone meetings between contracting
officer's technical representatives (COTR) at the region's MTFs and
lead agent staff. Lead agent officials in the Southeast Region also
told us that they used the system to track issues to ensure they were
being properly addressed and resolved, but that they did not organize
the issues by category or analyze for trends over time.
In the Southern California Region, the lead agent had implemented a
system specifically to track complaints. The system tracked
complaints (1) received by MTFs in the region if they concerned the
managed care support contractor, (2) received by the managed care
support contractor if they concerned an MTF, and (3) received by the
lead agent directly. The lead agent staff tracked and analyzed for
trends the complaints in this system by category of complaint. Lead
agent staff told us that they want to expand the system to include
more types of complaints in the future.
The Southwest Region lead agent asked the COTRs at the MTFs to
perform a number of contractor oversight functions and to report the
results monthly to the lead agent. Some of the issues that the COTR
reports raised were related to beneficiary complaints. The lead
agent staff then compiled the issues raised by the various monthly
reports of COTRs into a single letter to the region's managed care
support contractor asking for issue-by-issue responses.
Lead agent officials in the three regions we visited told us that
they use the complaints that they receive to identify and proactively
deal with issues before they become worse, as well as to monitor
overall TRICARE performance in their region. Lead agent staff said
that when their beneficiary complaint tracking indicates a possible
problem, they discuss the issue with the managed care support
contractor, MTF staff members, or both to help identify the cause and
discuss possible solutions. Lead agent staff also said that by
tracking complaints they are better able to identify the root causes
of problems in ways that surveys are not, although surveys can, on
the other hand, indicate how well DOD is fixing the problems
identified through complaints.
Lead agent officials told us they did not systematically report
beneficiary feedback-related issues to Health Affairs or the Surgeons
General. They said, however, that a number of regularly scheduled
video, telephone, and face-to-face meetings take place with Health
Affairs, service Surgeons General, and contractor staff and that at
these meetings some issues discussed may have emanated from
beneficiary comments. But, whether a particular issue is discussed
at these meetings is generally the result of a decision made by an
individual that the issue warrants the other participants' attention.
OTHER DOD OFFICES
------------------------------------------------------- Appendix III:3
Some issues communicated to Health Affairs, the service Surgeons
General, TSO, and the service Inspectors General come directly from
beneficiaries through letters and phone calls. Others are referred
through other means, such as priority correspondence, which is
referred from congressional and other offices. Some of these
complaints are from beneficiaries who have tried to get a problem
handled at a lower level, such as an MTF, but were not satisfied.
Others are from beneficiaries who simultaneously send their complaint
letters to as many places as possible.
Health Affairs, service Surgeons General, and TSO officials told us
their organizations have their own tracking systems for beneficiary
concerns that come to the attention of their respective offices.
Health Affairs officials told us they enter all beneficiary feedback
they receive--both directly sent and referred--into a tracking system
that notes the receipt date, which staffer was assigned to handle the
concern, the response due date, and a short issue description.
Officials told us that the system's purpose is to track response
timeliness and not to track or establish trends in issues by
category. Reports from the Health Affairs tracking system show that
the system's issue descriptions are not specific enough for tracking
or identifying trends in issues by category.
Service Surgeon General office officials described similar systems
for tracking the timeliness of the offices' responses to beneficiary
feedback. Also, staff from the Navy told us that they had begun to
track selected beneficiary concerns by type.
TSO also tracks beneficiary issues. According to officials there, a
large number of phone calls and letters come into that office and are
centrally tracked in a computer database. However, officials also
said that the categorization system they use puts issues only into
broad categories--such as "claims" or "policy questions"--which
limits the usefulness of the system for tracking issues by type.
Beneficiaries can also register their complaints with the Army or
Navy Inspector General.\27 These Inspectors General deal mostly with
misconduct allegations but, on occasion, they receive health care
service-related complaints. Officials at Inspector General offices
told us they track beneficiary concerns by nature of issue but report
health care issues only on an ad hoc basis. One official at an
Inspector General office told us that his office reports an issue to
the service Surgeon General only when it appears significant and
representative of a systemic issue.
--------------------
\27 The policy of the Air Force Inspector General is to refer
complaints about medical care issues to the medical chain of command.
An official at the Air Force Inspector General office said that his
office does not handle issues concerning any areas that have their
own internal grievance procedures. Therefore, because systems are in
place for people to complain directly to Air Force MTFs or to the Air
Force Surgeon General, the Air Force Inspector General does not get
involved in these issues.
MANAGED CARE SUPPORT
CONTRACTORS
------------------------------------------------------- Appendix III:4
DOD requires contractors to document and report statistics on the
nature and number of beneficiary contacts--including, but not limited
to, beneficiary complaints--as well as on the contractors' response
times to beneficiary inquiries. For example, DOD requires monthly
contractor reports to TSO on all phone calls received, local
contractor offices' walk-in traffic, and how long contractors take to
respond to priority correspondence items. For walk-in activity and
phone calls, DOD requires the reason for the person's visit or call,
but not identification of which calls or visits involved complaints
from beneficiaries. That is, a reason category called "enrollment"
would include calls or visits from beneficiaries who contacted the
contractor service center to enroll in TRICARE Prime as well as those
who expressed a complaint about some aspect of the enrollment
process.
Contractors are also required to report quality of health care issues
that they handle--and their actions in response to the issues--to
their lead agents. These quality of care issues include both
potential quality issues and issues that the contractor determines to
have already become quality of care problems. These issues may be
reported by hospital staff; identified through review of quality of
care indicators, such as incidents of post-operative infections; and
raised by beneficiaries through complaints.
In addition to the reports required by managed care support
contracts, contractors also gather feedback-related information for
their own use. One managed care support contractor's representatives
told us that the contractor categorizes all the complaints it
receives, whether over the phone, through the mail, or in person.
The contractor also analyzes the data to identify trends and reports
the results throughout the organization, including to senior
management. Another managed care support contractor's representative
told us his organization similarly tracks complaints received from
beneficiaries through calls to the contractor's toll-free telephone
number, as well as complaints raised with the contractor's field
staff when they determine the complaints to be serious enough to
warrant entering into the tracking system. The representatives told
us that their beneficiary feedback tracking systems are similar to
the systems used by their parent companies' civilian health care
operations. The contractors do not systematically report the results
of their internal tracking to DOD, although issues that the
contractors discover through their own systems may be discussed in ad
hoc letters to DOD.
(See figure in printed edition.)Appendix IV
COMMENTS FROM THE DEPARTMENT OF
DEFENSE
========================================================= Appendix III
*** End of document. ***