Medicare: Communications to Beneficiaries on the Prescription
Drug Benefit Could Be Improved (03-MAY-06, GAO-06-654).
On January 1, 2006, Medicare began providing coverage for
outpatient prescription drugs through its new Part D benefit.
Beneficiaries who enroll in Part D may choose a drug plan from
those offered by private plan sponsors under contract to the
Centers for Medicare & Medicaid Services (CMS), which administers
the Part D benefit. Beneficiaries have until May 15, 2006, to
enroll in the Part D benefit and select a plan without the risk
of penalties. GAO was asked to review the quality of CMS's
communications on the Part D benefit. GAO examined 70 CMS
publications to select 6 documents for review and contracted with
the American Institutes for Research to evaluate the clarity of
these texts; made 500 calls to the 1-800-MEDICARE help line; and
contracted with the Nielsen Norman Group to evaluate the
usability of the Medicare Web site.
-------------------------Indexing Terms-------------------------
REPORTNUM: GAO-06-654
ACCNO: A53099
TITLE: Medicare: Communications to Beneficiaries on the
Prescription Drug Benefit Could Be Improved
DATE: 05/03/2006
SUBJECT: Beneficiaries
Communication
Government information dissemination
Health care programs
Medicare
Prescription drugs
Program evaluation
Usability
Websites
Written communication
Customer service
Medicare Part D
******************************************************************
** This file contains an ASCII representation of the text of a **
** GAO Product. **
** **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced. Tables are included, but **
** may not resemble those in the printed version. **
** **
** Please see the PDF (Portable Document Format) file, when **
** available, for a complete electronic file of the printed **
** document's contents. **
** **
******************************************************************
GAO-06-654
* Results in Brief
* Conclusions
* Recommendations for Executive Action
* Agency Comments and Our Evaluation
* Written Documents
* The 1-800-MEDICARE Help Line
* Medicare Web Site
* State Health Insurance Assistance Programs
* GAO Contact
* Acknowledgments
* GAO's Mission
* Obtaining Copies of GAO Reports and Testimony
* Order by Mail or Phone
* To Report Fraud, Waste, and Abuse in Federal Programs
* Congressional Relations
* Public Affairs
Report to Congressional Requesters
United States Government Accountability Office
GAO
May 2006
MEDICARE
Communications to Beneficiaries on the Prescription Drug Benefit Could Be
Improved
GAO-06-654
Contents
Letter 1
Results in Brief 5
Conclusions 8
Recommendations for Executive Action 9
Agency Comments and Our Evaluation 9
Appendix I Briefing on Medicare Part D 13
Appendix II Objectives, Scope, and Methodology 61
Appendix III Comments from the Centers for Medicare & Medicaid Services 67
Appendix IV GAO Contact and Staff Acknowledgments 83
Tables
Table 1: Sample of Six Selected Documents 62
Table 2: Questions and Criteria Used to Evaluate Accuracy 64
Abbreviations
AIR American Institutes for Research CMS Centers for Medicare & Medicaid
Services CSR customer service representative MMA Medicare Prescription
Drug, Improvement, and Modernization Act of 2003 NN/g Nielsen Norman Group
SHIP State Health Insurance Assistance Program SMOG Simplified Measure of
Gobbledygook
This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed in
its entirety without further permission from GAO. However, because this
work may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this material
separately.
United States Government Accountability Office
Washington, DC 20548
May 3, 2006
Congressional Requesters
In the most significant change to the Medicare program since its
inception, the Medicare Prescription Drug, Improvement, and Modernization
Act of 2003 (MMA)1 established an outpatient prescription drug benefit in
Medicare, known as the Part D benefit. Coverage for this new benefit began
on January 1, 2006. Until this time, Medicare, the federal program that
finances health care benefits for about 42 million elderly and disabled
beneficiaries, had not generally provided coverage for outpatient
prescription drugs. Beneficiaries may choose a Part D plan2 from multiple
plans offered by private sponsors3 under contract to the Centers for
Medicare & Medicaid Services (CMS),4 the agency that is responsible for
administering the Medicare program, including the Part D benefit. These
plans differ in the drugs they cover and the pharmacies they use. In
addition, the costs to the enrollee for the monthly premium, the annual
deductible, and co-payments for covered drugs vary by plan. As of April
20, 2006, more than 30 million of Medicare's 42 million beneficiaries were
enrolled in a Part D plan or had other outpatient prescription drug
coverage. Beneficiaries have until May 15, 2006, to select a plan without
the risk of penalties in the form of higher premiums.
Given the newness and complexity of the Part D benefit, it is critical
that beneficiaries and their advisers, including members of their
families, understand the available options so that beneficiaries can make
informed decisions on whether to enroll in Part D, and if so, which drug
plan to choose. Beneficiaries need to compare drug plans in light of their
anticipated prescription drug needs and existing arrangements for paying
for these drugs. In addition to comparing costs and drug coverage,
beneficiaries need to consider whether the plans they are comparing have
contracted with a local or mail-order pharmacy that will provide a
convenient means of filling their prescriptions.
1Pub. L. No. 108-173, S: 101, 117 Stat. 2066, 2071-2152 (to be codified at
42 U.S.C. S:S: 1395w-101-1395w-152). MMA redesignated the previous part D
of title XVIII of the Social Security Act as part E and inserted a new
part D after part C.
2For Part D standard coverage, Medicare pays on average 75 percent of
prescription drug costs up to $2,250, after a $250 deductible.
Beneficiaries then pay their next $2,850 in drug costs. If their drug
costs exceed this amount, Medicare will pay about 95 percent of their
additional costs for the rest of the calendar year.
3Drug plan sponsors include insurance companies and other private
organizations.
4CMS is an agency in the Department of Health and Human Services.
As part of its responsibilities, CMS has undertaken outreach and education
efforts to provide beneficiaries and their advisers with the information
they need about Part D through various media, including written documents,
the 1-800-MEDICARE help line,5 and the Medicare Web site.6 As of December
2005, CMS has produced more than 70 written documents to explain Part D to
beneficiaries. Medicare & You-the beneficiary handbook-is the most widely
available of these documents and was sent directly to beneficiaries in
October 2005. Other CMS documents are targeted to specific groups of
beneficiaries, such as dual-eligible beneficiaries7 and beneficiaries with
Medicare Advantage or Medigap policies.8 Since March 1999, CMS has
administered its nationwide 1-800-MEDICARE help line to answer
beneficiaries' questions about the Medicare program. As of December 2005,
about 7,500 customer service representatives (CSR) were handling calls on
the help line, which operates 24 hours a day, 7 days a week, and is run by
two CMS contractors. Calls are answered by an automated system and are
routed to a CSR for specific questions, including those about Part D. CMS
provides CSRs with detailed scripts to use in answering the questions.
CSRs type in related keywords to generate a list of suggested scripts for
a given question, select the script they consider best suited to the
question, and read excerpts or the entire script. Call center contractors
write the scripts, and CMS checks them for accuracy and completeness. CSRs
can also consult other information sources, such as the Medicare Web site.
CMS does not allow CSRs to offer individualized guidance to callers,
including advice in choosing a drug plan. CMS's Medicare Web site provides
information about all aspects of the Medicare program. The Web site
contains basic information about the Part D benefit; suggests factors for
beneficiaries to consider when choosing a plan; describes situations
common to beneficiaries with guidance on next steps to take in deciding
whether to enroll and what plan to choose; lists frequently asked
questions; and allows users to view, print, or order publications. In
addition, the site contains information on cost, coverage, and convenience
of individual plans. There is also a tool that allows beneficiaries to
enroll directly in the plan they have chosen.
5In December 2004, we reported on the information being provided to
beneficiaries through the Medicare help line on eligibility, enrollment,
and benefits. (See GAO, Medicare: Accuracy of Responses from the
1-800-MEDICARE Help Line Should Be Improved, GAO-05-130 (Washington, D.C.:
Dec. 8, 2004).)
6The Medicare Web site is www.medicare.gov.
7Dual-eligible beneficiaries are Medicare beneficiaries who are also
eligible for Medicaid-the federal-state health program for low-income
individuals-and receive full Medicaid benefits for services not covered by
Medicare.
8Medicare Advantage replaced the Medicare+Choice managed care program and
expanded the availability of private health plan options to Medicare
beneficiaries. Medigap policies provide supplemental health coverage sold
by private insurers to help pay for Medicare cost-sharing requirements, as
well as for some services not provided by Medicare.
CMS has also arranged for State Health Insurance Assistance Programs
(SHIP) to provide Part D information on request to Medicare beneficiaries
and their advisers. Currently, CMS provides grants to the 54 SHIPs-one in
each state, the District of Columbia, the Virgin Islands, Puerto Rico, and
Guam. State SHIPs provide subgrants to over 1,300 local organizations to
assist SHIPs in their efforts. In total, SHIPs rely on over 12,000 trained
counselors, most of whom are volunteers, to provide free counseling and
assistance via telephone and face-to-face sessions, public education
presentations and programs, and media activities.
Widespread confusion among beneficiaries about the costs and coverage
under the new benefit has been reported by the media and others. For
example, according to an October 2005 survey by a research organization,
some beneficiaries are unaware of the penalties for late enrollment and
others did not realize that beneficiaries had to sign up for the benefit.9
In light of your interest in ensuring that Medicare beneficiaries receive
the information they need to make informed decisions, you asked us to
examine the quality of the information being provided on the Part D
benefit. In this report, we examined
o the extent to which CMS's written documents describe the Part D
benefit in a clear, complete, and accurate manner;
o the effectiveness of CMS's 1-800-MEDICARE help line in
providing accurate, complete, and prompt responses to callers
inquiring about the Part D benefit;
o whether CMS's Medicare Web site presents information on the
Part D benefit in a usable manner; and
o how CMS has used SHIPs to respond to the needs of Medicare
beneficiaries for information on the Part D benefit.
We briefed your staff regarding the results of our review on April
19, 2006. Appendix I contains information we provided during our
briefing to your staff.
To evaluate CMS's written documents describing the Part D benefit,
we examined 70 relevant CMS publications and selected a sample of
six documents for in-depth review. These documents represent a
variety of document types, content, and target audiences and
include Section 6 of the Medicare & You beneficiary handbook,
which discusses Part D. To assess the clarity of the sample
documents, we contracted with the American Institutes for Research
(AIR), a firm with experience in evaluating written documents. AIR
evaluated the texts by using three standard readability tests;10
60 commonly recognized good communications practices; and user
testing with 11 Medicare beneficiaries and 5 advisers to
beneficiaries, all of whom were asked to perform 18 specified
tasks related to enrollment, coverage, costs, penalty, and
informational resources and provide feedback about their
experiences. To evaluate completeness, we reviewed the sample
documents to determine if they included sufficient information for
the beneficiaries to identify (1) their next steps in deciding
whether to enroll and what plan to choose and (2) important
factors, such as penalty provisions, that could affect their
coverage decisions. To evaluate accuracy, we reviewed the sample
documents for consistency with MMA, regulations, and CMS guidance.
To assess the accuracy, completeness, and promptness of the help
line responses, we made 500 calls to 1-800-MEDICARE, posing one of
five questions about Part D in each call so that each question was
asked 100 times. To develop the questions, we considered topics
listed on the Medicare Web site and topics addressed in scripts
frequently accessed by CSRs. To develop our criteria for
evaluating the accuracy and completeness of CSRs' responses, we
used three resources: (1) the prescription drug finder tool on the
Medicare Web site, (2) the 1-800-MEDICARE scripts, and (3) input
from CMS officials. We also recorded the length of each call,
including wait times, and the time it took to be connected to a
CSR.
To assess whether the Medicare Web site presents information on
the Part D benefit in a usable manner, we contracted with the
Nielsen Norman Group (NN/g), a firm with expertise in Web design.
NN/g conducted three evaluations: (1) it calculated an overall
usability score for the site-considering factors such as site
navigation, customer support, and presentation of online forms-to
reflect the ease of finding necessary information and performing
various tasks; (2) it determined the usability of 137 detailed
aspects of the Web site, including aspects of Web design, online
tools, and writing style; and (3) it tested the ability of seven
participants (five beneficiaries and two advisers to
beneficiaries) to complete a total of 34 user tests to determine
the ease of performing a variety of Web-related tasks, such as
browsing the site and determining how to join a plan. We also
reviewed the results of CMS's analysis of its Web site's
compliance with requirements that federal government Web sites be
accessible to people with disabilities.
Finally, to examine how CMS has used SHIPs to meet the information
needs of beneficiaries regarding Part D, we obtained information
about SHIPs, their funding, changes made in response to the new
benefit, and the impact of Part D on the demand for SHIP services.
In addition, we interviewed CMS officials who monitor SHIP
activities as well as SHIP coordinators in the five states with
the largest populations of Medicare beneficiaries-California,
Florida, New York, Pennsylvania, and Texas.
We performed our work from November 2005 through May 2006 in
accordance with generally accepted government auditing standards.
For more information on our methodology, see appendix II.
The sample of CMS's written documents we reviewed describing the
Part D benefit to Medicare beneficiaries and their advisers were
largely complete and accurate, but the information these documents
presented lacked clarity. The documents were unclear in two ways.
First, about 40 percent of seniors read at or below the
fifth-grade level, but the reading levels of the documents ranged
from seventh grade to postcollege. As a result, documents at these
levels are not completely clear and understandable for many
seniors. Second, on average, the six documents did not comply with
about half of the 60 commonly recognized guidelines for good
communications. For example, although the documents included
concise and descriptive headings, they used too much technical
jargon and often did not define difficult terms, such as
formulary.11 The 11 beneficiaries and 5 advisers we tested
reported frustration with the documents' lack of clarity as they
encountered difficulties in understanding and attempting to
complete 18 specified tasks. For example, none of these
beneficiaries and only 2 of the advisers were able to complete the
task of computing their projected total out-of-pocket costs for a
plan that provided Part D's standard coverage. Only 1 of the 18
tasks was completed by all beneficiaries and advisers. Even those
who were able to complete a given task expressed confusion and
frustration as they worked to comprehend the relevant text.
Although the sample documents lacked clarity, the information
presented in them was generally complete. The documents informed
readers of next steps in determining whether to enroll and what
plan to choose, and of important factors that could affect their
coverage decisions. The information in the sample documents was
also generally accurate when evaluated for consistency with MMA,
implementing regulations, and agency guidance.
Responses to the 500 calls we placed to CMS's 1-800-MEDICARE help
line regarding the Part D benefit were frequently accurate and
complete. However, we nonetheless received a substantial number of
responses that were inaccurate, incomplete, or inappropriate and
that sometimes involved an extensive wait before we could speak to
a CSR. CSRs answered 67 percent of the calls accurately and
completely. Of the remainder, 18 percent of the calls received
inaccurate responses, 8 percent of the responses were
inappropriate given the question asked, and about 3 percent
received incomplete responses. In addition, about 5 percent of our
calls were not answered, primarily because of disconnections.12
Accuracy and completeness rates of CSRs' responses varied
significantly for the five questions we asked. For example, for
the question on whether a beneficiary qualifies for extra help,
CSRs provided an accurate and complete response 90 percent of the
time. However, for a question concerning which drug plan would
cost the least for a beneficiary with certain specified
prescription drug needs, the accuracy rate was 41 percent. CSRs
inappropriately responded 35 percent of the time that this
question could not be answered without personal identifying
information-such as the beneficiary's Medicare number or date of
birth-even though the CSRs could have answered our question using
CMS's Web-based prescription drug plan finder tool. The amount of
time we waited to speak with a CSR also varied, ranging from no
wait time to over 55 minutes. For 75 percent of the calls-374 of
the 500-we waited less than 5 minutes. For the remainder of the
calls, 62 were answered in less than 15 minutes, 39 calls were
answered in from 15 minutes to less than 25 minutes, and 25 led to
a wait of 25 minutes or more.
We found that the Part D benefit portion of the Medicare Web site
can be difficult to use. In our evaluation of overall
usability-the ease of finding needed information and performing
various tasks-we found usability scores of 47 percent for seniors
and 53 percent for younger adults, out of a possible 100 percent.
While there is no widely accepted benchmark for usability, these
scores indicate that using the site can be difficult. For example,
tools such as the drug plan finder were complicated to use, and
forms that collect information online from users were difficult to
correct if the user made an error. In our evaluation of the
usability of 137 detailed aspects of the Part D portion of the
site, including features of Web design and online tools, we found
that 70 percent of these aspects could be expected to cause users
confusion. For example, key functions of the prescription drug
plan finder tool, such as the "continue" and "choose a drug plan"
buttons, were often not visible on the page without scrolling
down. In our evaluation of the ability of seven participants to
collectively complete 34 user tests, we found that on average,
participants were able to proceed slightly more than halfway
through each test. In addition, CMS evaluated whether its Web site
complied with pertinent federal requirements regarding
accessibility for people with disabilities in March 2006. Although
CMS has established features to make information on its Web site
accessible to disabled users, it found that two requirements were
not met, making it difficult for the visually impaired to use. A
CMS official told us that the agency made the appropriate
corrections on April 20, 2006. Because of time constraints, we did
not verify that these corrections were made.
CMS relies on SHIPs to play a significant role in providing
counseling and education on the Part D benefit to Medicare
beneficiaries. CMS increased SHIP funding from $12 million for the
2003 SHIP grant year13 to $31.7 million for the 2005 grant year.
CMS kept funding relatively high for the 2006 grant year-$30
million-to ensure that SHIPs continued to play an important role
in educating beneficiaries about Part D. The number of
beneficiaries served by SHIPs has also increased. During the 2004
SHIP grant year, SHIPs served approximately 2.52 million people.
During the first 9 months of the 2005 SHIP grant year-when CMS was
gearing up its outreach and education on Part D-SHIPs served
approximately 3.3 million individuals, an increase of nearly
770,000 from the prior full grant year. CMS attributes the
increase in demand for SHIP services-as reflected in increases in
the number of calls, face-to-face assistance, and referrals from
the 1-800-MEDICARE help line-to beneficiaries' need for assistance
on Part D. The average number of calls per month referred from the
help line to SHIPs, for example, increased from about 16,000
referrals for May through September 2005 to an average of about
43,000 for October and November 2005, about the time Part D
enrollment began. According to CMS officials, this increased
demand can be attributed to callers seeking advice on choosing a
drug plan. Unlike CSRs on the help line, SHIP counselors can offer
individualized guidance to callers on enrollment and plan
selection. SHIP coordinators in the five states we contacted
confirmed that there was a substantial increase in the demand for
their services because of the new Part D benefit. For example, the
California SHIP served over 120,000 people in January 2006,
compared to about 35,000 served in all of 2005.
Within the past 6 months, millions of Medicare beneficiaries have
been making important decisions about their prescription drug
coverage and have needed access to information about the new Part
D benefit to make appropriate choices. CMS faced a tremendous
challenge in responding to this need and, within short time
frames, developed a range of outreach and educational materials to
inform beneficiaries and their advisers about Part D. To
disseminate these materials, CMS largely added information to
existing resources, including written documents, such as Medicare
& You; the 1-800-MEDICARE help line; the Medicare Web site; and
support for SHIPs. However, CMS has not ensured that its
communications to beneficiaries and their advisers are provided in
a manner that is consistently clear, complete, accurate, and
usable. Six months have passed since these materials were first
made available to beneficiaries, and their limitations could
result in confusion among those seeking to make coverage
decisions. Although the initial enrollment period for Part D will
end on May 15, 2006, CMS will continue to play a pivotal role in
providing beneficiaries with information about the drug benefit
during the year and in subsequent enrollment periods. CMS has an
opportunity to enhance its communications on the Part D benefit.
This would allow beneficiaries and their advisers to be better
prepared when deciding whether to enroll in the benefit, and if
enrolling, which drug plan to choose.
In order to improve the Part D benefit education and outreach
materials that CMS provides to Medicare beneficiaries, we are
recommending that the CMS Administrator take the following four
actions:
o Ensure that CMS's written documents describe the Part D benefit
in a manner that is consistent with commonly recognized
communications guidelines and that is responsive to the intended
audience's needs.
o Determine why CSRs frequently do not search for available drug
plans if the caller does not provide personal identifying
information.
o Monitor the accuracy and completeness of CSRs' responses to
callers' inquiries and identify tools targeted to improve their
performance in responding to questions concerning the Part D
benefit, such as additional scripts and training.
o Improve the usability of the Part D portion of the Medicare Web
site by refining Web-based tools, providing workable site
navigation features and links, and making Web-based forms easier
to use and correct.
We received written comments on a draft of this report from CMS
(see app. III). CMS said that it did not believe our findings
presented a complete and accurate picture of its Part D
communications activities. CMS discussed several concerns
regarding our findings on its written documents and the
1-800-MEDICARE help line. However, CMS did not disagree with our
findings regarding the Medicare Web site or the role ofSHIPs. CMS
also said that it supports the goals of our recommendationsand is
already taking steps to implement them, such as continually
enhancing and refining its Web-based tools.
CMS discussed concerns regarding the completeness and accuracy of
our findings in terms of activities we did not examine, as well as
those we did. CMS stated that our findings were not complete
because our report did not examine all of the agency's efforts to
educate Medicare beneficiaries and specifically mentioned that we
did not examine the broad array of communication tools it has made
available, including the development of its network of grassroots
partners throughout the country. We recognize that CMS has taken
advantage of many vehicles to communicate with beneficiaries and
their advisers. However, we focused our work on the four specific
mechanisms that we believed would have the greatest impact on
beneficiaries-written materials, the 1-800-MEDICARE help line, the
Medicare Web site, and the SHIPs. In addition, CMS stated that our
report is based on information from January and February 2006, and
that it has undertaken a number of activities since then to
address the problems we identified. Although we appreciate CMS's
efforts to improve its Part D communications to beneficiaries on
an ongoing basis, we believe it is unlikely that the problems we
identified in this report could have been corrected yet given
their nature and scope.
CMS raised two concerns with our examination of a sample of
written materials. First, it criticized our use of readability
tests to assess the clarity of the six sample documents we
reviewed. For example, CMS said that common multisyllabic words
would inappropriately inflate the reading level. However, we found
that reading levels remained high after adjusting for 26
multisyllabic words a Medicare beneficiary would encounter, such
as Social Security Administration. CMS also pointed out that some
experts find such assessments to be misleading. Because we
recognize that there is some controversy surrounding the use of
reading levels, we included two additional assessments to
supplement this readability analysis-the assessment of design and
organization of the sample documents based on 60 commonly
recognized communications guidelines and an examination of the
usability of six sample documents, involving 11 beneficiaries and
5 advisers.
Second, CMS expressed concern about our examination of the
usability of the six sample documents. The participating
beneficiaries and advisers were called on to perform 18 specified
tasks, after reading the selected materials, including a section
of the Medicare & You handbook. CMS suggested that the task asking
beneficiaries and advisers to calculate their out-of-pocket drug
costs was inappropriate because there are many other tools that
can be used to more effectively compare costs. We do not disagree
with CMS that there are a number of ways beneficiaries may
complete this calculation; however, we nonetheless believe that it
is important that beneficiaries be able to complete this task on
the basis of reading Medicare & You, which, as CMS points out, is
widely disseminated to beneficiaries, reaching all beneficiary
households each year. In addition, CMS noted that it was not able
to examine our detailed methodology regarding the clarity of
written materials-including assessments performed by one of our
contractors concerning readability and document design and
organization. We plan to share this information with CMS, once our
report has become public.
Finally, CMS took issue with one aspect of our evaluation of the
1-800-MEDICARE help line. Specifically, CMS said the 41 percent
accuracy rate associated with one of the five questions we asked
was misleading, because, according to CMS, we failed to analyze 35
of the 100 responses. However, we disagree. This question
addressed which drug plan would cost the least for a beneficiary
with certain specified prescription drug needs. We analyzed these
35 responses to this question and found the responses to be
inappropriate. The CSRs would not provide us with the information
we were seeking because we did not supply personal identifying
information, such as the beneficiary's Medicare number or date of
birth. We considered such responses inappropriate because the CSRs
could have answered this question without personal identifying
information by using CMS's Web-based prescription drug plan finder
tool. Although CMS said that it has emphasized to CSRs, through
training and broadcast messages, that it is permissible to provide
the information we requested without requiring information that
would personally identify a beneficiary, in these 35 instances,
the CSR simply told us that our question could not be answered.
CMS also said that the bulk of these inappropriate responses were
related to our request that the CSR use only brand-name drugs.
This is incorrect-none of these 35 responses were considered
incorrect or inappropriate because of a request that the CSR use
only brand-name drugs-as that was not part of our question.
As arranged with your offices, unless you publicly announce the
contents of this report earlier, we plan no further distribution
of it until 30 days after its date. At that time, we will send
copies of this report to the Secretary of Health and Human
Services, the Administrator of the Centers for Medicare & Medicaid
Services, and other interested parties. We will also make copies
available to others on request. In addition, the report will be
available at no charge on the GAO Web site at http://www.gao.gov.
If you or your staff have any questions about this report, please
contact me at (312) 220-7600 or [email protected]. Contact points
for our Offices of Congressional Relations and Public Affairs may
be found on the last page of this report. GAO staff who made major
contributions to this report are listed in appendix IV.
Leslie G. Aronovitz Director, Health Care
List of Requesters
The Honorable John D. Dingell Ranking Minority Member Committee on
Energy and Commerce House of Representatives
The Honorable Henry A. Waxman Ranking Minority Member Committee on
Government Reform House of Representatives
The Honorable Charles B. Rangel Ranking Minority Member Committee
on Ways and Means House of Representatives
The Honorable Sherrod Brown Ranking Minority Member Subcommittee
on Health Committee on Energy and Commerce House of
Representatives
The Honorable Pete Stark Ranking Minority Member Subcommittee on
Health Committee on Ways and Means House of Representatives
In this report, we assessed (1) the extent to which the Centers
for Medicare & Medicaid Services' (CMS) written documents describe
the Medicare Part D prescription drug benefit in a clear,
complete, and accurate manner; (2) the effectiveness of CMS's
1-800-MEDICARE help line in providing accurate, complete, and
prompt responses to callers inquiring about the Part D benefit;
(3) whether CMS's Medicare Web site presents information on the
Part D benefit in a usable manner; and (4) how CMS has used State
Health Insurance Assistance Programs (SHIP) to respond to the
needs of Medicare beneficiaries for information on the Part D
benefit. To obtain information on CMS's efforts to educate
beneficiaries about Part D, we interviewed agency officials
responsible for Part D written documents, the 1-800-MEDICARE help
line, the Medicare Web site, and SHIPs. Following our briefing of
congressional staff on April 19, 2006, the briefing slides were
updated to reflect CMS's reported correction to the Medicare Web
site to comply with section 508 of the Rehabilitation Act of
1973.1 We determined that the data used were sufficiently reliable
for the purposes of this report.
To assess the clarity, completeness, and accuracy of written
documents, we compiled a list of all available CMS-issued Part D
benefit publications intended to inform beneficiaries and their
advisers and selected a sample of 6 from the 70 CMS documents
available, as of December 7, 2005, for in-depth review, as shown
in table 1. The sample Part D documents were chosen to represent a
variety of publication types, such as frequently asked questions
and fact sheets available to beneficiaries about the Part D drug
benefit. We selected documents that targeted all beneficiaries or
those with unique drug coverage concerns, such as dual-eligibles
and beneficiaries with Medigap.2
Table 1: Sample of Six Selected Documents
Source: GAO.
aDual-eligible beneficiaries are Medicare beneficiaries who
receive full Medicaid benefits for services not covered by
Medicare.
bMedicare Advantage replaced the Medicare+ Choice managed care
program and expanded the availability of private health plan
options to Medicare beneficiaries.
To evaluate clarity, we contracted with the American Institutes
for Research (AIR)-a firm with experience in evaluating written
material. AIR evaluated the texts of the six sample documents
using three methodologies:
1. three standard readability tests;3
2. 60 commonly recognized written communications
guidelines, including practices to aid senior
readers; and
3. user testing with 11 Medicare beneficiaries and 5
advisers to beneficiaries, who performed 18 specified
tasks related to enrollment, coverage, cost, penalty,
and information resources and provided feedback about
their experiences.
We reviewed the sample documents for completeness to determine
whether they contained sufficient information to allow the
beneficiaries to identify (1) their next steps in determining
whether to enroll and what plan to choose and (2) important
factors, such as penalty provisions, that could affect their
coverage decisions. To identify those important factors associated
with the Part D benefit, we reviewed relevant laws, regulations,
and 1-800-MEDICARE scripts prepared for customer service
representatives (CSR) to read to callers and obtained information
from advocacy groups. To evaluate the accuracy of information, we
reviewed the sample materials for compliance with laws,
regulations, and CMS guidance.
To determine the accuracy and completeness of information provided
regarding the Part D benefit, we placed a total of 500 calls to
the 1-800-MEDICARE help line. We posed one of five questions about
Part D in each call, so that each question was asked 100 times.
Each question was pretested before we finalized its wording. We
randomly placed calls at different times of the day and different
days of the week from January 17 to February 7, 2006. Our calling
times were chosen to match the daily and hourly pattern of calls
reported by 1-800-MEDICARE in October 2005. We informed CMS
officials that we would be placing calls; however, we did not tell
them the questions we would ask or the specific dates and times
that we would be placing our calls.
To select the five questions, we considered topics identified in
the Medicare Web site's frequently asked questions. In addition,
we considered topics most frequently addressed by 1-800-MEDICARE
CSRs based on help line reports. To evaluate the accuracy of CSRs'
responses to our five questions, we used three resources: (1) the
prescription drug plan finder tool on the Medicare Web site, (2)
1-800-MEDICARE scripts, and (3) input obtained from CMS officials
on the criteria we used for evaluating CSR responses. Table 2
lists the questions we asked and the criteria we used to evaluate
the accuracy of responses.
9The Henry J. Kaiser Family Foundation, The Medicare Drug Benefit:
Beneficiaries Perspectives Just Before Implementation,
http://kff.org/kaiserpolls/med111005nr.cfm (downloaded Apr. 26, 2006).
10The three tests were the Flesch-Kincaid Grade Level, the SMOG
(Simplified Measure of Gobbledygook) Reading Grade Level, and the Fry
Readability Estimate. These tests use such measures as sentence length and
the number of syllables in a selection of text to arrive at a reading
level, which is expressed in terms of school grade level.
Results in Brief
11A formulary is a list of prescription drugs covered by a health plan.
12The percentages related to the responses we received to our 500 calls
exceed 100 percent because of rounding.
13A SHIP grant year begins on April 1 of the year the funds become
available.
Conclusions
Recommendations for Executive Action
Agency Comments and Our Evaluation
Appendix I: Briefing on Medicare Part D Appendix I: Briefing on Medicare
Part D
Appendix II: Methodology Appendix II: Objectives, Scope, and Methodology
Written Documents
129 U.S.C. S: 794d (2000).
2Medigap policies provide supplemental health coverage sold by private
insurers to help pay for Medicare cost-sharing requirements, as well as
for some services not provided by Medicare.
Document Target audience
Medicare & You, Section 6: Medicare All beneficiaries
Prescription Drug Coverage
Things to Think about When You Compare Plans All beneficiaries
Frequently Asked Questions about: Retiree Beneficiaries with employer
Prescription Drug Coverage & the New Medicare or union coverage
Prescription Drug Coverage
Introduction to the Auto-Enrollment Notice Dual-eligible beneficiariesa
Quick Facts about Medicare's New Coverage for Beneficiaries with Medicare
Prescription Drugs for People with a Medicare Advantageb
Health Plan with Prescription Drug Coverage
Do You Have a Medigap Policy with Beneficiaries with Medigap
Prescription Drug Coverage?
3The three tests were the Flesch-Kincaid Grade Level, the SMOG (Simplified
Measure of Gobbledygook) Reading Grade Level, and the Fry Readability
Estimate. The tests use such measures as sentence length and the number of
syllables in a selection of text to arrive at a reading level, which is
expressed in terms of school grade level.
The 1-800-MEDICARE Help Line
Table 2: Questions and Criteria Used to Evaluate Accuracy
Question Criteria
1. What drug plan can a An accurate and complete response would
beneficiary get that will identify the prescription drug plan that
cover all of his/her has the lowest estimated annual cost for
[specified] drugs at a the drugs the beneficiary uses.
[specified] pharmacy; have a
mail-order option; and cost
the least amount annually with
[or without] a deductible?
2. Can a beneficiary who is in An accurate and complete response would
a nursing home and not on indicate that a beneficiary can choose
Medicaid sign up for a whether to enroll in a Medicare
prescription drug plan? prescription drug plan.
3. Can a beneficiary enroll in An accurate and complete response would
the Medicare prescription drug inform the caller that enrolling for the
program and keep his/her prescription drug benefit would depend on
current Medigap policy? whether the beneficiary's Medigap plan was
creditable-that is, whether the coverage it
provided was at least as good as Medicare's
standard prescription drug coverage-or
noncreditable. The CSR response would also
mention that the beneficiary's Medigap plan
should have sent him/her information that
outlined options.
4. What options does a An accurate and complete response would
beneficiary, who has retiree indicate that a beneficiary has two
health insurance with options: (1) keep current health plan and
prescription drug coverage join the prescription drug plan later with
that is not as good as the a penalty or (2) drop current coverage and
Medicare prescription drug join a Medicare drug plan.
coverage, have as it relates
to the Medicare benefit?
5. How do I know if a An accurate and complete response would
beneficiary qualifies for refer the beneficiary to the Social
extra help? Security Administration.
Source: GAO.
When placing our calls, we identified ourselves as a beneficiary's
relative, but did not provide CSRs with specific identifying information,
such as a Medicare beneficiary number or date of birth. During our calls,
CSRs were not aware that their responses would be included in a research
study. We recorded the length of each call, including wait times, and the
time it took before being connected to a CSR. We evaluated the accuracy
and completeness of the responses by CSRs to the 500 calls by determining
whether key information was provided.
The results from our 500 calls are limited to those calls and are not
generalizable to the universe of calls made to the help line. The
questions we asked were limited to matters concerning the Part D benefit
and do not encompass all of the questions callers might ask.
Medicare Web Site
We contracted with the Nielsen Norman Group (NN/g)-a firm with expertise
in Web design-to assess the usability of the Part D information available
on the Medicare Web site. This study consisted of three separate
evaluations. First, NN/g compared the site's compliance with established
usability guidelines to determine a usability score to reflect the ease of
finding necessary information and performing various tasks. Specifically,
to determine the usability scores, NN/g evaluated various aspects of the
Web site using industry-recognized "good" Web design practices, as
indicated by the contractor, and the collective body of knowledge from
NN/g internal reports and experts, or NN/g usability guidelines.4
Second, NN/g determined the degree of difficulty associated with 137
detailed aspects of Web site design for the Part D portion of the site.
The 137 aspects fall into the following general categories:
o overall Web design (e.g., home page, navigation, search
function, graphics, and overall organization);
o tools (e.g., plan finder);
o writing style (e.g., content, tone, legibility, and
readability);
o accessibility (e.g., availability of a version of the Web site
for the blind); and
o languages (e.g., availability of languages other than English).
NN/g determined the difficulty level in using each of the 137
aspects. NN/g noted aspects that had good design and would not be
expected to cause confusion. For those aspects with a design that
would be expected to cause confusion, NN/g ranked the associated
difficulty level as high, medium, or low.5
Third, NN/g performed a qualitative evaluation on January 20 and
23, 2006, to test the ability of five Medicare beneficiaries and
two beneficiary advisers to perform specified tasks related to
Medicare beneficiaries using the Web site and to obtain feedback
about participants' experiences. While the results are not
statistically valid, these users provided important insights into
the usability of the Medicare Web site. Participants were asked to
"think out loud" as they worked through their tasks, while an NN/g
facilitator observed their behavior and took notes. NN/g gave each
task a score. At the end of their sessions, NN/g asked
participants for input regarding their confidence in the answers
they obtained from the Web site, and their overall satisfaction
and frustration levels associated with using the site.
Finally, we obtained the results of CMS's March 2006 review of its
Web site's compliance with section 508 of the Rehabilitation Act
of 1973, as amended. This law requires federal agencies to make
the information on their Web sites accessible to people with
disabilities. We also discussed the results of this review with
agency officials and followed up with them to determine the status
of CMS's corrective actions.
To determine the role of SHIPs in helping Medicare beneficiaries
understand Part D, we interviewed CMS officials who monitor SHIPs'
activities. We also reviewed information that we obtained from CMS
officials and other sources on the program, its funding, changes
made in response to the introduction of Part D, and the impact of
Part D on the demand for SHIP services. In addition, we
interviewed SHIP officials in California, Florida, New York,
Texas, and Pennsylvania-the five states with the largest Medicare
populations-to obtain information on the experience of their SHIPs
with Part D.
We conducted our work from November 2005 through May 2006 in
accordance with generally accepted government auditing standards.
Leslie G. Aronovitz (312) 220-7600 or [email protected]
In addition to the contact named above, Susan T. Anthony and
Geraldine Redican-Bigott, Assistant Directors; Ramsey L. Asaly;
Enchelle Bolden; Laura Brogan; Shaunessye D. Curry; Chir-Jen
Huang; M. Peter Juang; Ba Lin; Michaela M. Monaghan; Roseanne
Price; Pauline Seretakis; Margaret J. Weber; and Craig H. Winslow
made contributions to this report.
The Government Accountability Office, the audit, evaluation and
investigative arm of Congress, exists to support Congress in
meeting its constitutional responsibilities and to help improve
the performance and accountability of the federal government for
the American people. GAO examines the use of public funds;
evaluates federal programs and policies; and provides analyses,
recommendations, and other assistance to help Congress make
informed oversight, policy, and funding decisions. GAO's
commitment to good government is reflected in its core values of
accountability, integrity, and reliability.
The fastest and easiest way to obtain copies of GAO documents at
no cost is through GAO's Web site ( www.gao.gov ). Each weekday,
GAO posts newly released reports, testimony, and correspondence on
its Web site. To have GAO e-mail you a list of newly posted
products every afternoon, go to www.gao.gov and select "Subscribe
to Updates."
The first copy of each printed report is free. Additional copies
are $2 each. A check or money order should be made out to the
Superintendent of Documents. GAO also accepts VISA and Mastercard.
Orders for 100 or more copies mailed to a single address are
discounted 25 percent. Orders should be sent to:
U.S. Government Accountability Office 441 G Street NW, Room LM
Washington, D.C. 20548
To order by Phone: Voice: (202) 512-6000 TDD: (202) 512-2537 Fax:
(202) 512-6061
Contact:
Web site: www.gao.gov/fraudnet/fraudnet.htm E-mail:
[email protected] Automated answering system: (800) 424-5454 or
(202) 512-7470
Gloria Jarmon, Managing Director, [email protected] (202) 512-4400
U.S. Government Accountability Office, 441 G Street NW, Room 7125
Washington, D.C. 20548
Paul Anderson, Managing Director, [email protected] (202)
512-4800 U.S. Government Accountability Office, 441 G Street NW,
Room 7149 Washington, D.C. 20548
4These guidelines are presented in an NN/g report called Web Usability for
Senior Citizens: 46 Design Guidelines Based on Usability Studies with
People Age 65 and Older. For this study, NN/g conducted usability tests of
17 Web sites with 44 seniors. Based on the test findings, NN/g developed
46 design guidelines that would make Web sites more attractive to seniors.
5In addition, NN/g indicated cases where an aspect was not functioning
correctly from a Web site development standpoint by giving it a "bug"
mark.
State Health Insurance Assistance Programs
Appendix III: Comments from the Centers for Medicare & Medicaid Services
Appendix III: Comments from the Centers for Medicare & Medicaid Services
Appendix IV: St Appendix IV: GAO Contact and Staff Acknowledgments
GAO Contact
Acknowledgments
(290509)
GAO's Mission
Obtaining Copies of GAO Reports and Testimony
Order by Mail or Phone
To Report Fraud, Waste, and Abuse in Federal Programs
Congressional Relations
Public Affairs
www.gao.gov/cgi-bin/getrpt? GAO-06-654 .
To view the full product, including the scope
and methodology, click on the link above.
For more information, contact Leslie G. Aronovitz at [email protected] or
(312) 220-7600.
Highlights of GAO-06-654 , a report to congressional requesters
May 2006
MEDICARE
Communications to Beneficiaries on the Prescription Drug Benefit Could Be
Improved
On January 1, 2006, Medicare began providing coverage for outpatient
prescription drugs through its new Part D benefit. Beneficiaries who
enroll in Part D may choose a drug plan from those offered by private plan
sponsors under contract to the Centers for Medicare & Medicaid Services
(CMS), which administers the Part D benefit. Beneficiaries have until May
15, 2006, to enroll in the Part D benefit and select a plan without the
risk of penalties.
GAO was asked to review the quality of CMS's communications on the Part D
benefit. GAO examined 70 CMS publications to select 6 documents for review
and contracted with the American Institutes for Research to evaluate the
clarity of these texts; made 500 calls to the 1-800-MEDICARE help line;
and contracted with the Nielsen Norman Group to evaluate the usability of
the Medicare Web site.
What GAO Recommends
GAO is recommending that the CMS Administrator enhance the quality of its
communications by taking actions to improve written materials, its
1-800-MEDICARE help line, and the Medicare Web site. CMS said that GAO's
findings did not present a complete and accurate picture of its
activities. However, CMS said that it supports the goals of GAO's
recommendations and is already taking steps to implement them.
The information given in the six sample documents that GAO reviewed
describing the Part D benefit was largely complete and accurate, although
this information lacked clarity. The documents were unclear in two ways.
First, although about 40 percent of seniors read at or below the
fifth-grade level, the reading levels of these documents ranged from
seventh grade to postcollege. Second, on average, the six documents did
not comply with about half of 60 common guidelines for good communication.
For example, the documents used too much technical jargon and often did
not define difficult terms, such as formulary. Moreover, 16 beneficiaries
and advisers that GAO tested reported frustration with the documents' lack
of clarity and had difficulty completing the tasks assigned to them.
Although the documents lacked clarity, they informed readers of enrollment
steps and factors affecting coverage decisions and were consistent with
laws, regulations, and agency guidance.
Customer service representatives (CSR) responded to the 500 calls GAO
placed to CMS's 1-800-MEDICARE help line accurately and completely about
two-thirds of the time. Of the remainder, 18 percent of the calls received
inaccurate responses, 8 percent of the responses were inappropriate given
the question asked, and about 3 percent received incomplete responses. In
addition, about 5 percent of GAO's calls were not answered, primarily
because of disconnections. Accuracy and completeness rates of CSRs'
responses varied significantly across the five questions GAO asked. For
example, while CSRs provided accurate and complete responses to calls
about beneficiaries' eligibility for extra help 90 percent of the time,
the accuracy rate for calls concerning the drug plan that would cost the
least for a specified beneficiary was 41 percent. For this question, the
CSRs responded inappropriately for 35 percent of the calls by explaining
that they could not identify the least costly plan without the
beneficiary's personal information-even though CSRs had the information
needed to answer the question. The time GAO callers waited to speak with
CSRs also varied, ranging from no wait time to over 55 minutes. For 75
percent of the calls-374 of the 500-the wait was less than 5 minutes.
The Part D benefit portion of the Medicare Web site can be difficult to
use. GAO's test of the site's overall usability-the ease of finding needed
information and performing various tasks-resulted in scores of 47 percent
for seniors and 53 percent for younger adults, out of a possible 100
percent. While there is no widely accepted benchmark for usability, these
scores indicate that using the site can be difficult. For example, the
prescription drug plan finder was complicated to use and some of its key
functions, such as "continue" and "choose a drug plan," were often not
visible on the page without scrolling down.
*** End of document. ***