Medicare: Communications to Beneficiaries on the Prescription	 
Drug Benefit Could Be Improved (03-MAY-06, GAO-06-654). 	 
                                                                 
On January 1, 2006, Medicare began providing coverage for	 
outpatient prescription drugs through its new Part D benefit.	 
Beneficiaries who enroll in Part D may choose a drug plan from	 
those offered by private plan sponsors under contract to the	 
Centers for Medicare & Medicaid Services (CMS), which administers
the Part D benefit. Beneficiaries have until May 15, 2006, to	 
enroll in the Part D benefit and select a plan without the risk  
of penalties. GAO was asked to review the quality of CMS's	 
communications on the Part D benefit. GAO examined 70 CMS	 
publications to select 6 documents for review and contracted with
the American Institutes for Research to evaluate the clarity of  
these texts; made 500 calls to the 1-800-MEDICARE help line; and 
contracted with the Nielsen Norman Group to evaluate the	 
usability of the Medicare Web site.				 
-------------------------Indexing Terms------------------------- 
REPORTNUM:   GAO-06-654 					        
    ACCNO:   A53099						        
  TITLE:     Medicare: Communications to Beneficiaries on the	      
Prescription Drug Benefit Could Be Improved			 
     DATE:   05/03/2006 
  SUBJECT:   Beneficiaries					 
	     Communication					 
	     Government information dissemination		 
	     Health care programs				 
	     Medicare						 
	     Prescription drugs 				 
	     Program evaluation 				 
	     Usability						 
	     Websites						 
	     Written communication				 
	     Customer service					 
	     Medicare Part D					 

******************************************************************
** This file contains an ASCII representation of the text of a  **
** GAO Product.                                                 **
**                                                              **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced.  Tables are included, but    **
** may not resemble those in the printed version.               **
**                                                              **
** Please see the PDF (Portable Document Format) file, when     **
** available, for a complete electronic file of the printed     **
** document's contents.                                         **
**                                                              **
******************************************************************
GAO-06-654

     

     * Results in Brief
     * Conclusions
     * Recommendations for Executive Action
     * Agency Comments and Our Evaluation
     * Written Documents
     * The 1-800-MEDICARE Help Line
     * Medicare Web Site
     * State Health Insurance Assistance Programs
     * GAO Contact
     * Acknowledgments
     * GAO's Mission
     * Obtaining Copies of GAO Reports and Testimony
          * Order by Mail or Phone
     * To Report Fraud, Waste, and Abuse in Federal Programs
     * Congressional Relations
     * Public Affairs

Report to Congressional Requesters

United States Government Accountability Office

GAO

May 2006

MEDICARE

Communications to Beneficiaries on the Prescription Drug Benefit Could Be
Improved

GAO-06-654

Contents

Letter 1

Results in Brief 5
Conclusions 8
Recommendations for Executive Action 9
Agency Comments and Our Evaluation 9
Appendix I Briefing on Medicare Part D 13
Appendix II Objectives, Scope, and Methodology 61
Appendix III Comments from the Centers for Medicare & Medicaid Services 67
Appendix IV GAO Contact and Staff Acknowledgments 83

Tables

Table 1: Sample of Six Selected Documents 62
Table 2: Questions and Criteria Used to Evaluate Accuracy 64

Abbreviations

AIR American Institutes for Research CMS Centers for Medicare & Medicaid
Services CSR customer service representative MMA Medicare Prescription
Drug, Improvement, and Modernization Act of 2003 NN/g Nielsen Norman Group
SHIP State Health Insurance Assistance Program SMOG Simplified Measure of
Gobbledygook

This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed in
its entirety without further permission from GAO. However, because this
work may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this material
separately.

United States Government Accountability Office

Washington, DC 20548

May 3, 2006

Congressional Requesters

In the most significant change to the Medicare program since its
inception, the Medicare Prescription Drug, Improvement, and Modernization
Act of 2003 (MMA)1 established an outpatient prescription drug benefit in
Medicare, known as the Part D benefit. Coverage for this new benefit began
on January 1, 2006. Until this time, Medicare, the federal program that
finances health care benefits for about 42 million elderly and disabled
beneficiaries, had not generally provided coverage for outpatient
prescription drugs. Beneficiaries may choose a Part D plan2 from multiple
plans offered by private sponsors3 under contract to the Centers for
Medicare & Medicaid Services (CMS),4 the agency that is responsible for
administering the Medicare program, including the Part D benefit. These
plans differ in the drugs they cover and the pharmacies they use. In
addition, the costs to the enrollee for the monthly premium, the annual
deductible, and co-payments for covered drugs vary by plan. As of April
20, 2006, more than 30 million of Medicare's 42 million beneficiaries were
enrolled in a Part D plan or had other outpatient prescription drug
coverage. Beneficiaries have until May 15, 2006, to select a plan without
the risk of penalties in the form of higher premiums.

Given the newness and complexity of the Part D benefit, it is critical
that beneficiaries and their advisers, including members of their
families, understand the available options so that beneficiaries can make
informed decisions on whether to enroll in Part D, and if so, which drug
plan to choose. Beneficiaries need to compare drug plans in light of their
anticipated prescription drug needs and existing arrangements for paying
for these drugs. In addition to comparing costs and drug coverage,
beneficiaries need to consider whether the plans they are comparing have
contracted with a local or mail-order pharmacy that will provide a
convenient means of filling their prescriptions.

1Pub. L. No. 108-173, S: 101, 117 Stat. 2066, 2071-2152 (to be codified at
42 U.S.C. S:S: 1395w-101-1395w-152). MMA redesignated the previous part D
of title XVIII of the Social Security Act as part E and inserted a new
part D after part C.

2For Part D standard coverage, Medicare pays on average 75 percent of
prescription drug costs up to $2,250, after a $250 deductible.
Beneficiaries then pay their next $2,850 in drug costs. If their drug
costs exceed this amount, Medicare will pay about 95 percent of their
additional costs for the rest of the calendar year.

3Drug plan sponsors include insurance companies and other private
organizations.

4CMS is an agency in the Department of Health and Human Services.

As part of its responsibilities, CMS has undertaken outreach and education
efforts to provide beneficiaries and their advisers with the information
they need about Part D through various media, including written documents,
the 1-800-MEDICARE help line,5 and the Medicare Web site.6 As of December
2005, CMS has produced more than 70 written documents to explain Part D to
beneficiaries. Medicare & You-the beneficiary handbook-is the most widely
available of these documents and was sent directly to beneficiaries in
October 2005. Other CMS documents are targeted to specific groups of
beneficiaries, such as dual-eligible beneficiaries7 and beneficiaries with
Medicare Advantage or Medigap policies.8 Since March 1999, CMS has
administered its nationwide 1-800-MEDICARE help line to answer
beneficiaries' questions about the Medicare program. As of December 2005,
about 7,500 customer service representatives (CSR) were handling calls on
the help line, which operates 24 hours a day, 7 days a week, and is run by
two CMS contractors. Calls are answered by an automated system and are
routed to a CSR for specific questions, including those about Part D. CMS
provides CSRs with detailed scripts to use in answering the questions.
CSRs type in related keywords to generate a list of suggested scripts for
a given question, select the script they consider best suited to the
question, and read excerpts or the entire script. Call center contractors
write the scripts, and CMS checks them for accuracy and completeness. CSRs
can also consult other information sources, such as the Medicare Web site.
CMS does not allow CSRs to offer individualized guidance to callers,
including advice in choosing a drug plan. CMS's Medicare Web site provides
information about all aspects of the Medicare program. The Web site
contains basic information about the Part D benefit; suggests factors for
beneficiaries to consider when choosing a plan; describes situations
common to beneficiaries with guidance on next steps to take in deciding
whether to enroll and what plan to choose; lists frequently asked
questions; and allows users to view, print, or order publications. In
addition, the site contains information on cost, coverage, and convenience
of individual plans. There is also a tool that allows beneficiaries to
enroll directly in the plan they have chosen.

5In December 2004, we reported on the information being provided to
beneficiaries through the Medicare help line on eligibility, enrollment,
and benefits. (See GAO, Medicare: Accuracy of Responses from the
1-800-MEDICARE Help Line Should Be Improved, GAO-05-130 (Washington, D.C.:
Dec. 8, 2004).)

6The Medicare Web site is www.medicare.gov.

7Dual-eligible beneficiaries are Medicare beneficiaries who are also
eligible for Medicaid-the federal-state health program for low-income
individuals-and receive full Medicaid benefits for services not covered by
Medicare.

8Medicare Advantage replaced the Medicare+Choice managed care program and
expanded the availability of private health plan options to Medicare
beneficiaries. Medigap policies provide supplemental health coverage sold
by private insurers to help pay for Medicare cost-sharing requirements, as
well as for some services not provided by Medicare.

CMS has also arranged for State Health Insurance Assistance Programs
(SHIP) to provide Part D information on request to Medicare beneficiaries
and their advisers. Currently, CMS provides grants to the 54 SHIPs-one in
each state, the District of Columbia, the Virgin Islands, Puerto Rico, and
Guam. State SHIPs provide subgrants to over 1,300 local organizations to
assist SHIPs in their efforts. In total, SHIPs rely on over 12,000 trained
counselors, most of whom are volunteers, to provide free counseling and
assistance via telephone and face-to-face sessions, public education
presentations and programs, and media activities.

Widespread confusion among beneficiaries about the costs and coverage
under the new benefit has been reported by the media and others. For
example, according to an October 2005 survey by a research organization,
some beneficiaries are unaware of the penalties for late enrollment and
others did not realize that beneficiaries had to sign up for the benefit.9
In light of your interest in ensuring that Medicare beneficiaries receive
the information they need to make informed decisions, you asked us to
examine the quality of the information being provided on the Part D
benefit. In this report, we examined

           o  the extent to which CMS's written documents describe the Part D
           benefit in a clear, complete, and accurate manner;
           o  the effectiveness of CMS's 1-800-MEDICARE help line in
           providing accurate, complete, and prompt responses to callers
           inquiring about the Part D benefit;
           o  whether CMS's Medicare Web site presents information on the
           Part D benefit in a usable manner; and
           o  how CMS has used SHIPs to respond to the needs of Medicare
           beneficiaries for information on the Part D benefit.

           We briefed your staff regarding the results of our review on April
           19, 2006. Appendix I contains information we provided during our
           briefing to your staff.

           To evaluate CMS's written documents describing the Part D benefit,
           we examined 70 relevant CMS publications and selected a sample of
           six documents for in-depth review. These documents represent a
           variety of document types, content, and target audiences and
           include Section 6 of the Medicare & You beneficiary handbook,
           which discusses Part D. To assess the clarity of the sample
           documents, we contracted with the American Institutes for Research
           (AIR), a firm with experience in evaluating written documents. AIR
           evaluated the texts by using three standard readability tests;10
           60 commonly recognized good communications practices; and user
           testing with 11 Medicare beneficiaries and 5 advisers to
           beneficiaries, all of whom were asked to perform 18 specified
           tasks related to enrollment, coverage, costs, penalty, and
           informational resources and provide feedback about their
           experiences. To evaluate completeness, we reviewed the sample
           documents to determine if they included sufficient information for
           the beneficiaries to identify (1) their next steps in deciding
           whether to enroll and what plan to choose and (2) important
           factors, such as penalty provisions, that could affect their
           coverage decisions. To evaluate accuracy, we reviewed the sample
           documents for consistency with MMA, regulations, and CMS guidance.

           To assess the accuracy, completeness, and promptness of the help
           line responses, we made 500 calls to 1-800-MEDICARE, posing one of
           five questions about Part D in each call so that each question was
           asked 100 times. To develop the questions, we considered topics
           listed on the Medicare Web site and topics addressed in scripts
           frequently accessed by CSRs. To develop our criteria for
           evaluating the accuracy and completeness of CSRs' responses, we
           used three resources: (1) the prescription drug finder tool on the
           Medicare Web site, (2) the 1-800-MEDICARE scripts, and (3) input
           from CMS officials. We also recorded the length of each call,
           including wait times, and the time it took to be connected to a
           CSR.

           To assess whether the Medicare Web site presents information on
           the Part D benefit in a usable manner, we contracted with the
           Nielsen Norman Group (NN/g), a firm with expertise in Web design.
           NN/g conducted three evaluations: (1) it calculated an overall
           usability score for the site-considering factors such as site
           navigation, customer support, and presentation of online forms-to
           reflect the ease of finding necessary information and performing
           various tasks; (2) it determined the usability of 137 detailed
           aspects of the Web site, including aspects of Web design, online
           tools, and writing style; and (3) it tested the ability of seven
           participants (five beneficiaries and two advisers to
           beneficiaries) to complete a total of 34 user tests to determine
           the ease of performing a variety of Web-related tasks, such as
           browsing the site and determining how to join a plan. We also
           reviewed the results of CMS's analysis of its Web site's
           compliance with requirements that federal government Web sites be
           accessible to people with disabilities.

           Finally, to examine how CMS has used SHIPs to meet the information
           needs of beneficiaries regarding Part D, we obtained information
           about SHIPs, their funding, changes made in response to the new
           benefit, and the impact of Part D on the demand for SHIP services.
           In addition, we interviewed CMS officials who monitor SHIP
           activities as well as SHIP coordinators in the five states with
           the largest populations of Medicare beneficiaries-California,
           Florida, New York, Pennsylvania, and Texas.

           We performed our work from November 2005 through May 2006 in
           accordance with generally accepted government auditing standards.
           For more information on our methodology, see appendix II.

           The sample of CMS's written documents we reviewed describing the
           Part D benefit to Medicare beneficiaries and their advisers were
           largely complete and accurate, but the information these documents
           presented lacked clarity. The documents were unclear in two ways.
           First, about 40 percent of seniors read at or below the
           fifth-grade level, but the reading levels of the documents ranged
           from seventh grade to postcollege. As a result, documents at these
           levels are not completely clear and understandable for many
           seniors. Second, on average, the six documents did not comply with
           about half of the 60 commonly recognized guidelines for good
           communications. For example, although the documents included
           concise and descriptive headings, they used too much technical
           jargon and often did not define difficult terms, such as
           formulary.11 The 11 beneficiaries and 5 advisers we tested
           reported frustration with the documents' lack of clarity as they
           encountered difficulties in understanding and attempting to
           complete 18 specified tasks. For example, none of these
           beneficiaries and only 2 of the advisers were able to complete the
           task of computing their projected total out-of-pocket costs for a
           plan that provided Part D's standard coverage. Only 1 of the 18
           tasks was completed by all beneficiaries and advisers. Even those
           who were able to complete a given task expressed confusion and
           frustration as they worked to comprehend the relevant text.
           Although the sample documents lacked clarity, the information
           presented in them was generally complete. The documents informed
           readers of next steps in determining whether to enroll and what
           plan to choose, and of important factors that could affect their
           coverage decisions. The information in the sample documents was
           also generally accurate when evaluated for consistency with MMA,
           implementing regulations, and agency guidance.

           Responses to the 500 calls we placed to CMS's 1-800-MEDICARE help
           line regarding the Part D benefit were frequently accurate and
           complete. However, we nonetheless received a substantial number of
           responses that were inaccurate, incomplete, or inappropriate and
           that sometimes involved an extensive wait before we could speak to
           a CSR. CSRs answered 67 percent of the calls accurately and
           completely. Of the remainder, 18 percent of the calls received
           inaccurate responses, 8 percent of the responses were
           inappropriate given the question asked, and about 3 percent
           received incomplete responses. In addition, about 5 percent of our
           calls were not answered, primarily because of disconnections.12
           Accuracy and completeness rates of CSRs' responses varied
           significantly for the five questions we asked. For example, for
           the question on whether a beneficiary qualifies for extra help,
           CSRs provided an accurate and complete response 90 percent of the
           time. However, for a question concerning which drug plan would
           cost the least for a beneficiary with certain specified
           prescription drug needs, the accuracy rate was 41 percent. CSRs
           inappropriately responded 35 percent of the time that this
           question could not be answered without personal identifying
           information-such as the beneficiary's Medicare number or date of
           birth-even though the CSRs could have answered our question using
           CMS's Web-based prescription drug plan finder tool. The amount of
           time we waited to speak with a CSR also varied, ranging from no
           wait time to over 55 minutes. For 75 percent of the calls-374 of
           the 500-we waited less than 5 minutes. For the remainder of the
           calls, 62 were answered in less than 15 minutes, 39 calls were
           answered in from 15 minutes to less than 25 minutes, and 25 led to
           a wait of 25 minutes or more.

           We found that the Part D benefit portion of the Medicare Web site
           can be difficult to use. In our evaluation of overall
           usability-the ease of finding needed information and performing
           various tasks-we found usability scores of 47 percent for seniors
           and 53 percent for younger adults, out of a possible 100 percent.
           While there is no widely accepted benchmark for usability, these
           scores indicate that using the site can be difficult. For example,
           tools such as the drug plan finder were complicated to use, and
           forms that collect information online from users were difficult to
           correct if the user made an error. In our evaluation of the
           usability of 137 detailed aspects of the Part D portion of the
           site, including features of Web design and online tools, we found
           that 70 percent of these aspects could be expected to cause users
           confusion. For example, key functions of the prescription drug
           plan finder tool, such as the "continue" and "choose a drug plan"
           buttons, were often not visible on the page without scrolling
           down. In our evaluation of the ability of seven participants to
           collectively complete 34 user tests, we found that on average,
           participants were able to proceed slightly more than halfway
           through each test. In addition, CMS evaluated whether its Web site
           complied with pertinent federal requirements regarding
           accessibility for people with disabilities in March 2006. Although
           CMS has established features to make information on its Web site
           accessible to disabled users, it found that two requirements were
           not met, making it difficult for the visually impaired to use. A
           CMS official told us that the agency made the appropriate
           corrections on April 20, 2006. Because of time constraints, we did
           not verify that these corrections were made.

           CMS relies on SHIPs to play a significant role in providing
           counseling and education on the Part D benefit to Medicare
           beneficiaries. CMS increased SHIP funding from $12 million for the
           2003 SHIP grant year13 to $31.7 million for the 2005 grant year.
           CMS kept funding relatively high for the 2006 grant year-$30
           million-to ensure that SHIPs continued to play an important role
           in educating beneficiaries about Part D. The number of
           beneficiaries served by SHIPs has also increased. During the 2004
           SHIP grant year, SHIPs served approximately 2.52 million people.
           During the first 9 months of the 2005 SHIP grant year-when CMS was
           gearing up its outreach and education on Part D-SHIPs served
           approximately 3.3 million individuals, an increase of nearly
           770,000 from the prior full grant year. CMS attributes the
           increase in demand for SHIP services-as reflected in increases in
           the number of calls, face-to-face assistance, and referrals from
           the 1-800-MEDICARE help line-to beneficiaries' need for assistance
           on Part D. The average number of calls per month referred from the
           help line to SHIPs, for example, increased from about 16,000
           referrals for May through September 2005 to an average of about
           43,000 for October and November 2005, about the time Part D
           enrollment began. According to CMS officials, this increased
           demand can be attributed to callers seeking advice on choosing a
           drug plan. Unlike CSRs on the help line, SHIP counselors can offer
           individualized guidance to callers on enrollment and plan
           selection. SHIP coordinators in the five states we contacted
           confirmed that there was a substantial increase in the demand for
           their services because of the new Part D benefit. For example, the
           California SHIP served over 120,000 people in January 2006,
           compared to about 35,000 served in all of 2005.

           Within the past 6 months, millions of Medicare beneficiaries have
           been making important decisions about their prescription drug
           coverage and have needed access to information about the new Part
           D benefit to make appropriate choices. CMS faced a tremendous
           challenge in responding to this need and, within short time
           frames, developed a range of outreach and educational materials to
           inform beneficiaries and their advisers about Part D. To
           disseminate these materials, CMS largely added information to
           existing resources, including written documents, such as Medicare
           & You; the 1-800-MEDICARE help line; the Medicare Web site; and
           support for SHIPs. However, CMS has not ensured that its
           communications to beneficiaries and their advisers are provided in
           a manner that is consistently clear, complete, accurate, and
           usable. Six months have passed since these materials were first
           made available to beneficiaries, and their limitations could
           result in confusion among those seeking to make coverage
           decisions. Although the initial enrollment period for Part D will
           end on May 15, 2006, CMS will continue to play a pivotal role in
           providing beneficiaries with information about the drug benefit
           during the year and in subsequent enrollment periods. CMS has an
           opportunity to enhance its communications on the Part D benefit.
           This would allow beneficiaries and their advisers to be better
           prepared when deciding whether to enroll in the benefit, and if
           enrolling, which drug plan to choose.

           In order to improve the Part D benefit education and outreach
           materials that CMS provides to Medicare beneficiaries, we are
           recommending that the CMS Administrator take the following four
           actions:

           o  Ensure that CMS's written documents describe the Part D benefit
           in a manner that is consistent with commonly recognized
           communications guidelines and that is responsive to the intended
           audience's needs.
           o  Determine why CSRs frequently do not search for available drug
           plans if the caller does not provide personal identifying
           information.
           o  Monitor the accuracy and completeness of CSRs' responses to
           callers' inquiries and identify tools targeted to improve their
           performance in responding to questions concerning the Part D
           benefit, such as additional scripts and training.
           o  Improve the usability of the Part D portion of the Medicare Web
           site by refining Web-based tools, providing workable site
           navigation features and links, and making Web-based forms easier
           to use and correct.

           We received written comments on a draft of this report from CMS
           (see app. III). CMS said that it did not believe our findings
           presented a complete and accurate picture of its Part D
           communications activities. CMS discussed several concerns
           regarding our findings on its written documents and the
           1-800-MEDICARE help line. However, CMS did not disagree with our
           findings regarding the Medicare Web site or the role ofSHIPs. CMS
           also said that it supports the goals of our recommendationsand is
           already taking steps to implement them, such as continually
           enhancing and refining its Web-based tools.

           CMS discussed concerns regarding the completeness and accuracy of
           our findings in terms of activities we did not examine, as well as
           those we did. CMS stated that our findings were not complete
           because our report did not examine all of the agency's efforts to
           educate Medicare beneficiaries and specifically mentioned that we
           did not examine the broad array of communication tools it has made
           available, including the development of its network of grassroots
           partners throughout the country. We recognize that CMS has taken
           advantage of many vehicles to communicate with beneficiaries and
           their advisers. However, we focused our work on the four specific
           mechanisms that we believed would have the greatest impact on
           beneficiaries-written materials, the 1-800-MEDICARE help line, the
           Medicare Web site, and the SHIPs. In addition, CMS stated that our
           report is based on information from January and February 2006, and
           that it has undertaken a number of activities since then to
           address the problems we identified. Although we appreciate CMS's
           efforts to improve its Part D communications to beneficiaries on
           an ongoing basis, we believe it is unlikely that the problems we
           identified in this report could have been corrected yet given
           their nature and scope.

           CMS raised two concerns with our examination of a sample of
           written materials. First, it criticized our use of readability
           tests to assess the clarity of the six sample documents we
           reviewed. For example, CMS said that common multisyllabic words
           would inappropriately inflate the reading level. However, we found
           that reading levels remained high after adjusting for 26
           multisyllabic words a Medicare beneficiary would encounter, such
           as Social Security Administration. CMS also pointed out that some
           experts find such assessments to be misleading. Because we
           recognize that there is some controversy surrounding the use of
           reading levels, we included two additional assessments to
           supplement this readability analysis-the assessment of design and
           organization of the sample documents based on 60 commonly
           recognized communications guidelines and an examination of the
           usability of six sample documents, involving 11 beneficiaries and
           5 advisers.

           Second, CMS expressed concern about our examination of the
           usability of the six sample documents. The participating
           beneficiaries and advisers were called on to perform 18 specified
           tasks, after reading the selected materials, including a section
           of the Medicare & You handbook. CMS suggested that the task asking
           beneficiaries and advisers to calculate their out-of-pocket drug
           costs was inappropriate because there are many other tools that
           can be used to more effectively compare costs. We do not disagree
           with CMS that there are a number of ways beneficiaries may
           complete this calculation; however, we nonetheless believe that it
           is important that beneficiaries be able to complete this task on
           the basis of reading Medicare & You, which, as CMS points out, is
           widely disseminated to beneficiaries, reaching all beneficiary
           households each year. In addition, CMS noted that it was not able
           to examine our detailed methodology regarding the clarity of
           written materials-including assessments performed by one of our
           contractors concerning readability and document design and
           organization. We plan to share this information with CMS, once our
           report has become public.

           Finally, CMS took issue with one aspect of our evaluation of the
           1-800-MEDICARE help line. Specifically, CMS said the 41 percent
           accuracy rate associated with one of the five questions we asked
           was misleading, because, according to CMS, we failed to analyze 35
           of the 100 responses. However, we disagree. This question
           addressed which drug plan would cost the least for a beneficiary
           with certain specified prescription drug needs. We analyzed these
           35 responses to this question and found the responses to be
           inappropriate. The CSRs would not provide us with the information
           we were seeking because we did not supply personal identifying
           information, such as the beneficiary's Medicare number or date of
           birth. We considered such responses inappropriate because the CSRs
           could have answered this question without personal identifying
           information by using CMS's Web-based prescription drug plan finder
           tool. Although CMS said that it has emphasized to CSRs, through
           training and broadcast messages, that it is permissible to provide
           the information we requested without requiring information that
           would personally identify a beneficiary, in these 35 instances,
           the CSR simply told us that our question could not be answered.
           CMS also said that the bulk of these inappropriate responses were
           related to our request that the CSR use only brand-name drugs.
           This is incorrect-none of these 35 responses were considered
           incorrect or inappropriate because of a request that the CSR use
           only brand-name drugs-as that was not part of our question.

           As arranged with your offices, unless you publicly announce the
           contents of this report earlier, we plan no further distribution
           of it until 30 days after its date. At that time, we will send
           copies of this report to the Secretary of Health and Human
           Services, the Administrator of the Centers for Medicare & Medicaid
           Services, and other interested parties. We will also make copies
           available to others on request. In addition, the report will be
           available at no charge on the GAO Web site at http://www.gao.gov.

           If you or your staff have any questions about this report, please
           contact me at (312) 220-7600 or [email protected]. Contact points
           for our Offices of Congressional Relations and Public Affairs may
           be found on the last page of this report. GAO staff who made major
           contributions to this report are listed in appendix IV.

           Leslie G. Aronovitz Director, Health Care

           List of Requesters

           The Honorable John D. Dingell Ranking Minority Member Committee on
           Energy and Commerce House of Representatives

           The Honorable Henry A. Waxman Ranking Minority Member Committee on
           Government Reform House of Representatives

           The Honorable Charles B. Rangel Ranking Minority Member Committee
           on Ways and Means House of Representatives

           The Honorable Sherrod Brown Ranking Minority Member Subcommittee
           on Health Committee on Energy and Commerce House of
           Representatives

           The Honorable Pete Stark Ranking Minority Member Subcommittee on
           Health Committee on Ways and Means House of Representatives

           In this report, we assessed (1) the extent to which the Centers
           for Medicare & Medicaid Services' (CMS) written documents describe
           the Medicare Part D prescription drug benefit in a clear,
           complete, and accurate manner; (2) the effectiveness of CMS's
           1-800-MEDICARE help line in providing accurate, complete, and
           prompt responses to callers inquiring about the Part D benefit;
           (3) whether CMS's Medicare Web site presents information on the
           Part D benefit in a usable manner; and (4) how CMS has used State
           Health Insurance Assistance Programs (SHIP) to respond to the
           needs of Medicare beneficiaries for information on the Part D
           benefit. To obtain information on CMS's efforts to educate
           beneficiaries about Part D, we interviewed agency officials
           responsible for Part D written documents, the 1-800-MEDICARE help
           line, the Medicare Web site, and SHIPs. Following our briefing of
           congressional staff on April 19, 2006, the briefing slides were
           updated to reflect CMS's reported correction to the Medicare Web
           site to comply with section 508 of the Rehabilitation Act of
           1973.1 We determined that the data used were sufficiently reliable
           for the purposes of this report.

           To assess the clarity, completeness, and accuracy of written
           documents, we compiled a list of all available CMS-issued Part D
           benefit publications intended to inform beneficiaries and their
           advisers and selected a sample of 6 from the 70 CMS documents
           available, as of December 7, 2005, for in-depth review, as shown
           in table 1. The sample Part D documents were chosen to represent a
           variety of publication types, such as frequently asked questions
           and fact sheets available to beneficiaries about the Part D drug
           benefit. We selected documents that targeted all beneficiaries or
           those with unique drug coverage concerns, such as dual-eligibles
           and beneficiaries with Medigap.2

           Table 1: Sample of Six Selected Documents

           Source: GAO.

           aDual-eligible beneficiaries are Medicare beneficiaries who
           receive full Medicaid benefits for services not covered by
           Medicare.

           bMedicare Advantage replaced the Medicare+ Choice managed care
           program and expanded the availability of private health plan
           options to Medicare beneficiaries.

           To evaluate clarity, we contracted with the American Institutes
           for Research (AIR)-a firm with experience in evaluating written
           material. AIR evaluated the texts of the six sample documents
           using three methodologies:

                        1. three standard readability tests;3 
                        2. 60 commonly recognized written communications
                        guidelines, including practices to aid senior
                        readers; and
                        3. user testing with 11 Medicare beneficiaries and 5
                        advisers to beneficiaries, who performed 18 specified
                        tasks related to enrollment, coverage, cost, penalty,
                        and information resources and provided feedback about
                        their experiences.

           We reviewed the sample documents for completeness to determine
           whether they contained sufficient information to allow the
           beneficiaries to identify (1) their next steps in determining
           whether to enroll and what plan to choose and (2) important
           factors, such as penalty provisions, that could affect their
           coverage decisions. To identify those important factors associated
           with the Part D benefit, we reviewed relevant laws, regulations,
           and 1-800-MEDICARE scripts prepared for customer service
           representatives (CSR) to read to callers and obtained information
           from advocacy groups. To evaluate the accuracy of information, we
           reviewed the sample materials for compliance with laws,
           regulations, and CMS guidance.

           To determine the accuracy and completeness of information provided
           regarding the Part D benefit, we placed a total of 500 calls to
           the 1-800-MEDICARE help line. We posed one of five questions about
           Part D in each call, so that each question was asked 100 times.
           Each question was pretested before we finalized its wording. We
           randomly placed calls at different times of the day and different
           days of the week from January 17 to February 7, 2006. Our calling
           times were chosen to match the daily and hourly pattern of calls
           reported by 1-800-MEDICARE in October 2005. We informed CMS
           officials that we would be placing calls; however, we did not tell
           them the questions we would ask or the specific dates and times
           that we would be placing our calls.

           To select the five questions, we considered topics identified in
           the Medicare Web site's frequently asked questions. In addition,
           we considered topics most frequently addressed by 1-800-MEDICARE
           CSRs based on help line reports. To evaluate the accuracy of CSRs'
           responses to our five questions, we used three resources: (1) the
           prescription drug plan finder tool on the Medicare Web site, (2)
           1-800-MEDICARE scripts, and (3) input obtained from CMS officials
           on the criteria we used for evaluating CSR responses. Table 2
           lists the questions we asked and the criteria we used to evaluate
           the accuracy of responses.

9The Henry J. Kaiser Family Foundation, The Medicare Drug Benefit:
Beneficiaries Perspectives Just Before Implementation,
http://kff.org/kaiserpolls/med111005nr.cfm (downloaded Apr. 26, 2006).

10The three tests were the Flesch-Kincaid Grade Level, the SMOG
(Simplified Measure of Gobbledygook) Reading Grade Level, and the Fry
Readability Estimate. These tests use such measures as sentence length and
the number of syllables in a selection of text to arrive at a reading
level, which is expressed in terms of school grade level.

                                Results in Brief

11A formulary is a list of prescription drugs covered by a health plan.

12The percentages related to the responses we received to our 500 calls
exceed 100 percent because of rounding.

13A SHIP grant year begins on April 1 of the year the funds become
available.

                                  Conclusions

                      Recommendations for Executive Action

                       Agency Comments and Our Evaluation

Appendix I: Briefing on Medicare Part D Appendix I: Briefing on Medicare
Part D

Appendix II: Methodology Appendix II: Objectives, Scope, and Methodology

                               Written Documents

129 U.S.C. S: 794d (2000).

2Medigap policies provide supplemental health coverage sold by private
insurers to help pay for Medicare cost-sharing requirements, as well as
for some services not provided by Medicare.

Document                                      Target audience              
Medicare & You, Section 6: Medicare           All beneficiaries            
Prescription Drug Coverage                    
Things to Think about When You Compare Plans  All beneficiaries            
Frequently Asked Questions about: Retiree     Beneficiaries with employer  
Prescription Drug Coverage & the New Medicare or union coverage            
Prescription Drug Coverage                    
Introduction to the Auto-Enrollment Notice    Dual-eligible beneficiariesa 
Quick Facts about Medicare's New Coverage for Beneficiaries with Medicare  
Prescription Drugs for People with a Medicare Advantageb                   
Health Plan with Prescription Drug Coverage   
Do You Have a Medigap Policy with             Beneficiaries with Medigap   
Prescription Drug Coverage?                   

3The three tests were the Flesch-Kincaid Grade Level, the SMOG (Simplified
Measure of Gobbledygook) Reading Grade Level, and the Fry Readability
Estimate. The tests use such measures as sentence length and the number of
syllables in a selection of text to arrive at a reading level, which is
expressed in terms of school grade level.

                          The 1-800-MEDICARE Help Line

Table 2: Questions and Criteria Used to Evaluate Accuracy

Question                       Criteria                                    
1. What drug plan can a        An accurate and complete response would     
beneficiary get that will      identify the prescription drug plan that    
cover all of his/her           has the lowest estimated annual cost for    
[specified] drugs at a         the drugs the beneficiary uses.             
[specified] pharmacy; have a   
mail-order option; and cost    
the least amount annually with 
[or without] a deductible?     
2. Can a beneficiary who is in An accurate and complete response would     
a nursing home and not on      indicate that a beneficiary can choose      
Medicaid sign up for a         whether to enroll in a Medicare             
prescription drug plan?        prescription drug plan.                     
3. Can a beneficiary enroll in An accurate and complete response would     
the Medicare prescription drug inform the caller that enrolling for the    
program and keep his/her       prescription drug benefit would depend on   
current Medigap policy?        whether the beneficiary's Medigap plan was  
                                  creditable-that is, whether the coverage it 
                                  provided was at least as good as Medicare's 
                                  standard prescription drug coverage-or      
                                  noncreditable. The CSR response would also  
                                  mention that the beneficiary's Medigap plan 
                                  should have sent him/her information that   
                                  outlined options.                           
4. What options does a         An accurate and complete response would     
beneficiary, who has retiree   indicate that a beneficiary has two         
health insurance with          options: (1) keep current health plan and   
prescription drug coverage     join the prescription drug plan later with  
that is not as good as the     a penalty or (2) drop current coverage and  
Medicare prescription drug     join a Medicare drug plan.                  
coverage, have as it relates   
to the Medicare benefit?       
5. How do I know if a          An accurate and complete response would     
beneficiary qualifies for      refer the beneficiary to the Social         
extra help?                    Security Administration.                    

Source: GAO.

When placing our calls, we identified ourselves as a beneficiary's
relative, but did not provide CSRs with specific identifying information,
such as a Medicare beneficiary number or date of birth. During our calls,
CSRs were not aware that their responses would be included in a research
study. We recorded the length of each call, including wait times, and the
time it took before being connected to a CSR. We evaluated the accuracy
and completeness of the responses by CSRs to the 500 calls by determining
whether key information was provided.

The results from our 500 calls are limited to those calls and are not
generalizable to the universe of calls made to the help line. The
questions we asked were limited to matters concerning the Part D benefit
and do not encompass all of the questions callers might ask.

                               Medicare Web Site

We contracted with the Nielsen Norman Group (NN/g)-a firm with expertise
in Web design-to assess the usability of the Part D information available
on the Medicare Web site. This study consisted of three separate
evaluations. First, NN/g compared the site's compliance with established
usability guidelines to determine a usability score to reflect the ease of
finding necessary information and performing various tasks. Specifically,
to determine the usability scores, NN/g evaluated various aspects of the
Web site using industry-recognized "good" Web design practices, as
indicated by the contractor, and the collective body of knowledge from
NN/g internal reports and experts, or NN/g usability guidelines.4

Second, NN/g determined the degree of difficulty associated with 137
detailed aspects of Web site design for the Part D portion of the site.
The 137 aspects fall into the following general categories:

           o  overall Web design (e.g., home page, navigation, search
           function, graphics, and overall organization);
           o  tools (e.g., plan finder);
           o  writing style (e.g., content, tone, legibility, and
           readability);
           o  accessibility (e.g., availability of a version of the Web site
           for the blind); and
           o  languages (e.g., availability of languages other than English).

           NN/g determined the difficulty level in using each of the 137
           aspects. NN/g noted aspects that had good design and would not be
           expected to cause confusion. For those aspects with a design that
           would be expected to cause confusion, NN/g ranked the associated
           difficulty level as high, medium, or low.5

           Third, NN/g performed a qualitative evaluation on January 20 and
           23, 2006, to test the ability of five Medicare beneficiaries and
           two beneficiary advisers to perform specified tasks related to
           Medicare beneficiaries using the Web site and to obtain feedback
           about participants' experiences. While the results are not
           statistically valid, these users provided important insights into
           the usability of the Medicare Web site. Participants were asked to
           "think out loud" as they worked through their tasks, while an NN/g
           facilitator observed their behavior and took notes. NN/g gave each
           task a score. At the end of their sessions, NN/g asked
           participants for input regarding their confidence in the answers
           they obtained from the Web site, and their overall satisfaction
           and frustration levels associated with using the site.

           Finally, we obtained the results of CMS's March 2006 review of its
           Web site's compliance with section 508 of the Rehabilitation Act
           of 1973, as amended. This law requires federal agencies to make
           the information on their Web sites accessible to people with
           disabilities. We also discussed the results of this review with
           agency officials and followed up with them to determine the status
           of CMS's corrective actions.

           To determine the role of SHIPs in helping Medicare beneficiaries
           understand Part D, we interviewed CMS officials who monitor SHIPs'
           activities. We also reviewed information that we obtained from CMS
           officials and other sources on the program, its funding, changes
           made in response to the introduction of Part D, and the impact of
           Part D on the demand for SHIP services. In addition, we
           interviewed SHIP officials in California, Florida, New York,
           Texas, and Pennsylvania-the five states with the largest Medicare
           populations-to obtain information on the experience of their SHIPs
           with Part D.

           We conducted our work from November 2005 through May 2006 in
           accordance with generally accepted government auditing standards.

           Leslie G. Aronovitz (312) 220-7600 or [email protected]

           In addition to the contact named above, Susan T. Anthony and
           Geraldine Redican-Bigott, Assistant Directors; Ramsey L. Asaly;
           Enchelle Bolden; Laura Brogan; Shaunessye D. Curry; Chir-Jen
           Huang; M. Peter Juang; Ba Lin; Michaela M. Monaghan; Roseanne
           Price; Pauline Seretakis; Margaret J. Weber; and Craig H. Winslow
           made contributions to this report.

           The Government Accountability Office, the audit, evaluation and
           investigative arm of Congress, exists to support Congress in
           meeting its constitutional responsibilities and to help improve
           the performance and accountability of the federal government for
           the American people. GAO examines the use of public funds;
           evaluates federal programs and policies; and provides analyses,
           recommendations, and other assistance to help Congress make
           informed oversight, policy, and funding decisions. GAO's
           commitment to good government is reflected in its core values of
           accountability, integrity, and reliability.

           The fastest and easiest way to obtain copies of GAO documents at
           no cost is through GAO's Web site ( www.gao.gov ). Each weekday,
           GAO posts newly released reports, testimony, and correspondence on
           its Web site. To have GAO e-mail you a list of newly posted
           products every afternoon, go to www.gao.gov and select "Subscribe
           to Updates."

           The first copy of each printed report is free. Additional copies
           are $2 each. A check or money order should be made out to the
           Superintendent of Documents. GAO also accepts VISA and Mastercard.
           Orders for 100 or more copies mailed to a single address are
           discounted 25 percent. Orders should be sent to:

           U.S. Government Accountability Office 441 G Street NW, Room LM
           Washington, D.C. 20548

           To order by Phone: Voice: (202) 512-6000 TDD: (202) 512-2537 Fax:
           (202) 512-6061

           Contact:

           Web site: www.gao.gov/fraudnet/fraudnet.htm E-mail:
           [email protected] Automated answering system: (800) 424-5454 or
           (202) 512-7470

           Gloria Jarmon, Managing Director, [email protected] (202) 512-4400
           U.S. Government Accountability Office, 441 G Street NW, Room 7125
           Washington, D.C. 20548

           Paul Anderson, Managing Director, [email protected] (202)
           512-4800 U.S. Government Accountability Office, 441 G Street NW,
           Room 7149 Washington, D.C. 20548

4These guidelines are presented in an NN/g report called Web Usability for
Senior Citizens: 46 Design Guidelines Based on Usability Studies with
People Age 65 and Older. For this study, NN/g conducted usability tests of
17 Web sites with 44 seniors. Based on the test findings, NN/g developed
46 design guidelines that would make Web sites more attractive to seniors.

5In addition, NN/g indicated cases where an aspect was not functioning
correctly from a Web site development standpoint by giving it a "bug"
mark.

                   State Health Insurance Assistance Programs

Appendix III: Comments from the Centers for Medicare & Medicaid Services
Appendix III: Comments from the Centers for Medicare & Medicaid Services

Appendix IV: St Appendix IV: GAO Contact and Staff Acknowledgments

                                  GAO Contact

                                Acknowledgments

(290509)

GAO's Mission

Obtaining Copies of GAO Reports and Testimony

Order by Mail or Phone

To Report Fraud, Waste, and Abuse in Federal Programs

Congressional Relations

Public Affairs

www.gao.gov/cgi-bin/getrpt? GAO-06-654 .

To view the full product, including the scope

and methodology, click on the link above.

For more information, contact Leslie G. Aronovitz at [email protected] or
(312) 220-7600.

Highlights of GAO-06-654 , a report to congressional requesters

May 2006

MEDICARE

Communications to Beneficiaries on the Prescription Drug Benefit Could Be
Improved

On January 1, 2006, Medicare began providing coverage for outpatient
prescription drugs through its new Part D benefit. Beneficiaries who
enroll in Part D may choose a drug plan from those offered by private plan
sponsors under contract to the Centers for Medicare & Medicaid Services
(CMS), which administers the Part D benefit. Beneficiaries have until May
15, 2006, to enroll in the Part D benefit and select a plan without the
risk of penalties.

GAO was asked to review the quality of CMS's communications on the Part D
benefit. GAO examined 70 CMS publications to select 6 documents for review
and contracted with the American Institutes for Research to evaluate the
clarity of these texts; made 500 calls to the 1-800-MEDICARE help line;
and contracted with the Nielsen Norman Group to evaluate the usability of
the Medicare Web site.

What GAO Recommends

GAO is recommending that the CMS Administrator enhance the quality of its
communications by taking actions to improve written materials, its
1-800-MEDICARE help line, and the Medicare Web site. CMS said that GAO's
findings did not present a complete and accurate picture of its
activities. However, CMS said that it supports the goals of GAO's
recommendations and is already taking steps to implement them.

The information given in the six sample documents that GAO reviewed
describing the Part D benefit was largely complete and accurate, although
this information lacked clarity. The documents were unclear in two ways.
First, although about 40 percent of seniors read at or below the
fifth-grade level, the reading levels of these documents ranged from
seventh grade to postcollege. Second, on average, the six documents did
not comply with about half of 60 common guidelines for good communication.
For example, the documents used too much technical jargon and often did
not define difficult terms, such as formulary. Moreover, 16 beneficiaries
and advisers that GAO tested reported frustration with the documents' lack
of clarity and had difficulty completing the tasks assigned to them.
Although the documents lacked clarity, they informed readers of enrollment
steps and factors affecting coverage decisions and were consistent with
laws, regulations, and agency guidance.

Customer service representatives (CSR) responded to the 500 calls GAO
placed to CMS's 1-800-MEDICARE help line accurately and completely about
two-thirds of the time. Of the remainder, 18 percent of the calls received
inaccurate responses, 8 percent of the responses were inappropriate given
the question asked, and about 3 percent received incomplete responses. In
addition, about 5 percent of GAO's calls were not answered, primarily
because of disconnections. Accuracy and completeness rates of CSRs'
responses varied significantly across the five questions GAO asked. For
example, while CSRs provided accurate and complete responses to calls
about beneficiaries' eligibility for extra help 90 percent of the time,
the accuracy rate for calls concerning the drug plan that would cost the
least for a specified beneficiary was 41 percent. For this question, the
CSRs responded inappropriately for 35 percent of the calls by explaining
that they could not identify the least costly plan without the
beneficiary's personal information-even though CSRs had the information
needed to answer the question. The time GAO callers waited to speak with
CSRs also varied, ranging from no wait time to over 55 minutes. For 75
percent of the calls-374 of the 500-the wait was less than 5 minutes.

The Part D benefit portion of the Medicare Web site can be difficult to
use. GAO's test of the site's overall usability-the ease of finding needed
information and performing various tasks-resulted in scores of 47 percent
for seniors and 53 percent for younger adults, out of a possible 100
percent. While there is no widely accepted benchmark for usability, these
scores indicate that using the site can be difficult. For example, the
prescription drug plan finder was complicated to use and some of its key
functions, such as "continue" and "choose a drug plan," were often not
visible on the page without scrolling down.
*** End of document. ***