VA Health Care: VA Needs to Improve Accuracy of Reported Wait
Times for Blind Rehabilitation Services (22-JUL-04, GAO-04-949).
The Department of Veterans Affairs (VA) provides rehabilitation
services to legally blind veterans. These services are intended
to help them acquire the skills necessary to become more
independent. Almost all of VA's rehabilitation services for
legally blind veterans are provided at Blind Rehabilitation
Centers (BRC), an inpatient program. VA reported that the average
length of time a veteran waited to be admitted to a BRC increased
from 168 to 210 days from fiscal years 1999 through 2003. GAO was
asked to examine the accuracy of veterans' wait times for
admission to BRCs. GAO's objective was to determine whether the
average wait times for veterans seeking admission to BRCs
reported by VA were accurate. GAO reviewed VA policies and
procedures for determining the average length of time veterans
wait to be admitted to a BRC. GAO also visited 5 of VA's 10 BRCs
to evaluate the reliability of the data used to calculate wait
times.
-------------------------Indexing Terms-------------------------
REPORTNUM: GAO-04-949
ACCNO: A10972
TITLE: VA Health Care: VA Needs to Improve Accuracy of Reported
Wait Times for Blind Rehabilitation Services
DATE: 07/22/2004
SUBJECT: Data integrity
Performance measures
Program management
Rehabilitation programs
Statistical data
Veterans
Veterans benefits
Data collection
Health care facilities
Policies and procedures
******************************************************************
** This file contains an ASCII representation of the text of a **
** GAO Product. **
** **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced. Tables are included, but **
** may not resemble those in the printed version. **
** **
** Please see the PDF (Portable Document Format) file, when **
** available, for a complete electronic file of the printed **
** document's contents. **
** **
******************************************************************
GAO-04-949
United States Government Accountability Office
GAO
Report to Congressional Requesters
July 2004
VA HEALTH CARE
VA Needs to Improve Accuracy of Reported Wait Times for Blind Rehabilitation
Services
GAO-04-949
Highlights of GAO-04-949, a report to congressional requesters.
The Department of Veterans Affairs (VA) provides rehabilitation services
to legally blind veterans. These services are intended to help them
acquire the skills necessary to become more independent. Almost all of
VA's rehabilitation services for legally blind veterans are provided at
Blind Rehabilitation Centers (BRC), an inpatient program. VA reported that
the average length of time a veteran waited to be admitted to a BRC
increased from 168 to 210 days from fiscal years 1999 through 2003. GAO
was asked to examine the accuracy of veterans' wait times for admission to
BRCs. GAO's objective was to determine whether the average wait times for
veterans seeking admission to BRCs reported by VA were accurate. GAO
reviewed VA policies and procedures for determining the average length of
time veterans wait to be admitted to a BRC. GAO also visited 5 of VA's 10
BRCs to evaluate the reliability of the data used to calculate wait times.
GAO recommends that the Secretary of Veterans Affairs direct the Under
Secretary for Health to instruct (1) the program office to develop more
specific instructions for calculating wait times and (2) the BRCs to
adhere to these instructions by developing procedures to compile complete
and accurate information. VA concurred with GAO's recommendations.
July 2004
VA HEALTH CARE
VA Needs to Improve Accuracy of Reported Wait Times for Blind Rehabilitation
Services
GAO found that the average length of time VA reported that veterans wait
for admission to BRCs was inaccurate. Some data used to calculate wait
times were incomplete or incorrect. For example, at one BRC GAO found that
one or more of the data elements used to calculate the wait times-the date
the BRC received the application, the earliest admission date offered to
the veteran, and the date the veteran was admitted to the BRC-were missing
from 31 percent of the records and incorrect in 13 percent of the records.
GAO also found missing or inaccurate data at two other BRCs. In addition,
GAO found that BRCs used different procedures for their calculations,
which also contributed to the inaccurate average wait times. For example,
two BRCs correctly ended the wait times calculations on the earliest
admission date offered to the veteran, while the other three BRCs ended
the wait times calculations on the date the veteran was admitted to the
BRC. To enable VA to accurately assess wait times, it is essential for VA
to develop more comprehensive instructions to calculate average wait times
and for BRCs to adhere to them.
www.gao.gov/cgi-bin/getrpt?GAO-04-949.
To view the full product, including the scope and methodology, click on
the link above. For more information, contact Cynthia A. Bascetta at (202)
512-7101.
Contents
Letter 1
Results in Brief 2
Background 2
VA's Reported Wait Times for Accessing BRCs Were Inaccurate 4
Conclusions 5
Recommendations for Executive Action 6
Agency Comments 6
Appendix I Scope and Methodology
Appendix II Comments from the Department of Veterans Affairs
Table
Table 1: Location of VA's Blind Rehabilitation Centers and the Number of
Authorized Beds
Abbreviations
BRC Blind Rehabilitation Center
VA Department of Veterans Affairs
VIST Visual Impairment Service Team
This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed in
its entirety without further permission from GAO. However, because this
work may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this material
separately.
United States Government Accountability Office Washington, DC 20548
July 22, 2004
The Honorable Bob Graham
Ranking Minority Member
Committee on Veterans' Affairs
United States Senate
The Honorable Rob Simmons
Chairman
Subcommittee on Health
Committee on Veterans' Affairs
House of Representatives
The Department of Veterans Affairs (VA) provides rehabilitation training
services for legally blind1 veterans that are intended to enable them to
acquire the skills necessary to develop personal independence, such as
using magnification devices for reading. Almost all of VA's rehabilitation
services for legally blind veterans are provided at Blind Rehabilitation
Centers (BRC), a residential inpatient program. VA reported that the
average length of time a veteran waited to be admitted to a BRC increased
from 168 to 210 days from fiscal years 1999 through 2003. In 2003, VA
reported that approximately 2,100 legally blind veterans received care in
BRCs, with average wait times ranging from 77 to 343 days.
Congress has been concerned for more than 10 years about the length of
time veterans wait to receive training at BRCs. In 1993 and again in 1995,
Congress noted that veterans were unable to access services at BRCs in a
timely manner and encouraged VA to provide more services. You
expressed concern about the accuracy of the length of time VA reports
that veterans wait for admission to BRCs. Our objective was to determine
whether the average wait times for veterans seeking admission to BRCs
reported by VA were accurate.
1VA defines legal blindness as when the patient's best-corrected central
visual acuity with ordinary eyeglasses or contacts is 20/200 or less in
the better eye (as measured by the Snellen Visual Acuity Chart) or when
the field of useful vision is 20 degrees or less in the better eye. People
who are legally blind can only at best read the big "E" on the eye chart
or see as if they are looking through a paper towel tube, according to VA.
Results in Brief
Background
To assess the accuracy of VA's reported wait times, we visited 5 of VA's
10 BRCs that we selected based on differences in geographical location and
the number of beds available at the BRC. We met with BRC officials and
other VA employees who manage the health care provided to legally blind
veterans. We reviewed the procedures they followed to calculate veterans'
wait times and analyzed documents2 to evaluate the reliability of the data
used to calculate wait times at 3 of the BRCs. We were unable to assess
the data at the other 2 BRCs because application packages were not
available. After veterans were discharged, 1 BRC required that the
application packages be destroyed and the other BRC returned the packages
to the medical facility that referred the veteran. We also met with
officials from VA's Blind Rehabilitation Service Program Office (program
office), which has responsibility for blindness-related services, to
review its policies and procedures regarding wait times calculations. Our
review was conducted from September 2003 through July 2004 in accordance
with generally accepted government auditing standards. (See app. I for
additional information on our scope and methodology.)
The average length of time VA reported that veterans wait for admission to
BRCs was inaccurate. Some data used to calculate wait times-the date the
BRC received the application, the earliest admission date offered to the
veteran, and the date of admission-were incomplete or incorrect. In
addition, we found that BRCs used different procedures for their
calculations, which also contributed to the inaccurate average wait times.
We are making two recommendations to improve the accuracy of VA's reported
BRC wait times. VA agreed with our findings and recommendations and
indicated that it will provide a detailed action plan.
In 2003, VA estimated that about 157,000 veterans were legally blind,
about 44,000 of them were enrolled in its health care system, and about
2,100 received rehabilitation training in BRCs. The blind rehabilitation
program is designed to improve the quality of life for veterans who are
legally blind. VA's Visual Impairment Service Team (VIST) coordinators are
responsible for managing legally blind veterans' access to rehabilitation
services, including reaching a decision with the veteran about appropriate
training
2These documents included veterans' applications for BRC admission,
printouts from the BRC's electronic database used to manage the wait list,
and veterans' computerized medical records.
and other services. VIST coordinators prepare the veteran's application to
a BRC and ensure that the veteran has the medical examinations required
for admission, such as hearing examinations and low vision testing. VA's
10 BRCs provide training to legally blind veterans on an inpatient
residential basis. Table 1 lists the location of the BRCs and the number
of beds authorized for blind rehabilitation services.
Table 1: Location of VA's Blind Rehabilitation Centers and the Number of
Authorized Beds
Location Authorized bedsa
American Lake, Washingtonb
Augusta, Georgiab
Birmingham, Alabama
Hines, Illinoisb
Palo Alto, California
San Juan, Puerto Rico
Tucson, Arizonab
Waco, Texas
West Haven, Connecticut
b
West Palm Beach, Florida
Total 241
Source: VA.
aAs of May 2004.
bBRCs evaluated by GAO.
The training BRCs offer is comprehensive and individualized with the goal
of helping legally blind veterans meet their personal goals and achieve a
realistic level of independence. They offer a basic rehabilitation program
and computer training. The basic program trains veterans in such areas as
orientation and mobility (e.g., moving around the home or using a cane to
travel through different environments) or visual skills (e.g., using
closed circuit television or magnification devices to read or write
checks). Computer training teaches veterans how to operate a computer;
search the Internet; and send, receive, and read e-mail messages.
Each month, BRCs are required to submit a report to the program office
that includes their calculation of the average number of days veterans
waited to be admitted to a BRC for training during the past 6 months. VA
defines an applicant's wait time as the number of days that elapse from
the
VA's Reported Wait Times for Accessing BRCs Were Inaccurate
date the BRC receives the application to the earliest admission date
offered to the veteran.
Wait times reported by the program office were not accurate for two
reasons. First, we found that some data VA used to calculate wait times
were missing and some that it used were incorrect. Second, VA did not use
consistent procedures to calculate wait times.
Data Used to Report Wait Times Were Missing or Inaccurate
We determined that data elements BRCs used for calculating wait times for
both basic and computer training-the date the BRC received the
application, the earliest admission date offered to the veteran, and the
date the veteran was admitted to the BRC-were missing or inaccurate. For
example, during our review of data at one BRC, we found that one or more
of the data elements used to calculate the wait times were missing from 31
percent of the records and incorrect in 13 percent of the records. We also
noted missing or incorrect data elements during our review of records at
two other BRCs. Specifically, at one BRC we reviewed 30 records and found
missing or inaccurate data in 24 of them. At the other BRC, we found
inaccurate data in 8 of 16 records. At the remaining two BRCs, we could
not validate the completeness or accuracy of records used to calculate
wait times because application packages were not available. After veterans
completed their training, one BRC required that the application packages
be destroyed and the other BRC returned the packages to the medical
facility that referred the veteran. In addition, we found weaknesses in
the way the BRCs ensured complete and accurate data. For example, none of
the BRCs stated they validated data entry. Also, none of them checked
their databases for completeness (Is the data element filled in or has it
been left blank?) and reasonableness (Is the date the BRC received the
application earlier than the first admission date offered to the
veteran?). Further, one of them did not check for format accuracy (If the
data element is a date, is it formatted as a date, such as 5/24/2004?).
Procedures Used to We found that the five BRCs we visited followed
different procedures for Calculate Wait Times Were determining the average
time a veteran must wait for admission to a BRC. Inconsistent According to
VA instructions, the BRC is to average the number of days
that elapsed from the date the application was received to the earliest
admission date offered to the veteran for veterans admitted during the
past 6 months. However, the instructions do not stipulate whether the
application package must be complete before putting the veteran on the
wait list or the length of time BRCs allow VIST coordinators to submit the
missing information. This lack of specificity resulted in BRCs using
different procedures to calculate their wait times.
Although all five BRCs began their calculations when they received the
veteran's application-even if it was not complete-they allowed VIST
coordinators different lengths of time to provide the missing information.
According to the BRC officials at these locations, the BRCs accepted
incomplete packages as long as the basic information needed to process the
application was present, including documents to prove eligibility, a
recent physical examination, and an eye examination proving legal
blindness. For example, one BRC official stated that the results of
hearing examinations were not essential to approve applications but were
needed before veterans were admitted for training. VIST coordinators who
submitted incomplete applications could get their veterans on the wait
list earlier than VIST coordinators who waited to submit complete
applications. BRCs allowed different lengths of time for VIST coordinators
to provide missing information before removing veterans from the wait
list. For example, one BRC removed veterans with incomplete applications
from the wait list after 30 days, while another waited at least 6 months.
Not all BRCs used the same procedures to stop the wait times calculation.
The program office instructs BRCs to stop the wait times calculation on
the earliest admission date offered to the veteran. Two BRCs correctly
ended the wait times calculation on that date, while the other three ended
the wait times on the date the veteran was actually admitted, even if the
veteran had refused earlier admission dates. For example, one veteran was
offered an admission date of October 27th. The veteran declined that date
and two subsequently offered dates and was finally admitted on June 7th of
the following year. Because the veteran's first offer date was October
27th, according to the program office instructions the wait times should
have stopped on that date. Instead, the BRC included the additional time
between October 27th and June 7th in its wait times calculation. In this
example, the BRC calculation overstated the veteran's wait times by over 7
months.
Conclusions While VA reports BRC wait times for veterans, the information
does not accurately reflect the time veterans wait for admission. Data
used for calculations are often incomplete or inaccurate. In addition,
BRCs followed different procedures for calculating wait times because VA's
instructions do not specify whether application packages must be complete
before putting the veteran on the wait list or the length of time
Recommendations for Executive Action
Agency Comments
BRCs allow VIST coordinators to submit the missing information. For VA to
accurately assess wait times, it is essential for VA to develop a
consistent process for determining average wait times and for BRCs to
adhere to it.
To improve VA's ability to effectively manage care for legally blind
veterans, we recommend that the Secretary of Veterans Affairs direct the
Under Secretary for Health to instruct (1) the program office to develop
more specific instructions for calculating wait times and (2) the BRCs to
adhere to these instructions by developing procedures to compile complete
and accurate information on the length of time veterans wait for admission
to BRCs.
In commenting on the draft of this report, VA agreed with our findings and
conclusions. VA stated that the report accurately conveyed the variability
and complexity of reporting wait times for admission to BRCs. VA concurred
with our recommendations and indicated that it will provide a detailed
action plan. VA's written comments are reprinted in appendix II.
We are sending copies of this report to the Secretary of Veterans Affairs
and other interested parties. We also will make copies available to others
upon request. In addition, the report will be available at no charge at
the GAO Web site at http://www.gao.gov. If you or your staff have any
questions about this report, please call me at (202) 512-7101 or Michael
T. Blair, Jr. at (404) 679-1944. Cherie Starck, Cynthia Forbes, and Janet
Overton were key contributors to this report.
Cynthia A. Bascetta Director, Health Care-Veterans' Health and Benefits
Issues
Appendix I: Scope and Methodology
To determine whether VA accurately reports average wait times for veterans
seeking admission to Blind Rehabilitation Centers (BRC), we examined the
policies and procedures VA used to calculate the length of time veterans
wait for admission. We met with and obtained documentation from officials
at the Blind Rehabilitation Service Program Office and from the five BRCs
we visited. We selected the five BRCs located in Tucson, Arizona; West
Palm Beach, Florida; Augusta, Georgia; Hines, Illinois; and American Lake,
Washington, based on differences in geographical location and the number
of beds available at the BRC. We met with BRC officials and veterans' case
managers. We reviewed documentation on the procedures BRC staff followed
to calculate veterans' wait times to assess the quality of the data the
five BRCs maintained. We also reviewed veterans' paper application
packages for BRC admission, printouts from the BRC's electronic database
used to manage the wait list, and veteran's computerized medical records.
At the first BRC we visited, we compared the date the application was
received at the BRC in the electronic and paper files for a judgmentally
selected sample of 16 applicants. We identified data elements that were
inaccurate in either the paper or electronic records in 8 of the 16
records. At the second BRC, we chose a random sample of 100 veterans,
stratified by fiscal year, from the 578 that were admitted in fiscal years
2000 through 2003. A BRC official then provided printed copies of the data
from the electronic file used to calculate the wait times and the folders
containing the paper application packages. We reviewed the selected files
to compare the data elements used for calculating wait times-the date the
BRC received the application, the earliest admission date offered to the
veteran, and the date the veteran was admitted to the BRC-from the two
sources. We determined that one or more of these data elements were
missing from 31 percent of the records with a margin of error of plus or
minus 8 percent. Additionally, we found that one or more of these data
elements were inaccurate in 13 percent of the records with a margin of
error of plus or minus 6 percent.1
For the remaining three BRCs, we attempted to verify that the problems we
found at the first two BRCs were also found at the other facilities. At
the third BRC we randomly sampled 10 admissions each from fiscal years
1The categories of records with missing data elements and those with
inaccurate data elements are not mutually exclusive.
Appendix I: Scope and Methodology
2001 through 2003.2 A BRC official provided printed copies of the data
from the electronic file used to calculate the wait times and the folders
containing the paper application packages. We reviewed the two sources to
compare the data elements used for calculating wait times. To determine if
similar data problems also existed at this BRC, we compared the percentage
of inaccurate or missing data elements to a tolerable level of 10
percent-that is, not less than 10 percent of the records at this BRC had
one or more inaccurate or missing data elements. We found that 24 of the
30 records had inaccurate or missing data elements. We confirmed that the
percent of inaccurate and incomplete data elements was not less than 10
percent by performing a one-sided significance test at the 95-percent
confidence level. We were unable to assess the data at the other two BRCs
because application packages were not available. After veterans completed
their training, one BRC required that the application packages be
destroyed and the other BRC returned the packages to the medical facility
that referred the veteran. Our results from these five BRCs cannot be
generalized to other facilities.
2We had planned to sample from fiscal years 2000 through 2003 but records
from fiscal year 2000 had been destroyed.
Appendix II: Comments from the Department of Veterans Affairs
GAO's Mission The Government Accountability Office, the audit, evaluation
and investigative arm of Congress, exists to support Congress in meeting
its constitutional responsibilities and to help improve the performance
and accountability of the federal government for the American people. GAO
examines the use of public funds; evaluates federal programs and policies;
and provides analyses, recommendations, and other assistance to help
Congress make informed oversight, policy, and funding decisions. GAO's
commitment to good government is reflected in its core values of
accountability, integrity, and reliability.
Obtaining Copies of The fastest and easiest way to obtain copies of GAO
documents at no cost
is through GAO's Web site (www.gao.gov). Each weekday, GAO postsGAO
Reports and newly released reports, testimony, and correspondence on its
Web site. To Testimony have GAO e-mail you a list of newly posted products
every afternoon, go to
www.gao.gov and select "Subscribe to Updates."
Order by Mail or Phone The first copy of each printed report is free.
Additional copies are $2 each. A check or money order should be made out
to the Superintendent of Documents. GAO also accepts VISA and Mastercard.
Orders for 100 or more copies mailed to a single address are discounted 25
percent. Orders should be sent to:
U.S. Government Accountability Office 441 G Street NW, Room LM Washington,
D.C. 20548
To order by Phone: Voice: (202) 512-6000 TDD: (202) 512-2537 Fax: (202)
512-6061
To Report Fraud, Contact:
Waste, and Abuse in Web site: www.gao.gov/fraudnet/fraudnet.htm
E-mail: fraudnet@gao.govFederal Programs Automated answering system: (800)
424-5454 or (202) 512-7470
Congressional Gloria Jarmon, Managing Director, JarmonG@gao.gov (202)
512-4400 U.S. Government Accountability Office, 441 G Street NW, Room 7125
Relations Washington, D.C. 20548
Public Affairs Jeff Nelligan, Managing Director, NelliganJ@gao.gov (202)
512-4800 U.S. Government Accountability Office, 441 G Street NW, Room 7149
Washington, D.C. 20548
Presorted StandardEURPostage & Fees PaidEURGAOEURPermit No. GI00EUR
United StatesEURGovernment Accountability OfficeEURWashington, D.C.
20548-0001EUR
Official BusinessEURPenalty for Private Use $300EUR
Address Service RequestedEUR
*** End of document. ***