Veterans' Benefits: Improvements Needed in the Reporting and Use
of Data on the Accuracy of Disability Claims Decisions
(30-SEP-03, GAO-03-1045).
The Veterans Benefits Administration (VBA) has a large inventory
of claims for benefits under its compensation and pension
programs. The Secretary of the Department of Veterans Affairs has
pledged to substantially reduce this inventory in order to
improve timeliness. In response, VBA emphasized producing more
claims decisions per year. GAO was asked to ascertain how
accuracy has changed since VBA increased its emphasis on
production and to report on the agency's efforts to ensure the
accuracy of its decisions.
-------------------------Indexing Terms-------------------------
REPORTNUM: GAO-03-1045
ACCNO: A08610
TITLE: Veterans' Benefits: Improvements Needed in the Reporting
and Use of Data on the Accuracy of Disability Claims Decisions
DATE: 09/30/2003
SUBJECT: Pension claims
Standards evaluation
Veterans pensions
Veterans benefits
Decision making
Claims processing
Compensation claims
Accountability
Performance measures
VA Systematic Technical Accuracy Review
System
******************************************************************
** This file contains an ASCII representation of the text of a **
** GAO Product. **
** **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced. Tables are included, but **
** may not resemble those in the printed version. **
** **
** Please see the PDF (Portable Document Format) file, when **
** available, for a complete electronic file of the printed **
** document's contents. **
** **
******************************************************************
GAO-03-1045
United States General Accounting Office
GAO
Report to Congressional Requesters
September 2003
VETERANS' BENEFITS
Improvements Needed in the Reporting and Use of Data on the Accuracy of
Disability Claims Decisions
GAO-03-1045
Highlights of GAO-03-1045, a report to congressional requesters
The Veterans Benefits Administration (VBA) has a large inventory of claims
for benefits under its compensation and pension programs. The Secretary of
the Department of Veterans Affairs has pledged to substantially reduce
this inventory in order to improve timeliness. In response, VBA emphasized
producing more claims decisions per year. GAO was asked to ascertain how
accuracy has changed since VBA increased its emphasis on production and to
report on the agency's efforts to ensure the accuracy of its decisions.
GAO recommends that VBA
o report the accuracy of VBA disability compensation and pension claims
decisions to the Congress and other stakeholders in a manner that allows
for valid comparisons of accuracy across fiscal years and
o better hold regional offices accountable for the accuracy of their
claims decisions, by increasing the use of its regional office accuracy
data.
VBA concurred with GAO's recommendations and provided additional
information about its efforts to improve accuracy.
www.gao.gov/cgi-bin/getrpt?GAO-03-1045.
To view the full report, including the scope and methodology, click on the
link above. For more information, contact Cynthia A. Bascetta at (202)
512-7215 or bascettac@gao.gov.
September 2003
VETERANS' BENEFITS
Improvements Needed in the Reporting and Use of Data on the Accuracy of
Disability Claims Decisions
From fiscal years 2001 to 2002, VBA's accuracy of decision-making in the
disability compensation and pension benefit programs declined from 89
percent to 81 percent. The agency had reported a slight improvement in
accuracy between fiscal years 2001 and 2002-from 78 percent to 80 percent.
However, we found that these two annual figures were not comparable
because the agency had substantially changed the way it measured accuracy
for fiscal year 2002. Although VBA acknowledged a change in its accuracy
measure in its annual report to the Congress, the agency did not revise
its 2001 figure to allow for an appropriate comparison with 2002. VBA
officials GAO spoke with suggested several factors that may have
contributed to the decline in accuracy. We were not able to quantify the
relative contribution of these factors. These factors included VBA's
emphasis on increasing claims decisions, the specific processing
requirements of the Veterans Claims Assistance Act (VCAA) of 2000, and the
relative inexperience of VBA's claims processing staff.
To help ensure accountability for accuracy, VBA set accuracy standards for
its regional offices. Although VBA has regional office-level accuracy
data, it has not made full use of this information to encourage better
performance from regional offices with low accuracy scores. For example,
in fiscal year 2002, VBA did not require offices with poor accuracy to
prepare improvement plans and gave performance awards to two offices that
clearly failed to meet VBA's accuracy goal.
VBA Compensation and Pension Rating Products Decided and Accuracy Rates,
Fiscal Years 2001-2002
Number completed Percentage
(in thousands)
1,000 89 100
800 80
600 60
400 40
200 20
0 0
B
Production Accuracy
FY 2001 FY 2002
Source: GAO analysis of Veterans Benefits Administration data.
Note: Accuracy rates in the above chart are estimates based on samples of
rating products completed in fiscal years 2001 and 2002 and have margins
of error of about plus or minus 1 percentage point at the 95-percent level
of confidence. We determined these accuracy rates by comparing
substantially similar questions for the 2 fiscal years that focused on
whether the decision to grant or deny benefits was correct. The fiscal
year 2002 accuracy score includes a specific question on proper VCAA
pre-decision notification that VBA did not include in its fiscal year 2001
quality reviews.
Contents
Letter
Results in Brief
Background
Accuracy Declined in Fiscal Year 2002
VBA Could Have Done More to Hold Regional Offices Accountable
for Accuracy Conclusions Recommendations Agency Comments and Our Response
1
2 3 6
8 10 11 11
Appendix I Comments from the Veterans Benefits Administration
Figure
Figure 1: Fiscal Year 2001 STAR Rating Checklist-Criteria for a Correct
VBA Decision
Abbreviations
BVA Board of Veterans' Appeals
SIPA Systematic Individual Performance Assessment
STAR Systematic Technical Accuracy Review
VA Department of Veterans Affairs
VBA Veterans Benefits Administration
VCAA Veterans Claims Assistance Act of 2000
VSR veterans service representative
This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed in
its entirety without further permission from GAO. However, because this
work may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to
reproduce this material separately.
United States General Accounting Office Washington, DC 20548
September 30, 2003
The Honorable Arlen Specter
Chairman
The Honorable Bob Graham
Ranking Minority Member
Committee on Veterans' Affairs
United States Senate
The Honorable John D. Rockefeller IV
United States Senate
At the beginning of fiscal year 2002, over 400,000 claims for disability
compensation and pension benefits were waiting for Veterans Benefits
Administration (VBA) decisions. On average, veterans were waiting about
6 months and many were waiting more than a year. VBA has been making
a concerted effort to reduce the inventory of claims and complete
decisions in an average of 100 days by the end of fiscal year 2003. In
order
to improve the timeliness of its disability claims decisions, VBA
established a goal of completing about 839,000 decisions in fiscal year
2002
and reducing its inventory of pending claims to about 316,000 by the end
of that year. During fiscal year 2002, VBA increased its claims decisions
by
two-thirds from about 481,000 to about 797,000 and reduced its inventory
of pending claims by about one-fifth from about 421,000 to about 346,000.
For fiscal year 2003, VBA's goals are to produce about 806,000 decisions
and further reduce inventory to 250,000 claims.
In light of VBA's emphasis on completing more claims decisions more
quickly, you asked us to assist the Committee in its oversight of VBA by
examining the agency's efforts to ensure the accuracy of decision-making
in its disability compensation and pension benefits programs. In response,
we assessed (1) how accuracy may have changed since VBA increased its
emphasis on production and (2) how well VBA ensures the accuracy of its
decisions.
We reviewed VBA's efforts to measure and report accuracy for fiscal years
2001 and 2002.1 To determine how accuracy had changed over that time,
1This report focuses on the accuracy of rating decisions. Rating decisions
are primarily decisions on original claims for compensation and pension
benefits and reopened claims. For example, veterans may file reopened
claims if they believe that their service-connected conditions have
worsened.
Results in Brief
we independently calculated and compared VBA's accuracy rates for fiscal
years 2001 and 2002. As part of our analysis of how well VBA ensures
national and regional office accuracy, we reviewed VBA's accuracy
measurement system, known as the Systematic Technical Accuracy Review
(STAR) program, including how VBA incorporates accuracy into its
performance standards for both regional office directors and claims
processing staff. In addition, we examined VBA's criteria for regional
office performance awards and the results of reviews by VBA survey teams
of regional offices. We discussed factors affecting accuracy with staff at
VBA regional offices in Phoenix, Arizona; St. Petersburg, Florida; Boston,
Massachusetts; Pittsburgh, Pennsylvania; and Houston, Texas.2 We also
discussed accuracy issues with representatives of the American Legion,
Disabled American Veterans, Paralyzed Veterans of America, and Veterans of
Foreign Wars to gain their perspectives on VBA efforts to improve
accuracy. We conducted our review from January through August of 2003 in
accordance with generally accepted government auditing standards.
From fiscal years 2001 to 2002, VBA's accuracy of decision making in the
compensation and pension programs declined from 89 percent to 81 percent.3
We determined these accuracy rates by comparing substantially similar
questions for the 2 fiscal years that focused on whether the decision to
grant or deny benefits was correct. The agency had reported a slight
improvement in accuracy between fiscal years 2001 and 2002-from 78 percent
to 80 percent. However, we found that these two annual figures were not
comparable because the agency had substantially changed the way it
measured accuracy for fiscal 2002. Although VBA acknowledged a change in
its accuracy measure in its annual report to the Congress, the agency did
not revise its 2001 figure to allow for an appropriate comparison with
2002. VBA officials attributed the decline in accuracy during this period
to several factors. We were not able to quantify the relative contribution
of these factors. These factors included headquarters emphasis on
production, the specific processing requirements of the
2Except for Pittsburgh, we selected the regional offices we visited based
on a combination of their fiscal year 2002 rating accuracy and the number
of rating decisions they produced. We visited the Pittsburgh Regional
Office to observe how VBA conducts an inspection of a regional office's
compensation and pension operations as well as to discuss factors
affecting accuracy.
3The margin of error for rating accuracy VBA-wide for fiscal years 2001
and 2002 was about 1 percentage point at a 95-percent level of confidence.
Background
Veterans Claims Assistance Act of 2000, and the relative inexperience of
VBA's claims processing staff.
To help ensure accountability for accuracy, VBA set accuracy standards for
its regional offices. However, VBA has not fully used data gathered on the
accuracy of its regional office decisions to encourage better regional
office performance. For example, while some regional offices were required
to prepare plans to improve claims processing production, similar plans
were not required for offices to improve accuracy. Also, two regional
offices received overall performance awards despite not meeting VBA's
accuracy standard. Unless VBA strikes a balance between production goals
and accuracy standards, it cannot ensure that as it improves decision
timeliness, it will also improve decision accuracy.
This report contains recommendations to the Secretary of Veterans Affairs
to direct VBA to improve its reporting on and use of accuracy data. VBA
concurred with our recommendations and provided additional information
about its efforts to improve accuracy.
Through its disability compensation program, VBA pays monthly benefits to
veterans with service-connected disabilities (injuries or diseases
incurred or aggravated while on active military duty) according to the
severity of the disability. VBA's pension benefit program pays monthly
benefits to wartime veterans who have low incomes and are permanently and
totally disabled for reasons not service-connected.4 In addition, VBA pays
dependency and indemnity compensation to some deceased veterans' spouses,
children, and parents and to survivors of service members who died on
active duty. In fiscal year 2002, VBA paid over $22 billion in disability
compensation to an average of about 2.4 million veterans and over 300,000
survivors. VBA also paid over $3 billion in pensions to an average of
about 580,000 veterans and survivors.
The amount of disability compensation largely depends on the degree to
which a veteran is disabled. VBA determines the degree to which veterans
are disabled in 10 percent increments on a scale of 0 to 100 percent.
Basic monthly payments range from $104 for 10 percent disability to $2,193
for
4Veterans who are 65 years or older do not have to be permanently and
totally disabled to become eligible for pension benefits, as long as they
meet the other requirements for income and military service. VBA also pays
pensions to surviving spouses and unmarried children of deceased wartime
veterans.
100 percent disability. About 65 percent of veterans receiving disability
compensation have disabilities rated at 30 percent and lower; about 8
percent have disabilities rated at 100 percent. The most common
impairments for veterans who began receiving compensation in fiscal year
2001 were tinnitus (ex., a ringing in the ear), auditory acuity impairment
rated at 0 percent (i.e., mild hearing difficulties), skeletal conditions,
arthritis due to trauma, post-traumatic stress disorder, and scars.
Eligibility and priority for other Department of Veterans Affairs (VA)
benefits and services such as health care and vocational rehabilitation
are affected by these VA disability ratings.
When a veteran submits a claim to any of VBA's 57 regional offices, a
veterans service representative (VSR) is responsible for obtaining the
relevant evidence to evaluate the claim. Such evidence includes veterans'
military service records, medical examinations and treatment records from
VA medical facilities, and treatment records from private medical service
providers. Once a claim is developed (i.e., has all the necessary
evidence), a rating VSR, also called a rating specialist, evaluates the
claim, determines whether the claimant is eligible for benefits, and
assigns a rating based on degree of disability. Veterans with multiple
disabilities receive a single composite rating. For veterans claiming
pension eligibility, the regional office determines if the veteran served
in a period of war, is permanently and totally disabled for reasons not
service-connected, and meets the income thresholds for eligibility. A
veteran who disagrees with the regional office's decision for either
program can appeal sequentially to VA's Board of Veterans' Appeals (BVA),
the U.S. Court of Appeals for Veterans Claims, and the U.S. Court of
Appeals for the Federal Circuit.
In fiscal year 1999, VBA implemented a quality review system to improve
the measurement of the accuracy of its claims decisions. Under STAR, VBA
selects a random sample of completed claims decisions each month from each
of its 57 regional offices. STAR quality review staff review these claims
using a standard checklist. If a claim has any error, VBA counts the
entire claim as incorrect for accuracy rate computation purposes. STAR
then returns the case file and the results of the review to the regional
office that made the decision. If an error was found, the regional office
is required to either correct it or request reconsideration of the error
by STAR.
VBA made several significant changes to its STAR system in fiscal year
2002. First, in response to our March 1999 recommendation, VBA made the
STAR review staff organizationally independent of the regional offices to
ensure the independence of the reviews.5 Second, VBA more than doubled the
number of rating decisions it reviewed (from 3,209 in fiscal year 2001 to
6,646 in fiscal year 2002) in order to obtain more precise regional office
level accuracy scores. Third, VBA improved its key measure to focus more
on whether decisions to grant or deny benefits were correct. Previously,
VBA's key measure also included decision documentation and notification
issues. Figure 1 shows the areas covered by the checklist used to measure
accuracy in fiscal year 2001. The revised key accuracy measure includes
only the first four areas of the checklist.
Figure 1: Fiscal Year 2001 STAR Rating Checklist-Criteria for a Correct
VBA Decision
� Address all issues - The decision addressed all claimed
disabilities and benefits, and any disabilities or benefits that could be
inferred from the available evidence.
� Proper development - VBA made an adequate attempt to obtain all
evidence needed to decide the claim.
� Grant or deny - The decision to grant or deny was correct; if
granted, the percentage evaluation was correct.
� Award actions - Where benefits were granted, the effective dates
of benefits and payment amounts were correct.
� Reasons and bases - The decision discussed all applicable
evidence and the basis of the decision was explained.
� Notification - The claimant and power of attorney, if applicable,
were correctly notified of the decision and were notified of appeal
rights.
Source: Veterans Benefits Administration.
5For more information, see U.S. General Accounting Office, Veterans'
Benefits Claims: Further Improvements Needed in Claims-Processing
Accuracy, GAO/HEHS-99-35 (Washington, D.C.: Mar. 1, 1999) and Veterans'
Benefits: Quality Assurance for Disability Claims Processing, GAO-01-930R
(Washington, D.C.: Aug. 23, 2001).
Accuracy Declined in Fiscal Year 2002
From fiscal years 2001 to 2002, VBA's accuracy of decision making in the
compensation and pension programs declined from 89 percent to 81 percent.
We determined these accuracy rates by comparing substantially similar
questions for the 2 fiscal years that focused on whether the decision to
grant or deny benefits was correct.6 However, the agency reported that it
had improved its accuracy for that period. This discrepancy occurred
because VBA did not report comparable accuracy data for the 2 fiscal
years. VBA reported its fiscal year 2002 accuracy based on a revised
measure that excluded two areas of the checklist-reasons and bases and
notification. VBA then compared this to its fiscal year 2001 accuracy,
which included these two areas of the checklist. VBA officials attributed
the decline in accuracy during this period to several factors. We were not
able to quantify the relative contribution of these factors. These factors
included headquarters emphasis on production, the specific processing
requirements of the Veterans Claims Assistance Act of 2000, and the
relative inexperience of VBA's claims processing staff.
In fiscal year 2002, VBA made progress in increasing production of claims
decisions and reducing inventory. The number of claims decisions rose
substantially from about 481,000 in fiscal year 2001 to about 797,000 in
fiscal year 2002. During the same time period, the number of claims VBA
received increased from about 674,000 to 722,000. The higher level of
production permitted VBA to reduce its inventory of pending claims by 18
percent from about 421,000 to about 346,000.
Although accuracy actually declined, VBA reported in its performance
report to the Congress for fiscal year 2002 that the accuracy of its
rating decisions for compensation and pension benefits had improved
slightly. VBA reported that accuracy improved from 78 percent in fiscal
2001 to 80 percent in fiscal 2002.7 However, these rates were not
comparable because the new "benefit entitlement accuracy" rate for fiscal
year 2002 was based on fewer areas of the checklist that focused on
whether and to what extent the claimant received a benefit. In its fiscal
year 2002 performance report, VA made note of this change but did not
recalculate
6The fiscal year 2002 accuracy score includes a specific question on
proper VCAA pre-decision notification that VBA did not include in its
fiscal year 2001 quality reviews.
7Due to the time lag involved between receiving claims files and
conducting the accuracy reviews of disability compensation and pension
decisions as well as deadlines for incorporating data into its annual
accountability report, reported accuracy rates can differ from final rates
for the reported year. VBA's final rating accuracy rate for fiscal year
2002 was 81 percent.
the fiscal 2001 rate to provide a correct comparison. When we recomputed
the agency's accuracy rate for fiscal 2001 using its revised criteria, the
result was 89 percent, not 78 percent as reported. As a result, the
correct comparison showed a decline in accuracy from 89 percent to 81
percent. The agency corroborated our finding after performing its own
computation at our request.
VBA officials we spoke with cited a number of factors that affected
accuracy over the fiscal years 2001 to 2002 period. A key factor cited was
VBA's emphasis on production, such that accuracy became a lesser priority.
In 2001, the Secretary of Veterans Affairs set an ambitious goal to reduce
the agency's average time to complete a rating decision from 173 days (in
fiscal year 2000) to 100 days by the end of fiscal year 2003. Also, the
Secretary tasked VBA with reducing the inventory of rating claims to
250,000 by the end of September 2003. VBA managers responded by
emphasizing the completion of more rating cases each month. The result has
been a significant increase in production and a significant reduction in
the rating inventory since the end of fiscal year 2001.
In fiscal year 2002, VBA incorporated its new production targets into its
measures for regional office performance.8 Regional office directors
became accountable for specific targets for production, inventory
reduction, and timeliness improvement. The agency also established
regional office performance award criteria that gave more weight to
efficiency than accuracy. Three of the four criteria for cash awards were
based on production and timeliness and one on accuracy; bonuses could be
received without meeting the accuracy criteria.
VBA officials also attributed accuracy problems to the addition of more
specific processing requirements under the Veterans Claims Assistance Act
(VCAA) of 2000. The act broadened VBA's responsibility to assist veterans
- such as notifying them of needed evidence and helping them develop that
evidence for their claims. VBA incorporated VCAA requirements into its
STAR rating accuracy checklist in fiscal year 2002. In the first 4 months
of fiscal year 2002, VCAA errors accounted for almost half of all errors
identified by STAR rating reviewers. In April 2002, VBA required each
regional office to re-train its claims processing staff on the
8VBA established specific monthly production targets for its regional
offices in December 2001. For more information on these targets see U.S.
General Accounting Office, Veterans' Benefits: Despite Recent
Improvements, Meeting Claims Processing Goals Will Be Challenging,
GAO-02-645T (Washington, D.C.: Apr. 26, 2002).
VBA Could Have Done More to Hold Regional Offices Accountable for Accuracy
new requirements.9 Over the last 8 months of fiscal year 2002, the
proportion of VCAA-related errors declined to about one-third of all
errors.
VBA officials also cited the relative inexperience of VBA's claims
processing staff. Regional office officials noted that they had many VSRs
and rating specialists with insufficient training and experience to be
fully proficient at developing and making decisions on claims. For
example, officials at two offices we visited noted that their accuracy was
affected by large numbers of inexperienced staff, due to a large increase
in staff size or a relatively high rate of turnover. VBA has found that
attrition rates for new claims processing staff hired over a 3-year period
ranged from 0 percent to 49 percent at some regional offices.10
While VBA has established accuracy standards for its regional offices, it
has not made the best use of its accuracy data on regional offices. VBA
collected regional office accuracy data in fiscal year 2002 but has not
fully used the information to evaluate regional office performance,
correct errors, and identify needed training that could reduce errors.
Unless VBA better uses these data to hold regional offices accountable for
accuracy, it cannot ensure that as decisions become timelier, they also
become more accurate. VBA is developing an agencywide system for
evaluating individual claims processing employees' performance against
VBA's individual accuracy standards.
In fiscal year 2002, VBA improved its STAR review system to provide
independent review of decisions and more precise accuracy data at the
regional office level. Regional accuracy scores are estimates based on
samples of cases and revealed high and low performers relative to the
agency's fiscal year 2002 accuracy goal of 85 percent. Eleven regional
offices clearly failed to meet the goal while 3 clearly met or exceeded
the
9For more information on VCAA implementation, see U.S. General Accounting
Office, Veterans' Benefits: VBA's Efforts to Implement the Veterans Claims
Assistance Act Need Further Monitoring, GAO-02-412 (Washington, D.C.: July
1, 2002).
10For more information, see U.S. General Accounting Office, Veterans
Benefits Administration: Better Collection and Analysis of Attrition Data
Needed to Enhance Workforce Planning, GAO-03-491 (Washington, D.C.: Apr.
28, 2003).
goal.11 However, VBA managers did not fully use the regional
office-specific information generated by the STAR unit as a basis for
improving accuracy at the regional office level. For example, VBA did not
require regional offices that failed to meet the national accuracy goal of
85 percent to prepare strategies for improvement. In contrast, in fiscal
year 2002, 13 regional offices that did not meet established targets for
production, inventory, and timeliness were required to develop corrective
action plans. Also, as noted earlier, VBA did not make rating accuracy a
prerequisite for receiving a performance award. Three of the four criteria
for cash awards were based on production and timeliness and one on
accuracy; a bonus could be received without meeting the accuracy criteria.
In fiscal year 2002, 2 offices received performance awards despite clearly
not meeting VBA's accuracy goal. These offices had accuracy rates of 71
percent and 75 percent, respectively.
VBA's regional offices did not always properly address STAR errors that
were returned to them, a violation of VBA policy that regional offices
either correct errors or formally request that the STAR unit reconsider
its error call. VBA survey teams, which review judgmental samples of STAR
cases with errors, found that the 18 regional offices they visited from
October 2001 through December 2002 had failed to respond properly to over
40 percent of the errors reviewed. At 14 of the 18 offices, the survey
teams found problems significant enough to recommend improvements in STAR
error handling. For example, teams recommended that several offices
establish management controls to ensure that they properly address all
errors identified by the STAR unit. Effective April 1, 2003, VBA
established a new tracking system that required regional offices to report
to headquarters on actions taken to address all errors identified by the
STAR unit.
In addition to not correcting all errors, several of the regional offices
visited by the survey teams were not making adequate use of STAR errors to
identify trends that could be used to provide feedback and training to
claims processing staff. Officials at regional offices we visited
commented that this information is useful for identifying trends for
feedback and training purposes. For example, in fiscal year 2002, the
results of STAR reviews helped to alert management of the Boston Regional
Office that
11It could not be determined if the remaining offices met the accuracy
goal because of the margins of error associated with their accuracy
scores. In fiscal year 2002, the margins of error for rating accuracy for
each regional office ranged from 4 to 9 percentage points at the
95-percent level of confidence.
many of its claims processors were having difficulties with the new VCAA
legislation, permitting management to provide special training for the
Boston staff. The agency survey teams recommended that 7 of the regional
offices reviewed between October 2001 and December 2002 take steps to use
STAR reviews for feedback and training.
In addition to regional office-level accuracy standards, VBA has also
established quantifiable accuracy standards for its experienced claims
processing employees, as part of these employees' annual performance
evaluations.12 The accuracy standards for experienced rating specialists
require that they correctly decide 85 percent of 60 randomly selected
rating decisions per year (5 decisions per month).13 VBA is developing an
agencywide review process, the Systematic Individual Performance
Assessment (SIPA), to assess compliance with the individual accuracy
standards. VBA developed a uniform checklist, similar to the STAR
checklist for reviewing rating decisions, which offices could use to
assess rating specialists' accuracy. Unlike the STAR system, reviews are
conducted before decisions are finalized.
In April 2003, VBA surveyed regional offices to determine the status of
their efforts to measure individual accuracy and assess the staff
resources needed to implement SIPA VBA-wide. Individual reviews were being
conducted by supervisors, such as team coaches, and nonsupervisory staff,
such as Decision Review Officers, who are responsible for handling
appeals. At the time of our review, VBA was in the process of deciding who
would perform the reviews. In its efforts to develop SIPA as a consistent
review process, VBA is considering issues such as ensuring that consistent
feedback is provided to the employees and all regional offices are using
the same review checklists.
Conclusions While VBA's emphasis on production succeeded in increasing
the number of claims decisions made during fiscal year 2002, the accuracy
of those decisions declined. Because VBA did not report comparable
accuracy rates for fiscal years 2001 and 2002 in its performance and
accountability report, Members of Congress, Department of Veterans Affairs
management, and claimants did not know of this decline in accuracy. The
12In general, inexperienced claims processing employees have all of their
work reviewed.
13These standards, according to VBA officials, are intended to be floors;
individual regional offices can set higher standards.
lack of accurate performance data hampers stakeholders' ability to monitor
VBA's performance in serving the nation's veterans.
In addition, VBA could better use the accuracy data it has to hold its
regional offices more accountable for accuracy. While VBA has made
progress in measuring regional office accuracy, it has not made full use
of the results to reward high performing offices or to improve poorly
performing offices. VBA has made significant progress in getting its
compensation and pension claims workload under control to improve
timeliness for veterans waiting for decisions on their claims. However,
VBA's accuracy has declined. Unless VBA balances its emphasis on producing
more and faster decisions with an increased emphasis on accuracy, it will
be difficult to ensure that veterans are getting the benefits to which
they are entitled in a timely manner.
Recommendations
o
o
Agency Comments and Our Response
We recommend that the Secretary of Veterans Affairs direct the Under
Secretary for Benefits to
report the accuracy of VBA disability compensation and pension claims
decisions to the Congress and other stakeholders in a manner that allows
for valid comparisons of accuracy across fiscal years and
better hold regional offices accountable for the accuracy of their claims
decisions, by increasing the use of its regional office accuracy data,
while
at the same time maintaining an appropriate emphasis on production.
In its written comments on a draft of this report (see app. I), VBA
concurred with our recommendations. However, VBA suggested that more
effective use of STAR had been made to improve performance than our
report concludes. We did note that VBA used STAR data to improve
accuracy. Specifically, we noted in our report that VCAA-related errors
declined to about one-third of all errors after VBA identified the errors
through STAR and required retraining. We support VBA's continued efforts
to emphasize accuracy and timeliness as critical components of quality.
We will send copies of this report to the Secretary of Veterans Affairs,
appropriate congressional committees, and other interested parties. We
will also make copies of this report available to others on request. The
report will also be available at no charge on GAO's Web site at
http://www.gao.gov.
If you or your staffs have any questions regarding this report, please
call me at (202) 512-7215 or Irene Chu, Assistant Director, at (202)
512-7102. In addition to those named, Susan Bernstein, Kristine Braaten,
Kevin Jackson, Joseph Natalicchio, Martin Scire, and Greg Whitney made key
contributions to this report.
Cynthia A. Bascetta Director, Education, Workforce, and Income Security
Issues
GAO's Mission
Obtaining Copies of GAO Reports and Testimony
The General Accounting Office, the audit, evaluation and investigative arm
of Congress, exists to support Congress in meeting its constitutional
responsibilities and to help improve the performance and accountability of
the federal government for the American people. GAO examines the use of
public funds; evaluates federal programs and policies; and provides
analyses, recommendations, and other assistance to help Congress make
informed oversight, policy, and funding decisions. GAO's commitment to
good government is reflected in its core values of accountability,
integrity, and reliability.
The fastest and easiest way to obtain copies of GAO documents at no cost
is through the Internet. GAO's Web site (www.gao.gov) contains abstracts
and full-text files of current reports and testimony and an expanding
archive of older products. The Web site features a search engine to help
you locate documents using key words and phrases. You can print these
documents in their entirety, including charts and other graphics.
Each day, GAO issues a list of newly released reports, testimony, and
correspondence. GAO posts this list, known as "Today's Reports," on its
Web site daily. The list contains links to the full-text document files.
To have GAO e-mail this list to you every afternoon, go to www.gao.gov and
select "Subscribe to e-mail alerts" under the "Order GAO Products"
heading.
Order by Mail or Phone The first copy of each printed report is free.
Additional copies are $2 each. A check or money order should be made out
to the Superintendent of Documents. GAO also accepts VISA and Mastercard.
Orders for 100 or more copies mailed to a single address are discounted 25
percent. Orders should be sent to:
U.S. General Accounting Office 441 G Street NW, Room LM Washington, D.C.
20548
To order by Phone: Voice: (202) 512-6000 TDD: (202) 512-2537 Fax: (202)
512-6061
Contact:
To Report Fraud, Web site: www.gao.gov/fraudnet/fraudnet.htm
Waste, and Abuse in E-mail: fraudnet@gao.gov
Federal Programs Automated answering system: (800) 424-5454 or (202)
512-7470
Jeff Nelligan, Managing Director, NelliganJ@gao.gov (202) 512-4800
Public Affairs U.S. General Accounting Office, 441 G Street NW, Room 7149
Washington, D.C. 20548
*** End of document. ***