Veterans Benefits: VA Needs Plan for Assessing Consistency of
Decisions (19-NOV-04, GAO-05-99).
In the past, we have reported concerns about possible
inconsistencies in the disability decisions made by the 57
regional offices of the Department of Veterans Affairs (VA). In
2002, we reported that VA did not systematically assess the
consistency of decision making for any specific impairments
included in veterans' disability claims. We recommended that VA
conduct such assessments to help reduce any unacceptable
variations that VA might find among regional offices. VA agreed
that decision-making consistency is an important goal and
concurred in principle with our recommendation. However, VA did
not discuss how it would measure consistency. In January 2003, in
part because of concerns about consistency, we designated VA's
disability program, along with other federal disability programs,
as high-risk. In fiscal year 2005, VA estimates it will pay about
$25 billion in disability compensation benefits to about 2.7
million disabled veterans. In this context, we determined (1) the
actions that VA has taken to assess the consistency of regional
office decisions on disability compensation claims and (2) the
extent to which VA program data can be used to measure the
consistency of decision making among regional offices.
-------------------------Indexing Terms-------------------------
REPORTNUM: GAO-05-99
ACCNO: A13595
TITLE: Veterans Benefits: VA Needs Plan for Assessing
Consistency of Decisions
DATE: 11/19/2004
SUBJECT: Claims processing
Decision making
Disability benefits
Veterans benefits
Veterans disability compensation
VA Benefits Delivery Network System
******************************************************************
** This file contains an ASCII representation of the text of a **
** GAO Product. **
** **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced. Tables are included, but **
** may not resemble those in the printed version. **
** **
** Please see the PDF (Portable Document Format) file, when **
** available, for a complete electronic file of the printed **
** document's contents. **
** **
******************************************************************
GAO-05-99
United States Government Accountability Office
GAO
Report to Congressional Requesters
November 2004
VETERANS
BENEFITS
VA Needs Plan for Assessing Consistency of Decisions
GAO-05-99
Contents
Letter 1
Appendix I Briefing Slides
Abbreviations
BDN Benefits Delivery Network
STAR Systematic Technical Accuracy Review
VA Department of Veterans Affairs
This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed in
its entirety without further permission from GAO. However, because this
work may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this material
separately.
United States Government Accountability Office Washington, DC 20548
November 19, 2004
The Honorable Henry E. Brown, Jr.
Chairman
Subcommittee on Benefits
Committee on Veterans' Affairs
House of Representatives
The Honorable Mike Simpson
House of Representatives
In the past, we have reported concerns about possible inconsistencies in
the disability decisions made by the 57 regional offices of the Department
of Veterans Affairs (VA). In 2002, we reported that VA did not
systematically assess the consistency of decision making for any specific
impairments included in veterans' disability claims.1 We recommended
that VA conduct such assessments to help reduce any unacceptable
variations that VA might find among regional offices. VA agreed that
decision-making consistency is an important goal and concurred in
principle with our recommendation. However, VA did not discuss how it
would measure consistency.
In January 2003, in part because of concerns about consistency, we
designated VA's disability program, along with other federal disability
programs, as high-risk.2 In fiscal year 2005, VA estimates it will pay
about
$25 billion in disability compensation benefits to about 2.7 million
disabled
veterans. In this context, you asked us to determine (1) the actions that
VA
has taken to assess the consistency of regional office decisions on
disability compensation claims and (2) the extent to which VA program
data can be used to measure the consistency of decision making among
regional offices.
To address these issues, we (1) identified key data fields in VA's
Benefits
Delivery Network system-such as the level of benefits awarded for each
claimed impairment-which VA uses to manage the delivery of disability
1GAO, Veterans' Benefits: Quality Assurance for Disability Claims and
Appeals Processing Can Be Further Improved, GAO-02-806 (Washington, D.C.:
Aug. 16, 2002).
2GAO, High-Risk Series: An Update, GAO-03-119 (Washington, D.C.: Jan.
2003).
benefits to veterans; (2) obtained from VA an electronic file of these key
data fields for all veterans receiving compensation benefits as of March
2004; (3) conducted electronic testing of key data fields to determine
their reliability for identifying indications of possible inconsistency in
regional office decisions; and (4) reviewed VA records and documents and
interviewed VA officials. We conducted our review from November 2003
through October 2004 in accordance with generally accepted government
auditing standards. On October 28, 2004, we briefed your office on the
results of our work. This letter formally conveys the information provided
during that briefing. Appendix I contains the briefing slides.
In summary, we found that VA still does not systematically assess
decision-making consistency among the 57 regional offices. We also found
that data contained in VA's Benefits Delivery Network system, which was
designed for the purpose of paying benefits, do not provide a reliable
basis for identifying indications of possible decision-making
inconsistencies among regional offices. However, according to VA
officials, as of October 2004, a newly-implemented nationwide information
system (known as RBA 2000) could provide VA such an opportunity if the
system proves over time to reliably collect data needed to determine each
regional office's denial rates and average disability ratings for specific
impairments. VA will need to collect several years of data with RBA 2000
in order to have sufficient data to reliably identify indications of
impairment-specific inconsistencies among regional offices. Still, even if
the RBA 2000 system permits VA to identify indications of such
inconsistencies, VA will need to systematically study and determine the
extent and causes of such inconsistencies and identify ways to reduce any
variations among regional offices that VA may consider unacceptable.
We recommend that the Secretary of Veterans Affairs develop a plan, and
include it in VA's annual performance plan, that contains a detailed
description of how VA will (1) use data gathered through the new RBA 2000
system to identify indications of possible inconsistencies among regional
offices in the award and denial of disability compensation benefits for
specific impairments and (2) conduct systematic studies of consistency for
specific impairments for which RBA 2000 data reveal indications of
possible decision-making inconsistencies among regional offices.
In oral comments on a draft of this report, VA agreed with our findings
and conclusions and concurred with our recommendation. We also made
technical revisions as appropriate.
We are sending copies of this report to the Chairman and Ranking
Democratic Member, House Committee on Veterans' Affairs; the Chairman
and Ranking Democratic Member, Senate Committee on Veterans' Affairs;
and the Secretary of Veterans Affairs. We will also make copies available
upon request. In addition, the report will be available at no charge on
GAO's Web site at http://www.gao.gov.
If you or your staffs have any questions about this report, please contact
me on (202) 512-7215 or Irene Chu, Assistant Director, on (202) 512-7102.
Ira Spears, Joseph Natalicchio, Joan Vogel, Walter Vance, and Vanessa
Taylor also made key contributions to this report.
Cynthia A. Bascetta
Director, Education, Workforce,
and Income Security Issues
Appendix I: Briefing Slides
Data Limitations Hinder the Department
of Veterans Affairs' Ability to Assess the Consistency of
Decisions on Disability Compensation Claims
Briefing for Staff of
Representative Henry E. Brown, Jr.
Chairman, Subcommittee on Benefits,
House Committee on Veterans' Affairs
and
Representative Mike Simpson
Subcommittee on VA, HUD and Independent Agencies
House Committee on Appropriations
October 28, 2004
Data Limitations Hinder VA's Ability to Assess the Consistency of
Decisions on Disability Compensation Claims
o Key Questions
o Background
o Scope and Methodology
o Results in Brief o GAO Findings
o Conclusions
o Recommendation
2
Key Questions
o Since the issuance of our 2002 report, what actions has the Department
of Veterans Affairs (VA) taken to assess the consistency of regional
office decisions on disability compensation claims?
o To what extent does VA have program data that can be used to measure
the consistency of decision making among regional offices?
3
Background
VA claims adjudicators use judgment in making disability decisions, which
introduces an element of potential variability. Judgment is particularly
crucial when the adjudicator must
o assess the credibility of different sources of evidence;
o evaluate how much weight to assign differing sources of evidence; or
o assess some disabilities, such as mental disorders, for which the
disability standards are not entirely objective and require the use of
professional judgment.
4
Background
o In 1997, the National Academy of Public Administration reported that
VA's regulations were subject to varying interpretations and said
achieving consistency across 57 decentralized regional offices is
inherently difficult.
o In 2001, VA's Claims Processing Task Force questioned the consistency
of decisions because of factors such as differing interpretations of VA
guidance.
5
Background
o In 2002, we reported that VA did not systematically assess
decision-making consistency for any specific medical impairments, despite
concerns about possible inconsistencies in disability claims decisions
made by VA's 57 regional offices.
o VA's disability decision quality review program-known as Systematic
Technical Accuracy Review (STAR)- assesses the overall accuracy of all
disability decisions, but not the consistency of decisions overall or for
specific impairments.
6
Background
o We recommended in 2002 that VA assess decisionmaking consistency for
medical conditions requiring difficult judgment. We said VA could, for
example:
o develop hypothetical claims for a specific medical impairment,
o distribute these claims to multiple adjudicators, and
o analyze variations in decisions on these claims.
o VA could use these findings to reduce impairment-specific variations
among regional offices, if considered unacceptable, to levels VA believes
would be appropriate.
7
Background
o VA agreed that decision-making consistency is an important goal and
concurred in principle with our recommendation. VA added that it seeks to
ensure consistency through training and communication.
o However, VA did not discuss how it would measure consistency or
evaluate progress toward the goal of decision-making consistency.
8
Background
To decide a disability compensation claim, the regional office
o develops evidence,
o determines service connection of each claimed impairment,
o applies VA's medical criteria to evaluate the degree of disability due
to each service-connected impairment, and
o determines the veteran's overall degree of serviceconnected disability.
9
Background
After deciding a disability compensation claim, the regional office
o notifies the veteran of the decision and
o records the decision results in VA's Benefits Delivery Network (BDN)
system in order to begin paying benefits to the veteran.
Scope and Methodology
To do our work, we
o identified key BDN data fields, such as the level of benefits awarded
for each impairment, and obtained from VA an electronic file of these data
for all veterans receiving compensation benefits as of March 2004;
o conducted electronic testing of key BDN data fields to determine the
reliability of using these data to study the consistency of
decision-making among regional offices; and
o reviewed VA records and documents and interviewed VA officials.
11
Scope and Methodology
o We did not assess how well BDN supports the payment of benefits.
o We conducted our review from November 2003 through October 2004 in
accordance with generally accepted government auditing standards.
Results in Brief
o Since the issuance of our 2002 report, VA has not systematically
assessed the consistency of regional office decisions on specific
impairments.
o Existing compensation program data have limitations that preclude
identifying indications of decision-making inconsistency among regional
offices. However, VA is implementing a new data collection system that may
afford an opportunity to identify indications of inconsistency in the
future.
13
VA Has Not Assessed Consistency of Compensation Decisions among Regional
Offices
Although VA acknowledges that veterans are concerned about consistency, VA
has not taken any action to assess consistency.
o When we asked VA in 2004 about actions taken to assess consistency, VA
said it would continue to use its STAR system to assess the overall
accuracy of decisions for each regional office.
o However, STAR does not provide statistically meaningful data about the
consistency of decisions for specific impairments.
14
Data Limitations Prevent Reliable Identification of Potential
Inconsistency among Regional Offices
BDN-until recently VA's only nationwide administrative database for
recording the results of disability decisions-does not provide a reliable
basis for identifying indications of inconsistency in the disability
compensation program. BDN did not permit us to reliably
o identify dates of decisions so that we could examine only recent
decisions rather than decisions made many years ago,
o determine each regional office's average disability ratings for
specific impairments, or
o determine each regional office's denial rates for specific impairments.
Data Limitations Prevent Reliable Identification of Potential
Inconsistency among Regional Offices
We intended to identify indications of possible decision-making
inconsistency among regional offices using existing administrative data to
o examine original claims decisions made after fiscal year 2000 because
issues affecting decision-making consistency in the past may not be the
issues affecting consistency today and
o for selected impairments, such as post-traumatic stress disorder or an
undiagnosed illness, compare the
o average disability ratings of each regional office and
o service-connection denial rates of each regional office.
16
Data Limitations Prevent Reliable Identification of Indications of
Inconsistency among Regional Offices
We could not reliably group decisions by a relevant date in order to
identify original award decisions made after fiscal year 2000.
o No specific data field exists in BDN to record the date that the
veteran submitted his or her original claim or the date that the regional
office made the original award decision.
o Information in the field for recording the original award's effective
date (typically the original claim's submission date) could subsequently
be overwritten by another date or erased because of
o routine benefit changes made in a veteran's record or
o requirements of new court decisions or new laws.
17
Data Limitations Prevent Reliable Identification of Indications of
Inconsistency among Regional Offices
We could not compare consistency among regional offices because we could
not reliably identify the regional office that made the original award
decision in which each service-connected disability was rated.
o Some original decisions are made by a regional office other than the
one in the region where the veteran resides, but only the regional office
where the veteran resides is recorded in BDN because it has jurisdiction
over the veteran's claim file.
o If a veteran relocates to an area under the jurisdiction of another
regional office, the regional office identifier may be changed in BDN. If
so, BDN can no longer identify the office that made the original award.
18
Data Limitations Prevent Reliable Identification of Indications of
Inconsistency among Regional Offices
We could not use BDN to determine denial rates for specific impairments
because BDN
o does not maintain impairment-specific data for decisions in which
regional offices deny all disability benefits and
o does not capture impairment-specific decision data for any more than
six impairments per veteran, even though veterans may claim more than six
impairments.
19
Data Limitations Prevent Reliable Identification of Indications of
Inconsistency among Regional Offices
o VA is working on a new administrative data system-known as VETSNET-that
eventually will serve all VA benefits programs. According to VA, the
disability program portion of VETSNET will replace BDN in 2006.
o However, according to VA officials, as of October 2004, a nationwide
VETSNET subsystem known as RBA 2000, which collects disability decision
data, could provide a reliable basis for identifying impairmentspecific
indications of inconsistency among the 57 regional offices.
o Several years of data will need to be collected in RBA 2000 before its
data can be used to assess the consistency of decision making among the 57
regional offices.
Conclusions
o VA cannot provide reasonable assurance that similarly situated veterans
who submit claims for the same impairment to different regional offices
receive reasonably consistent decisions.
o RBA 2000 could allow VA to identify indications of possible
inconsistencies if it proves over time to reliably provide data that
enable VA to determine impairment-specific average disability ratings and
average denial rates for each regional office.
o However,VA still would need to take additional action, such as we
recommended in 2002, to determine the extent and causes of inconsistencies
for the impairments in question and to identify ways to reduce any
unacceptable levels of variation in the award and denial of disability
compensation benefits.
Recommendations for Executive Action
We recommend that the Secretary of Veterans Affairs develop a plan, and
include it in VA's annual performance plan, that contains a detailed
description of how VA will
o use data collected through RBA 2000 to identify indications of possible
inconsistencies among regional offices in the award and denial of benefits
for specific impairments and
o conduct systematic studies of consistency for specific impairments for
which RBA 2000 data reveal indications of inconsistencies among decisions
made by the regional offices.
(130341)
22
GAO's Mission The Government Accountability Office, the audit, evaluation
and investigative arm of Congress, exists to support Congress in meeting
its constitutional responsibilities and to help improve the performance
and accountability of the federal government for the American people. GAO
examines the use of public funds; evaluates federal programs and policies;
and provides analyses, recommendations, and other assistance to help
Congress make informed oversight, policy, and funding decisions. GAO's
commitment to good government is reflected in its core values of
accountability, integrity, and reliability.
Obtaining Copies of The fastest and easiest way to obtain copies of GAO
documents at no cost
is through GAO's Web site (www.gao.gov). Each weekday, GAO postsGAO
Reports and newly released reports, testimony, and correspondence on its
Web site. To Testimony have GAO e-mail you a list of newly posted products
every afternoon, go to
www.gao.gov and select "Subscribe to Updates."
Order by Mail or Phone The first copy of each printed report is free.
Additional copies are $2 each. A check or money order should be made out
to the Superintendent of Documents. GAO also accepts VISA and Mastercard.
Orders for 100 or more copies mailed to a single address are discounted 25
percent. Orders should be sent to:
U.S. Government Accountability Office 441 G Street NW, Room LM Washington,
D.C. 20548
To order by Phone: Voice: (202) 512-6000 TDD: (202) 512-2537 Fax: (202)
512-6061
To Report Fraud, Contact:
Waste, and Abuse in Web site: www.gao.gov/fraudnet/fraudnet.htm
E-mail: [email protected] Programs Automated answering system: (800)
424-5454 or (202) 512-7470
Congressional Gloria Jarmon, Managing Director, [email protected] (202)
512-4400 U.S. Government Accountability Office, 441 G Street NW, Room 7125
Relations Washington, D.C. 20548
Public Affairs Susan Becker, Acting Manager, [email protected] (202)
512-4800 U.S. Government Accountability Office, 441 G Street NW, Room 7149
Washington, D.C. 20548
*** End of document. ***