Veterans' Benefits: Quality Assurance for Disability Claims and
Appeals Processing Can Be Further Improved (16-AUG-02,
GAO-02-806).
For fiscal year 2002, the Department of Veterans Affairs (VA)
will pay $25 billion in cash disability benefits to 3.3 million
disabled veterans and their families. Veterans who are
dissatisfied with VA's 57 regional offices' decisions may file
appeals with VA's Board of Veteran's Appeals. In about half of
such appeals, the Board has either granted the benefits denied or
returned the cases to regional offices for rework. Additionally,
VA reported an accuracy rate of less than 70 percent for regional
office disability decisions when it tested a new quality
assurance program in fiscal year 1998. When the Board itself
denies benefits, veterans may appeal to the U.S. Court of Appeals
for Veterans Claims. In over half of the appeals, the Court has
either granted the benefits denied by the Board or returned the
decisions to the Board for rework. In fiscal year 1998, the Board
of Veteran's Appeals established a quantitative evaluation
program to score its decision-making accuracy and collect data to
improve decision-making. The accuracy measure used by the Board
understates its true accuracy rate because the calculations
include certain deficiencies, such as errors in a written
decision's format, which would not result in either a reversal or
a remand by the Court. VA does not assess the consistency of
decision-making across the regional office and Board disability
adjudicators, even though VA acknowledges that in many cases
adjudicators of equal competence could review the same evidence
but render different decisions. Even though available evidence
indicates that variations in decision-making occur across all
levels of VA adjudication, VA does not conduct systematic
assessments to determine the degree of variations that occurs for
specific impairments and to provide a basis for determining ways
to reduce such variations.
-------------------------Indexing Terms-------------------------
REPORTNUM: GAO-02-806
ACCNO: A04128
TITLE: Veterans' Benefits: Quality Assurance for Disability
Claims and Appeals Processing Can Be Further Improved
DATE: 08/16/2002
SUBJECT: Disability benefits
Veterans benefits
Claims processing
Decision making
Claims adjudicators
Performance measures
Veterans disability compensation
******************************************************************
** This file contains an ASCII representation of the text of a **
** GAO Product. **
** **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced. Tables are included, but **
** may not resemble those in the printed version. **
** **
** Please see the PDF (Portable Document Format) file, when **
** available, for a complete electronic file of the printed **
** document's contents. **
** **
******************************************************************
GAO-02-806
Report to the Ranking Democratic Member, Committee on Veterans* Affairs,
House of Representatives
United States General Accounting Office
GAO
August 2002 VETERANS* BENEFITS
Quality Assurance for Disability Claims and Appeals Processing Can Be
Further Improved
GAO- 02- 806
Page i GAO- 02- 806 Veterans* Disability Benefits Letter 1
Results in Brief 2 Background 3 The Board Understates Its Accuracy and
Does Not Capture Certain
Data for Improving Quality 6 VA Does Not Systematically Assess Consistency
of Decision
Making 11 Conclusions 13 Recommendations for Executive Action 14 Agency
Comments and Our Evaluation 15
Appendix I Steps in the Disability Claims and Appeals Process 17
Appendix II Board of Veterans* Appeals Illustrations of Difficult
Judgments Resulting in Decision- Making Variations 19
Appendix III Comments from the Department of Veterans Affairs 20
Tables
Table 1: Board*s Accuracy Rate Goals and Performance 8 Table 2: Board*s
Quality Review Criteria 9
Abbreviations
SSA Social Security Administration VA Department of Veterans Affairs VBA
Veterans Benefits Administration Contents
Page 1 GAO- 02- 806 Veterans* Disability Benefits
August 16, 2002 The Honorable Lane Evans Ranking Democratic Member
Committee on Veterans* Affairs House of Representatives
Dear Mr. Evans: For fiscal year 2002, the Department of Veterans Affairs
(VA) estimates that it will pay about $25 billion in cash disability
benefits to about 3.3 million disabled veterans and their families. When
veterans submit disability claims, VA*s 57 regional offices make decisions
to either grant or deny the requested benefits. Veterans who are
dissatisfied with regional office decisions may file appeals with VA*s
Board of Veterans* Appeals, and in about half of such appeals, the Board
has either granted the benefits denied by the regional offices or remanded
(returned) the cases to the regional offices for rework. Additionally, VA
reported an accuracy rate of less than 70 percent for regional office
disability decisions when it tested a new quality assurance program in
fiscal year 1998. As a result, we issued two prior reports at your request
on VA*s system for measuring and improving the accuracy of regional office
decisions. 1
Recently, questions have arisen about the quality of Board decisions. When
the Board itself denies benefits requested by veterans, they may appeal to
the U. S. Court of Appeals for Veterans Claims. In over half of such
appeals, the court has either granted the benefits denied by the Board or
remanded the decisions to the Board for rework. As a result, you now have
asked that we examine VA*s efforts to ensure the quality of decisions made
by the Board. Specifically, you asked that we assess the effectiveness of
VA*s system for (1) measuring and improving the accuracy of the Board*s
decisions and (2) determining the consistency (extent of variation) in
decision making across the spectrum of regional office and Board
disability adjudicators.
1 U. S. General Accounting Office, Veterans* Benefits Claims: Further
Improvements Needed in Claims- Processing Accuracy, GAO/ HEHS- 99- 35
(Washington, D. C.: Mar. 1, 1999), and Veterans* Benefits: Quality
Assurance for Disability Claims Processing,
GAO- 01- 930R (Washington, D. C.: Aug. 23, 2001).
United States General Accounting Office Washington, DC 20548
Page 2 GAO- 02- 806 Veterans* Disability Benefits
To address your request, we reviewed (1) data and documents of the Board
of Veterans* Appeals; (2) data and documents of the Veterans Benefits
Administration (VBA), the VA component responsible for the operations of
the 57 regional offices; (3) reports on studies of VA disability claims
processing conducted by the Veterans* Claims Adjudication Commission, the
National Academy of Public Administration, and VA*s Claims Processing Task
Force; and (4) data reported by the U. S. Court of Appeals for Veterans
Claims on its disposition of veterans* appeals. We also interviewed
officials of the Board and VBA about claims and appeals processing
accuracy and consistency issues. We conducted our review from October 2001
through June 2002 in accordance with generally accepted government
auditing standards.
In fiscal year 1998, the Board of Veterans* Appeals established its first
quantitative program to evaluate and score its decision- making accuracy
and to collect data to identify areas where the quality of decision making
needs improvement. The accuracy measure used by the Board understates its
true accuracy rate because the Board*s accuracy rate calculations include
certain deficiencies, such as errors in a written decision*s format, which
would not result in either a reversal or a remand by the Court. In fiscal
year 2001, if the Board had excluded format deficiencies from its accuracy
rate calculations, its accuracy rate would have been 92 percent versus the
reported rate of 87 percent. Even so, the Board*s quality assurance
program does not capture certain data that potentially could further help
improve the quality of the Board*s decisions. For example, the Board does
not record in its quality assurance database any information identifying
the specific medical issues involved in cases where a Board decision was
judged as being in error. Having such data could enhance the Board*s
ability to target training needed for its decision makers.
VA does not assess the consistency of decision making across the spectrum
of regional office and Board disability adjudicators, even though VA
acknowledges that in many cases two adjudicators of equal competence could
review the same evidence but render different decisions due to the
difficult judgments often required in decision making. The result is that
variations may occur in the benefits provided to veterans who have similar
impairments and circumstances. Even though available evidence provides
indications that variations in decision- making may occur across all
levels of VA adjudication, VA does not conduct systematic assessments to
determine the degree of variation that occurs for specific impairments and
to provide a basis for determining ways, if considered necessary, to
reduce such variation. Results in Brief
Page 3 GAO- 02- 806 Veterans* Disability Benefits
This report contains recommendations to the Secretary of VA concerning
actions to improve VA*s ability to better ensure the accuracy and
consistency of disability decisions made across the spectrum of decision
making in VA. In responding to a draft of this report, VA concurred fully
or in principle with our recommendations. However, although VA agreed that
consistency is an important goal, it did not fully respond to our
recommendation regarding consistency because it did not describe how it
will measure consistency and evaluate progress in reducing any
inconsistencies it may find. Instead, VA said that consistency is best
achieved through comprehensive training and communication among VA
components involved in the adjudication process and discussed efforts
underway to do so.
VA has two basic cash disability benefits programs. The compensation
program pays monthly benefits to eligible veterans who have
serviceconnected disabilities (injuries or diseases incurred or aggravated
while on active military duty). The payment amount is based on the
veteran*s degree of disability, regardless of employment status or level
of earnings. By contrast, the pension program assists permanently and
totally disabled wartime veterans under age 65 who have low incomes and
whose disabilities are not service- connected. 2 The payment amount is
determined on the basis of financial need.
VBA and the Board process and decide veterans* disability claims and
appeals on behalf of the Secretary. The claims process starts when
veterans submit claims to one of VBA*s 57 regional offices. (See app. I
for the overall flow of claims and appeals processing.) By law, regional
offices must assist veterans in supporting their claims. For example, for
a compensation claim, the regional office obtains records such as the
veteran*s existing service medical records, records of relevant medical
treatment or examinations provided at VA health- care facilities, and
other relevant records held by a federal department or agency. If
necessary, the regional office arranges a medical examination for the
claimant or obtains a medical opinion about the claim. The regional office
adjudicator then must
analyze the evidence for each claimed impairment (veterans claim an
average of about five impairments per claim);
2 Veterans age 65 or older do not have to be permanently and totally
disabled to become eligible for pension benefits, if they meet the income
and military service requirements. Background
Page 4 GAO- 02- 806 Veterans* Disability Benefits
determine whether each claimed impairment is service- connected (VA
grants service- connection for an average of about three impairments per
claim);
apply VA*s Rating Schedule which provides medical criteria for rating
the degree to which each service- connected impairment is disabling
(disability ratings can range from zero to 100 percent, in 10- percent
increments);
determine the overall disability rating that results from the
combination of service- connected impairments suffered by the veteran; and
notify the veteran of the decision. If a veteran disagrees with the
regional office*s decision, he or she begins the appeals process by
submitting a written Notice of Disagreement to the regional office. During
fiscal years 1999- 2000, the regional offices annually made an average of
about 616,000 decisions involving disability ratings, and veterans
submitted Notices of Disagreement in about 9 percent of these decisions.
Veterans can disagree with decisions for reasons other than the outright
denial of benefits that occurs, for example, in a compensation case when a
regional office decides an impairment claimed by a veteran is not service-
connected. The veteran also may believe the severity rating assigned to a
service- connected impairment is too low and ask for an increase in the
rating.
In response to a Notice of Disagreement, the regional office provides a
further written explanation of the decision, and if the veteran still
disagrees, the veteran may appeal to the Board. During fiscal years 1999-
2000, about 48 percent of the veterans who filed Notices of Disagreement
in decisions involving disability ratings went on to file appeals with the
Board. In fiscal year 2001, VBA began nationwide implementation of the
Decision Review Officer position in its regional offices. Now, before
appealing to the Board, a veteran may ask for a review by a Decision
Review Officer, who is authorized to grant the contested benefits based on
the same case record that the regional office relied on to make the
initial decision. VBA believes this process will result in fewer appeals
being filed with the Board.
Located in Washington, D. C., the Board is an administrative body whose
members are attorneys experienced in veterans* law and in reviewing
benefits claims. The Board*s members are divided into four decision teams,
with each team having up to 15 Board members and 61 staff attorneys. Each
team has primary responsibility for reviewing the appeals that originate
in an assigned group of regional offices. Board members*
Page 5 GAO- 02- 806 Veterans* Disability Benefits
decisions must be based on the law, regulations, precedent decisions of
the courts, and precedent opinions of VA*s General Counsel. 3 During the
Board*s appeals process, the veteran or the veteran*s representative may
submit new evidence and request a hearing.
During fiscal years 1999 and 2000, for all VA programs, the Board annually
decided an average of about 35,700 appeals, of which about 32,900 (92
percent) were disability compensation cases. The average appealed
compensation case contains three contested issues. As a result, in some
cases, the Board member may grant the requested benefits for some issues
but deny the requested benefits for others. During fiscal years 1999 and
2000, the Board in its initial decisions on appealed compensation cases
granted at least one of the requested benefits in about 24 percent of the
cases. In some instances, the Board member finds a case is not ready for a
final decision and returns (or remands) the case to the regional office to
obtain additional evidence and reconsider the veteran*s claim. During
fiscal years 1999 and 2000, respectively, the Board in its initial
decisions on appealed compensation cases remanded 38 percent and 34
percent of the cases. 4 After obtaining additional evidence for remanded
cases, if the regional office still denies the requested benefits, it
resubmits the case to the Board for a final decision.
If the Board denies benefits or grants less than the maximum benefit
available under the law, veterans may appeal to the U. S. Court of Appeals
3 According to VA, precedent opinions of the General Counsel are legal
opinions that interpret court decisions, laws, or regulations. These
opinions may apply such an interpretation to a certain set of facts, or
they may be used to implement an interpretation of law consistently across
VA. To the extent precedent opinions may contain any statements of VA
policy, such policy statements are not binding on the Board merely because
they appear in a precedent opinion. General Counsel precedent opinions
also are binding on regional office adjudicators.
4 Effective February 22, 2002, a new VA regulation allows the Board to
obtain evidence, clarify evidence, cure a procedural defect, or perform
most any other action essential for a proper appellate decision without
having to remand the appeal to the regional office. It also allows the
Board to consider additional evidence without having to refer the evidence
to the regional office for initial consideration and without having to
obtain the appellant*s waiver. Under this new regulation, the Board
expects that the proportion of appeals remanded to regional offices will
decline to about 12 percent. For those cases that previously would have
been remanded to regional offices, the Board provides information to VBA
on the reasons for actions taken by the Board itself under this new
regulation, such as obtaining new evidence or clarifying evidence.
Page 6 GAO- 02- 806 Veterans* Disability Benefits
for Veterans Claims. 5 The court is not part of VA and not connected to
the Board. During fiscal years 1999 and 2000, veterans filed appeals with
the court in an estimated 10 percent of the Board*s decisions. Unlike the
Board, the court does not receive new evidence, but considers the Board*s
decision, briefs submitted by the veteran and VA, oral arguments, if any,
and the case record that VA considered and that the Board had available.
The court may dismiss an appeal on procedural grounds such as lack of
jurisdiction, but in the cases decided on merit, the court may affirm the
Board*s decision (deny benefits), reverse the decision (grant benefits),
or remand the decision back to the Board for rework. During fiscal years
1999 and 2000, the court annually decided on merit an average of about
1,800 appealed Board decisions, and in about 67 percent of these cases,
the court remanded or reversed the Board*s decisions in part or in whole.
6 Under certain circumstances, a veteran who disagrees with a decision of
the court may appeal to the U. S. Court of Appeals for the Federal Circuit
and then to the Supreme Court of the United States.
In fiscal year 1998, the Board established the first quantitative quality
assurance program to evaluate and score the accuracy of its decisions and
to collect data to identify areas where the quality of decision- making
needs improvement. The accuracy measure used by the Board understates its
true accuracy rate because the Board*s accuracy rate calculations include
certain deficiencies that would not result in either a reversal or a
remand by the court. Even so, the Board*s quality assurance program does
not capture certain data that potentially could help improve the quality
of the Board*s decisions. Such data include information identifying the
specific medical issues involved in cases where a disability decision was
judged as being in error. Having such data could enhance the Board*s
ability to target training for its decision makers.
5 Before appealing to the court, a veteran may ask the Board itself to
review any previous Board decision on the basis of *clear and unmistakable
error.* During fiscal years 1999 and 2000, the Board revised a total of
only 23 of its decisions due to clear and unmistakable error. VA
regulations state that clear and unmistakable error is a specific and rare
kind of error, of fact or law, that when called to the attention of later
reviewers compels the conclusion, with which reasonable minds could not
differ, that the result would have been manifestly different if not for
the error. Generally, either the correct facts, as they were known at the
time, were not before the Board, or the statutory and regulatory
provisions in effect at the time were not correctly applied.
6 The court*s Annual Reports did not disaggregate data on remands and
reversals, but according to the Board, the court reversed Board decisions
in whole or in part in only about 1.6 percent of the cases. The Board
Understates Its Accuracy and Does Not Capture Certain Data for Improving
Quality
Page 7 GAO- 02- 806 Veterans* Disability Benefits
On the basis of the results of the quality assurance program it
established in fiscal year 1998, the Board estimated that 89 percent of
its decisions were accurate (or *deficiency- free*). Using these results
as a baseline, VA established performance accuracy goals for the Board.
One of the Board*s strategic performance goals is to make deficiency- free
decisions 95 percent of the time. To calculate its estimated overall
accuracy rate, the Board does quality reviews of selected Board decisions.
7 We reviewed the Board*s methods for selecting random samples and
calculating accuracy rates and concluded that the number of decisions
reviewed by the Board was sufficient to meet the Board*s goal for
statistical precision in estimating its accuracy rate. 8 However, we
brought to the Board*s attention some issues that caused the Board to fall
short of proper random sampling and accuracy rate calculation methods,
such as not ensuring that decisions made near the end of the fiscal year
are sampled or that the results from quality reviews are properly weighted
in the accuracy rate calculation formula. We do not believe the overall
accuracy rate reported by the Board for fiscal year 2001 would have been
materially different if these methodological issues had been corrected
earlier; however, if not corrected, these issues potentially could lead to
misleading accuracy rate calculations in the future. The Board agreed in
principle to correct these issues. As of June 2002, the Board had not yet
instituted corrective actions.
According to VA*s performance reports, the Board has come close but has
not achieved its annual interim goals for accuracy (see table 1). 9
However, in calculating its reported accuracy rates, the Board includes
deficiencies that are not *substantive** that is, they would not be
expected to result in
7 During our review, we brought to the Board*s attention certain issues
regarding the compliance of its quality assurance program with the
governmental internal control standard calling for separation of key
duties and the governmental performance audit standard calling for
organizational independence for agency employees who review and evaluate
program performance. These issues arose because certain Board members who
were directly involved in deciding veterans* appeals were also involved in
reviewing the accuracy of such decisions. Effective May 2002, the Board
took corrective actions to resolve these issues; now, all quality reviews
from which accuracy rates are determined are done by persons not directly
involved in deciding veterans* appeals.
8 The Board*s goal is that VA can have 95 percent confidence that the
Board*s true accuracy rate is no more than 5 percentage points higher or
lower than the estimated accuracy rate. 9 The Government Performance and
Results Act of 1993 requires VA, as well as other federal agencies, to
clearly define its mission, set goals, measure performance, and submit to
the Congress annual performance plans and annual reports on its success in
achieving program performance goals. The Board Established Its
First Quantitative Quality Assurance Program in 1998
The Board*s Actual Level of Accuracy Is Higher Than Reported
Page 8 GAO- 02- 806 Veterans* Disability Benefits
either a remand by the court or a reversal by the court. 10 Consequently,
the reported accuracy rates understate the Board*s level of accuracy that
would result if only substantive deficiencies were counted in the
calculation.
Table 1: Board*s Accuracy Rate Goals and Performance
In percentages
Accuracy rate) Accuracy Measure FY1999 FY 2000 FY 2001 FY 2002 FY 2003
Strategic goal
Accuracy rate goal 90.5 91.5 90 91 92 95 Reported performance (estimated)
84 86 87 a a a
a No data available. Source: VA budget submissions and performance plans,
fiscal years 2000- 2003.
Under its quality assurance program, the Board*s quality reviewers assess
the accuracy of selected decisions on the basis of six critical areas (see
table 2). One error (or deficiency) in any of these six areas means that a
decision fails the quality test. However, according to the Board, all six
areas would include certain deficiencies that are not substantive. In
particular, according to the Board, most deficiencies in the *format*
category are not substantive. In fiscal year 2001, the format category
accounted for about 38 percent of all recorded deficiencies. At our
request, the Board recalculated its accuracy rate for fiscal year 2001,
excluding format deficiencies, and the resulting accuracy rate was 92
percent, as compared with the reported accuracy rate of 87 percent.
Excluding all other nonsubstantive deficiencies presumably would have
resulted in an even higher accuracy rate. In contrast with the Board,
beginning in fiscal year 2002, VBA no longer includes nonsubstantive
deficiencies in its accuracy rate calculations; however, it continues to
monitor them. VBA took this action based on a recommendation by the 2001
VA Claims Processing Task Force, which said that mixing serious errors
with less significant deficiencies can obscure what is of real concern.
10 The court may set aside the Board's findings of fact that are "clearly
erroneous." A finding is clearly erroneous when, although there is
evidence to support it, the reviewing court on the entire evidence is left
with the definite and firm conviction that a mistake has been committed.
If there is a plausible basis in the record for the Board's factual
determinations, the court cannot overturn them, even if it might not have
reached the same determinations.
Page 9 GAO- 02- 806 Veterans* Disability Benefits
Table 2: Board*s Quality Review Criteria Six critical areas Criteria
Issues Did the decision maker identify and address all issues that were
explicitly expressed in, or could be inferred from, the evidence? Evidence
Did the decision maker account for all evidence, including the evidence in
favor of and the evidence
against the veteran*s claim? Laws and regulations Did the decision maker
cite and set forth all applicable laws and regulations? Reasons and bases
Did the decision maker fully and accurately coordinate the facts
(evidence) and laws and regulations with
each other and clearly explain how he or she reached the decision? Due
process Did the decision maker address all technical aspects of due
process such as accounting for jurisdictional
problems, hearing requests, and possible remand reasons? Format Did the
decision maker write the decision so that it meets the basic requirements
of format such as correct
grammar, spelling, decision structure, and requirements of statute?
Source: Board of Veterans* Appeals, 2002.
The Board*s quality review program subdivides the six critical areas shown
in table 2 into 31 subcategories. For example, if a quality reviewer
classifies an error as stemming from *reasons and bases,* the reviewer
must then indicate whether the error was due to misapplying legal
authority, failing to apply appropriate legal authority, using an
incorrect standard of proof, or providing an inadequate explanation for
the decision. This information is recorded in the Board*s quality review
database, providing the Board with data that can be analyzed to identify
training needed to improve quality.
However, the Board does not record in its quality review database any
information on the specific issue that prompted the appeal (such as
whether a disability is service- connected) or the specific medical
impairment to which an error is related. For example, a quality reviewer
might find an error in a Board decision for an appeal that involved four
separate medical impairments* two for which the veteran had requested
service connection and two others for which he had requested a disability
rating increase. On the basis of information that the quality review
database currently captures, however, the Board could not determine which
of the four impairments the error was related to, nor could the Board
determine whether the error was related to a request for serviceconnection
or an increased disability rating.
This is not the case, however, for Board decisions remanded by the Court
of Appeals for Veterans Claims. For these cases, the Board maintains a
separate database with information on the reasons that the court remands
decisions back to the Board for rework. For each issue that the court
remands in a compensation case, the Board records in the database such The
Board*s Quality Review
Program Does Not Capture Certain Data That Could Help Improve Quality
Page 10 GAO- 02- 806 Veterans* Disability Benefits
information as: (1) whether the issue involved a request for
serviceconnection or an increased rating, (2) the diagnostic code of the
impairment involved in each issue, and (3) the reason for the remand.
According to Board officials, being able to analyze the court*s reasons
for remands by type of decisional issue and type of impairment enhances
the Board*s ability to reduce remands from the court through appropriate
training.
VBA and the Board recognize that in some cases, different adjudicators
reviewing the same evidence can make differing judgments on the meaning of
the evidence, without either decision necessarily being wrong. In such
cases, VBA and the Board instruct quality reviewers not to record an
error. A hypothetical case provided by the Board furnishes an example. In
this case, a veteran files a claim in 1999 asserting he suffered a back
injury during military service but did not seek medical treatment at that
time. One of the veteran*s statements says he injured his back during
service in 1951, but another says he injured his back in 1953. An
adjudicator may find that this discrepancy in dates adversely affects the
claimant*s credibility about whether an injury actually occurred in
service, but the quality reviewer may consider the discrepancy to be
insignificant. Where such judgments are involved, the Board*s and VBA*s
quality review programs recognize that variations in judgment are to be
expected and are acceptable as long the degree of variation is within
reason. (App. II provides other examples of difficult judgments that could
result in decision- making variations and explains VA*s *benefit- of- the-
doubt* rule.)
The Board and VBA, however, differ in their approaches to collecting
information about cases where this type of variation occurs. In such
instances, the Board*s quality reviewers note why they believe an
alternative decision could have been made and send the explanation to the
deciding Board member. However, they do not enter any of this information
in the quality review database. In contrast, VBA recently instructed its
quality reviewers to enter such information in the VBA quality review
database, even though no error is recorded in the database. VBA believes
that by identifying and analyzing cases in which quality reviewers
believed the adjudicator*s judgment was pushing against the boundary of
reasonableness, it potentially can identify opportunities to improve the
quality of decision making by improving training.
Page 11 GAO- 02- 806 Veterans* Disability Benefits
Even though evidence suggests decision making across regional office and
Board adjudicators may not be consistent, VA does not systematically
assess decision making consistency to determine the degree of variation
that occurs for specific impairments and to provide a basis for
identifying steps that could be taken, if considered necessary, to reduce
such variation. In its 2003 performance plan, VA acknowledged that
veterans are concerned about the consistency of disability claims
decisions across the 57 regional offices. In a nationwide comparison, VBA
projected in its fiscal year 2001 Annual Benefits Report that the average
compensation payments per disabled veteran in fiscal year 2002 would range
from a low of $5,783 in one state to a high of $9,444 in another state.
According to a VBA official, this disparity in average payments per
veteran might be due in part to demographic factors such as differences in
the average age of veterans in each state. However, this disparity in
average payments per veteran also raises the possibility that when
veterans in the same age group submit claims for similar medical
conditions, the regional office in one state may tend to give lower
disability ratings than the regional office in another state.
Indeed, in 1997, the National Academy of Public Administration reviewed
disability claims processing and said VA needed to identify the degree of
decision- making variation expected for specific medical issues, set
consistency standards, and measure the level of consistency as part of the
quality review process or through testing of control cases in multiple
regional offices. Furthermore, in 2001, VA*s Claims Processing Task Force
said there was an apparent lack of uniformity among regional offices in
interpreting and complying with directives from VA headquarters and that
VA*s regulations and the procedures manual for regional offices were in
dire need of updating. The task force concluded that there was no
reasonable assurance that claims decisions would be made as uniformly and
fairly as possible to the benefit of the veteran. Even though such
concerns and issues exist, VA does not systematically assess the
decisionmaking consistency of regional office adjudicators.
Similarly, VA does not assess consistency between decisions made by
regional offices and the Board even though evidence suggests this issue
may warrant VA*s attention. Because veterans may submit new evidence
during the appeals process, one might assume that the Board generally
grants benefits denied by regional offices due to the impact of such new
evidence. However, an analysis in 1997 of about 50 decisions in which the
Board had granted benefits previously denied by regional offices yielded a
different viewpoint. Staff from both VBA and the Board reviewed these
cases and concluded that most of these Board decisions to grant benefits
VA Does Not
Systematically Assess Consistency of Decision Making
Page 12 GAO- 02- 806 Veterans* Disability Benefits
had been based on the same evidence that the regional offices had
considered in reaching their decisions to deny benefits. The reviewers
characterized the reason for the Board members* decisions to grant
benefits as a difference of opinion between the Board members and regional
office adjudicators in the weighing of evidence. Furthermore, even in
remanded compensation cases for which regional offices have obtained new
evidence in accordance with the Board*s remand instructions and then again
denied the benefits, the Board generally has granted benefits in about 26
percent of these cases after they have been resubmitted for a final
decision. This seems to indicate that, in these particular cases, Board
members in some way differed with regional office adjudicators on the
impact of the new evidence obtained by the regional offices before
resubmitting the remanded cases to the Board.
Available evidence also provides indications that the issue of variations
in decision making among the Board members themselves may warrant VA*s
attention in studies of consistency. Historically, there have been
variances in the rates at which the Board*s four decision teams have
remanded decisions to regional offices for rework. No systematic study has
been done to explain the variances in remand rates. Board officials said
that it is their perception that the remand rates vary among the Board*s
decision teams because the quality of claims processing varies among the
regional offices for which each team is responsible.
Similar concerns about consistency of claims adjudication in the Social
Security Administration (SSA) have prompted SSA to begin taking steps to
assess consistency issues in its disability program. As we reported in
1997, SSA*s primary effort to improve consistency has focused on
decisionmaking variations between its initial and appellate levels. 11 To
gather data on variations between these two levels, SSA instituted a
system in 1993 under which it selects random samples of final decisions
made by administrative law judges and reviews the entire decisional
history of each case at both the initial and appellate levels. The
reviewers examine adjudicative and procedural issues to address broad
program issues such as whether a claim could have been allowed earlier in
the process. Data captured through this system have provided a basis for
taking steps to clarify decision- making instructions and provide training
designed to
11 U. S. General Accounting Office, Social Security Disability: SSA Must
Hold Itself Accountable for Continued Improvement in Decision- Making,
GAO/ HEHS- 97- 102 (Washington, D. C.: Aug. 12, 1997).
Page 13 GAO- 02- 806 Veterans* Disability Benefits
improve consistency between the initial and appellate levels. However, no
systematic evaluations have been done to determine the effectiveness of
these actions. In its January 2001 disability management plan, SSA said
that it needed to take further steps to promote uniform and consistent
disability decisions across all geographic and adjudicative levels. 12
Opportunities exist to improve the quality of the Board*s reporting of
accuracy and decision making. The Board includes nonsubstantive
deficiencies in its accuracy rate calculation. By doing so, the Board may
be obscuring what is of real concern. In addition, the Board*s quality
assurance database does not capture data on specific medical disability
issues related to the reasons for errors found in Board decisions. Also,
in contrast with VBA, the Board*s quality assurance program does not
collect information on cases in which quality reviewers do not charge
errors but have differences of opinion with judgments made by Board
members. We believe that analysis of such data could lead to improvements
in quality through improved training or by clarifying regulations,
procedures, and policies.
Furthermore, because variations in decision making are to be expected due
to the difficult judgments that adjudicators often must make, one must ask
the questions: For a given medical condition, how much variation in
decision making exists and does the degree of variation suggest that VA
should take steps to reduce the level of variation? VA, however, does not
assess variation in decision making. None of the quality review efforts of
either VBA or the Board are designed to systematically assess the degree
to which veterans with similar medical conditions and circumstances may be
receiving different decisional outcomes or to help identify steps that
could reduce such variation if necessary. Without ongoing systematic
assessments of consistency across the continuum of decision making, VA
cannot adequately assure veterans that they can reasonably expect to
12 Since 1994, SSA has recognized the need to focus more attention on the
agency*s overall quality assurance program; however, SSA had made little
progress in this effort as of early 2002. Since then, the Commissioner of
SSA has appointed a senior manager for quality who reports directly to the
Commissioner and who is responsible for developing a proposal to establish
a quality- oriented approach to all SSA business processes. See U. S.
General Accounting Office, Social Security Disability: Disappointing
Results from SSA*s Efforts to Improve the Disability Claims Process
Warrant Immediate Attention, GAO- 02- 322 (Washington, D. C.: Feb. 27,
2002) and Social Security Disability: Efforts to Improve Claims Process
Have Fallen Short and Further Action is Needed, GAO- 02- 826T (Washington,
D. C.: June 11, 2002). Conclusions
Page 14 GAO- 02- 806 Veterans* Disability Benefits
receive consistent treatment of their claims across all decision- making
levels in VA.
We recognize that our recommendations will have to be implemented within
the context of VA*s current major efforts to reduce a large and persistent
backlog of disability claims and appeals and to reduce the average
processing time. Nevertheless, we believe it is critical that VA take the
necessary steps to support improvements in training and in regulations,
procedures, or policies that could enhance the quality of disability
decision making across the continuum of adjudication and to help provide
adequate assurance to veterans that they will receive consistent and fair
decisions as early as possible in the process. Indeed, maintaining and
improving quality should be of paramount concern while implementing a
major effort to reduce backlogs and processing time.
Accordingly, we recommend that the Secretary of VA direct the Chairman of
the Board of Veterans* Appeals to:
Revise the quality assurance program so that, similar to VBA, the
calculation of accuracy rates will take into account only those
deficiencies that would be expected to result in a reversal of a Board
decision by the U. S. Court of Appeals for Veterans Claims or result in a
remand by the court.
Revise the Board*s quality assurance program to record information in
the quality review database that would enable the Board to systematically
analyze case- specific medical disability issues related to specific
errors found in Board decisions in the same way that the Board is able to
analyze the reasons that the court remands Board decisions.
Monitor the experience of VBA*s quality assurance program in collecting
and analyzing data on cases in which VBA*s quality reviewers do not record
errors but have differences of opinion with regional office adjudicators
in the judgments made to reach a decision. If VBA finds that the analysis
of such data helps identify training that can improve the quality of
decision making, the Board should test such a process in its quality
assurance program to assess whether it would enable the Board to identify
training that could improve the quality of Board decisions.
We also recommend that the Secretary direct the Under Secretary for
Benefits and the Chairman of the Board of Veterans* Appeals to jointly
Recommendations for
Executive Action
Page 15 GAO- 02- 806 Veterans* Disability Benefits
establish a system to regularly assess and measure the degree of
consistency across all levels of VA adjudication for specific medical
conditions that require adjudicators to make difficult judgments. For
example, VA could develop sets of hypothetical claims for specific medical
issues, distribute such hypothetical claims to multiple adjudicators at
all decision- making levels, and analyze variations in outcomes for each
medical issue. Such a system should provide data to determine the degree
of variation in decision making and provide a basis to identify ways, if
considered necessary, to reduce such variation through training or
clarifying and strengthening regulations, procedures, and policies. Such a
system should also assess the effectiveness of actions taken to reduce
variation. If departmental consistency reviews reveal any systematic
differences among VA decision makers in the application of disability law,
regulations, or court decisions, the Secretary should, to the extent that
policy clarifications by VBA cannot resolve such differences, direct VA*s
General Counsel to resolve these differences through precedent legal
opinions if possible.
We received written comments on a draft of this report from VA (see app.
III). In its comments, VA concurred fully or in principle with our
recommendations. With regard to our first recommendation, VA said that the
Board intends to revise its quality review system to count only
substantive errors for computational and benchmarking purposes but will
continue to track all errors. On the basis of VA*s comments, we also
modified the report to accurately reflect the standard of review employed
by the U. S. Court of Appeals for Veterans Claims in reviewing Board
decisions. With regard to our second recommendation, VA said that it would
use its Veterans Appeals Control Locator System to gather information on
case- specific medical disability issues related to specific errors found
in Board decisions. VA questioned our basis for concluding that tracking
such information will yield useful data for improving the adjudication
system. As stated in the draft report, we based our recommendation on the
fact that the Board has already concluded that such information is
beneficial for analyzing the reasons for remands from the Court of Appeals
for Veterans Claims. With regard to our third recommendation, VA said
representatives of the Board and VBA will meet so that a system may be
established for the Board to access and review VBA*s methodology for
assessing, reporting, and evaluating instances of
*difference of opinion* between the quality reviewer and the decision
maker. Agency Comments
and Our Evaluation
Page 16 GAO- 02- 806 Veterans* Disability Benefits
In its comments, VA concurred in principle with our fourth recommendation.
VA agreed that consistency is an important goal and acknowledged that it
has work to do to achieve it. However, VA was silent on how it would
measure consistency for specific medical conditions that require
adjudicators to make difficult judgments. Instead, VA described the kinds
of actions underway that it believes will generally reduce inconsistency.
While we support these efforts, we maintain that without a way to evaluate
and measure consistency, VA will be unable to determine the extent to
which such efforts actually improve consistency of decisionmaking across
all levels of VA adjudication now and over time. Neither will VA have
information needed to identify ways to reduce decisionmaking variations
for specific medical conditions, if considered necessary.
As agreed with your office, unless you publicly announce its contents
earlier, we plan no further distribution of this report until 30 days
after its issue date. At that time, we will send copies of this report to
the Secretary of the Department of Veterans Affairs, appropriate
congressional committees, and other interested parties. We will also make
copies available to others upon request. In addition, the report will be
available at no charge on the GAO Web site at http:// www. gao. gov.
If you have questions about this report, please call me on (202) 512- 7101
or Irene Chu on (202) 512- 7102. Other key contributors were Ira Spears,
Steve Morris, Patrick diBattista, and Mark Ramage.
Sincerely yours, Cynthia A. Bascetta Director, Education, Workforce,
and Income Security Issues
Appendix I: Steps in the Disability Claims and Appeals Process
Page 17 GAO- 02- 806 Veterans* Disability Benefits
Note: We did not include claims and appeals data for fiscal year 2001 in
the basis for estimating the disposition of the 100,000 cases because of
anomalies related in whole or in part to the implementation of the
Veterans Claims Assistance Act of 2000. These fiscal year 2001 anomalies
included a substantial increase in the regional offices* inventory of
claims to be completed and substantial increases in the proportion of
appeals remanded by the Board and the Court. See U. S. General Accounting
Office, Veterans* Benefits: Despite Recent Improvements, Meeting Claims
Processing Goals Will Be Challenging, GAO- 02- 645T (Washington, D. C.:
Apr. 26, 2002). a The estimated disposition by VA*s regional offices of
the 100,000 claims (in boxes 1 and 2) is based
on data for claims involving disability ratings for fiscal years 1997 to
2000. During those years, veterans submitted Notices of Disagreement in
about 9 percent of the regional office decisions and went on to file
appeals with the Board in about 40 percent of the cases in which they had
submitted such notices. b Decisions appealed by veterans to the Board
reach one of the following dispositions before veterans can appeal to the
U. S. Court of Appeals for Veterans Claims: the Board makes an initial
decision to grant or deny requested benefits (boxes 4 and 5); the regional
office grants requested benefits in cases remanded by the Board for
further development and reconsideration (box 7); the Board grants or
denies benefits in remanded cases that regional offices resubmit to the
Board because the regional offices denied the requested benefits after
developing further evidence (boxes 4 and 5); the veteran withdraws his or
her appeal while the case is in remand status at the regional office (box
8); or the regional office closes the case while in remand status because
the veteran fails to respond to requests for information needed for the
appeal to proceed (box 8). Therefore, the estimated disposition the 3, 657
appealed cases is accounted for in boxes 4, 5, 7, and 8 (1,153 + 1,956 +
339 + 209 = 3,657).
Appendix I: Steps in the Disability Claims and Appeals Process
Illustration of the estimated disposition of 100,000 hypothetical
compensation claims.
Veterans Benefits Administration 57 Regional Offices Decide Claims and
Notify Veterans of Decisions (estimated disposition of 100,000
compensation claims filed with regional offices)
1. Veterans either agree with regional offices decisions or take no
further action in 90,880 cases. a 2. Veterans submit Notices of
Disagreement to regional
offices in 9,120 cases. In 3,657 of these cases, veterans go on to file
appeals with the Board. a
3. Board remands 1,311 cases to regional offices to develop further
evidence and reconsider their decisions (Board remands 211 of these cases
twice). c
4. Board grants at least one requested benefit in 1,153 cases (Board makes
269 of these grants after regional offices resubmit remanded cases). d
Board of Veterans' Appeals Board Members Review Regional Office Decisions
Appealed by Veterans (estimated disposition of 3,657 compensation cases
appealed to Board) b
5. Board denies all benefits in 1,956 cases (Board makes 494 of these
denials after regional offices resubmit remands). d
6. Regional offices obtain more evidence but deny requested benefits in
839 cases and resubmit these cases to the Board for a final decision (of
the 211 remanded twice, regional offices deny benefits in 135 and resubmit
them to Board). d
7. Regional offices obtain more evidence and grant requested benefits in
339 cases (47 of these 339 grants occur after the second remand). d
8. Veterans withdraw or regional offices close 209 cases (28 of these 209
withdrawals or closures occur after second remand). e
9. Veterans appeal 307 cases to U. S. Court of Appeals for Veterans
Claims. f
U. S. Court of Appeals for Veterans Claims Court Reviews Board Decisions
Appealed by Veterans (estimated disposition of 307 compensation cases
appealed to the court) f
10. Court dismisses 74 cases on procedural grounds.
11. Court affirms Board decisions in whole in 77 cases (all requested
benefits denied).
12. In whole or in part, Court reverses Board decisions (grants requested
benefits) or remands Board decisions in 156 cases. g
13. Veterans appeal 25 Court decisions to the U. S. Court of Appeals for
the Federal Circuit. 14. The Board or the regional offices grant or deny
benefits in
the cases remanded by the court. h
Appendix I: Steps in the Disability Claims and Appeals Process
Page 18 GAO- 02- 806 Veterans* Disability Benefits
c The estimate of 1, 311 remanded cases (in box 3) is based on Board data
for fiscal years 1999 and 2000. d On the basis of Board data for fiscal
years 1999 and 2000, in its initial decisions on appealed
compensation cases, the Board: (1) granted at least one of the requested
benefits in about 24 percent of the cases, (2) denied all requested
benefits in about 40 percent of the cases, and (3) remanded about 36
percent of the cases to regional offices for rework. After obtaining the
additional evidence required by the Board for remanded cases, the regional
offices granted requested benefits in about 22 percent of the remanded
cases and denied requested benefits in 64 percent of the cases. After
regional offices resubmitted denied cases to the Board for a final
decision, the Board granted at least one of the requested benefits in
about 26 percent of the cases, denied all benefits in about 49 percent,
and remanded about 25 percent once again to regional offices for further
rework. For this illustration, we assumed that the Board did not remand a
case more than two times. e On the basis of Board data for fiscal years
1999 and 2000, appellants withdrew about 13 percent of remanded
compensation cases while at the regional offices, and the regional offices
closed less than 1 percent of the remanded cases because appellants did
not respond to requests for information needed to proceed with the appeal.
f The estimate of 307 cases appealed to the U. S. Court of Appeals for
Veterans Claims (in box 9), the
court*s estimated disposition of these 307 cases (in boxes 10, 11, 12),
and the estimated number of decisions appealed to the U. S. Court of
Appeals for the Federal Circuit (in box 13) are based on fiscal years 1999
and 2000 data from the court*s annual reports. g The court*s annual
reports did not disaggregate data on reversals and remands. According to
the
Board, the court reversed Board decisions in whole or in part in only
about 1. 6 percent of the cases during fiscal years 1999 and 2000. h The
Board did not have any data that would provide a basis for estimating the
number of grants and denials made by the Board or regional offices in
cases remanded by the court (in box 14). Source: Prepared by GAO using
data from VBA, Board of Veterans* Appeals, and the U. S. Court of Appeals
for Veterans Claims.
Appendix II: Board of Veterans* Appeals Illustrations of Difficult
Judgments Resulting in Decision- Making Variations
Page 19 GAO- 02- 806 Veterans* Disability Benefits
Adjudicator*s task Examples of difficult judgments
Assessing credibility of sources of evidence To be granted benefits for
post- traumatic stress disorder, a veteran*s claim must have
credible evidence that a stressor occurred during military service. Assume
the record shows a claimant served in Vietnam as a supply specialist, and
he identified mortar attacks as a stressor. Reports prepared by his
military unit in Vietnam indicate a single enemy mortar attack occurred
where the claimant was stationed. The claimant*s testimony was vague about
the number and the time of the attacks. One adjudicator may rely on the
unit*s reports and conclude the claimant engaged in combat and is entitled
to have his lay statements accepted without further corroboration as
satisfactory evidence of the in- service stressor. Another adjudicator may
conclude that the claimant is not credible as to exposure to enemy fire
and require other credible supporting evidence that the in- service
stressor actually occurred. Evaluating and assigning weight to evidence
Assume an appeal for either service connection or a higher disability
rating has two
conflicting medical opinions, one provided by a medical specialist who
reviewed the claim file but did not actually examine the veteran and a
second opinion provided by a medical generalist who reviewed the file and
examined the veteran. One adjudicator could assign more weight to the
specialist*s opinion, while another could find the generalist*s opinion to
be more persuasive. Thus, depending on which medical opinion is given more
weight, one adjudicator could grant the claim and the other deny it. Yet,
a third adjudicator could find both opinions to be equally probative and
conclude that VA*s *benefit- of- the- doubt* rule requires that he decide
in favor of the veteran*s request for either service- connection or a
higher disability rating. Under the benefit- of- the- doubt rule, if an
adjudicator concludes that there is an approximate balance between the
evidence for and the evidence against a veteran*s claim, the adjudicator
must decide in favor of the veteran. Applying subjective standards in VA*s
Rating Schedule The Rating Schedule does not provide objective criteria
for rating the degree to which
certain spinal impairments limit a claimant*s motion. The adjudicator must
assess the evidence and draw a conclusion as to whether the limitation of
motion falls into one of three severity categories: *slight, moderate, or
severe.* Similarly, in assessing the severity of incomplete paralysis, the
adjudicator must draw a conclusion as to whether the veteran*s incomplete
paralysis falls into one of three severity categories: *mild,
moderate, or severe.* In each case, each severity category in itself
encompasses a range of severity, and the judgment as to whether a
claimant*s condition is severe enough to cross over from one severity
range into the next could vary in the minds of different adjudicators.
The Rating Schedule provides a formula for rating the severity of a
veteran*s occupational and social impairment due to a variety of mental
disorders. However, the formula actually is a nonquantitative,
behaviorally oriented framework for guiding adjudicators in making
judgments and drawing conclusions as to which of the following
characterizations best describes the degree to which a claimant is
occupationally and socially impaired: (1) totally impaired; (2) deficient
in most areas such as work, school, family relations, judgment, thinking,
or mood; (3) reduced reliability and productivity; (4) occasional decrease
in work efficiency and intermittent periods of inability to perform
occupational tasks; (5) mild or transient symptoms that decrease work
efficiency and ability to perform occupational tasks only during periods
of significant stress or symptoms can be controlled by continuous
medication, and (6) not severe enough to interfere with occupational or
social functioning or to require continuous medication.
Source: Board of Veterans* Appeals.
Appendix II: Board of Veterans* Appeals Illustrations of Difficult
Judgments Resulting in Decision- Making Variations
Appendix III: Comments from the Department of Veterans Affairs
Page 20 GAO- 02- 806 Veterans* Disability Benefits
Appendix III: Comments from the Department of Veterans Affairs
Appendix III: Comments from the Department of Veterans Affairs
Page 21 GAO- 02- 806 Veterans* Disability Benefits
Appendix III: Comments from the Department of Veterans Affairs
Page 22 GAO- 02- 806 Veterans* Disability Benefits
Appendix III: Comments from the Department of Veterans Affairs
Page 23 GAO- 02- 806 Veterans* Disability Benefits (130085)
The General Accounting Office, the investigative arm of Congress, exists
to support Congress in meeting its constitutional responsibilities and to
help improve the performance and accountability of the federal government
for the American people. GAO examines the use of public funds; evaluates
federal programs and policies; and provides analyses, recommendations, and
other assistance to help Congress make informed oversight, policy, and
funding decisions. GAO*s commitment to good government is reflected in its
core values of accountability, integrity, and reliability.
The fastest and easiest way to obtain copies of GAO documents at no cost
is through the Internet. GAO*s Web site (www. gao. gov) contains abstracts
and fulltext files of current reports and testimony and an expanding
archive of older products. The Web site features a search engine to help
you locate documents using key words and phrases. You can print these
documents in their entirety, including charts and other graphics.
Each day, GAO issues a list of newly released reports, testimony, and
correspondence. GAO posts this list, known as *Today*s Reports,* on its
Web site daily. The list contains links to the full- text document files.
To have GAO e- mail this list to you every afternoon, go to www. gao. gov
and select *Subscribe to daily E- mail alert for newly released products*
under the GAO Reports heading.
The first copy of each printed report is free. Additional copies are $2
each. A check or money order should be made out to the Superintendent of
Documents. GAO also accepts VISA and Mastercard. Orders for 100 or more
copies mailed to a single address are discounted 25 percent. Orders should
be sent to:
U. S. General Accounting Office 441 G Street NW, Room LM Washington, D. C.
20548
To order by Phone: Voice: (202) 512- 6000 TDD: (202) 512- 2537 Fax: (202)
512- 6061
Contact: Web site: www. gao. gov/ fraudnet/ fraudnet. htm E- mail:
fraudnet@ gao. gov Automated answering system: (800) 424- 5454 or (202)
512- 7470
Jeff Nelligan, managing director, NelliganJ@ gao. gov (202) 512- 4800 U.
S. General Accounting Office, 441 G Street NW, Room 7149 Washington, D. C.
20548 GAO*s Mission
Obtaining Copies of GAO Reports and Testimony
Order by Mail or Phone To Report Fraud, Waste, and Abuse in Federal
Programs
Public Affairs
*** End of document. ***