Veterans' Benefits: Quality Assurance for Disability Claims
Processing (23-AUG-01, GAO-01-930R).
In recent years, the accuracy of claims processing in the
Veterans Benefits Administration's (VBA) compensation and pension
programs has been the subject of concern and attention within the
Department of Veterans Affairs, Congress, and veteran's service
organizations. In October 1998, VBA implemented its Systematic
Technical Accuracy Review (STAR) system to improve the
measurement of the accuracy of claims processing. As designed by
VBA, the STAR system determines accuracy rates for (1) the nation
as a whole, (2) each VBA Service Delivery Network, and (3) each
VBA regional office. In March 1999, GAO reported that the
claims-processing accuracy rates for individual regional offices
are determined through quality reviews conducted by regional
office staff, who themselves are involved in processing claims
and who report to managers responsible for claims processing.
Such an arrangement does not meet the governmental internal
control standard calling for segregation of key duties, or the
performance audit standard calling for organizational
independence for those who review and evaluate program
performance. GAO recommended that VBA correct these reported
shortcomings to comply with these standards. Subsequently,
Congress enacted a provision in the Veteran's Millennium Health
Care and Benefits Act requiring VBA to have a quality assurance
program that meets both generally applicable government standards
for independence and internal controls for quality reviews of
government performance and results. Effective October 1, 2001,
VBA plans to implement a modification to the STAR system that
would bring it into compliance with the standards on segregation
of duties and organizational independence. VBA plans that all
STAR reviews will be conducted by staff of the Compensation and
Pension Service who themselves do not process claims and do not
report to managers responsible for claims processing. Staff of
VBA's regional offices would no longer be responsible for
conducting the STAR reviews that determine regional office
accuracy rates. If this modification is implemented as planned,
the STAR system would comply with the act's requirements
regarding segregation of duties and organizational independence.
-------------------------Indexing Terms-------------------------
REPORTNUM: GAO-01-930R
ACCNO: A01507
TITLE: Veterans' Benefits: Quality Assurance for Disability
Claims Processing
DATE: 08/23/2001
SUBJECT: Claims processing
Internal controls
Quality control
Veterans
Veterans benefits
VA Systematic Individual Performance
Assessment
VA Systematic Technical Accuracy Review
System
******************************************************************
** This file contains an ASCII representation of the text of a **
** GAO Testimony. **
** **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced. Tables are included, but **
** may not resemble those in the printed version. **
** **
** Please see the PDF (Portable Document Format) file, when **
** available, for a complete electronic file of the printed **
** document's contents. **
** **
******************************************************************
GAO-01-930R
GAO- 01- 930R VBA Disability Claims Processing United States General
Accounting Office
Washington, DC 20548
August 23, 2001 The Honorable Lane Evans Ranking Democratic Member Committee
on Veterans? Affairs House of Representatives
Subject: Veterans? Benefits: Quality Assurance for Disability Claims
Processing Dear Mr. Evans: In recent years, the accuracy of claims
processing in the Veterans Benefits Administration's (VBA) compensation and
pension programs has been the subject of concern and attention within the
Department of Veterans Affairs (VA), the Congress, and veterans' service
organizations. In October 1998, VBA implemented its Systematic Technical
Accuracy Review (STAR) system to improve the measurement of the accuracy of
claims processing. As designed by VBA, the STAR system determines accuracy
rates for (1) the nation as a whole, (2) each VBA Service Delivery Network,
and (3) each VBA regional office. 1
In March 1999, we reported to you that the claims- processing accuracy rates
for individual regional offices are determined through quality reviews
conducted by regional office staff, who themselves are involved in
processing claims and who report to managers responsible for claims
processing. 2 As we reported, such an arrangement does not meet the
governmental internal control standard calling for segregation of key
duties, or the performance audit standard calling for organizational
independence for those who review and evaluate program performance. 3 These
standards are intended to prevent bias and avoid compromised objectivity in
the efforts of individuals who perform key duties, such as reviewing and
evaluating program performance. We recommended that VBA correct the reported
shortcomings to comply with these standards. Subsequently, the Congress
enacted a
1 VBA has structured its 57 regional offices into nine Service Delivery
Networks. VBA expects the regional offices in each Network to collaborate,
provide mutual support, share resources, use teambased principles, and share
collective responsibility and accountability for the Network?s overall
performance of all assigned work.
2 Veterans? Benefits Claims: Further Improvements Needed in Claims-
Processing Accuracy
(GAO/ HEHS- 99- 35, Mar. 1, 1999). 3 Standards for Internal Control in the
Federal Government (GAO/ AIMD- 00- 21.3.1, Nov. 1999) and
Government Auditing Standards (U. S. General Accounting Office, Washington,
D. C., June 1994).
Page 2 GAO- 01- 930R VBA Disability Claims Processing
provision in the Veterans Millennium Health Care and Benefits Act (P. L.
106- 117, Nov. 30, 1999) requiring VBA to have a quality assurance program
that meets generally applicable governmental standards for independence and
internal controls for quality reviews of Government performance and results.
The conference report for this act specifically directed the Secretary to
design VBA's quality assurance program so that it complies with our
recommendation, which stated that VBA's quality assurance system should meet
the government's internal control standard on segregation of duties and the
program performance audit standard on organizational independence. 4
You asked that we determine whether VBA has a quality assurance system for
compensation and pension claims that complies with our recommendation. To
address this issue, we reviewed VBA documents related to the STAR system. At
VBA headquarters, we also interviewed VBA officials who oversee regional
office operations and officials in VBA's Compensation and Pension Service
who formulate program policy and guidance for the compensation and pension
programs and who are responsible for STAR reviews. 5 Our review was
conducted between June 2001 and August 2001 in accordance with generally
accepted government auditing standards.
Results in Brief
In brief, effective October 2001, VBA plans to implement a modification to
the STAR system that would bring it into compliance with our recommendation
regarding the standards on segregation of duties and organizational
independence. At that time, VBA plans that all STAR reviews will be
conducted by staff of the Compensation and Pension Service who themselves do
not process claims and do not report to managers responsible for claims
processing. Staff of VBA's regional offices would no longer be responsible
for conducting the STAR reviews that determine regional office accuracy
rates. As a result, if this modification is implemented as planned, the STAR
system would comply with our recommendation regarding segregation of duties
and organizational independence. We received comments from VA on a draft of
this letter, and VA agreed with our conclusions.
Background
The compensation program pays monthly benefits-- based on degree of
disability-- to veterans who have service- connected disabilities (injuries
or diseases incurred or aggravated while on active military duty),
regardless of whether these veterans are employed or have earnings. The
pension program pays monthly benefits-- based on financial need-- to wartime
veterans who have low incomes and who are permanently and totally disabled
for reasons not service- connected.
4 H. R. Rep. No. 106- 470, at 91- 92 (1999). 5 In addition to the
Compensation and Pension Service, VBA has four other Services that formulate
policy and guidance for its insurance, loan guaranty, vocational
rehabilitation and employment, and education programs. Section 801 of the
Veterans Millennium Health Care and Benefits Act permits VBA to establish a
quality assurance function in each of its five program Services. As agreed
with your office, we did not review the quality assurance systems for VBA's
other four program Services.
Page 3 GAO- 01- 930R VBA Disability Claims Processing
Veterans may submit their compensation and pension claims to any of VBA's 57
regional offices. In processing a veteran's claim, the regional office must
assist the veteran in obtaining evidence to support the claim. For example,
when a veteran submits a compensation claim, the regional office must obtain
records such as the veteran?s existing service medical records, records of
relevant medical treatment or examinations provided at VA health- care
facilities or at VA?s expense, and other relevant records held by a federal
department or agency. Also, if necessary to make a decision on the claim,
the regional office must arrange a medical examination or obtain a medical
opinion. The regional office then analyzes all the evidence, evaluates
service- connected impairments, and assigns a rating for the degree to which
the veteran is disabled. The regional office then notifies the veteran of
the decision. 6
Under the STAR system, VBA considers a claim to have been processed
accurately if the regional office determines basic eligibility correctly,
obtains all required medical and nonmedical documentary evidence, decides
service- connection correctly, gives the correct rating to each impairment,
determines the correct payment amount, and properly notifies the veteran of
the outcome of his or her claim. If a claim has any errors in any of these
areas, VBA counts the entire claim as incorrect for accuracy rate
computation purposes. For the nation as a whole, VBA reported an accuracy
rate of 59 percent for fiscal year 2000. VBA?s strategic goal is to achieve
a national accuracy rate of 96 percent by fiscal year 2006.
The accuracy rates determined through the STAR system are a key component of
VBA's system for holding program managers accountable for program
performance. In fiscal year 1999, VBA instituted a "balanced scorecard" for
measuring program performance at three organizational levels: the national
VBA headquarters level, the Service Delivery Network level, and the regional
office level. Under the balanced scorecard approach, claims processing
performance at each level is assessed on the basis of five performance
measures: accuracy, timeliness, unit cost, customer satisfaction, and
employee satisfaction and development. The balanced scorecard gives more
relative weight (28 percent) to the accuracy rate than to any of the other
four factors.
VBA's Planned Modification Would Bring the STAR System Into Compliance With
Our Recommendation Regarding Standards on Segregation of Duties and
Organizational Independence
In October 1998, when VBA implemented the STAR system for the compensation
and pension programs, VBA established a STAR unit in its Compensation and
Pension Service at VBA headquarters. Since its inception, this STAR unit,
which is staffed with Compensation and Pension Service employees, has
performed the quality
6 If a veteran disagrees with a regional office decision, the veteran may
file an appeal asking VA?s Board of Veterans? Appeals to review the
decision. The Board makes VA?s final decision on such appeals and can grant
or deny benefits. In some cases the Board may return (remand) the case to
the regional office to obtain further evidence and reconsider the decision.
In these cases, the regional office either grants the claim or returns it to
the Board for a final VA decision. Further, beyond appeals to the Board,
veterans can appeal first to the U. S. Court of Appeals for Veterans Claims,
then to the Court of Appeals for the Federal Circuit, and finally to the
Supreme Court of the United States.
Page 4 GAO- 01- 930R VBA Disability Claims Processing
reviews that are used to determine the accuracy rates for the nation and for
each of VBA's nine Service Delivery Networks. Because the Compensation and
Pension Service itself does not process claims and is not organizationally
responsible for claims processing in the regional offices, these national
and network reviews conducted by the Compensation and Pension Service's STAR
unit meet the standards on segregation of duties and organizational
independence (see fig. 1).
Figure 1: VBA Organization Chart Note: This is a simplified version of VBA?s
organization chart showing only the components needed to understand the
issues presented in this letter.
Source: VBA.
By contrast, as implemented in 1998, the STAR system does not provide
adequate segregation of duties, or organizational independence for the
quality reviewers who determine accuracy rates for individual regional
offices. Under the original design, VBA tasked each regional office with the
responsibility of self- reviewing its own work to determine its individual
accuracy rate. According to VBA, it took this approach due to the volume of
additional cases that the STAR unit in the Compensation and Pension Service
would have had to review to determine accuracy rates for each individual
regional office.
However, tasking the regional offices with reviewing their own work means
that the regional office staff members who process claims are themselves
responsible for reviewing the accuracy of their own office's claims
processing. These same staff report to regional office managers who are also
responsible for claims processing. Such an arrangement does not meet the
internal control standard calling for segregation of key duties, nor does it
meet the performance audit standard calling for
Page 5 GAO- 01- 930R VBA Disability Claims Processing
organizational independence of program reviewers. It does not meet these
standards because both the reviewers and their managers have an inherent
self- interest in having as high an accuracy rate as possible, which
potentially could bias the reviewers and influence them to overlook or
conceal errors.
Because such issues raise questions about the reliability of the 57 regional
offices' self- reported accuracy rates, VBA decided not to use these self-
reported accuracy rates in the regional offices' individual balanced
scorecards. Instead, to assess the performance of a specific regional
office, VBA inserts into that office's balanced scorecard the overall
average accuracy rate of its Service Delivery Network, as determined by the
STAR unit at VBA headquarters. In this way, the accuracy rate inserted into
the balanced scorecard for each individual regional office may not reflect
its actual performance. Consequently, a regional office whose actual
accuracy rate is below its network's average rate is not held appropriately
accountable for its performance.
In March 2001, however, the Under Secretary for Benefits announced a plan
that would correct these shortcomings and comply with our recommendation
regarding segregation of duties and organizational independence. The Under
Secretary stated that, to redirect regional office resources from quality
assurance reviews to claims processing, the STAR unit in the Compensation
and Pension Service would begin reviewing statistically representative
samples of cases for each regional office to determine the accuracy rates
for each office. Thus, under this planned arrangement, all of the STAR
reviews-- including those that determine regional office accuracy rates--
would be done by the STAR unit in the Compensation and Pension Service at
VBA headquarters. As we recommended, this planned approach would provide
adequate segregation of duties, as well as organizational independence for
the reviewers who determine regional office accuracy rates. VBA also plans
to use each regional office's own individual accuracy rate in the balanced
scorecard assessment for each office.
VBA plans to implement this new STAR system arrangement for regional offices
on October 1, 2001. VBA projects that the samples reviewed by the
Compensation and Pension Service's STAR unit will be large enough to produce
an annual accuracy rate for each regional office at the 95- percent
confidence level, plus or minus an 8- percentage point margin of error. 7
Doing so will require assigning up to 10 new fulltime staff in addition to
the 6 who have been reviewing cases since the STAR system was originally
implemented. 8 Under the original design, the Compensation and Pension
Service's STAR unit has been reviewing a total of about 7,000 cases per year
7 For cases involving ratings, the minimum annual sample size will be 120
cases per regional office. With this sample size, the expected margin of
sampling error at the 95- percent confidence level is plus or minus 8
percentage points. The sample size is calculated based on the assumption
that the accuracy rate will be 70 percent. If the actual accuracy rate turns
out to be 50 percent, the margin of sampling error at the 95- percent
confidence level would increase to plus or minus 9 percentage points.
8 The STAR unit at VBA headquarters actually has nine staff members who
review cases, but they spend about one- third of their time on duties other
than STAR reviews. Beginning October 1, 2001, these nine staff members will
work full- time on STAR reviews. VBA also plans to outbase six of its new
STAR review staff in the regional office in Nashville. They will report
directly to the STAR unit in VBA headquarters.
Page 6 GAO- 01- 930R VBA Disability Claims Processing
to determine accuracy rates for the nation and each Service Delivery
Network. However, with the additional responsibility of reviewing cases to
determine accuracy rates for each regional office, the STAR unit at VBA
headquarters will be responsible for reviewing between 10,000 and 12,000
additional cases per year.
Under this plan, the STAR unit will conduct all of the quality reviews that
determine the accuracy rates used to hold program managers accountable for
organizational performance. However, VBA also believes that regional offices
should have a role in the quality review process, and not simply be the
recipient of review results. As such, VBA plans to implement a Systematic
Individual Performance Assessment (SIPA) system in fiscal year 2002. Under
the SIPA system, each regional office will assess the accuracy performance
of individual employees to identify their training needs and ways to improve
the claims process. We support this effort.
Agency Comments
We received written comments from VA on a draft of this letter (see
enclosure). In its comments, VA said it was pleased that we had concluded
that planned modifications will bring VBA's STAR system into compliance with
our recommendation regarding the generally applicable governmental standards
on segregation of key duties and organizational independence for those who
review and evaluate program performance.
- - - - As agreed with your office, unless you publicly announce its
contents earlier, we plan no further distribution of this letter until 21
days after its issue date. At that time, we will send copies of this letter
to the Secretary of Veterans Affairs, the Under Secretary for Benefits, and
interested congressional committees. The letter is also available on GAO's
homepage at http:// www. gao. gov. If you or your staff have questions,
please call me on (202) 512- 7101 or Irene Chu on (202) 512- 7102. Ira
Spears and Steve Morris also were key contributors to this report.
Sincerely yours, Cynthia A. Bascetta Director, Education, Workforce,
and Income Security Issues
Page 7 GAO- 01- 930R VBA Disability Claims Processing
ENCLOSURE I ENCLOSURE I Comments from the Department of Veterans Affairs
(130053)
*** End of document. ***