    Veterans' Benefits: Improvements Needed in the Reporting and Use
    of Data on the Accuracy of Disability Claims Decisions
    (30-SEP-03, GAO-03-1045).

    The Veterans Benefits Administration (VBA) has a large inventory
    of claims for benefits under its compensation and	 pension
    programs. The Secretary of the Department of Veterans Affairs has
    pledged to substantially reduce this inventory in order to
    improve timeliness. In response, VBA emphasized producing more
    claims decisions per year. GAO was asked to ascertain how
    accuracy has changed since VBA increased its emphasis on
    production and to report on the agency's efforts to ensure the
    accuracy of its decisions.
    -------------------------Indexing Terms-------------------------
    REPORTNUM:   GAO-03-1045
        ACCNO:   A08610
      TITLE:     Veterans' Benefits: Improvements Needed in the Reporting
    and Use of Data on the Accuracy of Disability Claims Decisions
         DATE:   09/30/2003
      SUBJECT:   Pension claims
    	     Standards evaluation
    	     Veterans pensions
    	     Veterans benefits
    	     Decision making
    	     Claims processing
    	     Compensation claims
    	     Accountability
    	     Performance measures
    	     VA Systematic Technical Accuracy Review
    	     System

    ******************************************************************
    ** This file contains an ASCII representation of the text of a  **
    ** GAO Product.                                                 **
    **                                                              **
    ** No attempt has been made to display graphic images, although **
    ** figure captions are reproduced.  Tables are included, but    **
    ** may not resemble those in the printed version.               **
    **                                                              **
    ** Please see the PDF (Portable Document Format) file, when     **
    ** available, for a complete electronic file of the printed     **
    ** document's contents.                                         **
    **                                                              **
    ******************************************************************
    GAO-03-1045

    United States General Accounting Office

    GAO

                           Report to Congressional Requesters

    September 2003

    VETERANS' BENEFITS

        Improvements Needed in the Reporting and Use of Data on the Accuracy of
                              Disability Claims Decisions

    GAO-03-1045

    Highlights of GAO-03-1045, a report to congressional requesters

    The Veterans Benefits Administration (VBA) has a large inventory of claims
    for benefits under its compensation and pension programs. The Secretary of
    the Department of Veterans Affairs has pledged to substantially reduce
    this inventory in order to improve timeliness. In response, VBA emphasized
    producing more claims decisions per year. GAO was asked to ascertain how
    accuracy has changed since VBA increased its emphasis on production and to
    report on the agency's efforts to ensure the accuracy of its decisions.

    GAO recommends that VBA

    o  	report the accuracy of VBA disability compensation and pension claims
    decisions to the Congress and other stakeholders in a manner that allows
    for valid comparisons of accuracy across fiscal years and

    o  	better hold regional offices accountable for the accuracy of their
    claims decisions, by increasing the use of its regional office accuracy
    data.

    VBA concurred with GAO's recommendations and provided additional
    information about its efforts to improve accuracy.

    www.gao.gov/cgi-bin/getrpt?GAO-03-1045.

    To view the full report, including the scope and methodology, click on the
    link above. For more information, contact Cynthia A. Bascetta at (202)
    512-7215 or bascettac@gao.gov.

    September 2003

    VETERANS' BENEFITS

    Improvements Needed in the Reporting and Use of Data on the Accuracy of
    Disability Claims Decisions

    From fiscal years 2001 to 2002, VBA's accuracy of decision-making in the
    disability compensation and pension benefit programs declined from 89
    percent to 81 percent. The agency had reported a slight improvement in
    accuracy between fiscal years 2001 and 2002-from 78 percent to 80 percent.
    However, we found that these two annual figures were not comparable
    because the agency had substantially changed the way it measured accuracy
    for fiscal year 2002. Although VBA acknowledged a change in its accuracy
    measure in its annual report to the Congress, the agency did not revise
    its 2001 figure to allow for an appropriate comparison with 2002. VBA
    officials GAO spoke with suggested several factors that may have
    contributed to the decline in accuracy. We were not able to quantify the
    relative contribution of these factors. These factors included VBA's
    emphasis on increasing claims decisions, the specific processing
    requirements of the Veterans Claims Assistance Act (VCAA) of 2000, and the
    relative inexperience of VBA's claims processing staff.

    To help ensure accountability for accuracy, VBA set accuracy standards for
    its regional offices. Although VBA has regional office-level accuracy
    data, it has not made full use of this information to encourage better
    performance from regional offices with low accuracy scores. For example,
    in fiscal year 2002, VBA did not require offices with poor accuracy to
    prepare improvement plans and gave performance awards to two offices that
    clearly failed to meet VBA's accuracy goal.

    VBA Compensation and Pension Rating Products Decided and Accuracy Rates,
    Fiscal Years 2001-2002

    Number completed Percentage

    (in thousands)
    1,000                  89          100
    800 80
    600 60
    400 40
    200 20
    0 0
                                 B

    Production Accuracy

    FY 2001 FY 2002

    Source: GAO analysis of Veterans Benefits Administration data.

    Note: Accuracy rates in the above chart are estimates based on samples of
    rating products completed in fiscal years 2001 and 2002 and have margins
    of error of about plus or minus 1 percentage point at the 95-percent level
    of confidence. We determined these accuracy rates by comparing
    substantially similar questions for the 2 fiscal years that focused on
    whether the decision to grant or deny benefits was correct. The fiscal
    year 2002 accuracy score includes a specific question on proper VCAA
    pre-decision notification that VBA did not include in its fiscal year 2001
    quality reviews.

    Contents

      Letter

    Results in Brief
    Background
    Accuracy Declined in Fiscal Year 2002
    VBA Could Have Done More to Hold Regional Offices Accountable

    for Accuracy Conclusions Recommendations Agency Comments and Our Response

    1

    2 3 6

    8 10 11 11

    Appendix I 	Comments from the Veterans Benefits Administration

      Figure

    Figure 1: Fiscal Year 2001 STAR Rating Checklist-Criteria for a Correct
    VBA Decision

    Abbreviations

    BVA Board of Veterans' Appeals
    SIPA Systematic Individual Performance Assessment
    STAR Systematic Technical Accuracy Review
    VA Department of Veterans Affairs
    VBA Veterans Benefits Administration
    VCAA Veterans Claims Assistance Act of 2000
    VSR veterans service representative

    This is a work of the U.S. government and is not subject to copyright
    protection in the United States. It may be reproduced and distributed in
    its entirety without further permission from GAO. However, because this
    work may contain copyrighted images or other material, permission from the
    copyright holder may be necessary if you wish to

                          reproduce this material separately.

    United States General Accounting Office Washington, DC 20548

    September 30, 2003

    The Honorable Arlen Specter
    Chairman
    The Honorable Bob Graham
    Ranking Minority Member
    Committee on Veterans' Affairs
    United States Senate

    The Honorable John D. Rockefeller IV
    United States Senate

    At the beginning of fiscal year 2002, over 400,000 claims for disability
    compensation and pension benefits were waiting for Veterans Benefits
    Administration (VBA) decisions. On average, veterans were waiting about
    6 months and many were waiting more than a year. VBA has been making
    a concerted effort to reduce the inventory of claims and complete
    decisions in an average of 100 days by the end of fiscal year 2003. In
    order
    to improve the timeliness of its disability claims decisions, VBA
    established a goal of completing about 839,000 decisions in fiscal year
    2002
    and reducing its inventory of pending claims to about 316,000 by the end
    of that year. During fiscal year 2002, VBA increased its claims decisions
    by
    two-thirds from about 481,000 to about 797,000 and reduced its inventory
    of pending claims by about one-fifth from about 421,000 to about 346,000.
    For fiscal year 2003, VBA's goals are to produce about 806,000 decisions
    and further reduce inventory to 250,000 claims.

    In light of VBA's emphasis on completing more claims decisions more
    quickly, you asked us to assist the Committee in its oversight of VBA by
    examining the agency's efforts to ensure the accuracy of decision-making
    in its disability compensation and pension benefits programs. In response,
    we assessed (1) how accuracy may have changed since VBA increased its
    emphasis on production and (2) how well VBA ensures the accuracy of its
    decisions.

    We reviewed VBA's efforts to measure and report accuracy for fiscal years
    2001 and 2002.1 To determine how accuracy had changed over that time,

    1This report focuses on the accuracy of rating decisions. Rating decisions
    are primarily decisions on original claims for compensation and pension
    benefits and reopened claims. For example, veterans may file reopened
    claims if they believe that their service-connected conditions have
    worsened.

      Results in Brief

    we independently calculated and compared VBA's accuracy rates for fiscal
    years 2001 and 2002. As part of our analysis of how well VBA ensures
    national and regional office accuracy, we reviewed VBA's accuracy
    measurement system, known as the Systematic Technical Accuracy Review
    (STAR) program, including how VBA incorporates accuracy into its
    performance standards for both regional office directors and claims
    processing staff. In addition, we examined VBA's criteria for regional
    office performance awards and the results of reviews by VBA survey teams
    of regional offices. We discussed factors affecting accuracy with staff at
    VBA regional offices in Phoenix, Arizona; St. Petersburg, Florida; Boston,
    Massachusetts; Pittsburgh, Pennsylvania; and Houston, Texas.2 We also
    discussed accuracy issues with representatives of the American Legion,
    Disabled American Veterans, Paralyzed Veterans of America, and Veterans of
    Foreign Wars to gain their perspectives on VBA efforts to improve
    accuracy. We conducted our review from January through August of 2003 in
    accordance with generally accepted government auditing standards.

    From fiscal years 2001 to 2002, VBA's accuracy of decision making in the
    compensation and pension programs declined from 89 percent to 81 percent.3
    We determined these accuracy rates by comparing substantially similar
    questions for the 2 fiscal years that focused on whether the decision to
    grant or deny benefits was correct. The agency had reported a slight
    improvement in accuracy between fiscal years 2001 and 2002-from 78 percent
    to 80 percent. However, we found that these two annual figures were not
    comparable because the agency had substantially changed the way it
    measured accuracy for fiscal 2002. Although VBA acknowledged a change in
    its accuracy measure in its annual report to the Congress, the agency did
    not revise its 2001 figure to allow for an appropriate comparison with
    2002. VBA officials attributed the decline in accuracy during this period
    to several factors. We were not able to quantify the relative contribution
    of these factors. These factors included headquarters emphasis on
    production, the specific processing requirements of the

    2Except for Pittsburgh, we selected the regional offices we visited based
    on a combination of their fiscal year 2002 rating accuracy and the number
    of rating decisions they produced. We visited the Pittsburgh Regional
    Office to observe how VBA conducts an inspection of a regional office's
    compensation and pension operations as well as to discuss factors
    affecting accuracy.

    3The margin of error for rating accuracy VBA-wide for fiscal years 2001
    and 2002 was about 1 percentage point at a 95-percent level of confidence.

    Background

    Veterans Claims Assistance Act of 2000, and the relative inexperience of
    VBA's claims processing staff.

    To help ensure accountability for accuracy, VBA set accuracy standards for
    its regional offices. However, VBA has not fully used data gathered on the
    accuracy of its regional office decisions to encourage better regional
    office performance. For example, while some regional offices were required
    to prepare plans to improve claims processing production, similar plans
    were not required for offices to improve accuracy. Also, two regional
    offices received overall performance awards despite not meeting VBA's
    accuracy standard. Unless VBA strikes a balance between production goals
    and accuracy standards, it cannot ensure that as it improves decision
    timeliness, it will also improve decision accuracy.

    This report contains recommendations to the Secretary of Veterans Affairs
    to direct VBA to improve its reporting on and use of accuracy data. VBA
    concurred with our recommendations and provided additional information
    about its efforts to improve accuracy.

    Through its disability compensation program, VBA pays monthly benefits to
    veterans with service-connected disabilities (injuries or diseases
    incurred or aggravated while on active military duty) according to the
    severity of the disability. VBA's pension benefit program pays monthly
    benefits to wartime veterans who have low incomes and are permanently and
    totally disabled for reasons not service-connected.4 In addition, VBA pays
    dependency and indemnity compensation to some deceased veterans' spouses,
    children, and parents and to survivors of service members who died on
    active duty. In fiscal year 2002, VBA paid over $22 billion in disability
    compensation to an average of about 2.4 million veterans and over 300,000
    survivors. VBA also paid over $3 billion in pensions to an average of
    about 580,000 veterans and survivors.

    The amount of disability compensation largely depends on the degree to
    which a veteran is disabled. VBA determines the degree to which veterans
    are disabled in 10 percent increments on a scale of 0 to 100 percent.
    Basic monthly payments range from $104 for 10 percent disability to $2,193
    for

    4Veterans who are 65 years or older do not have to be permanently and
    totally disabled to become eligible for pension benefits, as long as they
    meet the other requirements for income and military service. VBA also pays
    pensions to surviving spouses and unmarried children of deceased wartime
    veterans.

    100 percent disability. About 65 percent of veterans receiving disability
    compensation have disabilities rated at 30 percent and lower; about 8
    percent have disabilities rated at 100 percent. The most common
    impairments for veterans who began receiving compensation in fiscal year
    2001 were tinnitus (ex., a ringing in the ear), auditory acuity impairment
    rated at 0 percent (i.e., mild hearing difficulties), skeletal conditions,
    arthritis due to trauma, post-traumatic stress disorder, and scars.
    Eligibility and priority for other Department of Veterans Affairs (VA)
    benefits and services such as health care and vocational rehabilitation
    are affected by these VA disability ratings.

    When a veteran submits a claim to any of VBA's 57 regional offices, a
    veterans service representative (VSR) is responsible for obtaining the
    relevant evidence to evaluate the claim. Such evidence includes veterans'
    military service records, medical examinations and treatment records from
    VA medical facilities, and treatment records from private medical service
    providers. Once a claim is developed (i.e., has all the necessary
    evidence), a rating VSR, also called a rating specialist, evaluates the
    claim, determines whether the claimant is eligible for benefits, and
    assigns a rating based on degree of disability. Veterans with multiple
    disabilities receive a single composite rating. For veterans claiming
    pension eligibility, the regional office determines if the veteran served
    in a period of war, is permanently and totally disabled for reasons not
    service-connected, and meets the income thresholds for eligibility. A
    veteran who disagrees with the regional office's decision for either
    program can appeal sequentially to VA's Board of Veterans' Appeals (BVA),
    the U.S. Court of Appeals for Veterans Claims, and the U.S. Court of
    Appeals for the Federal Circuit.

    In fiscal year 1999, VBA implemented a quality review system to improve
    the measurement of the accuracy of its claims decisions. Under STAR, VBA
    selects a random sample of completed claims decisions each month from each
    of its 57 regional offices. STAR quality review staff review these claims
    using a standard checklist. If a claim has any error, VBA counts the
    entire claim as incorrect for accuracy rate computation purposes. STAR
    then returns the case file and the results of the review to the regional
    office that made the decision. If an error was found, the regional office
    is required to either correct it or request reconsideration of the error
    by STAR.

    VBA made several significant changes to its STAR system in fiscal year
    2002. First, in response to our March 1999 recommendation, VBA made the
    STAR review staff organizationally independent of the regional offices to
    ensure the independence of the reviews.5 Second, VBA more than doubled the
    number of rating decisions it reviewed (from 3,209 in fiscal year 2001 to
    6,646 in fiscal year 2002) in order to obtain more precise regional office
    level accuracy scores. Third, VBA improved its key measure to focus more
    on whether decisions to grant or deny benefits were correct. Previously,
    VBA's key measure also included decision documentation and notification
    issues. Figure 1 shows the areas covered by the checklist used to measure
    accuracy in fiscal year 2001. The revised key accuracy measure includes
    only the first four areas of the checklist.

    Figure 1: Fiscal Year 2001 STAR Rating Checklist-Criteria for a Correct
    VBA Decision

    � Address all issues - The decision addressed all claimed
    disabilities and benefits, and any disabilities or benefits that could be
    inferred from the available evidence.

    � Proper development - VBA made an adequate attempt to obtain all
    evidence needed to decide the claim.

    � Grant or deny - The decision to grant or deny was correct; if
    granted, the percentage evaluation was correct.

    � Award actions - Where benefits were granted, the effective dates
    of benefits and payment amounts were correct.

    � Reasons and bases - The decision discussed all applicable
    evidence and the basis of the decision was explained.

    � Notification - The claimant and power of attorney, if applicable,
    were correctly notified of the decision and were notified of appeal
    rights.

    Source: Veterans Benefits Administration.

    5For more information, see U.S. General Accounting Office, Veterans'
    Benefits Claims: Further Improvements Needed in Claims-Processing
    Accuracy, GAO/HEHS-99-35 (Washington, D.C.: Mar. 1, 1999) and Veterans'
    Benefits: Quality Assurance for Disability Claims Processing, GAO-01-930R
    (Washington, D.C.: Aug. 23, 2001).

      Accuracy Declined in Fiscal Year 2002

    From fiscal years 2001 to 2002, VBA's accuracy of decision making in the
    compensation and pension programs declined from 89 percent to 81 percent.
    We determined these accuracy rates by comparing substantially similar
    questions for the 2 fiscal years that focused on whether the decision to
    grant or deny benefits was correct.6 However, the agency reported that it
    had improved its accuracy for that period. This discrepancy occurred
    because VBA did not report comparable accuracy data for the 2 fiscal
    years. VBA reported its fiscal year 2002 accuracy based on a revised
    measure that excluded two areas of the checklist-reasons and bases and
    notification. VBA then compared this to its fiscal year 2001 accuracy,
    which included these two areas of the checklist. VBA officials attributed
    the decline in accuracy during this period to several factors. We were not
    able to quantify the relative contribution of these factors. These factors
    included headquarters emphasis on production, the specific processing
    requirements of the Veterans Claims Assistance Act of 2000, and the
    relative inexperience of VBA's claims processing staff.

    In fiscal year 2002, VBA made progress in increasing production of claims
    decisions and reducing inventory. The number of claims decisions rose
    substantially from about 481,000 in fiscal year 2001 to about 797,000 in
    fiscal year 2002. During the same time period, the number of claims VBA
    received increased from about 674,000 to 722,000. The higher level of
    production permitted VBA to reduce its inventory of pending claims by 18
    percent from about 421,000 to about 346,000.

    Although accuracy actually declined, VBA reported in its performance
    report to the Congress for fiscal year 2002 that the accuracy of its
    rating decisions for compensation and pension benefits had improved
    slightly. VBA reported that accuracy improved from 78 percent in fiscal
    2001 to 80 percent in fiscal 2002.7 However, these rates were not
    comparable because the new "benefit entitlement accuracy" rate for fiscal
    year 2002 was based on fewer areas of the checklist that focused on
    whether and to what extent the claimant received a benefit. In its fiscal
    year 2002 performance report, VA made note of this change but did not
    recalculate

    6The fiscal year 2002 accuracy score includes a specific question on
    proper VCAA pre-decision notification that VBA did not include in its
    fiscal year 2001 quality reviews.

    7Due to the time lag involved between receiving claims files and
    conducting the accuracy reviews of disability compensation and pension
    decisions as well as deadlines for incorporating data into its annual
    accountability report, reported accuracy rates can differ from final rates
    for the reported year. VBA's final rating accuracy rate for fiscal year
    2002 was 81 percent.

    the fiscal 2001 rate to provide a correct comparison. When we recomputed
    the agency's accuracy rate for fiscal 2001 using its revised criteria, the
    result was 89 percent, not 78 percent as reported. As a result, the
    correct comparison showed a decline in accuracy from 89 percent to 81
    percent. The agency corroborated our finding after performing its own
    computation at our request.

    VBA officials we spoke with cited a number of factors that affected
    accuracy over the fiscal years 2001 to 2002 period. A key factor cited was
    VBA's emphasis on production, such that accuracy became a lesser priority.
    In 2001, the Secretary of Veterans Affairs set an ambitious goal to reduce
    the agency's average time to complete a rating decision from 173 days (in
    fiscal year 2000) to 100 days by the end of fiscal year 2003. Also, the
    Secretary tasked VBA with reducing the inventory of rating claims to
    250,000 by the end of September 2003. VBA managers responded by
    emphasizing the completion of more rating cases each month. The result has
    been a significant increase in production and a significant reduction in
    the rating inventory since the end of fiscal year 2001.

    In fiscal year 2002, VBA incorporated its new production targets into its
    measures for regional office performance.8 Regional office directors
    became accountable for specific targets for production, inventory
    reduction, and timeliness improvement. The agency also established
    regional office performance award criteria that gave more weight to
    efficiency than accuracy. Three of the four criteria for cash awards were
    based on production and timeliness and one on accuracy; bonuses could be
    received without meeting the accuracy criteria.

    VBA officials also attributed accuracy problems to the addition of more
    specific processing requirements under the Veterans Claims Assistance Act
    (VCAA) of 2000. The act broadened VBA's responsibility to assist veterans
    - such as notifying them of needed evidence and helping them develop that
    evidence for their claims. VBA incorporated VCAA requirements into its
    STAR rating accuracy checklist in fiscal year 2002. In the first 4 months
    of fiscal year 2002, VCAA errors accounted for almost half of all errors
    identified by STAR rating reviewers. In April 2002, VBA required each
    regional office to re-train its claims processing staff on the

    8VBA established specific monthly production targets for its regional
    offices in December 2001. For more information on these targets see U.S.
    General Accounting Office, Veterans' Benefits: Despite Recent
    Improvements, Meeting Claims Processing Goals Will Be Challenging,
    GAO-02-645T (Washington, D.C.: Apr. 26, 2002).

      VBA Could Have Done More to Hold Regional Offices Accountable for Accuracy

    new requirements.9 Over the last 8 months of fiscal year 2002, the
    proportion of VCAA-related errors declined to about one-third of all
    errors.

    VBA officials also cited the relative inexperience of VBA's claims
    processing staff. Regional office officials noted that they had many VSRs
    and rating specialists with insufficient training and experience to be
    fully proficient at developing and making decisions on claims. For
    example, officials at two offices we visited noted that their accuracy was
    affected by large numbers of inexperienced staff, due to a large increase
    in staff size or a relatively high rate of turnover. VBA has found that
    attrition rates for new claims processing staff hired over a 3-year period
    ranged from 0 percent to 49 percent at some regional offices.10

    While VBA has established accuracy standards for its regional offices, it
    has not made the best use of its accuracy data on regional offices. VBA
    collected regional office accuracy data in fiscal year 2002 but has not
    fully used the information to evaluate regional office performance,
    correct errors, and identify needed training that could reduce errors.
    Unless VBA better uses these data to hold regional offices accountable for
    accuracy, it cannot ensure that as decisions become timelier, they also
    become more accurate. VBA is developing an agencywide system for
    evaluating individual claims processing employees' performance against
    VBA's individual accuracy standards.

    In fiscal year 2002, VBA improved its STAR review system to provide
    independent review of decisions and more precise accuracy data at the
    regional office level. Regional accuracy scores are estimates based on
    samples of cases and revealed high and low performers relative to the
    agency's fiscal year 2002 accuracy goal of 85 percent. Eleven regional
    offices clearly failed to meet the goal while 3 clearly met or exceeded
    the

    9For more information on VCAA implementation, see U.S. General Accounting
    Office, Veterans' Benefits: VBA's Efforts to Implement the Veterans Claims
    Assistance Act Need Further Monitoring, GAO-02-412 (Washington, D.C.: July
    1, 2002).

    10For more information, see U.S. General Accounting Office, Veterans
    Benefits Administration: Better Collection and Analysis of Attrition Data
    Needed to Enhance Workforce Planning, GAO-03-491 (Washington, D.C.: Apr.
    28, 2003).

    goal.11 However, VBA managers did not fully use the regional
    office-specific information generated by the STAR unit as a basis for
    improving accuracy at the regional office level. For example, VBA did not
    require regional offices that failed to meet the national accuracy goal of
    85 percent to prepare strategies for improvement. In contrast, in fiscal
    year 2002, 13 regional offices that did not meet established targets for
    production, inventory, and timeliness were required to develop corrective
    action plans. Also, as noted earlier, VBA did not make rating accuracy a
    prerequisite for receiving a performance award. Three of the four criteria
    for cash awards were based on production and timeliness and one on
    accuracy; a bonus could be received without meeting the accuracy criteria.
    In fiscal year 2002, 2 offices received performance awards despite clearly
    not meeting VBA's accuracy goal. These offices had accuracy rates of 71
    percent and 75 percent, respectively.

    VBA's regional offices did not always properly address STAR errors that
    were returned to them, a violation of VBA policy that regional offices
    either correct errors or formally request that the STAR unit reconsider
    its error call. VBA survey teams, which review judgmental samples of STAR
    cases with errors, found that the 18 regional offices they visited from
    October 2001 through December 2002 had failed to respond properly to over
    40 percent of the errors reviewed. At 14 of the 18 offices, the survey
    teams found problems significant enough to recommend improvements in STAR
    error handling. For example, teams recommended that several offices
    establish management controls to ensure that they properly address all
    errors identified by the STAR unit. Effective April 1, 2003, VBA
    established a new tracking system that required regional offices to report
    to headquarters on actions taken to address all errors identified by the
    STAR unit.

    In addition to not correcting all errors, several of the regional offices
    visited by the survey teams were not making adequate use of STAR errors to
    identify trends that could be used to provide feedback and training to
    claims processing staff. Officials at regional offices we visited
    commented that this information is useful for identifying trends for
    feedback and training purposes. For example, in fiscal year 2002, the
    results of STAR reviews helped to alert management of the Boston Regional
    Office that

    11It could not be determined if the remaining offices met the accuracy
    goal because of the margins of error associated with their accuracy
    scores. In fiscal year 2002, the margins of error for rating accuracy for
    each regional office ranged from 4 to 9 percentage points at the
    95-percent level of confidence.

    many of its claims processors were having difficulties with the new VCAA
    legislation, permitting management to provide special training for the
    Boston staff. The agency survey teams recommended that 7 of the regional
    offices reviewed between October 2001 and December 2002 take steps to use
    STAR reviews for feedback and training.

    In addition to regional office-level accuracy standards, VBA has also
    established quantifiable accuracy standards for its experienced claims
    processing employees, as part of these employees' annual performance
    evaluations.12 The accuracy standards for experienced rating specialists
    require that they correctly decide 85 percent of 60 randomly selected
    rating decisions per year (5 decisions per month).13 VBA is developing an
    agencywide review process, the Systematic Individual Performance
    Assessment (SIPA), to assess compliance with the individual accuracy
    standards. VBA developed a uniform checklist, similar to the STAR
    checklist for reviewing rating decisions, which offices could use to
    assess rating specialists' accuracy. Unlike the STAR system, reviews are
    conducted before decisions are finalized.

    In April 2003, VBA surveyed regional offices to determine the status of
    their efforts to measure individual accuracy and assess the staff
    resources needed to implement SIPA VBA-wide. Individual reviews were being
    conducted by supervisors, such as team coaches, and nonsupervisory staff,
    such as Decision Review Officers, who are responsible for handling
    appeals. At the time of our review, VBA was in the process of deciding who
    would perform the reviews. In its efforts to develop SIPA as a consistent
    review process, VBA is considering issues such as ensuring that consistent
    feedback is provided to the employees and all regional offices are using
    the same review checklists.

    Conclusions 	While VBA's emphasis on production succeeded in increasing
    the number of claims decisions made during fiscal year 2002, the accuracy
    of those decisions declined. Because VBA did not report comparable
    accuracy rates for fiscal years 2001 and 2002 in its performance and
    accountability report, Members of Congress, Department of Veterans Affairs
    management, and claimants did not know of this decline in accuracy. The

    12In general, inexperienced claims processing employees have all of their
    work reviewed.

    13These standards, according to VBA officials, are intended to be floors;
    individual regional offices can set higher standards.

    lack of accurate performance data hampers stakeholders' ability to monitor
    VBA's performance in serving the nation's veterans.

    In addition, VBA could better use the accuracy data it has to hold its
    regional offices more accountable for accuracy. While VBA has made
    progress in measuring regional office accuracy, it has not made full use
    of the results to reward high performing offices or to improve poorly
    performing offices. VBA has made significant progress in getting its
    compensation and pension claims workload under control to improve
    timeliness for veterans waiting for decisions on their claims. However,
    VBA's accuracy has declined. Unless VBA balances its emphasis on producing
    more and faster decisions with an increased emphasis on accuracy, it will
    be difficult to ensure that veterans are getting the benefits to which
    they are entitled in a timely manner.

    Recommendations

    o

    o

      Agency Comments and Our Response

    We recommend that the Secretary of Veterans Affairs direct the Under
    Secretary for Benefits to

    report the accuracy of VBA disability compensation and pension claims
    decisions to the Congress and other stakeholders in a manner that allows
    for valid comparisons of accuracy across fiscal years and
    better hold regional offices accountable for the accuracy of their claims
    decisions, by increasing the use of its regional office accuracy data,
    while
    at the same time maintaining an appropriate emphasis on production.

    In its written comments on a draft of this report (see app. I), VBA
    concurred with our recommendations. However, VBA suggested that more
    effective use of STAR had been made to improve performance than our
    report concludes. We did note that VBA used STAR data to improve
    accuracy. Specifically, we noted in our report that VCAA-related errors
    declined to about one-third of all errors after VBA identified the errors
    through STAR and required retraining. We support VBA's continued efforts
    to emphasize accuracy and timeliness as critical components of quality.

    We will send copies of this report to the Secretary of Veterans Affairs,
    appropriate congressional committees, and other interested parties. We
    will also make copies of this report available to others on request. The
    report will also be available at no charge on GAO's Web site at
    http://www.gao.gov.

    If you or your staffs have any questions regarding this report, please
    call me at (202) 512-7215 or Irene Chu, Assistant Director, at (202)
    512-7102. In addition to those named, Susan Bernstein, Kristine Braaten,
    Kevin Jackson, Joseph Natalicchio, Martin Scire, and Greg Whitney made key
    contributions to this report.

    Cynthia A. Bascetta Director, Education, Workforce, and Income Security
    Issues

      GAO's Mission

    Obtaining Copies of GAO Reports and Testimony

    The General Accounting Office, the audit, evaluation and investigative arm
    of Congress, exists to support Congress in meeting its constitutional
    responsibilities and to help improve the performance and accountability of
    the federal government for the American people. GAO examines the use of
    public funds; evaluates federal programs and policies; and provides
    analyses, recommendations, and other assistance to help Congress make
    informed oversight, policy, and funding decisions. GAO's commitment to
    good government is reflected in its core values of accountability,
    integrity, and reliability.

    The fastest and easiest way to obtain copies of GAO documents at no cost
    is through the Internet. GAO's Web site (www.gao.gov) contains abstracts
    and full-text files of current reports and testimony and an expanding
    archive of older products. The Web site features a search engine to help
    you locate documents using key words and phrases. You can print these
    documents in their entirety, including charts and other graphics.

    Each day, GAO issues a list of newly released reports, testimony, and
    correspondence. GAO posts this list, known as "Today's Reports," on its
    Web site daily. The list contains links to the full-text document files.
    To have GAO e-mail this list to you every afternoon, go to www.gao.gov and
    select "Subscribe to e-mail alerts" under the "Order GAO Products"
    heading.

    Order by Mail or Phone 	The first copy of each printed report is free.
    Additional copies are $2 each. A check or money order should be made out
    to the Superintendent of Documents. GAO also accepts VISA and Mastercard.
    Orders for 100 or more copies mailed to a single address are discounted 25
    percent. Orders should be sent to:

    U.S. General Accounting Office 441 G Street NW, Room LM Washington, D.C.
    20548

    To order by Phone: 	Voice: (202) 512-6000 TDD: (202) 512-2537 Fax: (202)
    512-6061

    Contact:

    To Report Fraud, Web site: www.gao.gov/fraudnet/fraudnet.htm

      Waste, and Abuse in E-mail: fraudnet@gao.gov

    Federal Programs Automated answering system: (800) 424-5454 or (202)
    512-7470

    Jeff Nelligan, Managing Director, NelliganJ@gao.gov (202) 512-4800

    Public Affairs 	U.S. General Accounting Office, 441 G Street NW, Room 7149
    Washington, D.C. 20548
    *** End of document. ***