Veterans Affairs: Improved Planning Needed to Guide Development
and Implementation of Education Benefits System (31-JUL-07,
GAO-07-1045).
The Department of Veterans Affairs (VA) administers programs that
provide educational resources to veterans, service members,
reservists, and eligible family members of veterans. Under an
initiative that the Veterans Benefits Administration (VBA) began
in 1999, called The Education Expert System (TEES), VA is moving
its education benefits claims processing systems to new
technology platforms and replacing the current systems with a new
system that is to add more automated processes and eliminate most
human intervention. GAO was requested to (1) report on the
results of TEES development to date and identify the remaining
work to complete the project, (2) identify VBA's plans for
completing and implementing TEES, and (3) assess the feasibility
of developing and deploying TEES before the completion of VBA's
Finance and Accounting System. To accomplish these objectives,
GAO analyzed relevant documentation, interviewed program and
project officials, and observed processing systems at VA regional
offices.
-------------------------Indexing Terms-------------------------
REPORTNUM: GAO-07-1045
ACCNO: A73690
TITLE: Veterans Affairs: Improved Planning Needed to Guide
Development and Implementation of Education Benefits System
DATE: 07/31/2007
SUBJECT: Claims processing
Management information systems
Program evaluation
Program management
Strategic information systems planning
Systems conversions
Veterans benefits
Veterans education
Program implementation
The Education Expert System
******************************************************************
** This file contains an ASCII representation of the text of a **
** GAO Product. **
** **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced. Tables are included, but **
** may not resemble those in the printed version. **
** **
** Please see the PDF (Portable Document Format) file, when **
** available, for a complete electronic file of the printed **
** document's contents. **
** **
******************************************************************
GAO-07-1045
* [1]Conclusions
* [2]Recommendations
* [3]Agency Comments and Our Evaluation
* [4]GAO Contact
* [5]Staff Acknowledgments
* [6]GAO's Mission
* [7]Obtaining Copies of GAO Reports and Testimony
* [8]Order by Mail or Phone
* [9]To Report Fraud, Waste, and Abuse in Federal Programs
* [10]Congressional Relations
* [11]Public Affairs
Report to the Subcommittee on Economic Opportunity, Committee on Veterans'
Affairs, House of Representatives
United States Government Accountability Office
GAO
July 2007
VETERANS AFFAIRS
Improved Planning Needed to Guide Development and Implementation of
Education Benefits System
GAO-07-1045
Contents
Letter 1
Conclusions 3
Recommendations 3
Agency Comments and Our Evaluation 4
Appendix I Veterans Affairs: Improved Planning Needed to Guide
Implementation of The Education Expert System (TEES) 6
Appendix II Comments from the Department of Veterans Affairs 42
Appendix III GAO Contact and Staff Acknowledgments 46
This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed in
its entirety without further permission from GAO. However, because this
work may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this material
separately.
Abbreviations
FAS Finance and Accounting System
OMB Office of Management and Budget
TEES The Education Expert System
VA Department of Veterans Affairs
VBA Veterans Benefits Administration
United States Government Accountability Office
Washington, DC 20548
July 31, 2007
The Honorable Stephanie Herseth Sandlin Chair The Honorable John Boozman
Ranking Member Subcommittee on Economic Opportunity Committee on Veterans'
Affairs House of Representatives The Honorable Stephanie Herseth Sandlin
Chair The Honorable John Boozman Ranking Member Subcommittee on Economic
Opportunity Committee on Veterans' Affairs House of Representatives
Under an initiative called The Education Expert System (TEES), the
Department of Veterans Affairs (VA) is taking steps intended to improve
its education benefits delivery services by moving its education claims
processing systems to new technology platforms and a new architecture. The
effort, which the Veterans Benefits Administration (VBA) initiated in
December 1999, is expected to enable faster and more accurate processing
of education claims by replacing the existing education benefits systems
within VBA's antiquated Benefits Delivery Network with a new rules-based
system that will add more automated capabilities and eliminate most human
intervention. When it began the initiative, VBA had planned to complete
the new system by September 2005; however, in 2004, the department
refocused and rebaselined the system's development effort. VA currently
estimates that the TEES initiative will be completed by 2011. Under an
initiative called The Education Expert System (TEES), the Department of
Veterans Affairs (VA) is taking steps intended to improve its education
benefits delivery services by moving its education claims processing
systems to new technology platforms and a new architecture. The effort,
which the Veterans Benefits Administration (VBA) initiated in December
1999, is expected to enable faster and more accurate processing of
education claims by replacing the existing education benefits systems
within VBA's antiquated Benefits Delivery Network with a new rules-based
system that will add more automated capabilities and eliminate most human
intervention. When it began the initiative, VBA had planned to complete
the new system by September 2005; however, in 2004, the department
refocused and rebaselined the system's development effort. VA currently
estimates that the TEES initiative will be completed by 2011.
At your request, we examined VA's actions on the TEES initiative.
Specifically, our objectives were to At your request, we examined VA's
actions on the TEES initiative. Specifically, our objectives were to
o report the results of VBA's development of TEES to date, and
identify the remaining work needed to complete the system; o
report the results of VBA's development of TEES to date, and
identify the remaining work needed to complete the system;
o identify VBA's plans for completing and implementing TEES; and
o identify VBA's plans for completing and implementing TEES; and
o assess the feasibility of developing and deploying TEES
application processing functions before the completion of VBA's
Finance and Accounting System (FAS). o assess the feasibility of
developing and deploying TEES application processing functions
before the completion of VBA's Finance and Accounting System
(FAS).
To address these objectives, we obtained and analyzed documentation
describing the planned and ongoing activities for TEES and FAS, including
functional requirements documents, system development and code conversion
contracts for the initiative, regional office processing metrics, and cost
information. We supplemented our analysis with interviews of To address
these objectives, we obtained and analyzed documentation describing the
planned and ongoing activities for TEES and FAS, including functional
requirements documents, system development and code conversion contracts
for the initiative, regional office processing metrics, and cost
information. We supplemented our analysis with interviews of education
program officials, project managers, and other officials responsible for
developing the new system. In addition, we observed education claims
processing and the systems currently used to support claims processing at
two of VA's four regional offices--Atlanta, Georgia and Muskogee,
Oklahoma.^1 We performed our work at VA's headquarters in Washington,
D.C., and at the Atlanta and Muskogee Regional Offices from October 2006
through May 2007, in accordance with generally accepted government
auditing standards.
^1Education benefit claims are processed at four VA regional offices.
On May 25, 2007, we provided your offices with briefing slides (included
as appendix I) that outlined the results of our study. Subsequently, we
met with your staff on June 15, 2007, to further discuss our findings and
recommendations. The purpose of this report is to provide the published
briefing slides to you and to officially transmit our recommendations to
the Secretary of Veterans Affairs.
In summary, our study highlighted the following:
o VBA has, to date, enhanced education benefits claims processing
by developing certain functionalities to allow information to be
captured in an electronic format. For example, it has developed
automated systems that allow (1) education institutions to provide
online enrollment certifications, (2) students to provide online
and telephonic verification of enrollment, and (3) the public to
inquire about approved academic programs, licensing and
certification programs, and national exams. However, VBA has not
taken action on various other initiatives that it identified as
necessary to complete the new system and eliminate most human
intervention. These initiatives include (1) moving the
functionality provided by its aging, stand-alone system used to
process flight school, on-the-job training, and apprenticeships
claims to a platform compliant with the department's enterprise
architecture; (2) moving the processing and payment functionality
used for many of the education claims from its current antiquated
system to new technology; and (3) creating a rules-based claims
processing system that will process all education benefits and
types of training.
o VBA does not have an integrated project plan for the TEES
initiative. According to agency officials, the plan that had been
developed in 2001 has not been updated since 2004, when program
goals were modified. Agency officials told us the most current
detailed information on planned TEES-related efforts is the fiscal
year 2008 TEES Exhibit 300, submitted to the Office of Management
and Budget (OMB), which includes an initiative that could
duplicate the Benefits Delivery Network code conversion effort.
Because VBA does not have an integrated project management plan,
it lacks
o critical elements needed to effectively guide the
initiative to completion--such as a full description
of the scope of the system development efforts,
identified stakeholders' responsibilities, documented
risks, and performance metrics; and
o an overall approach for coordinating its various
education claims initiatives, such as its Benefits
Delivery Network code conversion effort and the
development of a rules-based system to improve
education claims processing.
o Since much of the functionality for FAS has been developed, VBA
could implement a rules-based system that is integrated with FAS.
However, while the current strategy calls for using FAS as the
payment system for TEES, the existing functional requirements for
an interface between these two systems, which were developed in
2003, have not yet been assessed and updated. Subject matter
experts needed to define business requirements for interfacing the
systems have not been assigned by a key stakeholder. Thus, it is
uncertain when VBA will validate and update functional
requirements for this interface.
Conclusions
While VBA has completed several new initiatives to provide more
automated assistance for processing education claims, the absence
of an integrated project plan for TEES could place the department
at risk of wasting millions of dollars on education claims
processing initiatives that may overlap or be duplicative.
Further, without updated requirements, VA's success in interfacing
the new system with FAS, and ultimately processing education
benefits payments more efficiently, could be jeopardized.
Recommendations
To ensure the successful implementation of a system that provides
consistent and effective delivery of education benefits payments,
we are recommending that the Secretary of Veterans Affairs, before
approving any new funding for TEES and related systems, direct
VA's Chief Information Officer to take the following three
actions:
o Develop a comprehensive, integrated TEES project plan that
1. defines critical elements, including but not
limited to project scope, budget, schedule, risk,
planning for data management, needed knowledge and
skills, and stakeholder involvement; and
2. addresses redundancy between the requests for
developing a TEES rules-based system, the
department-level code conversion effort, and
integration of the TEES development with FAS.
o Once an integrated plan among related TEES initiatives has been
completed and documented, revise the OMB Exhibit 300s for TEES and
related efforts to ensure that duplication in efforts and budgets
is eliminated and that funding is consistent with timelines.
o Obtain stakeholders' commitment of resources to ensure that
functional requirements are revalidated and updated as necessary
to provide an interface between FAS and the future VBA education
systems.
Agency Comments and Our Evaluation
We received written comments on a draft of this report from the
Deputy Secretary of Veterans Affairs. (The department's comments
are reproduced in appendix II.) In the comments, VA generally
agreed with our findings and concurred with our recommendations.
The department acknowledged that there is still much to be done to
ensure that TEES is properly planned and developed. The comments
described actions planned that respond to our recommendations,
such as obtaining independent contractor support to conduct a TEES
program assessment and validation of consolidated functional
requirements in conjunction with defining and implementing a
program-level governance organization. In addition, the department
stated that its Education Project Management Service would work
with the responsible project manager to ensure TEES timelines,
requirements, risks, and budget requirements are accurately
reflected in VA's FY09 OMB Exhibit 300 submission. Further, it
stated that Education Project Management Service would coordinate
with related program offices, such as Benefits Delivery Network
Rehosting and the Finance and Accounting System, to ensure that
intersections and dependencies have been identified and accounted
for in all schedules, and eliminate the possibility of redundant
efforts. If the planned actions are properly implemented and
monitored by agency leadership, they should help to ensure that
TEES will be successfully completed. The department also provided
technical comments on the draft report, which we incorporated as
appropriate.
We are sending copies of this report to the Chairman and Ranking
Member of the Committee on Veterans' Affairs. We are also sending
copies to the Secretary of Veterans Affairs and other appropriate
congressional committees. We will make copies available to other
interested parties upon request. Copies of this report will also
be made available at no charge on GAO's Web site at
http://www.gao.gov.
Should you or your staff have any questions about this report,
please contact me at (202) 512-6304 or [email protected]. Contact
points for our Offices of Congressional Relations and Public
Affairs may be found on the last page of this report. GAO staff
who made major contributions to this report are listed in appendix
III.
Valerie C. Melvin
Director, Human Capital and Management Information Systems Issues
Appendix I: Veterans Affairs: Improved Planning Needed to Guide
Implementation of The Education Expert System (TEES)
Appendix II: Comments from the Department of Veterans Affairs
Appendix III: GAO Contact and Staff Acknowledgments
GAO Contact
Valerie C. Melvin (202) 512-6304 or melvinv @gao.gov
Staff Acknowledgments
In addition to the contact named above, key contributions to this
report were made by Barbara Oliver, Assistant Director; Charles
Youman; B. Scott Pettis; Matthew R. Grote; and Amos A. Tevelow.
GAO's Mission
The Government Accountability Office, the audit, evaluation and
investigative arm of Congress, exists to support Congress in
meeting its constitutional responsibilities and to help improve
the performance and accountability of the federal government for
the American people. GAO examines the use of public funds;
evaluates federal programs and policies; and provides analyses,
recommendations, and other assistance to help Congress make
informed oversight, policy, and funding decisions. GAO's
commitment to good government is reflected in its core values of
accountability, integrity, and reliability.
Obtaining Copies of GAO Reports and Testimony
The fastest and easiest way to obtain copies of GAO documents at
no cost is through GAO's Web site ( [12]www.gao.gov ). Each
weekday, GAO posts newly released reports, testimony, and
correspondence on its Web site. To have GAO e-mail you a list of
newly posted products every afternoon, go to [13]www.gao.gov and
select "Subscribe to Updates."
Order by Mail or Phone
The first copy of each printed report is free. Additional copies
are $2 each. A check or money order should be made out to the
Superintendent of Documents. GAO also accepts VISA and Mastercard.
Orders for 100 or more copies mailed to a single address are
discounted 25 percent. Orders should be sent to:
U.S. Government Accountability Office 441 G Street NW, Room LM
Washington, D.C. 20548
To order by Phone: Voice: (202) 512-6000 TDD: (202) 512-2537 Fax:
(202) 512-6061
To Report Fraud, Waste, and Abuse in Federal Programs
Contact:
Web site: [14]www.gao.gov/fraudnet/fraudnet.htm E-mail:
[15][email protected] Automated answering system: (800) 424-5454 or
(202) 512-7470
Congressional Relations
Gloria Jarmon, Managing Director, [16][email protected] (202)
512-4400 U.S. Government Accountability Office, 441 G Street NW,
Room 7125 Washington, D.C. 20548
Public Affairs
Paul Anderson, Managing Director, [17][email protected] (202)
512-4800 U.S. Government Accountability Office, 441 G Street NW,
Room 7149 Washington, D.C. 20548
(310907)
[18]www.gao.gov/cgi-bin/getrpt?GAO-07-1045 .
To view the full product, including the scope
and methodology, click on the link above.
For more information, contact Valerie Melvin at (202) 512-6304 or
[email protected].
Highlights of [19]GAO-07-1045 , a report to the Subcommitee on Economic
Opportunity, Committee on Veterans' Affairs, House of Representatives
July 2007
VETERANS AFFAIRS
Improved Planning Needed to Guide Development and Implementation of
Education Benefits System
The Department of Veterans Affairs (VA) administers programs that provide
educational resources to veterans, service members, reservists, and
eligible family members of veterans. Under an initiative that the Veterans
Benefits Administration (VBA) began in 1999, called The Education Expert
System (TEES), VA is moving its education benefits claims processing
systems to new technology platforms and replacing the current systems with
a new system that is to add more automated processes and eliminate most
human intervention. GAO was requested to (1) report on the results of TEES
development to date and identify the remaining work to complete the
project, (2) identify VBA's plans for completing and implementing TEES,
and (3) assess the feasibility of developing and deploying TEES before the
completion of VBA's Finance and Accounting System. To accomplish these
objectives, GAO analyzed relevant documentation, interviewed program and
project officials, and observed processing systems at VA regional offices.
[20]What GAO Recommends
To ensure the successful implementation of TEES, GAO is recommending that
the Secretary of Veterans Affairs, among other actions, require the
development of a comprehensive, integrated project plan to coordinate and
manage the initiative. Commenting on a draft of this report, VA concurred
with GAO's recommendations and described actions planned to address them.
To date, as part of the TEES project, VBA has enhanced education benefits
claims processing by developing certain capabilities that allow
information to be captured in an electronic format. However, VBA has not
taken action on various other initiatives that it identified as necessary
to complete the project. These initiatives include (1) moving the
functionality provided by its aging, stand-alone system used to process
flight school, on-the-job training, and apprenticeship claims to a
platform compliant with the department's enterprise architecture; (2)
moving the processing and payment functionality used for many of the
education claims from its current antiquated system to new technology; and
(3) creating a rules-based claims processing system that will process all
education benefits and types of training.
VBA does not have an integrated project management plan for the TEES
initiative. According to agency officials, a project management plan
originally developed in 2001 has not been updated since 2004. Because VBA
does not have an integrated project management plan, it lacks critical
elements needed to effectively guide the project to completion, such as a
full description of its scope; identification of stakeholders'
responsibilities, documented risks, and performance metrics; and an
overall approach for coordinating its various education claims improvement
initiatives. Moreover, without an integrated project management plan, VBA
risks initiating work that could be duplicative and overlapping.
VBA's Finance and Accounting System (FAS) is substantially complete and
can currently pay new compensation claims; all functionality for
compensation and pension payments is scheduled to be completed by August
2008. Thus, since much of the functionality for FAS has been developed,
VBA could implement a rules-based system that integrates with FAS.
However, while the current strategy calls for using FAS as the payment
system for TEES, the existing functional requirements for an interface
between the two systems, which were developed in 2003, have not been
assessed and updated. Subject matter experts needed to define business
requirements for interfacing the systems have not been assigned by a key
stakeholder. As a result, it is uncertain when VBA will validate and
update functional requirements for the FAS interface with TEES.
References
Visible links
12. http://www.gao.gov/
13. http://www.gao.gov/
14. http://www.gao.gov/fraudnet/fraudnet.htm
15. mailto:[email protected]
16. mailto:[email protected]
17. mailto:[email protected]
18. http://www.gao.gov/cgi-bin/getrpt?GAO-07-1045
19. http://www.gao.gov/cgi-bin/getrpt?GAO-07-1045
*** End of document. ***