Information Technology: OMB Can More Effectively Use Its
Investment Reviews (21-APR-05, GAO-05-571T).
Federal spending on information technology (IT) is over $60
billion this year and is expected to continue to rise.
Accordingly, it is essential that federal IT investments are
managed efficiently. Of the 1,200 major IT projects in the
President's Budget for Fiscal Year 2005, OMB stated that it had
placed about half--621 projects, representing about $22
billion--on a Management Watch List to focus attention on
mission-critical IT investments that need management
improvements. GAO was asked to testify on the findings and
recommendations made in a report that it recently completed
(GAO-05-276), which describes and assesses OMB's processes for
(1) placing projects on its Management Watch List and (2)
following up on corrective actions established for projects on
the list.
-------------------------Indexing Terms-------------------------
REPORTNUM: GAO-05-571T
ACCNO: A22117
TITLE: Information Technology: OMB Can More Effectively Use Its
Investment Reviews
DATE: 04/21/2005
SUBJECT: Agency evaluation
Information technology
Presidential budgets
Program evaluation
Regulatory agencies
Data collection
Monitoring
Risk management
Corrective action
Investment management
OMB Management Watch List
******************************************************************
** This file contains an ASCII representation of the text of a **
** GAO Product. **
** **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced. Tables are included, but **
** may not resemble those in the printed version. **
** **
** Please see the PDF (Portable Document Format) file, when **
** available, for a complete electronic file of the printed **
** document's contents. **
** **
******************************************************************
GAO-05-571T
United States Government Accountability Office
GAO Testimony Before the Committee on Government Reform, House of
Representatives
For Release on Delivery
Expected at
10 a.m. EDT on Thursday,
April 21, 2005
INFORMATION
TECHNOLOGY
OMB Can More Effectively Use Its Investment Reviews
Statement of David A. Powner, Director, Information Technology Management Issues
GAO-05-571T
[IMG]
April 21, 2005
INFORMATION TECHNOLOGY
OMB Can More Effectively Use Its Management Reviews
What GAO Found
For the fiscal year 2005 budget, OMB developed processes and criteria for
including investments on its Management Watch List. In doing so, it
identified opportunities to strengthen investments and promote
improvements in IT management. However, it did not develop a single,
aggregate list identifying the projects and their weaknesses. Instead, OMB
officials told GAO that to identify projects with weaknesses, individual
analysts used scoring criteria that the office established for evaluating
the justifications for funding that federal agencies submit for major
projects. These analysts, each of whom is typically responsible for
several federal agencies, were then responsible for maintaining
information on these projects. To derive the total number of projects on
the list for fiscal year 2005, the office polled its individual analysts
and compiled the result. However, OMB officials told GAO that because they
did not see such an activity as necessary, they did not compile a single
list. Accordingly, OMB has not fully exploited the opportunity to use its
watch list as a tool for analyzing IT investments on a governmentwide
basis.
OMB asked agencies to take corrective actions to address weaknesses
associated with projects on the Management Watch List, but it did not
develop a structured, consistent process for deciding how to monitor
agency corrective actions. According to OMB officials, decisions on
monitoring of progress were typically made by the staff with
responsibility for reviewing individual agency budget submissions,
depending on the staff's insights into agency operations and objectives.
Because it did not consistently require or monitor agency follow-up
activities, OMB did not know whether the project risks that it identified
through its Management Watch List were being managed effectively,
potentially leaving resources at risk of being committed to poorly planned
and managed projects. In addition, because it did not consistently monitor
the follow-up performed on projects on the Management Watch List, OMB
could not readily tell GAO which of the 621 projects received follow-up
attention.
To help enable OMB to take advantage of the potential benefits of using
the Management Watch List as a tool for analyzing and following up on
investments, GAO's report included recommendations that OMB develop a
single, aggregate Management Watch List and that it develop and use
criteria for prioritizing and monitoring the projects on the list. GAO
also recommended that the office use the prioritized list for reporting to
the Congress as part of its statutory reporting responsibilities. In
commenting on a draft of this report, OMB did not agree that the
aggregated governmentwide list recommended by GAO is necessary for
adequate oversight and management. However, GAO continues to believe that
an aggregated Management Watch List would contribute to OMB's ability to
analyze IT investments governmentwide and track progress in addressing
deficiencies.
United States Government Accountability Office
Mr. Chairman and Members of the Committee:
Thank you for the opportunity to participate in the Committee's hearing on
processes that the Office of Management and Budget (OMB) has developed as
part of its efforts to identify and follow up on information technology
(IT) projects that need management improvements.
As you know, the President's Budget for Fiscal Year 2005 requested over
$60 billion to fund IT, and that figure is expected to rise throughout the
rest of the decade. OMB stated that of the nearly 1,200 major IT projects
in the fiscal year 2005 budget, it had placed about half-621 projects,
representing about $22 billion-on its Management Watch List. For fiscal
year 2006, 342 of 1,087 IT projects (representing about $15 billion) were
placed on the watch list.
At your request, we performed a review of OMB's processes for (1) placing
projects on its Management Watch List and (2) following up on corrective
actions established for projects on the list. Today I am summarizing the
findings and recommendations of that report, which is being released
today.1
Results in Brief
For the fiscal year 2005 budget, OMB developed processes and criteria for
including IT projects (investments) on its Management Watch List. In doing
so, it identified opportunities to strengthen investments and promote
improvements in IT management. However, OMB did not develop a single,
aggregate list identifying the projects and their weaknesses. Instead,
according to OMB officials, individual OMB analysts assigned scores to the
justifications for funding (known as exhibit 300s) that are submitted by
federal agencies. (These scores were based on criteria
1 GAO, Information Technology: OMB Can Make More Effective Use of Its
Investment Reviews, GAO-05-276 (Washington, D.C.: Apr. 15, 2005).
established in the office's Circular A-11.) OMB delegated individual
analysts on its staff, each of whom is typically assigned responsibility
for several federal agencies, with maintaining, for their respective
agencies, information for the IT projects included on the list. To derive
the total number of projects on the list that OMB reported (621 for fiscal
year 2005), OMB polled its individual analysts and compiled the numbers.
According to OMB officials, they did not construct a single list of
projects meeting their watch list criteria because they did not see such
an activity as necessary for performing OMB's predominant mission: to
assist in overseeing the preparation of the federal budget and to
supervise agency budget administration. Thus, OMB did not exploit the
opportunity to use the list as a tool for analyzing IT investments on a
governmentwide basis, limiting its ability to identify and report on the
full set of IT investments requiring corrective actions.
OMB asked agencies to take corrective actions to address weaknesses
associated with projects on the Management Watch List, but it did not
develop a structured, consistent process for deciding how to follow up on
these actions. According to OMB officials, decisions on follow-up and
monitoring of specific projects were typically made by the OMB staff with
responsibility for reviewing individual agency budget submissions,
depending on the staff's insights into agency operations and objectives.
Because it did not consistently monitor the follow-up performed, OMB could
not tell us which of the 621 projects identified on the fiscal year 2005
list received follow-up attention, and it did not know whether the
specific project risks that it identified through its Management Watch
List were being managed effectively. This approach could leave resources
at risk of being committed to poorly planned and managed projects. Thus,
OMB was not using its Management Watch List as a tool for improving IT
investments on a governmentwide basis and focusing attention where it was
most needed.
To enable OMB to take advantage of the potential benefits of using the
Management Watch List as a tool for analyzing and following up on IT
investments, we recommended in our report that OMB develop a single,
aggregate Management Watch List, and that it develop and use criteria for
prioritizing and monitoring the projects on the list. We also recommended
that the office use the prioritized
list for reporting to the Congress as part of its statutory reporting
responsibilities. In commenting on a draft of this report, OMB did not
agree that the aggregated governmentwide list recommended by GAO is
necessary for adequate oversight and management. However, GAO continues to
believe that an aggregated Management Watch List would contribute to OMB's
ability to analyze IT investments governmentwide and track progress in
addressing deficiencies.
Background
The President's Budget for Fiscal Year 2005 identified approximately $60
billion for IT projects. In that budget, OMB stated that, of approximately
1,200 major IT projects, about half-621 projects, representing about $22
billion-were on a Management Watch List. In testimony in March 2004,2 OMB
officials explained that the fiscal year 2005 budget process required
agencies to successfully correct project weaknesses and business case
deficiencies of projects on the Management Watch List; otherwise, OMB
would limit agencies' spending on new starts and other developmental
activities.
In the most recent budget, that for fiscal year 2006, OMB continued its
use of a Management Watch List. This budget includes 1,087 IT projects,
totaling about $65 billion. Of this total, 342 projects, representing
about $15 billion, are on the Management Watch List. The budget also
stated that projects on the Management Watch List had to address
performance, security, or other related issues before funding would be
obligated in fiscal year 2006.
2On March 3, 2004, OMB's Deputy Director for Management and its
Administrator for Electronic Government and Information Technology
testified at a hearing conducted by the Subcommittee on Technology,
Information Policy, Intergovernmental Relations and the Census, Committee
on Government Reform, House of Representatives. The hearing topic was
"Federal Information Technology Investment Management, Strategic Planning,
and Performance Measurement: $60 Billion Reasons Why."
According to OMB officials, the office identifies projects for the
Management Watch List through their evaluation of justifications for
funding that agencies submit for major IT projects as part of the budget
development process. This evaluation is carried out as part of OMB's
predominant mission: to assist the President in overseeing the preparation
of the federal budget and to supervise budget administration in executive
branch agencies. OMB is also responsible for evaluating the effectiveness
of agency programs, policies, and procedures; assessing competing funding
demands among agencies; and setting funding priorities. Finally, OMB is
responsible for overseeing and coordinating the administration's policies
regarding procurement, financial management, information, and regulations.
In each of these three areas of responsibility, OMB's role is to help
improve administrative management, to develop better performance measures
and coordinating mechanisms, and to reduce unnecessary burden on the
public.
To drive improvement in the implementation and management of IT projects,
the Congress enacted the Clinger-Cohen Act in 1996, which expanded the
responsibilities of the agencies and OMB under the Paperwork Reduction
Act.3 Under the act, agencies are required to engage in capital planning
and performance- and results-based management. OMB is required to
establish processes to analyze, track, and evaluate the risks and results
of major capital investments in information systems made by executive
agencies. OMB is also required to report to the Congress on the net
program performance benefits achieved as a result of major capital
investments in information systems that are made by executive agencies.4
In response to the Clinger-Cohen Act and other statutes, OMB developed
section 300 of Circular A-11. This section provides policy for planning,
budgeting, acquisition, and management of federal capital assets and
instructs agencies on budget justification and
344 U.S.C. S: 3504(a)(1)(B)(vi) (OMB); 44 U.S.C. S: 3506(h)(5) (agencies).
4These requirements are specifically described in the Clinger-Cohen Act,
40 U.S.C. S: 11302(c).
reporting requirements for major IT investments.5 Section 300 defines the
budget exhibit 300 as a document that agencies submit to OMB to justify
resource requests for major IT investments. This reporting mechanism (part
of the budget formulation and review process) is intended to enable an
agency to demonstrate to its own management, as well as to OMB, that it
has employed the disciplines of good project management; developed a
strong business case for the investment; and met other Administration
priorities in defining the cost, schedule, and performance goals proposed
for the investment. The exhibit 300 includes information that is intended,
among other things, to help OMB and the agencies identify and correct
poorly planned or performing investments (i.e., investments that are
behind schedule, over budget, or not delivering expected results) and real
or potential systemic weaknesses in federal information resource
management (e.g., project manager qualifications).
According to OMB's description of its processes, agencies' exhibit 300
business cases are reviewed by OMB analysts from its four statutory
offices-the Offices of E-Government and Information Technology (e-Gov),
Information and Regulatory Affairs (OIRA), Federal Financial Management,
and Federal Procurement Policy- and its Resource Management Offices (RMO).
Within OIRA, each of about 12 analysts is responsible for overseeing IT
projects for a specific agency or (more commonly) several agencies.
According to OMB officials, the OIRA and e-Gov analysts, along with RMO
program examiners, evaluate and score agency exhibit 300 business cases as
part of the development of the President's Budget. The results of this
review are provided to agencies through what is called the "passback"
process. That is, OMB passes the requests back to agencies with its
evaluation, which identifies any areas requiring remediation.
5OMB Circular A-11 defines a major IT investment as an investment that
requires special management attention because of its importance to an
agency's mission or because it is an integral part of the agency's
enterprise architecture, has significant program or policy implications,
has high executive visibility, or is defined as major by the agency's
capital planning and investment control process.
The integrity of this review process presupposes that the exhibit 300s are
accurate. In response to a request from this committee, we are currently
reviewing the quality of the information that underlies exhibit 300s at
several agencies. We will be reporting on this work in the fall of this
year.
OMB Established Processes and Criteria for Identifying Weak
Projects, but It Did Not Use an Aggregate List to Perform Its
Analysis or Oversight According to OMB officials, including the Deputy
Administrator of OIRA and the Chief of the Information Technology and
Policy Branch, OMB staff identified projects for the Management Watch List
through their evaluation of the exhibit 300s that agencies submit for
major IT projects as part of the budget development process. The OMB
officials added that the scoring of agency exhibit 300s is based on
guidance in OMB Circular A-116 that is intended to ensure that agency
planning and management of capital assets are consistent with OMB policy
and guidance.
As described in Circular A-11, the scoring of a business case consists of
individual scoring for 10 categories, as well as a total composite score
of all the categories. (Examples of these 10 categories are performance
goals, security and privacy, performance-based management system-including
the earned value management system7-and support of the President's
Management Agenda.) According to Circular A-11, scores range from 1 to 5,
with 5 indicating investments whose business cases provided the best
justification and 1 the least.
6These scoring criteria are presented in Office of Management and Budget
Circular A-11, Part 7, Planning, Budgeting, Acquisition, and Management of
Capital Assets (July 2004).
7Earned value management is a project management tool that integrates the
investment scope of work with schedule and cost elements for investment
planning and control. This method compares the value of work accomplished
during a given period with that of the work expected in the period.
Differences in expectations are measured in both cost and schedule
variances.
OMB officials said that, for fiscal year 2005, an IT project was placed on
the Management Watch List if its exhibit 300 business case received a
total composite score of 3 or less, or if it received a score of 3 or less
in the areas of performance goals, performance-based management systems,
or security and privacy, even if its overall score was a 4 or 5. OMB
reported that agencies with weaknesses in these three areas were to submit
remediation plans addressing the weaknesses.
According to OMB management, individual analysts were responsible for
evaluating projects and determining which projects met the criteria to be
on the Management Watch List for their assigned agencies. To derive the
total number of projects on the list that were reported for fiscal year
2005, OMB polled the individual analysts and compiled the numbers.
OMB officials said that they did not aggregate these projects into a
single list describing projects and their weaknesses, because they did not
see such an activity as necessary in performing OMB's predominant mission.
Further, OMB officials stated that the limited number of analysts involved
enabled them to explore governmentwide issues using ad hoc queries and to
develop approaches to address systemic problems without the use of an
aggregate list. They pointed at successes in improving IT management, such
as better compliance with security requirements, as examples of the
effectiveness of their current approach.
Nevertheless, OMB has not fully exploited the opportunity to use its
Management Watch List as a tool for analyzing IT investments on a
governmentwide basis. According to the Clinger-Cohen Act, OMB is required
to establish processes to analyze, track, and evaluate the risks and
results of major IT capital investments made by executive agencies, which
aggregation of the Management Watch List would facilitate. Without
aggregation, OMB's ability to conduct governmentwide analysis is limited,
since no governmentwide dataset exists-only a set of subordinate datasets
in the hands of individual analysts. In addition, each time an up-to-date
report is required, OMB must query all its analysts to assemble an
aggregate response; thus, the office cannot efficiently identify, analyze,
and report on the full set of IT investments requiring corrective actions.
OMB's Monitoring of Projects Was Inconsistent, and Agency Follow
up Activities Were Not Tracked Centrally OMB asked agencies to take
corrective actions to address weaknesses associated with projects on the
Management Watch List, but it did not develop a structured, consistent
process or criteria for deciding how to follow up on these actions.
Instead, OMB officials, including the Deputy Administrator of OIRA and the
Chief of the Information Technology and Policy Branch, said that the
decision on whether and how to follow up on a specific project was
typically made jointly between the OIRA analyst and the RMO program
examiner who had responsibility for the individual agency, and that
follow-up on specific projects was driven by a number of factors, only one
of which was inclusion on the Management Watch List. According to these
officials, those Management Watch List projects that did receive specific
follow-up attention received feedback through the passback process,
through targeted evaluation of remediation plans designed to address
weaknesses, and through the apportioning of funds so that the use of
budgeted dollars was conditional on appropriate remediation plans being in
place.8
These officials also said that follow-up of some Management Watch List
projects was done through quarterly e-Gov Scorecards; these are reports
that use a red/yellow/green scoring system to illustrate the results of
OMB's evaluation of the agencies' implementation of e-government criteria
in the President's Management Agenda. OMB determines the scores in
quarterly reviews, in which it evaluates agency progress toward
agreed-upon goals along several dimensions. The e-gov scores are part of
the input to the quarterly reporting on the President's Management Agenda.
OMB officials also stated that those Management Watch List projects that
did receive follow-up attention were not tracked centrally, but only by
the individual OMB analysts. Accordingly, OMB could not readily tell us
which of the 621 watch list projects
8The authority for apportioning funds is specifically described in the
Clinger-Cohen Act, 40 U.S.C. S: 11303(b)(5)(B)(ii).
for fiscal year 2005 were followed up on, nor could it use the list to
describe the relationship between its follow-up activities and the changes
in the numbers of projects on the watch list between fiscal year 2005 (621
projects) and fiscal year 2006 (342 projects).
OMB does not have specific criteria for prioritizing follow-up on
Management Watch List projects. Without specific criteria, OMB staff may
be agreeing to commit resources to follow up on projects that did not
represent OMB's top priorities from a governmentwide perspective. For
example, inconsistent attention to OMB priorities, such as earned value
management, could undermine the objectives that OMB set in these areas. In
addition, major projects with significant management deficiencies may have
continued to absorb critical agency resources.
In order for OMB management to have assurance that IT program deficiencies
are addressed, it is critical that corrective actions associated with
Management Watch List projects be monitored. Such monitoring is
instrumental in ensuring that agencies address and resolve weaknesses
found in exhibit 300s, which may indicate underlying weaknesses in project
planning or management. Tracking agency follow-up activities is essential
to enabling OMB to determine progress on both specific projects and
governmentwide trends. Without tracking specific follow-up activities, OMB
could not readily ascertain whether the risks that it identified through
its Management Watch List were being managed effectively; if they were
not, funds were potentially being spent on poorly planned and managed
projects.
In summary, OMB's scoring of agency IT budget submissions and
identification of weaknesses has resulted in opportunities to strengthen
investments. However, the office has not taken the next step-to develop a
single, aggregate list identifying the projects and their weaknesses-and
it has not developed a structured, consistent process for deciding how to
follow up on corrective actions. OMB's approach does not fully exploit the
insights developed through the scoring process, and it may leave
unattended weak projects consuming significant budget dollars. Developing
an aggregated list
would help OMB to realize more fully the potential benefits of using the
Management Watch List as a tool for monitoring and analyzing IT
investments governmentwide. Accordingly, in our report we recommended that
the Director of OMB take the following four actions:
0M Develop a central list of projects and their deficiencies.
0M Use the list as the basis for selecting projects for follow-up and for
tracking follow-up activities;
0M to guide follow-up, develop specific criteria for prioritizing the IT
projects included on the list, taking into consideration such factors as
the relative potential financial and program benefits of these IT
projects, as well as potential risks.
0M Analyze the prioritized list to develop governmentwide and agency
assessments of the progress and risks of IT investments, identifying
opportunities for continued improvement.
0M Report to the Congress on progress made in addressing risks of major IT
investments and management areas needing attention.
In commenting on a draft of this report, OMB's Administrator of the Office
of E-Government and Information Technology expressed appreciation for our
review of OMB's use of its Management Watch List. However, the
Administrator disagreed with our assessment that an aggregated
governmentwide list is necessary to perform adequate oversight and
management, and that OMB does not know whether risks are being addressed.
Nonetheless, based on OMB's inability to easily report which of the 621
investments on the Management Watch List remained deficient or how much of
the $22 billion cited in the President's Budget remained at risk, we
continue to believe that an aggregate list would facilitate OMB's ability
to track progress.
Mr. Chairman, that concludes my testimony. I would be pleased to answer
any questions that you and the other Members of the Committee may have.
Contact and Acknowledgements
For further information, please contact David A. Powner at (202) 512-9286
or Lester Diamond at (202) 512-7957. We can also be reached by e-mail at
pownerd@gao.gov or diamondl@gao.gov. Key contributors to this testimony
were William G. Barrick, Barbara Collier, Lester Diamond, and Sandra Kerr.
This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed in
its entirety without further permission from GAO. However, because this
work may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this material
separately.
GAO's Mission
Obtaining Copies of GAO Reports and Testimony
The Government Accountability Office, the audit, evaluation and
investigative arm of Congress, exists to support Congress in meeting its
constitutional responsibilities and to help improve the performance and
accountability of the federal government for the American people. GAO
examines the use of public funds; evaluates federal programs and policies;
and provides analyses, recommendations, and other assistance to help
Congress make informed oversight, policy, and funding decisions. GAO's
commitment to good government is reflected in its core values of
accountability, integrity, and reliability.
The fastest and easiest way to obtain copies of GAO documents at no cost
is through GAO's Web site (www.gao.gov). Each weekday, GAO posts newly
released reports, testimony, and correspondence on its Web site. To have
GAO e-mail you a list of newly posted products every afternoon, go to
www.gao.gov and select "Subscribe to Updates."
Order by Mail or Phone The first copy of each printed report is free.
Additional copies are $2 each. A check or money order should be made out
to the Superintendent of Documents. GAO also accepts VISA and Mastercard.
Orders for 100 or more copies mailed to a single address are discounted 25
percent. Orders should be sent to:
U.S. Government Accountability Office 441 G Street NW, Room LM Washington,
D.C. 20548
To order by Phone: Voice: (202) 512-6000 TDD: (202) 512-2537 Fax: (202)
512-6061
To Report Fraud, Contact:
Waste, and Abuse in Web site: www.gao.gov/fraudnet/fraudnet.htm
E-mail: fraudnet@gao.govFederal Programs Automated answering system: (800)
424-5454 or (202) 512-7470
Gloria Jarmon, Managing Director, JarmonG@gao.gov (202)
512-4400Congressional U.S. Government Accountability Office, 441 G Street
NW, Room 7125 Relations Washington, D.C. 20548
Public Affairs Paul Anderson, Managing Director, AndersonP1@gao.gov (202)
512-4800 U.S. Government Accountability Office, 441 G Street NW, Room 7149
Washington, D.C. 20548
PRINTED ON RECYCLED PAPER
*** End of document. ***