Information Technology Management: Improvements Needed in
Strategic Planning, Performance Measurement, and Investment
Management Governmentwide (03-MAR-04, GAO-04-478T).
The federal government spends billions of dollars annually on
information technology (IT) investments that are critical to the
effective implementation of major government programs. To help
agencies effectively manage their substantial IT investments, the
Congress has established a statutory framework of requirements
and roles and responsibilities relating to information and
technology management, that addresses, for example, (1) IT
strategic planning/performance measurement (which defines what an
organization seeks to accomplish, identifies the strategies it
will use to achieve desired results, and then determines how well
it is succeeding in reaching resultsoriented goals and achieving
objectives) and (2) IT investment management (which involves
selecting, controlling, and evaluating investments). GAO was
asked to summarize its January 2004 report on IT strategic
planning/performance measurement and investment management
(Information Technology Management: Governmentwide Strategic
Planning, Performance Measurement, and Investment Management Can
Be Further Improved, GAO-04-49, January 12, 2004) and to discuss
how agencies can improve their performance in these areas.
-------------------------Indexing Terms-------------------------
REPORTNUM: GAO-04-478T
ACCNO: A09419
TITLE: Information Technology Management: Improvements Needed in
Strategic Planning, Performance Measurement, and Investment
Management Governmentwide
DATE: 03/03/2004
SUBJECT: Agency evaluation
Agency missions
Federal agencies
Financial management
General management reviews
Information technology
Performance measures
Strategic planning
******************************************************************
** This file contains an ASCII representation of the text of a **
** GAO Product. **
** **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced. Tables are included, but **
** may not resemble those in the printed version. **
** **
** Please see the PDF (Portable Document Format) file, when **
** available, for a complete electronic file of the printed **
** document's contents. **
** **
******************************************************************
GAO-04-478T
United States General Accounting Office
Testimony
Before the Subcommittee on Technology,
Information Policy, Intergovernmental
Relations and the Census, Committee on
Government Reform, House of Representatives
For Release on Delivery
Expected at 1:00 p.m. EST
Wednesday, March 3, 2004 INFORMATION TECHNOLOGY
MANAGEMENT
Improvements Needed in Strategic Planning, Performance Measurement, and
Investment Management Governmentwide
Statement of David A. Powner, Director, Information Technology Management Issues
GAO-04-478T
Highlights of GAO-04-478T, testimony before the Subcommittee on
Technology, Information Policy, Intergovernmental Relations and the
Census, Committee on Government Reform, House of Representatives
The federal government spends billions of dollars annually on information
technology (IT) investments that are critical to the effective
implementation of major government programs. To help agencies effectively
manage their substantial IT investments, the Congress has established a
statutory framework of requirements and roles and responsibilities
relating to information and technology management, that addresses, for
example, (1) IT strategic planning/performance measurement (which defines
what an organization seeks to accomplish, identifies the strategies it
will use to achieve desired results, and then determines how well it is
succeeding in reaching resultsoriented goals and achieving objectives) and
(2) IT investment management (which involves selecting, controlling, and
evaluating investments).
March 2004
INFORMATION TECHNOLOGY MANAGEMENT
Improvements Needed in Strategic Planning, Performance Measurement, and
Investment Management Governmentwide
GAO recently reported that the use of important IT strategic planning/
performance measurement and investment management practices by 26 major
federal agencies was mixed (see figure below). For example, agencies
generally had IT strategic plans and goals, but these goals were not
always linked to specific performance measures that were tracked. Agencies
also largely had IT investment management boards, but no agency had the
practices associated with the oversight of IT investments fully in place.
Although they could not always provide an explanation, agencies cited a
variety of reasons for not having practices fully in place, including that
the chief information officer position had been vacant and that the
process was being revised. By improving their IT strategic planning,
performance measurement, and investment management, agencies can better
ensure that they are being responsible stewards of the billions of dollars
for IT that they have been entrusted with through the wise investment of
these monies.
To help agencies improve in these areas, GAO has made numerous
recommendations to agencies and issued guidance. For example, in the
January 2004 report, GAO made recommendations to the 26 agencies regarding
practices that were not fully in place. In addition, today GAO is
releasing the latest version of its Information Technology Investment
Management (ITIM) framework, which identifies critical processes for
selecting, controlling, and evaluating IT investments and organizes them
into a framework of increasingly mature stages; thereby providing agencies
a road map for improving IT investment management processes in a
systematic and organized manner.
GAO was asked to summarize its January 2004 report on IT strategic
planning/performance measurement and investment management (Information
Technology Management: Governmentwide Strategic Planning, Performance
Measurement, and Investment Management Can Be Further Improved, GAO-04-49,
January 12, 2004) and to discuss how agencies can improve their
performance in these areas.
www.gao.gov/cgi-bin/getrpt?GAO-04-478T.
To view the full product, click on the link above. For more information,
contact David Powner at (202) 512-9286 or [email protected].
Percentage of Agencies' Use of IT Strategic Planning/Performance
Measurement Practices (left) and Investment Management Practices (right)a
aPercentages do not add to 100 percent due to rounding.
Note: Yes-the practice was in place. Partially-the agency has some, but
not all, aspects of the practice in place. Examples of circumstances in
which the agency would receive this designation include when (1) some, but
not all, of the elements of the practice were in place; (2) the agency
documented that it has the information or process in place but it was not
in the prescribed form (e.g., in a specific document as required by law or
the Office of Management and Budget); (3) the agency's documentation was
in draft form; or (4) the agency had a policy related to the practice, but
evidence supported that it had not been completely or consistently
implemented. No- the practice was not in place. Not applicable-the
practice was not relevant to the agency's particular
circumstances.
Mr. Chairman and Members of the Subcommittee:
Thank you for the opportunity to join in today's hearing on the
government's information technology (IT) management. This is a critical
topic because, according to the President's most recent budget, the
federal government spends billions of dollars annually on IT-reportedly
investing about $57 billion in fiscal year 2003.1 Yet these dollars are
not always managed wisely. For example, the Administration reported that
of the $60 billion in IT investments requested for fiscal year 2005, $22
billion- representing 621 major projects-are currently on its "Management
Watch List."2 This list includes mission-critical projects that need
improvement in the areas of performance measures, earned value
management,3 and/or IT security.
To help agencies effectively manage their substantial IT investments, the
Congress has established a statutory framework of requirements and roles
and responsibilities relating to information and technology management
through laws such as the Paperwork Reduction Act of 19954 and the
Clinger-Cohen Act of 1996. This framework addresses, for example, IT
strategic planning/performance measurement (which defines what an
organization seeks to accomplish, identifies the strategies it will use to
achieve desired results, and then determines how well it is succeeding in
reaching results-oriented goals and achieving objectives), and investment
1Office of Management and Budget, Budget of the U.S. Government, Fiscal
Year 2005, Report on IT Spending for the Federal Government for Fiscal
Years 2003, 2004, and 2005. We did not verify this data.
2Office of Management and Budget, Budget of the U.S. Government, Fiscal
Year 2005, Analytical Perspectives. We did not verify these data.
3Earned value management is a project management tool that integrates the
investment scope of work with schedule and cost elements for optimum
investment planning and control.
4The Paperwork Reduction Act of 1995 revised the information resources
management responsibilities established under the Paperwork Reduction Act
of 1980, as amended in 1986.
Results in Brief
management (which involves selecting, 5 controlling,6 and evaluating7
investments).
At your request, today I will summarize our recently issued report8 on the
extent to which 26 agencies9 had in place 30 important practices
associated with key legislative and other requirements for IT strategic
planning/performance measurement and IT investment management (app. I
lists the 30 practices). I will also discuss how agencies can improve
their performance in these areas.
The use of important IT strategic planning/performance measurement and
investment management practices-identified based on legislation, policy,
and guidance-by the agencies in our review was mixed; collectively the
agencies had less than 50 percent of the practices fully in place. For
example, agencies generally had IT strategic plans and goals, but these
goals were not always linked to specific performance measures that were
tracked. Without enterprisewide performance measures that are tracked
against actual results, agencies lack critical information about whether
their overall IT activities are achieving expected goals. In the
investment
5During the selection phase the organization (1) identifies and analyzes
each project's risks and returns before committing significant funds to
any project and (2) selects those IT projects that will best support its
mission needs.
6During the control phase the organization ensures that, as projects
develop and investment expenditures continue, the project is continuing to
meet mission needs at the expected levels of cost and risk. If the project
is not meeting expectations or if problems have arisen, steps are quickly
taken to address the deficiencies.
7During the evaluation phase, actual versus expected results are compared
once projects have been fully implemented. This is done to (1) assess the
project's impact on mission performance, (2) identify any changes or
modifications to the project that may be needed, and (3) revise the
investment management process based on lessons learned.
8U.S. General Accounting Office, Information Technology Management:
Governmentwide Strategic Planning, Performance Measurement, and Investment
Management Can Be Further Improved, GAO-04-49 (Washington, D.C.: Jan. 12,
2004).
9We reviewed 23 entities identified in 31 U.S.C. 901 and the 3 military
services. These were the Departments of Agriculture, the Air Force, the
Army, Commerce, Defense, Education, Energy, Health and Human Services,
Housing and Urban Development, the Interior, Justice, Labor, the Navy,
State, Transportation, the Treasury, and Veterans Affairs; and the
Environmental Protection Agency, General Services Administration, National
Aeronautics and Space Administration, National Science Foundation, Nuclear
Regulatory Commission, Office of Personnel Management, Small Business
Administration, Social Security Administration, and U.S. Agency for
International Development.
management area, the agencies largely had IT investment management boards,
but no agency had the practices associated with the oversight of IT
investments fully in place. Executive-level oversight of project-level
management activities provides organizations with increased assurance that
each investment will achieve the desired cost, benefit, and schedule
results. Although they could not always provide an explanation, agencies
cited a variety of reasons for not having practices fully in place, such
as that the chief information officer (CIO) position had been vacant and
that their process was being revised. Regardless of the reason, these
practices are important ingredients for ensuring effective strategic
planning, performance measurement, and investment management, which, in
turn, make it more likely that the billions of dollars in government IT
investments will not be wasted.
To help agencies improve their performance in the IT strategic
planning/performance measurement and IT investment management areas, we
made numerous recommendations to each of the 26 agencies we reviewed. In
addition, at today's hearing we are releasing the latest version of our
Information Technology Investment Management (ITIM) framework.10 First
issued as an exposure draft in May 2000, this version of the ITIM includes
lessons learned from our use of the framework in our agency reviews and by
users of the framework. The framework identifies critical processes for
the successful selection, control, and evaluation of IT investments and
organizes them into a framework of increasingly mature stages. ITIM offers
organizations a road map for improving their IT investment management
processes in a systematic and organized manner.
Background Advances in the use of IT and the Internet are continuing to
change the way that federal agencies communicate, use, and disseminate
information; deliver services; and conduct business. For example,
electronic government (e-government) has the potential to help build
better relationships between government and the public by facilitating
timely and efficient interaction with citizens. To help agencies more
effectively manage IT, the Congress has established a statutory framework
of requirements and roles and responsibilities relating to information and
technology management. In particular, the Paperwork Reduction Act of
10U.S. General Accounting Office, Information Technology Investment
Management: A Framework for Assessing and Improving Process Maturity,
GAO-04-394G (Washington, D.C.: March 2004).
1995 and the Clinger-Cohen Act of 1996 require agency heads, acting
through agency CIOs to, among other things,
o better link their IT planning and investment decisions to program
missions and goals;
o develop and maintain a strategic information resources management
(IRM) plan that describes how IRM activities help to accomplish agency
missions;
o develop and maintain an ongoing process to establish goals for
improving IRM's contribution to program productivity, efficiency, and
effectiveness; methods for measuring progress toward these goals; and
clear roles and responsibilities for achieving these goals;
o develop and implement a sound IT architecture;
o implement and enforce IT management policies, procedures, standards,
and guidelines;
o establish policies and procedures for ensuring that IT systems provide
reliable, consistent, and timely financial or program performance data;
and
o implement and enforce applicable policies, procedures, standards, and
guidelines on privacy, security, disclosure, and information sharing.
Nevertheless, the agencies face significant challenges in effectively
planning for and managing their IT. Such challenges can be overcome
through the use of a systematic and robust management approach that
addresses critical elements such as IT strategic planning and investment
management.
Agencies Did Not Always Have Strategic Planning/Performance Measurement and
Investment Management Practices in Place
Federal agencies did not always have in place important practices
associated with IT laws, policies, and guidance related to strategic
planning/performance measurement and investment management (see fig. 1). A
well-defined strategic planning process helps to ensure that an agency's
IT goals are aligned with its strategic goals. Moreover, establishing
performance measures and monitoring actual-versusexpected performance
using those measures can help to determine whether IT is making a
difference in improving performance. Finally, an IT investment management
process is an integrated approach to managing
investments that provides for the continuous identification, selection,
control, life-cycle management, and evaluation of IT investments.
Figure 1: Percentage of Agencies' Use of 12 IT Strategic
Planning/Performance Measurement Practices (left) and 18 Investment
Management Practices (right)a
aPercentages do not add to 100 percent due to rounding.
Note: Yes-the practice was in place. Partially-the agency has some, but
not all, aspects of the practice in place. Examples of circumstances in
which the agency would receive this designation include when (1) some, but
not all, of the elements of the practice were in place; (2) the agency
documented that it has the information or process in place but it was not
in the prescribed form (e.g., in a specific document as required by law or
the Office of Management and Budget); (3) the agency's documentation was
in draft form; or (4) the agency had a policy related to the practice, but
evidence supported that it had not been completely or consistently
implemented. No-the practice was not in place. Not applicable-the practice
was not relevant to the agency's particular circumstances.
Agency IT officials could not always identify why practices were not in
place, but in those instances in which reasons were identified, a variety
of explanations were provided; for example, that the CIO position had been
vacant, that not including a requirement in the agency's guidance was an
oversight, or that the process was being revised. Nevertheless, these
practices are based on law, executive orders, Office of Management and
Budget (OMB) policies, and our guidance, and are also important
ingredients in ensuring effective strategic planning, performance
measurement, and investment management that, in turn, make it more likely
that the billions of dollars in government IT investments will be wisely
spent.
Agencies' Use of IT Strategic Planning/Performance Measurement Practices
Was Uneven
Critical aspects of the strategic planning/performance measurement area
include documenting the agency's IT strategic planning processes,
developing IRM plans, establishing goals, and measuring performance to
evaluate whether goals are being met. Although the agencies often had
these practices, or elements of these practices, in place, additional work
remains, as demonstrated by the following examples:
o Strategic planning process. Strategic planning defines what an
organization seeks to accomplish and identifies the strategies it will use
to achieve desired results. A defined strategic planning process allows an
agency to clearly articulate its strategic direction and to establish
linkages among planning elements such as goals, objectives, and
strategies. About half of the agencies had fully documented their
strategic planning processes. Such processes are an essential foundation
for ensuring that IT resources are effectively managed.
o Strategic IRM plans. The Paperwork Reduction Act requires that
agencies indicate in strategic IRM plans how they are applying information
resources to improve the productivity, efficiency, and effectiveness of
government programs. An important element of a strategic plan is that it
presents an integrated system of high-level decisions that are reached
through a formal, visible process. The Paperwork Reduction Act also
requires agencies to develop IRM plans in accordance with OMB's guidance.
However, OMB does not provide cohesive guidance on the specific contents
of IRM plans. Accordingly, although agencies generally provided OMB with a
variety of planning documents to meet its requirement that they submit an
IRM plan, these plans were generally limited to IT strategic or
e-government issues and did not address other elements of IRM, as defined
by the Paperwork Reduction Act. In particular, these plans generally
include individual IT projects and initiatives, security, and enterprise
architecture elements but do not often address other information
functions-such as information collection, records management, and
privacy-or the coordinated management of all information functions.
OMB IT staff agreed that the agency has not set forth guidance on the
contents of agency IRM plans in a single place, stating that its focus has
been on looking at agencies' cumulative results and not on planning
documents. These staff also noted that agencies account for their IRM
activities through multiple documents (e.g., Information Collection
Budgets11 and Government Paperwork Elimination Act12 plans). Nevertheless,
half the agencies indicated a need for OMB to provide additional guidance
on the development and content of IRM plans. Accordingly, we recommended
that OMB develop and disseminate to agencies guidance on developing IRM
plans.
o IT goals. The Paperwork Reduction Act and the Clinger-Cohen Act
require agencies to establish goals that address how IT contributes to
program productivity, efficiency, effectiveness, and service delivery to
the public. We have previously reported that leading organizations define
specific goals, objectives, and measures, use a diversity of measure
types, and describe how IT outputs and outcomes impact operational
customer and agency program delivery requirements.13 The agencies
generally had the types of goals outlined in the Paperwork Reduction Act
and the Clinger-Cohen Act. However, five agencies did not have one or more
of the goals required by the Paperwork Reduction Act and the Clinger-Cohen
Act. It is important that agencies specify clear goals and objectives to
set the focus and direction for IT performance.
o IT performance measures. The Paperwork Reduction Act, the
Clinger-Cohen Act, and an executive order14 require agencies to establish
a variety of IT performance measures-such as those related to how IT
contributes to program productivity, efficiency, and effectiveness-and to
monitor the actual-versus-expected performance using those measures.
Although the agencies largely had one or more of the required performance
measures in place, these measures were not always linked to the agencies'
enterprisewide IT goals. Moreover, few agencies monitored
actual-versusexpected performance for all of their enterprisewide IT
goals. Specifically, although some agencies tracked actual-versus-expected
outcomes for the IT performance measures in their performance plans or
accountability
11Each year, OMB's Office of Information and Regulatory Affairs publishes
an Information Collection Budget by gathering data from executive branch
agencies on the total number of burden hours it approved for collection of
information at the end of the fiscal year and agency estimates of the
burden for the coming fiscal year.
12In fulfilling its responsibilities under this act, OMB requires agencies
to report to OMB on their plans for providing the public with the option
of submitting, maintaining, and disclosing required information
electronically, instead of on paper.
13U.S. General Accounting Office, Executive Guide: Measuring Performance
and Demonstrating Results of Information Technology Investments,
GAO/AIMD-98-89 (Washington, D.C.: March 1998).
14Executive Order 13103, Computer Software Piracy (September 30, 1998).
reports and/or for specific IT projects, they generally did not track the
performance measures that were specified in their IRM plans. As we have
previously reported, an effective IT performance management system
offers a variety of benefits, including serving as an early warning
indicator
of problems and the effectiveness of corrective actions; providing input
to
resource allocation and planning; and providing periodic feedback to
employees, customers, stakeholders, and the general public about the
quality, quantity, cost, and timeliness of products and services.15
Moreover,
without enterprisewide performance measures that are tracked against
actual results, agencies lack critical information about whether their
overall IT activities are achieving expected goals.
o Benchmarking. The Clinger-Cohen Act requires agencies to
quantitatively benchmark agency process performance against public- and
private-sector organizations, where comparable processes and organizations
exist. Benchmarking is used because there may be external organizations
that have more innovative or more efficient processes than their own
processes. Seven agencies in our review had mechanisms in place-such as
policies and strategies-related to benchmarking their IT processes. In
general, however, agencies' benchmarking decisions were ad hoc. Few
agencies had developed a mechanism to identify comparable external
private- or public-sector organizations and processes and/or had policies
related to benchmarking, although all but 10 of the agencies provided
examples of benchmarking that they had performed. Our previous study of IT
performance measurement at leading organizations found that they had spent
considerable time and effort comparing their performance information with
that of other organizations.16
Agency IT officials could not identify why strategic planning/performance
measurement practices were not in place in all cases, but in those
instances in which reasons were identified, a variety of explanations were
provided. For example, reasons cited by agency IT officials included that
they lacked the support from agency leadership, that the agency had not
been developing IRM plans until recently and recognized that the plan
needed further refinement, that the process was being revised, and that
requirements were evolving.
15GAO/AIMD-98-89. 16GAO/AIMD-98-89.
Without strong strategic management practices, it is less likely that IT
is being used to maximize improvement in mission performance. Moreover,
without enterprisewide performance measures that are being tracked against
actual results, agencies lack critical information about whether their
overall IT activities, at a governmentwide cost of billions of dollars
annually, are achieving expected goals.
Agencies' Use of IT Investment Management Practices Was Mixed
Critical aspects of IT investment management include developing
wellsupported proposals, establishing investment management boards, and
selecting and controlling IT investments. The agencies' use of practices
associated with these aspects of investment management was wideranging, as
follows:
o IT investment proposals. Various legislative requirements, an
executive order, and OMB policies provide minimum standards that govern
agencies' consideration of IT investments. In addition, we have issued
guidance to agencies for selecting, controlling, and evaluating IT
investments.17 Such processes help ensure, for example, that investments
are cost-beneficial and meet mission needs and that the most appropriate
development or acquisition approach is chosen. The agencies in our review
had mixed results when evaluated against these various criteria. For
example, the agencies almost always required that proposed investments
demonstrate that they support the agency's business needs, are
cost-beneficial, address security issues, and consider alternatives.
However, they were not as likely to have fully in place the Clinger-Cohen
Act requirement that agencies follow, to the maximum extent practicable, a
modular, or incremental, approach when investing in IT projects.
Incremental investment helps to mitigate the risks inherent in large IT
acquisitions/developments by breaking apart a single large project into
smaller, independently useful components with known and defined
relationships and dependencies.
o Investment management boards. Our investment management guide states
that establishing one or more IT investment board(s) is a key component of
the investment management process. Such executive-level boards, made up of
business-unit executives, concentrate management's attention on assessing
and managing risks and regulating the trade-offs between continuing to
fund existing operations and developing new performance capabilities.
Almost all of the agencies in our review had one or more enterprise-level
investment management board. However, the
17For example, see GAO-04-394G.
investment management boards for six agencies were not involved, or the
agency did not document the boards' involvement, in the control phase.
Maintaining responsibility for oversight with the same body that selected
the investment is crucial to fostering a culture of accountability by
holding the investment board that initially selected an investment
responsible for its ongoing success.
o Selection of IT investments. During the selection phase of an IT
investment management process, the organization (1) selects projects that
will best support its mission needs and (2) identifies and analyzes each
project's risks and returns before committing significant funds. To
achieve desired results, it is important that agencies have a selection
process that, for example, uses selection criteria to choose the IT
investments that best support the organization's mission and that
prioritizes proposals. Twentytwo agencies used selection criteria in
choosing their IT investments. In addition, about half the agencies used
scoring models18 to help choose their investments.
o Control over IT investments. During the control phase of the IT
investment management process, the organization ensures that, as projects
develop and as funds are spent, the project is continuing to meet mission
needs at the expected levels of cost and risk. If the project is not
meeting expectations or if problems have arisen, steps are quickly taken
to address the deficiencies. In general, the agencies were weaker in the
practices pertaining to the control phase of the investment management
process than to the selection phase and no agency had the practices
associated with the control phase fully in place. In particular, the
agencies did not always have important mechanisms in place for agencywide
investment management boards to effectively control investments, including
decision-making rules for project oversight, early warning mechanisms,
and/or requirements that corrective actions for underperforming projects
be agreed upon and tracked. Executive level oversight of project-level
management activities provides an organization with increased assurance
that each investment will achieve the desired cost, benefit, and schedule
results.
18With a scoring model, the assessment body typically attaches numerical
scores and "relative value" weights to each of the individual selection
criteria. Investments are then assessed relative to these scores and then
against weights associated with each individual criterion. Finally, the
weighted scores are summed to create a numerical value for each
investment.
Among the variety of reasons that agencies cited for not having IT
investment management practices fully in place were that the CIO position
had been vacant, that not including a requirement in the IT investment
management guide was an oversight, and that the process was being revised.
However, in some cases agencies could not identify why certain practices
were not in place. It is important that agencies address their
shortcomings, because only by effectively and efficiently managing their
IT resources through a robust investment management process can they gain
opportunities to make better allocation decisions among many investment
alternatives and to further leverage their IT investments.
Improving Agencies' IT Strategic Planning/Performance Measurement and
Investment Management
To help agencies improve their IT strategic planning/performance
measurement and investment management, we have made numerous
recommendations to agencies and issued guidance. Specifically, in our
January 2004 report we made recommendations to the 26 agencies in our
review regarding practices that were not fully in place. These
recommendations addressed issues such as IT strategic planning;
establishing and linking enterprisewide goals and performance measures and
tracking progress against these measures; and selecting, controlling, and
evaluating investments. By implementing these recommendations, agencies
can better ensure that they are using strategic planning, performance
measurement, and investment management practices that are consistent with
IT legislation, executive orders, OMB policies, and our guidance.
Another mechanism that agencies can use to improve their IT management is
to apply the management frameworks and guides that we have issued, which
are based on our research into IT management best practices and our
evaluations of agency IT management performance.19 In this vein, today we
are releasing the latest version of our ITIM framework.20 This framework
identifies and organizes critical processes for selecting, controlling,
and evaluating IT investments into a framework of increasingly mature
stages (see fig. 2).
19For example, see U.S. General Accounting Office, Information Technology:
A Framework for Assessing and Improving Enterprise Architecture Management
(Version 1.1),
GAO-03-584G (Washington, D.C.: April 2003) and GAO/AIMD-98-89.
20GAO-04-394G.
Figure 2: The ITIM Stages of Maturity with Critical Processes
First issued as an exposure draft in May 2000, this new version of the
ITIM includes lessons learned from our use of the framework in our agency
reviews and from lessons conveyed to us by users of the framework. In
addition, in order to validate the appropriateness of our changes and to
gain the advantage of their experience, we had the new version reviewed by
several outside experts who are familiar with the ITIM exposure draft and
with investment management in a broad array of public and private
organizations.
ITIM can be used to analyze an organization's investment management
processes and to determine its level of maturity. The framework is useful
to many federal agencies because it provides: (1) a rigorous, standardized
tool for internal and external evaluations of an agency's IT investment
management process; (2) a consistent and understandable mechanism for
reporting the results of these assessments to agency executives, Congress,
and other interested parties; and (3) a road map that agencies can use for
improving their investment management processes. Regarding the first two
points, we and selected agency Inspectors General have used the ITIM to
evaluate and report on the investment management processes of several
agencies.21 Concerning the third point, a number of agencies have
recognized the usefulness of the ITIM framework and have used it to
develop and enhance their investment management strategies. For example,
one agency uses the framework to periodically review its IT investment
management capabilities and has developed an action plan to move through
the stages of maturity.
In summary, our January 2004 report indicates that the federal government
can significantly improve its IT strategic planning, performance
measurement, and investment management. Such improvement would better
ensure that agencies are being responsible stewards of the billions of
dollars for IT with which they have been entrusted, by helping them to
invest these monies wisely. This can be accomplished, in part, through the
expeditious implementation of our recommendations and the adoption of best
practices, which we have incorporated into our IT management frameworks
and guides such as the ITIM.
Mr. Chairman, this completes my prepared statement. I would be happy to
respond to any questions that you or other Members of the Subcommittee may
have at this time.
21For example, see U.S. General Accounting Office, Information Technology:
Departmental Leadership Crucial to Success of Investment Reforms at
Interior, GAO-03-1028 (Washington, D.C.: Sept. 12, 2003); Bureau of Land
Management: Plan Needed to Sustain Progress in Establishing IT Investment
Management Capabilities, GAO-03-1025 (Washington, D.C.: Sept. 12, 2003);
United States Postal Service: Opportunities to Strengthen IT Investment
Management Capabilities, GAO-03-3 (Washington, D.C.: Oct. 15, 2002);
Information Technology: DLA Needs to Strengthen Its Investment Management
Capability, GAO-02-314 (Washington, D.C.: Mar. 15, 2002); and Information
Technology: INS Needs to Strengthen Its Investment Management Capability,
GAO-01-146 (Washington, D.C.: Dec. 29, 2000).
Contacts If you have any questions regarding this statement, please
contact me at (202) 512-9286 or by e-mail at [email protected]. Specific
questions related to our January 2004 report may also be directed to Linda
Lambert at (202) 512-9556 or via e-mail at [email protected] or Mark Shaw
at (202) 512-6251 or via e-mail at [email protected]. Questions related to the
ITIM framework can be directed to Lester Diamond at (202) 512-7957 or via
email at [email protected].
Appendix I: Information Technology (IT) Strategic Planning/Performance
Measurement and Investment Management Practices
Table 1 describes the 12 IT strategic planning/performance measurement and
the 18 IT investment management practices that we used in our January 2004
report on the government's performance in these areas.1 We identified
these 30 practices after reviewing major legislative requirements (e.g.,
the Paperwork Reduction Act of 1995 and the Clinger-Cohen Act of 1996),
executive orders, Office of Management and Budget policies, and our own
guidance.
Table 1: IT Strategic Planning/Performance Measurement and Investment Management
Practices
Practice
Number Practice Description
IT Strategic Planning/Performance Measurement Practices
The agency has documented its IT strategic management process, including,
at a minimum,
o the responsibilities and accountability for IT resources across the
agency, including the relationship between the chief information officer
(CIO), chief financial officer (CFO), and mission/program officials; and
o the method by which the agency defines program information needs and
develops strategies, systems, and capabilities to meet those needs.
The agency has documented its process to integrate IT management
operations and decisions with organizational planning, budget, financial
management, human resources management, and program decisions.
The agency requires that information security management processes be
integrated with strategic and operational planning processes.
The agency has a process that involves the CFO, or comparable official, to
develop and maintain a full and accurate accounting of IT-related
expenditures, expenses, and results.
The agency prepares an enterprisewide strategic information resources
management (IRM) plan that, at a minimum,
o describes how IT activities will be used to help accomplish agency
missions and operations, including related resources; and
o identifies major IT acquisition program(s) or any phase or increment
of that program that has significantly deviated from the cost,
performance, or schedule goals established for the program.
The agency's performance plan required under GPRA includes
o a description of how IT supports strategic and program goals,
o the resources and time periods required to implement the information
security program plan required by the Federal Information Security
Management Act (FISMA), and
o a description of major IT acquisitions contained in the capital asset
plan that will bear significantly on the achievement of a performance
goal.
1.7 The agency has a documented process to
o develop IT goals in support of agency needs,
o measure progress against these goals, and
o assign roles and responsibilities for achieving these goals.
1U.S. General Accounting Office, Information Technology Management:
Governmentwide Strategic Planning, Performance Measurement, and Investment
Management Can Be Further Improved, GAO-04-49 (Washington, D.C.: Jan. 12,
2004).
Practice
Number Practice Description
1.8 The agency has established goals that, at a minimum, address how IT
contributes to
o program productivity,
o efficiency,
o effectiveness, and
o service delivery to the public (if applicable).
The agency has established IT performance measures and monitors
actual-versus-expected performance that at least addresses
o how IT contributes to program productivity,
o how IT contributes to the efficiency of agency operations,
o how IT contributes to the effectiveness of agency operations,
o service delivery to the public (if applicable),
o how electronic government initiatives enable progress toward agency
goals and statutory mandates,
o the performance of IT programs (e.g., system development and
acquisition projects), and
o agency compliance with federal software piracy policy.
1.10 The agency has developed IT performance measures that align with and
support the goals in the GPRA performance plan.
1.11 The agency developed an annual report, included as part of its
budget submission, that describes progress in achieving goals for
improving the efficiency and effectiveness of agency operations and, as
appropriate, the delivery of services to the public through the effective
use of IT.
1.12 The agency requires that its IT management processes be benchmarked
against appropriate processes and/or organizations from the public and
private sectors in terms of cost, speed, productivity, and quality of
outputs and outcomes where comparable processes and organizations in the
public or private sectors exist.
IT Investment Management Practices
The agency has a documented IT investment management process that, at a
minimum,
o specifies the roles of key people (including the CIO) and groups within
the IT investment management process, o outlines significant events and
decision points,
o identifies external and environmental factors that influence the
process,
o explains how the IT investment management process is coordinated with
other organizational plans and processes, and
o describes the relationship between the investment management process
and the agency's enterprise architecture.
The agency established one or more agencywide IT investment management
boards responsible for selecting, controlling, and evaluating IT
investments that, at a minimum,
o have final project funding decision authority (or provide
recommendations) over projects within their scope of authority, and
o are composed of key business unit executives.
2.3 The agencywide board(s) work processes and decision-making processes
are described and documented.
2.4 If more than one IT investment management board exists in the
organization (e.g., at the component level), the organization has
o documented policies and procedures that describe the processes for
aligning and coordinating IT investment decision making,
o criteria for determining where in the organization different types of
IT investment decisions are made, and
o processes that describe how cross-functional investments and decisions
(e.g., common applications) are handled.
Practice
Number Practice Description
2.5
As part of its investment management process, the agency has available an
annually updated comprehensive inventory of its major information systems
that includes major national security systems and interfaces.
2.6
A standard, documented procedure is used so that developing and
maintaining the inventory is a repeatable event, which produces inventory
data that are timely, sufficient, complete, and compatible.
2.7 The IT asset inventory is used as part of managerial decision making.
2.8 Proposed IT investments are required to document that they have
addressed the following items during project planning:
o that the project supports the organization's business and mission needs
and meets users' needs,
o whether the function should be performed by the public or private
sector,
o whether the function or project should be performed or is being
performed by another agency,
o that alternatives have been considered, and
o how security will be addressed.
In considering a proposed IT project, the agency requires that the project
demonstrate that it is economically beneficial through the development of
a business case that at least addresses costs, benefits, schedule, and
risks.
2.10 In considering a proposed IT project, the agency requires that the
project demonstrate that it is consistent with federal and agency
enterprise architectures.
2.11 The agency requires that the proposed IT investment, at a minimum,
o support work processes that it has simplified or redesigned to reduce
costs and improve effectiveness, and
o make maximum use of commercial-off-the-shelf (COTS) software. 2.12 The
agency has established project selection criteria distributed throughout
the organization that include, at a minimum,
o cost, benefit, schedule, and risk elements;
o measures such as net benefits, net risks, and risk-adjusted return on
investment; and
o qualitative criteria for comparing and prioritizing alternative
information systems investment projects.
2.13 The agency has established a structured selection process that, at a
minimum,
o selects IT proposals using selection criteria;
o identifies and addresses possible IT investments and proposals that
are conflicting, overlapping, strategically unlinked, or redundant;
o prioritizes proposals; and
o is integrated with budget, financial, and program management decisions.
2.14 Agency policy calls for investments to be modularized (e.g., managed
and procured in well-defined useful segments or modules that are short in
duration and small in scope) to the maximum extent achievable.
2.15 The agencywide investment management board(s) has written policies
and procedures for management oversight of IT projects that cover, at a
minimum,
o decision-making rules for project oversight that allow for terminating
projects, when appropriate;
o current project data, including expected and actual cost, schedule,
and performance data, to be provided to senior management periodically and
at major milestones;
o criteria or thresholds related to deviations in cost, schedule, or
system capability actuals versus expected project performance; and
o the generation of an action plan to address a project's problem(s) and
track resolution.
Practice
Number Practice Description
2.16 The agencywide investment management board(s) established an
oversight mechanism of funded investments that, at a minimum,
o determines whether mission requirements have changed;
o determines whether the investment continues to fulfill ongoing and
anticipated mission requirements;
o determines whether the investment is proceeding in a timely manner
toward agreed-upon milestones;
o employs early warning mechanisms that enable it to take corrective
action at the first sign of cost, schedule, or performance slippages; and
o includes the use of independent verification and validation (IV&V)
reviews of under-performing projects, where appropriate.
2.17 Corrective actions for under-performing projects are agreed upon,
documented, and tracked by the agencywide investment management board(s).
2.18 The agencywide investment management board(s) requires that
postimplementation reviews be conducted to
o validate expected benefits and costs and
o document and disseminate lessons learned.
Source: GAO.
This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed in
its entirety without further permission from GAO. However, because this
work may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this material
separately.
GAO's Mission
Obtaining Copies of GAO Reports and Testimony
The General Accounting Office, the audit, evaluation and investigative arm
of Congress, exists to support Congress in meeting its constitutional
responsibilities and to help improve the performance and accountability of
the federal government for the American people. GAO examines the use of
public funds; evaluates federal programs and policies; and provides
analyses, recommendations, and other assistance to help Congress make
informed oversight, policy, and funding decisions. GAO's commitment to
good government is reflected in its core values of accountability,
integrity, and reliability.
The fastest and easiest way to obtain copies of GAO documents at no cost
is through the Internet. GAO's Web site (www.gao.gov) contains abstracts
and fulltext files of current reports and testimony and an expanding
archive of older products. The Web site features a search engine to help
you locate documents using key words and phrases. You can print these
documents in their entirety, including charts and other graphics.
Each day, GAO issues a list of newly released reports, testimony, and
correspondence. GAO posts this list, known as "Today's Reports," on its
Web site daily. The list contains links to the full-text document files.
To have GAO e-mail this list to you every afternoon, go to www.gao.gov and
select "Subscribe to e-mail alerts" under the "Order GAO Products"
heading.
Order by Mail or Phone The first copy of each printed report is free.
Additional copies are $2 each. A check or money order should be made out
to the Superintendent of Documents. GAO also accepts VISA and Mastercard.
Orders for 100 or more copies mailed to a single address are discounted 25
percent. Orders should be sent to:
U.S. General Accounting Office 441 G Street NW, Room LM Washington, D.C.
20548
To order by Phone: Voice: (202) 512-6000 TDD: (202) 512-2537 Fax: (202)
512-6061
Contact:
To Report Fraud, Web site: www.gao.gov/fraudnet/fraudnet.htm
Waste, and Abuse in E-mail: [email protected]
Federal Programs Automated answering system: (800) 424-5454 or (202)
512-7470
Jeff Nelligan, Managing Director, [email protected] (202) 512-4800
Public Affairs U.S. General Accounting Office, 441 G Street NW, Room 7149
Washington, D.C. 20548
*** End of document. ***