Information Technology: HHS Has Several Investment Management
Capabilities in Place, but Needs to Address Key Weaknesses
(28-OCT-05, GAO-06-11).
The Department of Health and Human Services (HHS) is one of the
largest federal agencies, the nation's largest health insurer,
and the largest grant- making agency in the federal government.
The department manages over 300 programs that serve to improve
the health and well-being of the American public and is comprised
of several component agencies covering a wide range of activities
including conducting and sponsoring medical and social science
research, guarding against the outbreak of infectious diseases,
assuring the safety of food and drugs, and providing health care
services and insurance. It also manages and funds a variety of
information technology (IT) initiatives ranging from those
facilitating the payment of claims for Medicare and Medicaid
services to those supporting health surveillance and
communications. In fiscal year 2006, the department plans to
spend over $5 billion on information technology--the third
largest IT expenditure in the federal budget. As we agreed with
Congress, our objectives were to (1) assess the department's
capabilities for managing its IT investments and (2)determine any
plans the department might have for improving those capabilities.
To address these objectives, we analyzed documents and
interviewed agency officials to (1)validate and update HHS's
self-assessments of key practices in the framework and
(2)evaluate HHS's plans for improving its capabilities.
-------------------------Indexing Terms-------------------------
REPORTNUM: GAO-06-11
ACCNO: A40577
TITLE: Information Technology: HHS Has Several Investment
Management Capabilities in Place, but Needs to Address Key
Weaknesses
DATE: 10/28/2005
SUBJECT: Information resources management
Information technology
IT investment management
IT policies
IT standards
Management information systems
Policy evaluation
Program evaluation
Systems design
Systems evaluation
HHS Unified Financial Management System
******************************************************************
** This file contains an ASCII representation of the text of a **
** GAO Product. **
** **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced. Tables are included, but **
** may not resemble those in the printed version. **
** **
** Please see the PDF (Portable Document Format) file, when **
** available, for a complete electronic file of the printed **
** document's contents. **
** **
******************************************************************
GAO-06-11
United States Government Accountability Office
GAO
Report to the Chairman, Committee on
Finance, U.S. Senate
October 2005
INFORMATION
TECHNOLOGY
HHS Has Several Investment Management Capabilities in Place, but Needs to
Address Key Weaknesses
a
GAO-06-11
[IMG]
October 2005
INFORMATION TECHNOLOGY
HHS Has Several Investment Management Capabilities in Place, but Needs to
Address Key Weaknesses
What GAO Found
Judged against the criteria of GAO's framework for information technology
investment management (ITIM), which measures the maturity of an
organization's investment management processes, HHS has established 63
percent of the foundational practices that it needs to manage its IT
investments individually; and 30 percent to manage its investments as a
portfolio (see table below). Specifically, HHS has implemented processes
to ensure that projects support business needs and meet users'
requirements, established a process for selecting investments, and has
created portfolio selection criteria. However, weaknesses remain in
several areas. The department's senior investment board does not regularly
review component agencies' IT investments, leaving close to 90 percent of
its discretionary investments without an appropriate level of executive
oversight. In addition, HHS does not evaluate the performance of its
portfolio on a continuing basis or conduct postimplementation reviews.
Finally, HHS currently has no structured mechanism in place to ensure that
the component agencies are defining and implementing investment processes
that are aligned with those of the department. Until HHS establishes the
practices it needs to effectively manage its IT investments, executives
cannot be assured that they are appropriately selecting, managing, and
evaluating the mix of investments that will maximize returns to the
organization, taking into account the appropriate level of risk.
HHS has initiated efforts to improve its investment management processes,
but has not coordinated these and additional efforts that would be needed
to address the weaknesses we identify in a comprehensive plan that defines
and prioritizes improvements to the investment process. Such a plan is
instrumental in helping HHS to coordinate and guide its improvement
efforts and sustain its commitment to the efforts already under way.
Without such a plan and procedures for implementing it, the department
risks being unable to effectively establish mature investment management
capabilities. As a result, executives may not be able to make informed and
prudent investment decisions in managing HHS's multibillion-dollar IT
budget.
HHS's Current IT Investment Management Capabilities
Stage 2: Building the Percentage Stage 3: Developing a
investment foundation
of key practices executed complete investment portfolio Percentage of key
practices executed
Instituting the 63 Defining the portfolio 71
investment criteria
board
Meeting business needs 100 Creating the portfolio 43
Selecting an 70 Evaluating the portfolio 0
investment
Providing investment 0 Conducting 0
oversight postimplementation reviews
Capturing investment 83
information
Overall 63 Overall 30
Source: GAO.
United States Government Accountability Office
Contents
Letter
Results in Brief
Background
HHS Has Established Many Key Practices for Managing Its
Investments, but Has Provided Limited Guidance and Oversight to Component
Agencies Processes HHS Does Not Have a Plan to Coordinate and Guide
Improvement
Efforts Conclusions Recommendations for Executive Action Agency Comments
1 2 4
19
47 48 48 50
Appendixes
Appendix I: Objectives, Scope, and Methodology 53
Appendix II: Comments from the Department of Health and Human Services 56
Appendix III: GAO Contact and Staff Acknowledgments 60
Tables Table 1:
Table 2:
Table 3:
Table 4: Table 5: Table 6: Table 7: Table 8: Table 9:
Estimated IT Budget of HHS Component Agencies for
Fiscal Year 2006
Stage 2 Critical Processes-Building the Investment
Foundation
Summary of Results for Stage 2 Critical Processes and Key
Practices
Instituting the Investment Board
Meeting Business Needs
Selecting an Investment
Providing Investment Oversight
Capturing Investment Information
Stage 3 Critical Processes-Developing a Complete
Investment Portfolio
7
20
21 25 27 30 33 36
37
38 40 42 44 45
Table 10: Summary of Results for Stage 3 Critical Processes and Key
Practices
Table 11: Defining the Portfolio Criteria
Table 12: Creating the Portfolio
Table 13: Evaluating the Portfolio
Table 14: Conducting Postimplementation Reviews
Contents
Simplified HHS Organizational Chart HHS
Figures Figure 1: Figure 2: Discretionary IT Investments for Fiscal 5 6
Year 2006
Figure 3: Detailed Breakdown of HHS's Investment
Management
Process 15
Figure 4: The Five ITIM Stages of Maturity with
Critical
Processes 18
Abbreviations
CPIC Capital Planning and Investment Control
CIO Chief Information Officer
HHS Department of Health and Human Services
IT information technology
ITIM information technology investment management framework
ITIRB Information Technology Investment Review Board
PMT Portfolio Management Tool
PIR postimplementation reviews
This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed in
its entirety without further permission from GAO. However, because this
work may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this material
separately.
A
United States Government Accountability Office Washington, D.C. 20548
October 28, 2005
The Honorable Charles E. Grassley Chairman, Committee on Finance United
States Senate
Dear Mr. Chairman:
The Department of Health and Human Services (HHS) is one of the largest
federal agencies, the nation's largest health insurer, and the largest
grantmaking agency in the federal government. The department manages over
300 programs that serve to improve the health and well-being of the
American public and is comprised of several component agencies covering a
wide range of activities including conducting and sponsoring medical and
social science research, guarding against the outbreak of infectious
diseases, assuring the safety of food and drugs, and providing health care
services and insurance. It also manages and funds a variety of information
technology (IT) initiatives ranging from those facilitating the payment of
claims for Medicare and Medicaid services to those supporting health
surveillance and communications. In fiscal year 2006, the department plans
to spend over $5 billion on information technology-the third largest IT
expenditure in the federal budget.1
This report is one of two we prepared in response to your request that we
evaluate HHS's information technology investment management capabilities.2
It focuses on HHS's processes for making IT investment management
decisions and evaluates how well these processes compare with the accepted
practices presented in our IT investment management (ITIM) framework.3
This framework provides a method for evaluating and
1Office of Management and Budget, Budget of the U.S. Government, Fiscal
Year 2006, Report on IT Spending for the Federal Government for Fiscal
Years 2004, 2005, and 2006.
We did not verify these data.
2Our second report, GAO, Information Technology: Centers for Medicare &
Medicaid Services Needs to Establish Critical Investment Management
Capabilities, GAO-06-12 (Washington, D.C.: Oct. 28, 2005), addresses (1)
the agency's capabilities for managing its IT investments, (2) determining
any plans the agency might have for improving these capabilities, and (3)
examining the agency's process for approving and monitoring the state
Medicaid management systems it funds.
3GAO, Information Technology Investment Management: A Framework for
Assessing and Improving Process Maturity, GAO-04-394G (Washington, D.C.:
March 2004).
assessing how well an agency is selecting and managing its IT resources.
As we agreed with your office, our objectives were to (1) assess the
department's capabilities for managing its IT investments and (2)
determine any plans the department might have for improving those
capabilities. To address these objectives, we analyzed documents and
interviewed agency officials to (1) validate and update HHS's
selfassessments of key practices in the framework and (2) evaluate HHS's
plans for improving its capabilities. We performed our work from January
through September 2005, in accordance with generally accepted government
auditing standards. Appendix I contains details about our objectives,
scope, and methodology.
Results in Brief Because of the management attention that has been given
to IT investment management, HHS has established over half of the
foundational practices needed to manage its IT investments individually
and about 30 percent of the key practices needed to effectively manage its
portfolio of investments. For example, HHS has implemented many of the
practices required to ensure that (1) projects support business needs and
meet users' requirements, (2) a well-defined and disciplined process is
used to select IT investments, (3) investment information is captured in a
repository for decision makers, and (4) IT portfolio selection criteria
are developed and maintained. However, critical weaknesses remain in
several areas. Specifically, HHS lacks
o business representation on its senior IT investment review board of
component agencies to carry out its full scope of responsibilities,
o an established process for the IT investment board to regularly review
a defined set of the component agencies' IT investments and maintain
visibility of other investments,
o criteria for assessing portfolio performance or regularly reviewing the
performance of the organization's investment portfolio, and
o processes for conducting postimplementation reviews (PIR) of its IT
investments.
The department also does not have a structured mechanism in place for
ensuring that component agencies define and implement investment
management processes that are aligned with those of the department. Until
the department fully establishes all foundational and portfolio-level
practices and establishes a mechanism to ensure that component agencies
define and implement processes that are aligned with those of the
department, executives cannot be assured that they are appropriately
selecting, managing, and evaluating the mix of investments that will
maximize returns to the organization, taking into account the appropriate
level of risk.
HHS has initiated steps to improve its investment management process;
however, these steps do not fully address the weaknesses we identify in
this report, nor are they coordinated along with other needed improvement
efforts into a plan that (1) is based on an assessment of strengths and
weaknesses; (2) specifies measurable goals, objectives, and milestones;
(3) specifies needed resources; (4) assigns clear responsibility and
accountability for accomplishing tasks; and (5) is approved by senior
management. Without such a plan and procedures for implementing it, the
department risks being unable to effectively establish mature investment
management capabilities. As a result, executives may not be able to make
informed and prudent investment decisions in managing the department's
annual multibillion-dollar IT budget.
To further strengthen HHS's investment management capability, we are
recommending that the department develop and implement a plan aimed at
addressing the weaknesses that we identify in this report.
In commenting on a draft of this report, HHS generally agreed with our
findings and recommendations and stated that it will leverage the report
in its efforts to improve its investment management processes. However, it
expressed differing perspectives on the inclusion of component agency
business representation on the investment review and the performance of
postimplementation reviews. Specifically, the department commented that it
used a hierarchy of investment reviews combined with investment review
board members representing mission support areas such as Finance,
Acquisition, and Human Resources, to provide a structure for making the
business decisions regarding the department's investments. Nevertheless,
we reiterate the importance of having business representation from
component agencies to make these decisions. In addition, the department
stated that it was performing postimplementation reviews in an informal
manner through closeout reviews of investments that have recently been
implemented and annual reviews of systems in operations and maintenance.
However, neither of these reviews currently identify lessons learned or
capture benefits realized, key elements of postimplementation reviews.
Background
HHS's Mission, Organizational Structure, and Use of IT
HHS is the primary organization within the federal government that is
devoted to protecting the health of Americans. It provides essential human
services, such as ensuring food and drug safety and assisting needy
families. HHS administers more grant dollars than all other federal
agencies combined, providing over $200 billion of the more than $350
billion in federal funds that were awarded to states and other entities in
fiscal year 2002, the most recent year for which these data are available.
For fiscal year 2005, HHS had a budget of $581 billion and a workforce of
over 67,000 employees.
To accomplish its mission, HHS is comprised of 12 component agencies4 and
several staff offices that cover a wide range of activities-including
conducting and sponsoring medical and social science research, guarding
against the outbreak of infectious diseases, assuring the safety of food
and drugs, and providing health care services and insurance. The Office of
the Secretary consists of several staff divisions and offices, including
the Office of the Assistant Secretary for Budget, Technology, and Finance.
The HHS Office of the Chief Information Officer (CIO) is located within
this staff office (see fig. 1).
4HHS refers to its component agencies as operating divisions.
Figure 1: Simplified HHS Organizational Chart
Source: HHS.
Information technology investments play a critical role in helping HHS
carry out its diverse mission. According to the President's most recent
budget, HHSexpects to spend about $5 billion in IT in fiscal year 2006,
making the department's IT investment budget the third largest in the
federal government.As figure 2 illustrates, approximately $3 billion is
designated as grants to states for investments for Medicaid programs and
other purposes, such as child support enforcement systems. Approximately
$2 billion is for discretionary investment spending, of which 89 percent
is used to fund IT investments for component agencies; 7 percent is
invested in HHS enterprisewide initiatives;5 and 4 percent is used to fund
other initiatives, including Office of the Inspector General IT
investments.
Figure 2: HHS Discretionary IT Investments for Fiscal Year 2006 (in
millions) Total: $2,312
Other - $101
HHS enterprisewide initiatives - $162
Component agency initiatives - $2,049
Source: GAO analysis of Office of Management and Budget data.
Table 1 provides additional information about the component agencies and
their estimated IT budget for fiscal year 2006.
5Enterprisewide initiatives are mission-support andadministrative systems
that are used by all component agencies.
Table 1: Estimated IT Budget of HHS Component Agencies for Fiscal Year 2006
Estimated budget
for FY 2006
Component agency Mission (in millions)a
Centers for Medicare & To administer the Medicare program and work in $780
partnership with the states to
Medicaid Services administer Medicaid and the State Children's
Health Insurance Program. The agency
also enforces health insurance portability
standards and is responsible for
implementing a number of statutory provisions
that have been enacted in recent years,
including the Medicare Prescription Drug,
Improvement, and Modernization Act of
2003.
To extend healthy life and reduce the
National Institutes of burdens of illness and disability by $479
pursuing
Health fundamental knowledge about the nature and
behavior of living systems and the
application of that knowledge.
To promote health and quality of life by
Centers for Disease preventing and controlling disease, injury, $309b
and
Control and Prevention disability.
To protect the public health by ensuring the $194
Food and Drug safety, efficacy, and security of human and
Administration veterinary drugs, biological products,
medical devices, the nation's food supply,
cosmetics, and products that emit radiation.
To improve the quality, safety, efficiency, $65
Agency for Healthcare and effectiveness of health care for all
Research and Quality Americans.
Indian Health Service To raise the physical, mental, social, and $57
spiritual health of American Indians and
Alaska Natives.
Health Resources and To provide national leadership, program $51
resources, and services needed to improve
Services Administration access to culturally competent, quality
health care.
To provide a full range of program support $44
Program Support Center services to all components of HHS and other
federal agencies, primarily in the areas of
Human Resources, Health Resources,
Acquisition Services, Administrative
Services, and Financial Management.
To build resilience and facilitate recovery
Substance Abuse and for people with or at risk for substance $35
abuse
Mental Health Services and mental illness.
Administration
Administration for Children To administer federal programs that promote
the economic and social well-being of $34 and Families families, children,
individuals, and communities.
Administration on Aging To promote the dignity and independence of $2
older people, and to help society prepare
for an aging population by serving as an
advocate for older people, and by overseeing
the development of a comprehensive and
coordinated system of care that is responsive
to the needs and preferences of older people
and their family caregivers.
To provide health information and take
Agency for Toxic public health actions in order to $0b
prevent harmful
Substances and Disease exposures and disease related to toxic
substances.
Registry
Total $2.0 billion
Source: GAO analysis based on Office of Management and Budget and HHS
data.
aOffice of Management and Budget, Budget of the U.S. Government, Fiscal
Year 2006, Report on IT Spending for the Federal Government for Fiscal
Years 2004, 2005, and 2006. We did not verify these data.
bThe Agency for Toxic Substances and Disease Registry investments are
included in the total for Centers for Disease Control and Prevention.
HHS' investments reflect the diversity of the department's missions and
operating environments. For example, HHS currently has several
enterprisewide IT initiatives that enable stakeholders to advance the
causes of better health, safety, and well-being for American people. These
initiatives include:
o
Unified FinancialManagement System, a new core financial system, to help
management monitor budgets, conduct operations, evaluate program
performance, and make financial and programmatic decisions. As a core
financial system, it will interface with an estimated 110 other HHS
information systems.6
o
The Office of the Assistant Secretary for Public Health Emergency
Preparedness maintains a command center where it can coordinate the
response to public health emergencies from one centralized location. This
center is equipped with satellite teleconferencing capability, broadband
Internet hookups, and analysis and tracking software.
In addition, HHS's component agencies have several projects and systems
that are critical to the effective implementation of HHS'smission,
including the following:
o
The Food and Drug Administration's Automated Drug Information Management
System isto be developed asafully electronic information management system
that will receive, evaluate, and disseminate information about
investigational and marketing submissions for human drugs and therapeutic
biologics.
o
The National Institutes of Health's major IT initiative, the Clinical
Research Information System, is a comprehensive effort to modernize the
systems that support clinical care and the agency's collection of research
data for the intramural clinical research programs.
6GAO, Financial Management Systems: Lack of Disciplined Processes Puts
Implementation of HHS' Financial System at Risk, GAO-04-1008 (Washington,
D.C.: Sept. 23, 2004).
o
The Centers for Disease Control and Prevention's major IT initiative,
Public Health InformationNetwork,is a nationalinitiative to implement a
multiorganizational business and technical architecture for public health
information systems.7
Prior Reviews Identified Weaknesses in HHS's IT Investment Management
Process
In January 2004, we reported8 on a broad view of the government's
implementation of investment management practices at 26 major departments
and agencies, including HHS. We also reported-and HHS acknowledged-that
there were serious weaknesses in investment management. Notably, the
department had not yet established selection criteria for project
investments or a requirement that investments support work processes that
have been simplified or redesigned. In addition, the department did not
have decision-making rules to guide oversight of IT investments, review
projects at major milestones, or systematically track corrective actions.
Accordingly, we made several recommendations, including that HHS revise
its investment management policy and require PIRs to address validating
benefits and costs. In response to our recommendations, the department has
been modifying several of its investment management policies, including
its capital planning and investment control guidance and its governance
policies.
More recently, in June 2005, we reported9 that the HHS IT Investment
Review Board had conducted only budgetary reviews of the Centers for
Disease Control and Prevention's Public Health Information Network and
someofits initiatives, untilthis pastFebruary, when HHS initiated steps
for better monitoring of system development projects. We concluded that
until management implements a systematicmethod for ITinvestment reviews,
it willhave difficulty minimizing risks while maximizing returns on these
critical public health investments.
7GAO, Information Technology: Federal Agencies Face Challenge in
Implementing Initiatives to Improve Public Health Infrastructure,
GAO-05-308(Washington, D.C.: June 10, 2005).
8GAO, Information Technology Management: Governmentwide Strategic
Planning, Performance, Measurement, and Investment Management Can Be
Further Improved,
GAO-04-49 (Washington, D.C.: Jan. 12, 2004).
9GAO-05-308.
HHS's Approachto Investment Management
HHS has several groups and individuals involved in managing both the
enterprisewide and component agency IT investments.10 They are involved
from reviewing and approving a proposed IT project, through the process of
budgeting for it, monitoring it through implementation, and evaluating it
at its conclusion. The composition, roles, and responsibilities of these
individuals and groups are described below:
Information Technology Investment Review Board (ITIRB)-Chaired by HHS's
CIO, this board is responsible for selecting, controlling, and evaluating
alldepartmental IT investments. Members include the Deputy Assistant
Secretary for Budget, Finance, Performance and Planning; the Directors for
Acquisition Management Policy and Human Resources; and the component
agency CIOs. The board is supported by an executive secretary whois
responsiblefor, among other things, managing theflow of IT investment
documentation, scheduling meetings, and assisting the members in preparing
for their meetings. Currently, this board reviews all enterprisewide
investments and delegates responsibilities for component agency
investments to each individual component agencies investment review boards
in accordance with departmental policies and procedures.
CIO Council-Also chaired by the HHS CIO and comprised of component agency
CIOs, this boardadvises the HHS ITIRB on thetechnical soundness of all IT
investments that require departmental review and provides recommendations
regarding, among other things, technical aspects of affordability,
soundness of design, risk, and compliance with architectural and security
standards.
Critical Partners-Comprised of departmental officials from various
functional areas, including enterprise architecture, security and privacy,
acquisition management, finance, budget, human resources, and
e-government; this group is responsible for ensuring that most
investments11 comply with the HHS policy in each of the functional areas
and for advising the HHS ITIRB and individual IT investment managers on
10We did not evaluate HHS administrative processes for managing IT grants
to states because accordingto officials, both the department andcomponent
agenciesCIOsare not directlyinvolved in the approval oroversight of those
IT investments.
11According to HHSIT officials,for the fiscal year2006 budgetformulation,
the business casesand Selectformswereupdated for investments that
represented 80percent of the entire HHS IT portfolio dollar value. The
remaining 20percent are nonmajor investments requesting lessthan
$4.5million infiscal year 2006.
issues in their areas of expertise. Each review results in a determination
whether the investment is approved, conditionally approved, or not
approved. A not approved result is flagged for executive review.
Business Case Quality Review Team-Comprised of component agency officials,
this group evaluates the justifications for IT investments-both formal
business cases and information documented in the department's portfolio
management tool's Select forms-against the criteria used by the Office of
Management and Budget's to evaluate business cases12 agencies submit to
the office as part of the formulation of the federal budget13 and provides
recommendations for improving these justifications.
Capital Planning and Investment Control (CPIC) Reengineering/Portfolio
Management Tool (PMT) Implementation Team-Chaired by the Office ofthe CIO
officials withrepresentatives from the Critical Partners and the Business
Case Quality Review Team, this group advises the board on issues regarding
investment management policies and procedures and the implementation of
the department's portfolio management tool.
Investment Managers-Responsible for managing investments in accordance
with approved cost, schedule, and performance baselines, and for
maintaining information on project status, control, performance, risk, and
corrective actions.
Process for Managing Investments
The department has defined a three-phase process for managing investments
that involves selecting proposed projects and reselecting ongoing projects
(select phase), controlling ongoing projects through development (control
phase), and evaluating projects that have been deployed (evaluate phase).
The department retains direct management of HHS enterprisewide IT
investments and delegates considerable authority for other investments to
component agencies. Specifically, the department selects ongoing and new
component agency investments through the
12These business casesare generallyreferredto as "exhibit 300s."
13The Office of Management and Budgetevaluates the business cases against
the following 10 criteria: acquisition strategy, project
(investment)management, enterprise architecture, alternatives analysis,
riskmanagement, performance goals, securityand privacy,
performance-basedmanagement system, life-cycle costs formulation, and
support the President's Management Agenda.
process for selecting enterprisewide IT investments described below.
Controlling and evaluating component agency IT investments are delegated
to the component agencies, which are requiredby the department tofollow a
process similar to the one described below.
Each phase of the process for enterprisewide investments is comprised of
multiple steps that set out requirements needed for the HHS ITIRB to make
the decision to move forward with the project.
Thepurpose of the select phase is to ensure that HHS chooses the projects
that best support its mission and applies resources to the most important
and valuable investments. The select phase is also intended to help the
department justifybudget requests by demonstrating sound business cases
and project plans. To select investments, HHShas established two separate
components-investment screening for new investment proposals and
investment scoring and screening for ongoing investments.
During the new investment screening, the investment manager is expected to
develop a project prospectus, which identifies a specific business need
and preliminary, high-level system requirements. A high-level
determination of resource and schedule requirements is also to be
conducted as part of the business needidentification activities. Approval
of the project prospectus by the HHS ITIRB signifies that the agency
agrees that the need is critical enough to proceed to the next step in
which the business case is developed. During business case development,
the investment manager is required to develop the business case, which
establishes the lifecycle cost, schedule, benefits, and performance
baselines and includes an analysis for each investment to identify
alternatives that may satisfy the needs of the department. In addition,
the investment managerssign a document called the accountability agreement
form to accept responsibility for reporting on the project status in
achieving performance baselines throughout the remaining phases of the
investment management process.
After the project is initially approvedby the HHS ITIRB, the business
cases and Select forms for most IT investments are updated annually as
part of the budget formulation process. (The Select forms are a collection
offorms with HHS's portfolio management tool that capture investment data
to justify funding and ensure adequate project planning during the select
phase.) The first step within the annual budget formulation process
requires that all component agencies use the Select forms to report the
project cost estimates that best represent the level of funding required
to meet program or business needs.At this point, the CriticalPartnersand
the Business Case Quality Review Team score and rank the Select forms
using the department's portfolio management tool14 to create a single HHS
portfolio as well as component agency portfolios to provide
recommendationsto the component agencies for making final adjustments to
their portfolio ranking.
Once the component agencies have made the appropriate changes, the Office
of the CIO develops prioritized ITportfolios for HHS as a whole as well as
each component agency to present to the HHS ITIRB. The departmental board
and CIO Council review and comment on the prioritized portfolio and submit
it to the Secretary's Budget Council for input into their budget
deliberations. The Secretary's Budget Council then makes recommendations
to the Secretary regarding HHS and component agencies' budgets. Finally,
the department submits its approved Secretary's IT budget to the Office of
Management and Budget for inclusion in the President's Budget.
Once selectedfor inclusion inthe department'sITportfolio, each project is
to be managed by an investment manager and reviewed by the ITIRB on a
quarterly basis throughout the end of development. The board performs
reviews of projects that deviate from predetermined budget, schedule, or
performance milestones established in the business case and works with the
investment managers to develop a correction action plan. The ITIRB must
also decide whether to continue to fund the project; rebaseline the scope,
schedule, or budget; or to terminate the project.
Once a project has been fully implemented, the HHS ITIRB is to conduct
annual reviews of all HHS enterprisewide steady state investments-that is,
investments in operations and maintenance-to determine whether they
14The department's portfolio management toolwasimplemented in May 2004and
hasnot been usedyet tosupport the entire investment management process.
continue to meet the business needs. In addition, investments that have
recently completed implementation or a significant phase are to undergo
PIRs to evaluate actual development events against project management
plans and to identify lessons learned that can be applied to current and
future investments.
Figure 3 illustrates HHS's investment management process phases and steps.
The highlighted steps represent the activities that the department
conducts for both enterprisewide and component agency investments.
Figure 3: Detailed Breakdown of HHS's Investment Management Process
Source: GAO analysis based on HHS documents.
Business Need (Project Prospectus Development)
A specific business need and preliminary,
high-level systems requirements are identified and
general scope of the project is established.
Step 1 (Submission to HHS Office of the CIO and
Review Groups)
All projects must be reviewed and prioritized for
funding. The investment review board must evaluate
each project concept based on the project's impact
and the probable benefit to the department. The
review will result in a rating for each system and
then used to prioritize candidate IT investments.
Specifically, component agencies submit their
draft IT portfolio to HHS Office of the CIO that
are to be included for the annual budget
formulation process.The Business Case Quality
Review Team and the Critical Partners are expected
to score and rank both the department and
component agencies' IT investments using the
portfolio management tool and provide
recommendations to the component agencies to make
final adjustments to their portfolio ranking.
Based on these reviews, the Office of the CIO
develops a prioritized HHS IT portfolio to present
to the HHS ITIRB for review.
ITIM Maturity Framework The ITIM framework is a maturity model composed of
five progressive stages of maturity that an agency can achieve in its
investment management capabilities.15 It was developed on the basis of our
research into the IT investment management practices of leading
private-and publicsector organizations. In each of the five stages, the
framework identifies critical processes for making successful IT
investments. The maturity stages are cumulative; that is, in order to
attain a higher stage the agency must have institutionalized all of the
critical processes at the lower stages, in addition to the higher stage
critical processes.
The framework can be used to assess the maturity of an agency's investment
management processes and as a tool for organizational improvement. The
overriding purpose of the framework is to encourage investment processes
that increase business value and mission performance, reduce risk, and
increase accountability and transparency in the decision process. We have
used the framework in several of our evaluations,16 and a number of
agencies have adopted it. These agencies have used ITIM for purposes
ranging from self-assessment to redesign of their IT investment management
processes.
ITIM's five maturity stages represent steps toward achieving stable and
mature processes for managing IT investments. Each stage builds on the
lower stages; the successful attainment of each stage leads to improvement
in the organization's ability to manage its investments. With the
exception of the first stage, each maturity stage is composed of "critical
processes" that must be implemented and institutionalized in order for the
organization to achieve that stage. These critical processes are further
broken down into key practices that describe the types of activities that
an organization should be performing to successfully implement each
critical process. It is not unusual for an organization to be performing
key
15GAO-04-394G.
16GAO, Information Technology: DLA Needs to Strengthen Its Investment
Management Capability, GAO-02-314 (Washington, D.C.: Mar. 15, 2002); GAO,
United States Postal Service: Opportunities to Strengthen IT Investment
Management Capabilities, GAO-03-3 (Washington, D.C.: Oct. 15, 2002); GAO,
Information Technology: Departmental Leadership Crucial to Success of
Investment Reforms at Interior, GAO-03-1028 (Washington, D.C.: Sept. 12,
2003); GAO, Bureau of Land Management: Plan Needed to Sustain Progress in
Establishing IT Investment Management Capabilities, GAO-03-1025
(Washington, D.C.: Sept. 12, 2003); and GAO, Information Technology: FAA
Has Many Investment Management Capabilities in Place, but More Oversight
of Operational Systems Is Needed, GAO-04-822 (Washington, D.C.: Aug. 20,
2004).
practices from more than one maturity stage at the same time, but efforts
to improve investment management capabilities should focus on implementing
all lower stage practices before addressing higher stage practices.
In the ITIM framework, Stage 2 critical processes lay the foundation for
sound IT investment processes by helping the agency to attain successful,
predictable, and repeatable investment control processes at the project
level. Specifically, Stage 2 encompasses building a sound investment
management foundation by establishing basic capabilities for selecting new
IT projects. It also involves developing the capability to control
projects so that they finish predictably within established cost and
schedule expectations and the capability to identify potential exposures
to risk and put in place strategies to mitigate that risk. The basic
selection processes established in Stage 2 lays the foundation for more
mature selection capabilities in Stage 3, which represents a major step
forward in maturity, in which the agency moves from project-centric
processes to a portfolio approach, evaluating potential investments by how
well they support the agency's missions, strategies, and goals.
Stage 3 requires that an organization continually assess both proposed and
ongoing projects as parts of a complete investment portfolio-an integrated
and competing set of investment options. It focuses on establishing a
consistent, well-defined perspective on the IT investment portfolio and
maintaining mature, integrated selection (and reselection), control, and
evaluation processes, which are to be evaluated during PIRs. This
portfolio perspective allows decision makers to consider the interaction
among investments and the contributions to organizational mission goals
and strategies that could be made by alternative portfolio selections,
rather than to focus exclusively on the balance between the costs and
benefits of individual investments.
Stages 4 and 5 require the use of evaluation techniques to continuously
improve both the investment portfolio and the investment processes in
order to better achieve strategic outcomes. At Stage 4 maturity, an
organization has the capacity to conduct IT succession activities and,
therefore, can plan and implement the deselection of obsolete, high-risk,
or low-value IT investments. An organization with Stage 5 maturity
conducts proactive monitoring for breakthrough information technologies
that will enable it to change and improve its business performance.
Organizations implementing Stages 2 and 3 have in place the selection,
control, and
evaluation processes that are required by the Clinger-Cohen Act of 1996.17
Stages 4 and 5 define key attributes that are associated with the most
capable organizations.
Figure 4 shows the five ITIM stages of maturity and the critical processes
associated with each stage.
Figure 4: The Five ITIM Stages of Maturity with Critical Processes
Source: GAO.
As defined by the model, each critical process consists of "key practices"
that must be executed to implement the critical process.
17The Clinger-Cohen Act of 1996, 40 U.S.C. S:S: 11311-11313.
HHS Has Established Many Key Practices for Managing Its Investments, but
Has Provided Limited Guidance and Oversight to Component Agencies
Processes
In order to have the capabilities to effectively manage IT investments, an
agency, at a minimum, should, (1) build an investment foundation by
putting basic, project-level control and selection practices in place
(Stage 2 capabilities) and (2) manage its projects as a portfolio of
investments, treating them as an integrated package of competing
investment options and pursuing those that best meet the strategic goals,
objectives, and mission of the agency (Stage 3 capabilities). These
practices may be executed at various organizational levels of the agency,
including at the component level. However, overall responsibility for
their success remains at the department level. Therefore, at a minimum,
the department should effectively oversee component agencies' IT
investment management processes.
HHS has executed 24 of the 38 key practices that the ITIM framework
requires to build a foundation for IT investment management (Stage 2) and
8 of the 27 key practices required to manage investments as a portfolio
(Stage 3). However, the department has only provided limited oversight of
component agencies' ITIM processes. Until HHS implements and oversees a
stable investment management process throughout the department, it will
lack essential management controls over all of its IT investments, and it
will be unable to ensure that it is appropriately selecting, managing, and
evaluating the mix of investments that will maximize returns to the
organization, taking into account the appropriate level of risk.
HHS Has Established Over Half of the Foundational Practices Needed to
Manage Its Investments
At the ITIM Stage 2 level of maturity, an organization has attained
repeatable, successful IT project-level investment control processes and
basic selection processes. Through these processes, the organization can
identify expectation gaps early and take the appropriate steps to address
them. According to the ITIM, critical processes at Stage 2 include (1)
defining IT investment board18 operations, (2) identifying the business
needs for each IT investment, (3) developing a basic process for selecting
new IT proposals and reselecting ongoing investments, (4) developing
project-level investment control processes, and (5) collecting information
18An IT investment board is a decision-making body, made up of senior
program, financial, and information managers, that is responsible for
making decisions about IT projects and systems on the basis of comparisons
and trade-offs among competing projects, with an emphasis on meeting
mission goals.
about existing investments to inform investment management decisions. Table 2
describes the purpose of each of these Stage 2 critical processes.
Table 2: Stage 2 Critical Processes-Building the Investment Foundation Critical
process Purpose
Instituting the investment board To define and establish an appropriate IT
investment management structure and the processes for selecting,
controlling, and evaluating IT investments.
Meeting business needs To ensure that IT projects and systems support the
organization's business needs and meet users' needs.
Selecting an investment To ensure that a well-defined and disciplined
process is used to select new IT proposals and reselect ongoing
investments.
Providing investment oversight To review the progress of IT projects and
systems, using predefined criteria and checkpoints, in meeting cost,
schedule, risk, and benefit expectations and to take corrective action
when these expectations are not being met.
Capturing investment information To make available to decision makers
information to evaluate the impacts and opportunities created by proposed
(or continuing) IT investments.
Source: GAO.
In the federal government, the agency head and the CIO are responsible for
effectively managing information technology.19 The agency head, through
the department-level CIO, is responsible for providing leadership and
oversight for foundational critical processes by ensuring that written
policies and procedures are established, repositories of information are
created that support investment decision making, resources are allocated,
responsibilities are assigned, and all the activities are properly carried
out where they may be most effectively executed. In a large and diverse
organization such as HHS, it is especially critical that the CIO create
this structure and framework to ensure that the organization is
effectively managing its investments at every level. This means that the
CIO must ensure that component agencies have investment management
processes in place that adequately support the department's investment
management process to make certain that funds are being expended on
component agency investments that will fulfill mission needs.
1940 U.S.C. S: 11312(b)(1).
Because of the management attention that has been given to IT investment
management, the department has put in place over half of the key practices
needed to establish the investment foundation. The department has
satisfied all of the key practices associated with ensuring that projects
and systems support organizational needs and meet users' needs. It has
satisfied most of the key practices associated with identifying and
collecting investment information, selecting new proposals20 and
reselecting ongoing investments, and instituting the department's
investment review board.
However, because of its limited involvement in overseeing component agency
investments, the department has not executed any of the key practices
related to providing investment oversight.
Table 3 summarizes the status of HHS's critical processes for Stage 2 and
shows how many key practices HHS has executed in managing its IT
investments.
Table 3: Summary of Results for Stage 2 Critical Processes and Key
Practices
Critical process
Key practices executed
Total required by critical process
Percentage of key practices executed
Instituting the investment
board 5 8
Meeting business needs 7 7 100
Selecting an investment 7 10
Providing investment
oversight 0 7
Capturing investment
information 5 6
Total 24 38
Source: GAO.
20According to the ITIM, "new" proposals include both (1) previously
submitted IT proposals that were not originally selected for funding and
(2) IT proposals that have never been submitted.
HHS Has Established an Investment Review Board, but It Is Operating
without a Comprehensive Process Guide
The establishment of decision-making bodies or boards is a key component
of the IT investment management process. At the Stage 2 level of maturity,
organizations define one or more boards, provide resources to support the
boards' operations, and appoint members who have expertise in both
operational and technical aspects of proposed investments. The boards
should operate according to a written IT investment process guide that is
tailored to the organization's unique characteristics, thus ensuring that
consistent and effective management practices are implemented across the
organization.21 The organization selects board members to ensure that they
are knowledgeable about policies and procedures for managing investments.
Organizations at the Stage 2 level of maturity also take steps to ensure
that executives and line managers support and carry out the decisions of
the investment board. According to the ITIM, organizations should (1) use
an investment management guide as an authoritative document to initiate
and manage investment processes and (2) provide a comprehensive foundation
for the policies and procedures that are developed for all of the other
related processes. (The complete list of key practices is provided in
table 4.)
The department has executed 5 of the 8 key practices for this critical
process. The department established an IT investment review board as its
corporate-level investment board that consists of senior officials,
including the CIO and the Deputy Assistant Secretaries for Budget,
Finance, and Performance & Planning. The board is adequately resourced,
with most support being provided by the Office of the CIO, whose
responsibilities include developing and modifying the department's
criteria for selecting, controlling, and evaluating potential and existing
IT investments. In addition, the CIO Council reviews the enterprisewide
investments for technical soundness and provides its recommendations to
the board. The Critical Partners and Business Case Quality Review Team
provide additional support to the board by reviewing and scoring most of
their IT investments.
21According to the ITIM, a process is a sequence of steps performed for a
given purpose, and a process guide is a document that specifically defines
the manner in which the general IT investment guidance will be implemented
within the organization.
To ensure that the board's decisions are carried out for enterprisewide
investments, the ITIRB approves an accountability agreement document and
business case that identify the benefits, costs, and schedule for the
approved investments. The board then monitors the investments through the
end of development. HHS requires the component agencies to follow a
similar process in accordance with departmental policies and procedures.
We verified that an accountability agreement document was signed and the
business case identified performance expectations for the two
enterprisewide IT investments we reviewed-Public Key Infrastructure and
Enterprise Architecture initiatives.22 Additionally, the board has
oversight of the development and maintenance of the documented IT
investment process through the CPIC Reengineering/PMT Implementation Team,
who provides investment management policy change recommendations to the
board for approval.
Although HHS has implemented these key practices, it does not have a
comprehensive organization-specific process guide to direct the operations
of the investment board. While the Information Resources Management
policy, guidelines, and standard operating procedures provide general
guidance on the organization's investment management process, they do not
reflect the current investment management process. Moreover, they do not
constitute an IT investment process guide because they do not sufficiently
define the investment process. Specifically, the policies and procedures
do not include information on the roles of the key players such as the CIO
Council, Critical Partners, Business Case Quality Review Team, or the
component agency investment review boards. In addition, they do not
identify the manner in which investment board's processes are to be
coordinated with other key organizational plans and processes (such as the
budget formulation process). HHS has recently drafted a revised investment
management policy addressing many of these weaknesses; however, it has not
been finalized, and HHS officials could not provide a final issuance date.
Without a comprehensive investment management process guide, the
department lacks the assurance that IT investment activities will be
coordinated and performed in a consistent and costeffective manner.
22We reviewed two enterprisewide projects-HHS Public Key Infrastructure
and HHS Enterprise Architecture initiative, and two component agency
projects-National Institutes of Health's Electronic Research
Administration and Food and Drug Administration's Mission Accomplishment
and Regulatory Compliance Services. The projects are described in appendix
I.
Moreover, while HHS has established an IT investment board, the board does
not have business representation (that is, mission representation) from
component agencies. Instead, Chief Information Officers represent the
component agencies. According to HHS's CIO, the membership of the board is
adequate for carrying out the investment activities it currently
performs-primarily focusing on enterprisewide IT investments. However,
because allocating resources among major IT investments may require
fundamental trade-offs among a multitude of business objectives, portfolio
management decisions are essentially business decisions, and therefore
require sufficient business representation on the board. Until the
department adjusts its board membership to include business representation
from component agencies, it will not have assurance that it includes those
executives who are in the best position to make the full range of
decisions needed to enable the agency to meet its mission most
effectively, particularly as it begins to execute its full range of
responsibility.
Finally, the HHS ITIRB is not operating according to its assigned
authority and responsibility. The department's investment management
policy and the HHS ITIRB's charter state that the board has oversight
responsibility for both enterprisewide and a defined set of component
agency IT investments, including projects that are high risk,
crosscutting, and require review by the Office of Management and Budget.
However, the board currently oversees only enterprisewide IT investments.
According to HHS officials, the department has delegated authority to the
component agencies to conduct investment reviews; however, the board does
not have a mechanism in place for ensuring that component agencies are
conducting such reviews in accordance with department policies and
procedures. Until the board operates according to its assigned authority,
it cannot ensure that component agency investments are properly aligned
with the organization's objectives or reviewed by the appropriate board.
Table 4 shows the rating for each key practice required to institute the
investment board. Each of the "executed" ratings shown below represents
instances where, on the basis of the evidence provided by HHS officials,
we concluded that the specific key practices were executed by the
organization.
Table 4: Instituting the Investment Board
Type of
practice Key practice Rating Summary of evidence
Organizational 1. An enterprisewide IT investment Not Although HHS has an
enterprisewide IT investment board that
commitments board composed of senior executives executed is responsible
for defining and implementing the organization's from IT and business
units is responsible IT investment governance process and consists of the
for defining and implementing the department's senior executives from IT
and other supporting organization's IT investment governance units,
including the CIO, Deputy Assistant Secretaries for process. Budget,
Finance, Performance & Planning, and the component agencies' CIO, the
board does not have business representation from component agencies.
2. The organization has a documented Not IT investment process directing
each executed investment board's operations.
Although the Information Resources Management policy, guidelines, and
standard operating procedures provide general guidance on the department's
investment management process, these policies and procedures do not
reflect the department's current investment management process. In
addition, these documents do not constitute an investment management
process guide in that they do not (1) include information on the roles of
key working groups involved in the organization's IT investment processes
or (2) identify the manner in which investment board's processes are to be
coordinated with other key organizational plans and processes (such as the
budget formulation process) or component agency investment management
processes. HHS is currently revising its documented IT investment process
to reflect its current investment management practices.
Prerequisites 1. Adequate resources, including people, funding, and tools,
are provided for supporting the operations of each IT investment board.
Executed Adequate resources are provided to support the ITIRB's
operations. The executive secretariat provides operations support such as
scheduling meetings and managing the flow of IT investment documentation.
The CIO Council performs technical reviews of enterprisewide IT
investments and provides recommendations to the ITIRB. The Critical
Partners rank and score most IT investments from a functional perspective,
while the Business Case Quality Review Team ranks and scores these
investments against the Office of Management and Budget Exhibit 300
quality criteria.
2. The board members understand the organization's IT investment
management policies and procedures and the tools and techniques used in
the board's decision-making process.
Executed HHS ITIRB members understand the investment board's policies and
procedures and the tools and techniques used in the board's
decision-making process. High-level training has been provided to members
during past board meetings on an informal basis.
3. Each board's span of authority and responsibility is defined to
minimize overlaps or gaps among the boards.
Executed HHS' investment board, the ITIRB, is responsible for defining and
implementing the organization's IT investment governance process.
(Continued From Previous Page)
Type of
practice Key practice Rating Summary of evidence
Activities 1. The enterprisewide Executed While the HHS ITIRB does not
investment board directly oversee the
has oversight development and maintenance
responsibilities for the of HHS's documented
development and investment process, it is
maintenance of the involved in this process
through the
organization's documented CPIC Reengineering/PMT
IT Implementation Team, who
provides
investment process. investment management policy
change recommendations to
the HHS ITIRB for approval.
2. Each investment board operates in Not While, the HHS ITIRB's charter
assigns the board authority and accordance with its assigned authority
executed responsibility for reviewing both the enterprisewide and a and
responsibility. defined set of component agency IT investments, the board
primarily focuses on enterprisewide IT investments.
3. The organization has established Executed HHS ITIRB has established
management controls such as the management controls for ensuring that
accountability agreement document for ensuring that the investment boards'
decisions are carried board's decisions regarding the enterprisewide IT
investments, out. which it directly reviews, are carried out.
For the two enterprisewide projects we reviewed, we verified that
management controls were established through the accountability agreement
document and business cases.
HHS Has a Process for Ensuring That Its Investments Support Business Needs
and Meet Users' Needs
Source: GAO.
Defining business needs for each IT project helps to ensure that projects
and systems support an organization's business needs and meet users'
needs. This critical process ensures that an organization's business
objectives and its IT management strategy are linked. According to the
ITIM, effectively meeting business needs requires, among other things, (1)
documenting business needs with stated goals and objectives; (2)
identifying specific users and other beneficiaries of IT projects and
systems; (3) providing adequate resources to ensure that projects and
systems support the organization's business needs and meet users' needs;
and (4) periodically evaluating the alignment of IT projects and systems
with the organization's strategic goals and objectives. (The complete list
of key practices is provided in table 5.)
The department has in place all of the key practices for meeting business
needs. Specifically, HHS has policy and procedures that call for business
needs to be identified in the business case or the portfolio management
tool's Select forms for both proposed and ongoing enterprisewide and
component agency IT projects. Resources devoted to ensuring that IT
projects and systems support the organization's business needs and meet
users' needs include the Business Case Quality Review Team, the Critical
Partners, the portfolio management tool, and detailed procedures and
associated templates for developing business cases. HHS's specific
business mission, with stated goals and objectives, is defined in the HHS
Strategic Plan for fiscal years 2004 through 2009.
Further, HHS defines and documents business needs for both proposed and
ongoing enterprisewide and component agency IT projects, and identifies
users and other beneficiaries during its selection activities. In
addition, according to HHS IT officials, end users participate in project
management throughout the IT project's life cycle. For the four projects
we reviewed, we verified that business needs and specific users and other
beneficiaries were identified and documented in the business case or in
the Select forms within HHS's portfolio management tool. In addition, end
users are involved in project management throughout the life cycle of the
enterprisewide investments. For example, users of HHS's Public Key
Infrastructure and Enterprise Architecture initiatives participate in
project management through integrated project teams, which meet
approximately once a month and are comprised of representatives from the
component agencies. Because the department has executed all of the key
practices associated with identifying business needs, it has increased
confidence that its IT projects will meet both business needs and users'
needs.
Table 5 shows the rating for each key practice required to meet business
needs and summarizes the evidence that supports these ratings.
Table 5: Meeting Business Needs
Type of
practice Key practice Rating Summary of evidence
Organizational 1. The organization has documented Executed HHS has
policies and procedures for ensuring that IT projects and
commitments policies and procedures for identifying IT systems support the
department's ongoing and future business projects or systems that support
the needs. organization's ongoing and future business needs.
Prerequisites 1. The organization has a documented Executed The HHS
Strategic Plan for fiscal years 2004 through 2009 business mission with
stated goals and defines the agency's mission goals and objectives.
objectives.
2. Adequate resources, including people, Executed HHS has adequate
resources for ensuring that its IT projects and funding, and tools, are
provided for systems support the organization's business needs and meet
ensuring that IT projects and systems users' needs. They include Business
Case Quality Review Team, support the organization's business Critical
Partners, and the portfolio management tool. Also, HHS needs and meet
users' needs. has templates for developing business cases and training
manuals on the use of the portfolio management tool.
(Continued From Previous Page)
Type of
practice Key practice Rating Summary of evidence
Activities 1. The organization Executed HHS policies and procedures
defines and call for business needs for
documents business needs enterprisewide and component
for both agency ongoing and proposed
IT
proposed and ongoing IT projects and systems to be
projects and specified in the business
case or
systems. Select forms.
We verified that business
needs were defined and
documented
within the business case or
Select forms in the portfolio
management tool for the four
projects we reviewed.
2. The organization identifies specific Executed HHS policy and procedures
call for specific users and other users and other beneficiaries of IT
beneficiaries of both enterprisewide and component agency IT projects and
systems. projects and systems to be identified in the business case and
Select forms.
We verified that customers and stakeholders were defined and documented
within the business case or Select forms in the portfolio management tool
for the four projects we reviewed.
3. Users participate in project Executed According to HHS IT officials,
end users participate in project management throughout an IT project's
management throughout an IT project's or system's life cycle. or system's
life cycle.
We verified that users participated in project management throughout the
life cycle of the two enterprisewide projects we reviewed. According to
HHS Office of the CIO, user participation in project management is not
addressed at the department level for the two component agency projects we
reviewed since it is delegated to the component agency.
4. The investment board periodically Executed The ITIRB evaluates the
alignment of both HHS enterprisewide
evaluates the alignment of its IT projects and component agency IT systems
through the annual budget
and systems with the organization's formulation process and takes
corrective action when
strategic goals and objectives and takes misalignment occurs.
corrective actions when misalignment
occurs.
HHS Is Selecting New Investments and Reselecting Ongoing Investments, but
Lacks a Fully Documented Process for Doing So
Source: GAO.
Selecting new IT proposals and reselecting ongoing investments require a
well-defined and disciplined process to provide the agency's investment
boards, business units, and developers with a common understanding of the
process and the cost, benefit, schedule, and risk criteria that will be
used both to select new projects and to reselect ongoing projects for
continued funding. According to the ITIM, this critical process requires,
among other things, (1) making funding decisions for new proposals
according to an established process; (2) providing adequate resources for
investment selection activities; (3) using a defined selection process to
select new investments and reselect ongoing investments; (4) establishing
criteria for analyzing, prioritizing, and selecting new IT investments and
for
reselecting ongoing investments; and (5) creating a process for ensuring
that the criteria change as organizational objectives change. (The
complete list of key practices is provided in table 6.)
HHS has executed 7 of the 10 key practices associated with selecting an
investment. For example, resources devoted to selection activities include
the Critical Partners, Business Case Quality Review Team, and portfolio
management tool, which contains several forms for selecting IT projects
and systems. HHS also has detailed procedures for using its portfolio
management tool and developing business cases. The criteria for analyzing,
prioritizing, selecting and reselecting new and ongoing investments
address the President's Management Agenda, HHS strategic goals, and IT
strategic goals, value, and risk. They are incorporated into the
department's portfolio management tool and are reviewed by the investment
review board and adjusted within the tool annually at the beginning of
each budget cycle to reflect organizational objectives. This year, HHS
added additional criteria-a quality score.
HHS uses its annual budget formulation process to select both
enterprisewide and component agency proposed and ongoing IT investments.
We verified that the four projects we reviewed were reselected by the
department using the annual budget formulation process.
Although HHS has the above strengths, the department has not executed any
of the practices associated with documenting policies and procedures.
Specifically, HHS has not fully documented its process for selecting new
IT proposals and reselecting ongoing IT investments. Although a number of
documents address investment selection, they are not linked to provide
decision makers with a clear understanding of the selection and
reselection processes. In addition, they do not define the roles and
responsibilities for all key players involved in these processes.
Moreover, although the HHS Office of the CIO works directly with the
department's Office of the Budget, HHS does not have policies and
procedures documenting the integration of funding with the process of
selecting and reselecting investments. Until the department fully
documents policies and procedures for selecting new IT proposals and
reselecting ongoing IT investments, the department will not be adequately
certain that it is consistently and objectively selecting and reselecting
investments that best meet the needs and priorities of the department.
Table 6 shows the rating for each key practice required to select an
investment and summarizes the evidence that supports these ratings.
Table 6: Selecting an Investment
Type of
practice Key practice Rating Summary of evidence
Organizational 1. The organization has documented Not Although HHS has a
number of documents that address commitments policies and procedures for
selecting executed investment selection, they are not linked to provide
decision
new IT proposals. makers with a common understanding of the selection
process. In addition, these documents do not define the roles and
responsibilities for each participating unit involved in the project
selection process.
2. The organization has documented Not Although HHS has a number of
documents that address
policies and procedures for reselectinga executed investment reselection,
they are not linked to provide the decision
ongoing IT investments. makers with a common understanding of the
selection process. In
addition, these documents do not define the roles and responsibilities for
each participating unit involved in the project selection process.
3. The organization has policies and Not Although the HHS Office of the
CIO works directly with the procedures for integrating funding with
executed department's Office of the Budget, HHS does not have policies and
the process of selecting an investment. procedures documenting the
integration of funding with the
process of selecting and reselecting investments.
Prerequisites 1. Adequate resources, including people, Executed Adequate
resources are provided for identifying and selecting IT funding, and
tools, are provided for projects and systems. They include the Critical
Partners, Business identifying and selecting IT projects and Case Quality
Review Team, and the department's portfolio systems. management tool,
which contains several forms for selecting IT projects and systems.
2. Criteria for analyzing, prioritizing, and Executed HHS has established
criteria for analyzing, prioritizing, and selecting new IT investment
selecting enterprisewide and component agency new IT opportunities have
been established. investments. The department selects new IT proposals and
reselects ongoing investments using the same criteria, which are
incorporated into its portfolio management tool.
3. Criteria for analyzing, prioritizing, and Executed HHS has established
criteria for analyzing, prioritizing, and reselecting IT investment
opportunities reselecting both enterprisewide and component agency IT have
been established. investments. The department selects new IT proposals and
reselects ongoing investments using the same criteria, which are
incorporated into its portfolio management tool.
4. A mechanism exists to ensure that the Executed The HHS ITIRB reviews
and adjusts criteria annually at the start of criteria continue to reflect
organizational each budget cycle and updates the portfolio management tool
to objectives. reflect HHS's objectives.
(Continued From Previous Page)
Type of
practice Key practice Rating Summary of evidence
HHS uses its annual budget
Activities 1. The organization uses Executed formulation process to select
its defined new IT
selection process, investments.
including predefined
selection criteria, to
select new IT
We verified that the four
investments. projects we reviewed were
selected using
the annual budget formulation
activities.
2. The organization uses the defined Executed HHS uses its annual budget
formulation process to reselect
selection process, including predefined ongoing IT investments.
selection criteria, to reselect ongoing IT
investments. We verified that the four projects we reviewed were
reselected
using the annual budget formulation activities.
3. Executives' funding decisions are Executed The HHS ITIRB makes funding
decisions for new and ongoing IT
aligned with selection decisions. investments through the department's
budget formulation process, which is used to select both enterprisewide
and component agency investments.
Source: GAO.
aAccording to the GAO ITIM framework, reselecting is the periodic
reconsideration of an investment's continuing value to the organization
and the decision to continue funding. It is a recurring process that
continues for as long as a project is receiving funding.
HHS Does Not Have a Process An organization should effectively oversee its
IT projects throughout all for Effectively Overseeing Its phases of their
life cycles. Its investment board should observe each Component Agency IT
project's performance and progress toward predefined cost and schedule
Investments
expectations as well as each project's anticipated benefits and risk
exposure. This does not mean that a departmental board, such as the ITIRB,
should micromanage each project to provide effective oversight; rather it
means that the departmental board should be actively involved in all IT
investments and proposals that are high cost or high risk or have
significant scope and duration and at a minimum, should, have a mechanism
for maintaining visibility of other investments. The board should also
employ early warning systems that enable it to take corrective actions at
the first sign of cost, schedule, and performance slippages. According to
the ITIM, effective project oversight requires, among other things, (1)
having written policies and procedures for management oversight; (2)
developing and maintaining an approved management plan for each IT
project; (3) making up-to-date cost and schedule data for each project
available to the oversight boards; (4) having regular reviews by each
investment board of each project's performance against stated
expectations; and (5) ensuring that corrective actions for each
underperforming project are documented, agreed to, implemented, and
tracked until the desired outcome is achieved. (The complete list of key
practices is provided in table 7.)
The department has not executed any of the seven key practices associated
with effective project oversight, primarily because of its limited role in
overseeing component agency IT investments. Specifically, while the
department has documented standard operating procedures and instructional
memorandums for oversight of enterprisewide IT investments, they are not
comprehensive in that they do not specify the board's responsibilities for
investment oversight; procedural rules for the ITIRB operations and
decision making during project oversight; or policies and procedures for
overseeing component agency IT investments.
The HHS ITIRB is currently performing regular reviews23 of enterprisewide
IT projects and systems against stated expectations through reports that
are available to decision makers on the HHS Intranet. However, the
department is not regularly reviewing component agency investments that
are high risk, crosscutting, and require review by the Office of
Management and Budget, although their policy calls for it. The board also
does not have a mechanism for maintaining visibility of other component
agency investments.
The department delegates oversight of these investments to the component
agencies but believes it is nonetheless effectively overseeing component
agency investments through (1) reviews of these investments as part of the
annual Critical Partner and Business Case Quality reviews performed during
the annual selection process and the use of (2) earned value management
data.24 Although the annual reviews may provide insight into the status of
investments, they are not frequent enough to allow for timely
identification of problems. Moreover, while HHS officials told us that
staff responsible for collecting earned value management data on component
agency investments share significant concerns about the data with the
ITIRB, they did not have formal documentation clearly supporting this
issue. In addition, formal procedures for elevating issues to the board
have not been developed. In the absence of effective board oversight, HHS
executives will not have the information they need to determine whether
23HHS conducts quarterly reviews on its enterprisewide investments during
the period of development and annual reviews of its steady state
enterprisewide investments, that is, those systems that have completed
development and become operational.
24Earned value management is a project management tool that integrates the
investment scope of work with schedule and cost elements for investment
planning and control. This method compares the value of work accomplished
during a given period with that of the work expected in the period.
Differences in expectations are measured in both cost and schedule
variances.
component agency projects are being developed on schedule and within
budget. In addition, the department will run the risk that underperforming
component agency projects will not be identified in time for corrective
actions to be taken.
We verified that HHS provided oversight for the two enterprisewide
investments, but had delegated oversight activities for the two component
agency investments we reviewed.
Table 7 shows the rating for each key practice required to provide
investment oversight and summarizes the evidence that supports these
ratings.
Table 7: Providing Investment Oversight
Type of
practice Key practice Rating Summary of evidence
Organizational 1. The organization has documented Not Although HHS has
developed standard operating procedures and
commitment policies and procedures for management executed instructional
memorandums for oversight of enterprisewide IT oversight of IT projects
and systems. projects and systems, they do not (1) specify the HHS ITIRB's
responsibilities when providing investment oversight within its domain or
(2) procedural rules for the ITIRB's operations and for decision making
during project oversight. In addition, HHS does not have policies and
procedures for management oversight of component agency investments.
Prerequisites 1. Adequate resources, including people, funding, and tools,
are provided for IT project oversight.
Not executed Although HHS has adequate resources for providing oversight
for enterprisewide IT investments, the department does not have adequate
resources for providing oversight for component agency IT investments.
2. IT projects and systems, including
Not HHS's policy calls for an accountability agreement document and
those in steady state (operations and maintenance), maintain approved
project management plans that include expected cost and schedule
milestones and measurable benefit and risk expectations.
executed
business case, including cost, benefit, schedule, and risk expectations,
to be available to the ITIRB after approval of an enterprisewide IT
projects and systems, but there is no similar requirement for component
agency IT projects and systems.
We verified that HHS provided oversight for the two enterprisewide
investments, but had delegated oversight activities for the two component
agency investments we reviewed.
(Continued From Previous Page)
Type of
practice Key practice Rating Summary of evidence
Activities 1. Data on actual performance (including Not cost, schedule,
benefit, and risk executed performance) are provided to the appropriate IT
investment board.
Data on actual performance of enterprisewide IT investments are provided
to the HHS ITIRB; however, the ITIRB does not regularly receive data on
actual performance of a defined set of component agencies' IT investments
and maintain visibility of other investments.
We verified that the two enterprisewide projects provide quarterly reports
to the ITIRB. For the component agency projects we reviewed, this activity
is delegated to the component agency and is not addressed at the
department level.
2. Using verified data, each investment Not HHS ITIRB quarterly reviews
performance of enterprisewide IT board regularly reviews the performance
executed investments under development and annually reviews of IT projects
and systems against stated enterprisewide IT investment in their
operational phase of their life expectations. cycles; however, the
investment board does not have a process
for regularly reviewing the performance of a defined set of component
agency investments and maintaining visibility of other investments.
3. For each underperforming IT project or Not The HHS ITIRB takes
appropriate actions to correct or terminate
system, appropriate actions are taken to executed the enterprisewide IT
projects or systems. However, it does not
correct or terminate the project or system take actions to correct or
terminate underperforming component
in accordance with defined criteria and agency investments because it does
not regularly review these
the documented policies and procedures investments' performance.
for management oversight.
4. The investment board regularly tracks Not The HHS ITIRB maintains
meeting minutes for enterprisewide IT the implementation of corrective
actions executed investments to ensure that corrective actions are
implemented for each underperforming project until the and tracked until
the desired outcome is achieved. However, it actions are completed. does
not take actions to correct or terminate underperforming
component agency investments because it does not regularly
review these investments' performance.
Source: GAO.
HHS Has a Defined Process for To make good IT investment decisions, an
organization must be able to Capturing Investment acquire pertinent
information about each investment and store that
Information
information in a retrievable format. During this critical process, an
organization identifies its IT assets and creates a comprehensive
repository of investment information. This repository provides information
to investment decision makers to help them evaluate the potential impacts
and opportunities created by proposed or continuing investments. It can
provide insights into major IT cost and management drivers and trends. The
repository can take many forms and need not be centrally located, but the
collection method should, at a minimum, identify each IT investment and
its associated components. This critical process may be satisfied by the
information contained in the organization's current enterprise
architecture, augmented by additional information-such as financial
information and information on risk and benefits-that the investment board
may require to
ensure that informed decisions are being made. According to the ITIM,
effectively managing this repository requires, among other things, (1)
developing written policies and procedures for identifying and collecting
the information; (2) assigning responsibilities for ensuring that the
information being collected meets the needs of the investment management
process; (3) identifying IT projects and systems and collecting relevant
information to support decisions about them; and (4) making the
information easily accessible to decision makers and others. (The complete
list of key practices is provided in table 8.)
HHS has executed 5 of the 6 key practices for capturing investment
information. For example, the department has several documents that define
the policies and procedures for identifying and collecting investment
information in its repositories and also assign responsibility to the HHS
CIO for ensuring that the information collected during project and systems
identification meets the needs of the investment management process. HHS
maintains a portfolio management tool, which serves as the primary
repository for identifying and collecting information about both
department and component agency IT projects and systems. The department's
portfolio management tool is easily accessible to decision makers at both
the department and component level and the Office of the CIO has provided
decision makers with various training manuals and guidance memorandums. In
addition, the department also identifies and collects information about
enterprisewide IT investments using its Intranet. Further, the department
recently began collecting earned value information through spreadsheets on
major HHS IT investments that compares planned and actual cost and
schedule information. These repositories are easily accessible to the
board members.
The key practice HHS has not executed has to do with the captured
investment information not yet being used by the HHS ITIRB to fully
support decisions about component agency investments. For example, the
earned value investment data received from each component agency has not
been used by the HHS ITIRB for control and evaluation decisions. According
to agency officials, the department has recently begun monitoring the
earned value data to identify investments that report cost and schedule
variances and these officials acknowledge a need to formalize the process
for doing so. Until HHS's decision makers use the information in the
repository to fully support the investment management process, it will be
unable to effectively evaluate the impacts and opportunities created by
proposed or continuing investments.
Table 8 shows the rating for each key practice required to capture
investment information and summarizes the evidence that supports these
ratings.
Table 8: Capturing Investment Information
Type of
practice Key practice Rating Summary of evidence
Organizational 1. The organization has documented policies Executed The
department has documented policies and procedures for
commitments and procedures for identifying and collecting identifying and
collecting information about IT projects and information about IT projects
and systems to systems to support the investment management process.
support the investment management process.
2. An official is assigned responsibility for Executed The HHS CIO is
responsible for ensuring that the information
ensuring that the information collected during collected during project
and systems identification meets the
project and systems identification meets the needs of the investment
management process.
needs of the investment management
process.
Prerequisite 1. Adequate resources, including people, Executed According
to the HHS IT officials, adequate resources are funding, and tools, are
provided for identifying provided for identifying IT projects and systems
and IT projects and systems and collecting collecting relevant investment
information about them. relevant investment information about them.
Activities 1. The organization's IT projects and systems are identified,
and specific information is collected to support decisions about them.
Executed HHS's portfolio management tool identifies and collects
information about both department and component agency IT projects and
systems to support the investment management process as it currently
exists. The department also identifies and collects relevant investment
information for the enterprisewide IT investments through the HHS Intranet
and component agency IT investments through spreadsheets that capture
earned value data.
We verified that HHS's portfolio management tool identifies and contains
investment information for the four projects we reviewed.
2. The information that has been collected is Executed IT investment
decision makers at both the department and easily accessible and
understandable to component agency level have access to HHS's portfolio
decision makers and others. management tool that is used to capture IT
project and
system information. Instructions on the use and navigation through the
portfolio management system are available to investment management
decision makers. In addition, the HHS ITIRB can also access the
enterprisewide IT investment information posted on the HHS Intranet.
3. The information repository is used by Not While HHS identifies and
collects information about IT investment decision makers and others to
executed projects and systems to support the investment management support
investment management. process, this information has not been used by the
HHS
ITIRB to fully support the control and evaluate decisions for
component agency IT investments.
Source: GAO.
HHS Has Some of the Capabilities Needed to Manage IT Investments as a
Portfolio
Once an agency has attained Stage 2 maturity, it needs to implement
critical processes for managing its investments as a portfolio (Stage 3).
An IT investment portfolio is an integrated, agencywide collection of
investments that are assessed and managed collectively based on common
criteria. Managing investments as a portfolio is a conscious, continuous,
and proactive approach to allocating limited resources among an
organization's competing initiatives in light of the relative benefits
expected from these investments. Taking an agencywide perspective enables
an organization to consider its investments comprehensively, so that
collectively the investments optimally address the organization's
missions, strategic goals, and objectives. Managing IT investments as a
portfolio also allows an organization to determine its priorities and make
decisions about which projects to fund and continue to fund based on
analyses of the relative organizational value and risks of all projects,
including projects that are proposed, under development, and in operation.
Although investments may initially be organized into subordinate
portfolios-based on, for example, business lines or life cycle stages-and
managed by subordinate investment boards; they should ultimately be
aggregated into this enterprise-level portfolio.
According to the ITIM framework, Stage 3 maturity includes (1) defining
the portfolio criteria, (2) creating the portfolio, (3) evaluating the
portfolio, and (4) conducting postimplementation reviews. Table 9
summarizes the purpose of each critical process in Stage 3.
Table 9: Stage 3 Critical Processes-Developing a Complete Investment Portfolio
Critical process Purpose
Defining the portfolio criteria To ensure that the organization develops
and maintains IT portfolio selection criteria that support its mission,
organizational strategies, and business priorities.
Creating the portfolio To ensure that IT investments are analyzed
according to the organization's portfolio selection criteria and to ensure
that an optimal IT investment portfolio with manageable risks and returns
is selected and funded.
Evaluating the portfolio To review the performance of the organization's
investment portfolio(s) at agreed-upon intervals and to adjust the
allocation of resources among investments as necessary.
Conducting postimplementation To compare the results of recently
implemented investments with the expectations that were set reviews for
them and to develop a set of lessons learned from these reviews.
Source: GAO.
HHS has executed 8 of the 27 key practices required by Stage 3. For
example, the department's core IT portfolio selection criteria, including
cost, benefit, schedule, and risk are approved by the HHS ITIRB. In
addition, the investment board examines the mix of new and ongoing
investments and their respective data and analyses to select investments
to fund. However, many key practices still need to be executed before HHS
can effectively manage its IT investments from a portfolio perspective.
For example, HHS has not addressed any of the key practices related to
evaluating the portfolio or conducting PIRs. Until HHS fully implements
the critical processes associated with managing its investments as a
complete portfolio, it will not have the data it needs to make informed
decisions about competing investments.
Table 10 summarizes the status of HHS's critical processes for Stage 3,
showing how many associated key practices it has executed.
Table 10: Summary of Results for Stage 3 Critical Processes and Key
Practices
Critical process
Key practices executed
Total required by critical process
Percentage of key practices executed
Defining the portfolio criteria 5 7
Creating the portfolio 3 7
Evaluating the portfolio 0 7
Conducting
postimplementation reviews 0 6
Total 8 27
Source: GAO.
Process for Modifying IT To manage IT investments effectively, an
organization needs to establish Portfolio Selection Criteria Is rules or
"portfolio selection criteria" for determining how to allocate scarce
Not Institutionalized
funding to existing and proposed investments. Thus, developing an IT
investment portfolio requires defining appropriate cost, benefit,
schedule, and risk criteria with which to evaluate individual investments
in the context of all other investments. To ensure that the organization's
strategic goals, objectives, and mission will be satisfied by its
investments, the criteria should have an enterprisewide perspective.
Further, if an organization's mission or business needs and strategies
change, criteria for selecting investments should be reexamined and
modified as appropriate. Portfolio selection criteria should be
disseminated throughout the organization to ensure that decisions
concerning investments are made in a consistent manner and that this
critical process is institutionalized. To
achieve this result, project management personnel and others should be
aware of the criteria and address the criteria in funding submissions for
projects. Resources required for this critical process typically include
the time and attention of executives involved in the process, adequate
funding, and supporting tools. (The complete list of key practices is
provided in table 11.)
The department has executed 5 of the 7 key practices for this critical
process. For example, responsibility has been assigned to the HHS Lead
Capital Planner for managing the development and modification of the IT
portfolio selection criteria, and adequate resources have been committed
for portfolio selection activities, including the Critical Partners,
portfolio management tool project manager, and the Office of the CIO
staff. Moreover, the project management personal and other stakeholders
are aware of the portfolio selection criteria that are embedded into the
department's portfolio management tool and also contained within policies
and procedures.
Finally, the HHS ITIRB approves the core IT selection criteria, including
cost, benefit, schedule, and risk criteria, based on the organization's
mission, goals, strategies, and priorities. Beginning in fiscal year 2004,
HHS began scoring and ranking approximately 80 percent of its IT
investments against alignment, value, and risk criteria in order to
determine a priority score, which is the sum of alignment, value, and risk
criteria scores, weighted for relative importance. Similarly, for the
fiscal year 2007 budget formulation process, HHS began collecting
investment information on the business case quality, Critical Partner
reviews, and cost and schedule variance to determine a quality score,
which is the sum of the business case quality, Critical Partner reviews,
and cost and schedule variance scores, weighted for relative importance.
The HHS ITIRB evaluates and annually adjusts its portfolio selection
criteria within the portfolio management tool.
Despite these important steps in defining portfolio selection criteria,
weaknesses remain. The department has not developed policies or procedures
for modifying the portfolio selection criteria to reflect changes to HHS
mission, goals, strategies, and priorities. In addition, the HHS ITIRB
began reviewing the IT portfolio selection criteria this year. However,
the process for modifying portfolio selection criteria is not
institutionalized because the process to do so was only used once and
there are no documented policies and procedures to ensure that it will be
used again. Until HHS defines and implements the practices required for
defining the portfolio criteria definition, it will not have the tool it
needs to select
investments that support its mission, organizational strategies, and
business priorities.
Table 11 shows the rating for each key practice required to define
portfolio selection criteria and summarizes the evidence that supports
these ratings.
Table 11: Defining the Portfolio Criteria
Type of
practice Key practice Rating Summary of evidence
Organizational commitments 1. The organization has documented policies and
procedures for creating and modifying IT portfolio selection criteria. Not
executed While HHS has policies and procedures for creating IT portfolio
selection criteria, the department lacks policies and procedures for
modifying the portfolio selection criteria. 2. Responsibility is assigned
to an individual or group for managing the development and modification of
the IT portfolio selection criteria.
Executed The HHS Lead Capital Planner is responsible for managing the
development and modification of the IT portfolio selection criteria.
Prerequisites 1. Adequate resources, including people, Executed Adequate
resources have been committed for portfolio selection funding, and tools,
have been committed criteria activities. They include the Critical
Partners, portfolio for portfolio selection criteria activities.
management tool project manager, and the Office of the CIO staff.
2. A working group has been designated Executed The CPIC Reengineering/PMT
Implementation Team conducts to be responsible for developing and weekly
teleconferences with HHS component agencies to modifying the IT portfolio
selection coordinate investment management issues, including the criteria.
development and modification of IT portfolio selection criteria.
According to HHS IT officials, this group will evolve into the Policy
Advisory Board, which, among other things, will formalize the IT portfolio
selection criteria activities.
1. The enterprisewide The HHS ITIRB approves the
Activities investment board Executed core IT portfolio selection
criteria,
approves the core IT including cost, benefit,
portfolio selection schedule, and risk criteria,
based on the
criteria, including CBSR organization's mission,
criteria, based goals, strategies, and
priorities.
on the organization's
mission, goals,
strategies, and
priorities.
2. Project management personnel and Executed Project management personnel
and other stakeholders are aware other stakeholders are aware of the of
the portfolio selection criteria, which are embedded into HHS's portfolio
selection criteria. portfolio management tool and contained in policies
and
procedures.
3. The enterprisewide investment board Not The HHS ITIRB began reviewing
the IT portfolio selection criteria regularly reviews the IT portfolio
executed this year. However, the process for modifying the portfolio
selection criteria, using cumulative selection criteria is not
institutionalized because it was only used experience and event-driven
data, and once and there are no documented policies and procedures to
modifies the criteria as appropriate. ensure that it will be used again.
Source: GAO.
Process for Creating a Portfolio At Stage 3, organizations create a
portfolio of IT investments to ensure that
Is Not Documented IT investments are analyzed according to the
organization's portfolio selection criteria and to ensure that an optimal
IT investment portfolio with
manageable risks and returns is selected and funded. According to ITIM,
creating the portfolio requires organizations to, among other things,
document policies and procedures for analyzing, selecting, and maintaining
the portfolio; provide adequate resources, including people, funding, and
tools for creating the portfolio; and capture the information used to
select, control, and evaluate the portfolio and maintain it for future
reference. In creating the portfolio, the investment board must also (1)
examine the mix of new and ongoing investments, and their respective data
and analyses and select investments for funding and (2) approve or modify
the performance expectations for the IT investments they have selected.
(The complete list of key practices is provided in table 12.)
HHS has executed 3 of the 7 key practices associated with creating the
portfolio. Beginning in fiscal year 2004, the department began to create a
portfolio by using its portfolio management tool to collect cost, benefit,
schedule, risk, strategic alignment, and enterprise architecture
information on investments accounting for 80 percent of the dollar value
of the HHS IT investment portfolio. Each component agency's IT portfolio
is displayed in priority order along with where each investment falls
within the overall IT portfolio. Further, according to HHS IT officials,
the agency has adequate resources for portfolio selection activities,
including the Critical Partners, the portfolio management tool project
manager, and the Office of the CIO staff. These officials also stated that
HHS ITIRB members are also knowledgeable about the process of creating a
portfolio.
Nevertheless, HHS has a number of significant weaknesses in the way it
creates a portfolio. First, it does not have policies and procedures that
sufficiently address this critical process. Although the department has
policies and procedures for creating IT portfolio selection criteria, they
lack policies and procedures for using these criteria to analyze, select,
and maintain the investment portfolio. Second, even though the HHS ITIRB
has quarterly reviews to compare project and system performance with
expectations for enterprisewide IT investments, the board is not provided
with information comparing the performance of component agency investments
against expectations. In addition, the board approves or modifies the
performance expectations for the enterprisewide IT investments it has
selected, but does not regularly approve or modify the performance
expectations for component agency IT investments or ensure that this is
done. Moreover, as previously mentioned, investment information has not
been used to fully support control and evaluate decisions for component
agency investments. Unless HHS defines and implements the practices for
creating a comprehensive portfolio of IT
investments, it will not be able to determine whether it has selected the
mix of investments that best meets its needs considering resource and
funding constraints.
Table 12 shows the rating for each key practice required to create a
portfolio and summarizes the evidence that supports these ratings.
Table 12: Creating the Portfolio
Type of
practice Key practice Rating Summary of evidence
Organizational 1. The organization has documented Not While HHS has
policies and procedures for creating IT portfolio
commitment policies and procedures for analyzing, executed selection
criteria, the department lacks policies and procedures selecting, and
maintaining the investment for using these criteria to analyze, select,
and maintain the portfolio. investment portfolio.
Prerequisites 1. Adequate resources, including people, Executed According
to HHS IT officials, adequate resources have been funding, and tools, are
provided for the committed for portfolio selection criteria activities.
They include process of creating the portfolio. the Critical Partners,
portfolio management tool project manager, and Office of the CIO staff.
2. Board members are knowledgeable Executed HHS ITIRB members are
knowledgeable about the process of
about the process of creating a portfolio. creating a portfolio; they have
now gone through the process twice.
3. The investment board is provided with Not While the investment board is
provided with information information comparing project and system
executed comparing HHS enterprisewide project and system performance
performance with expectations. with expectations, it is not provided with
information comparing
the performance of component agency investments against
expectations.
Activities 1. Each IT investment board examines the Executed The ITIRB
examines a mix of new and ongoing investments mix of new and ongoing
investments and through the department's portfolio management tool, which
is their respective data and analyses and used to analyze, prioritize, and
select investments for funding. selects investments for funding.
2. Each investment board approves or Not While the HHS ITIRB approves the
performance expectations for modifies the performance expectations for
executed its enterprisewide IT investments, it does not have a similar its
selected IT investments. process for approving the performance
expectations for
component agency IT investments or ensuring that this is done.
3. Information used to select, control, and Not Although HHS is capturing
investment information, the evaluate the portfolio is captured and
executed information is not yet used to fully support control and evaluate
maintained for future reference. decisions about component agency
investments.
Source: GAO.
Criteria for Portfolio This critical process builds upon the Stage 2
critical process, Providing Performance Evaluations Are Investment
Oversight, by adding the elements of portfolio performance to Not Yet
Developed or Regularly an organization's investment control capacity.
Compared with less mature Modified organizations, Stage 3 organizations
will have the foundation they need to
control the risks faced by each investment and to deliver benefits that
are
linked to mission performance. In addition, a Stage 3 organization will
have the benefit of performance data generated by Stage 2 processes.
Executivelevel oversight of risk management outcomes and incremental
benefit accumulation provides the organization with increased assurance
that each IT investment will achieve the desired results. (The complete
list of key practices is provided in table 13.)
HHS has not executed any of the seven key practices for evaluating a
portfolio. It has yet to develop policies and procedures that address
performance oversight from a portfolio perspective. Moreover, while the
department annually reviews its portfolio as part of its selection
process, it does not evaluate the investment portfolio on a continuing
basis to assess its performance. Finally, the results of Providing
Investment Oversight reviews from Stage 2 are important to this critical
process. However, as previously mentioned, while the HHS ITIRB has
oversight of enterprisewide investments, it does not regularly review a
defined set of component agencies' investments and maintain visibility of
other investments. Although the department's portfolio management tool has
the ability to summarize performance metrics for each investment and
quickly understand the status of each investment and any potential
emerging problem area, the tool is currently only being used on an ad hoc
basis to make portfolio oversight decisions. Defining and implementing
processes to evaluate the performance of its entire portfolio would
provide HHS with greater assurance that it is controlling the risks and
achieving the benefits associated with the mix of investments it has
selected.
Table 13 shows the rating for each key practice required to evaluate the
portfolio and summarizes the evidence that supports these ratings.
Table 13: Evaluating the Portfolio
Type of
practice Key practice Rating Summary of evidence
Organizational 1. The organization has documented Not HHS does not have
policies and procedures for reviewing,
commitment policies and procedures for reviewing, executed evaluating, and
improving the performance of its portfolio. evaluating, and improving the
performance of its portfolio(s).
Prerequisites 1. Adequate resources, including people, Not Although HHS
annually reviews its portfolio as part of its selection funding, and tools
have been provided for executed process, it does not evaluate the
performance on a continuing reviewing the investment portfolio and its
basis. projects.
2. Board members are familiar with the Not Although HHS annually reviews
its portfolio as part of its selection process for evaluating and
improving the executed process, it does not evaluate the performance on a
continuing portfolio's performance. basis.
3. Results of relevant Providing Not While the HHS ITIRB has oversight of
enterprisewide investments, Investment Oversight reviews from Stage
executed it does not effectively oversee its component agency IT 2 are
provided to the investment board. investments.
4. Criteria for assessing portfolio Not HHS does not have criteria for
assessing portfolio performance.
performance are developed, reviewed, executed
and modified at regular intervals to reflect
current performance expectations.
Activities 1. IT portfolio performance measurement Not HHS does not have
criteria for assessing portfolio performance. data are defined and
collected consistent executed with portfolio performance criteria.
2. Adjustments to the IT investment Not Although HHS annually reviews its
portfolio as part of its selection portfolio are executed in response to
executed process, it does not evaluate the performance on a continuing
actual portfolio performance. basis.
Source: GAO.
Process for Conducting The purpose of a PIR is to evaluate an investment
after it has completed Postimplementation Reviews Is development (that is,
after its transition from the implementation phase to
Not Defined
the operations and maintenance phase) in order to validate actual
investment results. This review is conducted to (1) examine differences
between estimated and actual investment costs and benefits and possible
ramifications for unplanned funding needs in the future and (2) extract
"lessons learned" about the investment selection and control processes
that can be used as the basis for management improvements. Similarly, PIRs
should be conducted for investment projects that were terminated before
completion, to readily identify potential management and process
improvements. (The complete list of key practices is provided in table
14.)
HHS has not executed the six key practices for conducting PIRs. Although
its policy calls for postimplementation reviews of IT investments that
have recently completed implementation of the entire investment or a
significant
phase of the investment, the department does not have specific procedures
for conducting such reviews, including specifying who conducts and
participates in the PIR, what information is presented in a PIR, or how
results are to be disseminated to decision makers. To date, HHS has
conducted closeout reviews of two enterprisewide investments following
their implementation; however, while these reports do cover investment
cost expectations, they cannot be considered PIRs because the reports do
not address general conclusions, lessons learned, or schedule deviations.
Unless PIRs are conducted on a regular basis, HHS will not be able to
effectively evaluate the results of its IT investments to determine
whether continuation, modification, or termination of an IT investment
would be necessary in order to meet stated HHS mission objectives.
Table 14 shows the rating for each key practice required to conduct PIRs
and summarizes the evidence that supports these ratings.
Table 14: Conducting Postimplementation Reviews
Type of
practice Key practice Rating Summary of evidence
Organizational 1. The organization has documented policies and procedures
for Not Although, HHS has policy for conducting
commitment conducting PIRs. executed PIRs, the department does not have
associated procedures for conducting such reviews.
Prerequisites 1. Adequate resources, including people, funding, and tools,
have Not HHS is not conducting PIRs. been provided for conducting PIRs.
executed
2. Individuals assigned to the investment board to conduct PIRs Not HHS is
not conducting PIRs. should be familiar with both the policies and the
procedures for executed conducting such reviews.
Activities 1. The investment board identifies which projects will have a
PIR Not HHS is not conducting PIRs. conducted. executed
2. Quantitative and qualitative investment data are collected, Not HHS is
not conducting PIRs. evaluated for reliability, and analyzed during the
PIRs. executed
3. Lessons learned and recommendations for improving the Not HHS is not
conducting PIRs.
investment process are developed during the PIR, documented, executed
and then distributed to all stakeholders.
Source: GAO.
HHS Has Provided Limited Guidance to and Oversight of Component Agencies'
Investment Management Processes
The ability of a department-level CIO to effectively oversee IT investment
management processes throughout the agency depends on the existence of
appropriate management structures with adequate authorities and sufficient
guidance. Under the Clinger-Cohen Act of 1996, the CIO of each agency is
responsible for effectively managing all of the agency's IT resources. To
comply with the act, HHS designates its CIO to be responsible for ensuring
that the component agencies are defining and implementing effective
investment management processes that are appropriately aligned with the
department's processes.
Although each component agency has staff responsible for gathering,
maintaining, and analyzing IT investment information, the HHS Office of
the CIO has the responsibility to define and implement overall HHS IT
investment management practices, and monitor component agency investment
management practices to ensure a cohesive departmental process and the
capability exists to carry out the process. In accordance with this, the
department's investment management policies and guidelines state that the
component agencies are to establish and manage investment management
processes and governance structures that are aligned with the department's
policies and procedures. However, as mentioned in previous sections, the
department's investment management policies and procedures have several
weaknesses. For example, HHS does not have a set of documented procedures
that provide decision makers with a clear understanding of the selection
and reselection process.
Moreover, HHS currently has no structured mechanism in place to ensure
that the component agencies are adhering to the department's policies and
procedures. According to HHS officials, the CIO has the authority to audit
a component agencies IT investment management process. However, they were
unable to provide us evidence of having performed any such audits. These
officials also stated that the department's portfolio management tool is
another method that will enable HHS to oversee component-level investment
management processes. However, since not all component agencies are using
the portfolio management tool to individually make select, control, and
evaluate decisions, its usefulness in this regard is limited. Until the
department develops a mechanism for ensuring that component agencies
define and implement investment management processes that align with those
of the department, it is running the risk that effective processes are
being institutionalized at both the department and the component agency
level. In addition, the department will be unable to ensure that it is
optimizing its investments in IT and effectively assessing and managing
the risks of these investments.
HHS Does Not Have a Plan to Coordinate and Guide Improvement Efforts
HHS has initiated several efforts to improve its investment management
process. Specifically, it has drafted a revised investment management
guide that addresses the weaknesses with current guidance that we identify
in this report. In addition, in February 2005, HHS incorporated
capabilities into its portfolio management tool to enhance performance of
control and evaluate functions. Specifically, the tool now has the
capabilities to produce (1) scorecards to provide data for each investment
in a portfolio, allowing cross investment comparisons on data elements
collected; (2) investor maps to provide a graphical depiction of a
portfolio in terms of up to six data categories, with the ability to show
target and actual values; and (3) a workbook module to track the
identification and resolution of issues that may arise regarding the
management of an investment or set of investments.
Although HHS has initiated these efforts, they only fully address 2 of the
14 Stage 2 key practices the department did not execute.
o The draft investment management guidance, when finalized, will address
weaknesses associated with one of the key practices for instituting the
investment board by reflecting the current management process, including
information on the roles of key working groups involved in the
organization's IT investment processes, and identifying the manner in
which investments board's processes are to be coordinated with other key
organizational plans and processes. The guidance will also address the
integration of the funding and selection processes, a key practice the
department has not executed that is associated with selecting an
investment.
o The enhanced portfolio management tool capabilities will enhance the
department's ability to oversee investments' performance and position the
board to perform portfolio evaluation activities, but they will not fully
address any of the weaknesses we identify.
HHS has not coordinated these and additional efforts that would address
the weaknesses we identify in this report in a comprehensive plan that (1)
specifies measurable goals, objectives, and milestones; (2) specifies
needed resources; (3) assigns clear responsibility and accountability for
accomplishing tasks; and (4) is approved by senior management. We have
previously reported that such a plan is instrumental in helping agencies
coordinate and guide improvement efforts. Until HHS develops a plan that
would allow for the systematic prioritization, sequencing, and evaluation
of
improvement efforts, the agency risks not being able to effectively
establish the mature investment management processes that result in
greater certainty about the outcomes of future IT investments.
Conclusions Because of the attention that has been given to investment
management, HHS has established several of the practices needed to
effectively manage its investments. These practices have strengthened the
department's basic capabilities for selecting and controlling projects and
begun to equip the department with the capabilities it needs to make
informed decisions about competing investments. However, several
significant weaknesses remain in the foundational practices needed to
manage individual investments, the portfolio-level investments needed to
manage investments as a collection, and in the level of guidance and
oversight provided to component agency investment management processes.
These weaknesses hamper the department's ability to ensure that it is
managing the mix of investments that will maximize returns to the
organization, taking into account the appropriate level of risk.
Critical to HHS's success, going forward will be the development of an
implementation plan that (1) is based on an assessment of strengths and
weaknesses; (2) specifies measurable goals, objectives, and milestones;
(3) specifies needed resources; (4) assigns clear responsibility and
accountability for accomplishing tasks; and (5) is approved by senior
management. Although the department has initiated improvement efforts, it
has not developed a comprehensive plan to guide these and other efforts
needed to improve its investment management process. Without such a plan
and procedures for implementing it, it is unlikely that the department
will effectively establish mature investment management capability. As a
result, HHS will continue to be challenged in its ability to make informed
and prudent investment decisions in managing its annual
multibillion-dollar IT budget.
Recommendations for To strengthen HHS's investment management capability
and address the weaknesses discussed in this report, we recommend that the
Secretary of
Executive Action the Department of Health and Human Services direct the
Chief Information Officer to develop and implement a plan for improving
the department's IT investment management processes. The plan should
address the weaknesses described in this report, beginning with those we
identified in our Stage 2 analysis and continuing with those we identified
in our Stage 3
analysis. The plan should, at a minimum, provide for accomplishing the
following:
In Stage 2
o Develop comprehensive guidance and additional supporting guidance that
defines and describes the complete investment management process, unifies
existing processes enterprisewide, reflects changes in processes as they
occur; define the operations and decision-making processes of the HHS
investment review board and other management entities, such as the
component agencies, involved in managing IT investments.
o Ensure that HHS's investment review board's membership includes
business representation of its component agencies as it begins to execute
its full range of responsibilities.
o Develop well-defined and disciplined written procedures that outline
the process for selecting new IT proposals, reselecting ongoing IT
investments, and integrating funding with the process of selecting an
investment.
o Establish a process for the investment board to regularly review and
track the performance of a defined set of component agency IT systems
against expectations, and take corrective actions when these expectations
are not being met; and establish a mechanism for maintaining visibility
into other investments.
In Stage 3
o Develop and implement policies and procedures for modifying IT
portfolio selection criteria.
o Develop policies and procedures for using the portfolio selection
criteria to create its portfolio.
o Develop, review, and modify criteria for assessing portfolio
performance at regular intervals to reflect current performance
expectations.
o Define and implement processes for carrying out PIRs for all IT
investments.
We also recommend that the HHS Secretary direct the CIO to ensure that the
plan draws together ongoing efforts and additional efforts that are needed
to address the weaknesses identified in this report. The plan should also
(1) specify measurable goals, objectives, and milestones; (2) specify
needed resources; (3) assign clear responsibility and accountability for
accomplishing tasks; and (4) be approved by senior management.
Finally, to improve the department oversight of its component agency
investment management process, we are recommending that the HHS Secretary
direct the HHS CIO to establish a mechanism for ensuring component
agencies define and implement investment management processes that are
aligned with those of the department.
Agency Comments The Department of Health and Human Services's Inspector
General provided written comments on a draft of this report (reprinted in
app. II). In these comments, HHS generally agreed with our findings and
recommendations and stated that the report represented a fair assessment
of the department's progress in IT investment management. The department
added that it will leverage the report in its efforts to improve its
investment management processes.
HHS expressed differing perspectives on the inclusion of component agency
business representation on the investment review board and the performance
of postimplementation reviews. Specifically, regarding business
representation on the board, the department commented that it used a
hierarchy of investment reviews (with the first review occurring at the
component agency) combined with ITIRB members representing mission support
areas, such as Finance, Acquisition, and Human Resources, to provide a
structure for making the business decisions regarding the department's
investments. We disagree with the department that this arrangement
provides an adequate structure for managing the department's investments.
Because allocating resources among major IT investments may require
fundamental trade-offs among a multitude of business objectives, portfolio
management decisions are essentially business decisions, and therefore
require sufficient business representation on the board. CIOs and
executives responsible for missionsupport functions do not constitute
sufficient business representation because, by virtue of their
responsibilities, they are not in the best position to make business
decisions. Portfolio management decisions are better made by executives
with business line decision-making authority.
Regarding PIRs, HHS commented that it was currently informally performing
them by conducting closeout reviews of recently implemented investments
and annual reviews of systems in operations and maintenance. PIRs are
conducted to determine whether cost, benefit, schedule, and risk
expectations that were set for investments were achieved and develop
lessons learned about the investment selection and control processes that
can be used as the basis for management improvements. However, neither the
closeout reviews, nor the reviews of systems in operations and
maintenance, are addressing all these elements. Specifically, as we stated
in our report, the closeout reviews do not address schedule deviations,
determine whether the benefits were achieved, or identify lessons learned.
In addition, the reviews of projects in operations and maintenance do not
capture the benefits realized or identify lessons learned.
Commenting on departmental-level oversight of component agency
investments, HHS stated that it agrees with our recommendation to improve
its oversight of component agency investments. It stated that it would use
a number of mechanisms to do this, including performing audits to ensure
alignment of component agency's processes with those of the department,
using earned value management data to identify potential performance
problems with most investments, and directly reviewing investments
determined to be of high priority. We agree with HHS that these steps
would help address some of the weaknesses in project oversight that we
identify in this report.
As agreed with your office, unless you publicly announce its contents
earlier, we plan no further distribution of this report until 30 days from
the date of this report. At that time, we will send copies to other
interested congressional committees, the Secretary of Health and Human
Services, and other interested parties. We will also make copies available
to others upon request. In addition, the report will be available at no
charge on the GAO Web site at http://www.gao.gov.
Should you or your offices have questions on matters discussed in this
report, please contact me at (202) 512-9286 or [email protected]. Contact
points for our Offices of Congressional Relations and Public Affairs may
be found on the last page of this report. GAO staff who made major
contributions to this report are listed in appendix III.
Sincerely yours,
David A. Powner Director, Information Technology Management Issues
Appendix I
Objectives, Scope, and Methodology
The objectives of our review were to (1) assess the Department of Health
and Human Services's capabilities for managing its IT investments and (2)
determine any plans HHS might have for improving those capabilities.
To address our first objective, we reviewed the results of the
department's self-assessment of Stages 2 and 3 practices using our ITIM
framework and validated and updated the results of the self-assessment
through document reviews and interviews with officials. We reviewed
written policies, procedures, and guidance and other documentation
providing evidence of executed practices, including HHS's Capital Planning
and Investment Control Policy and Guidelines, standard operating
procedures, portfolio management tool training manuals, and various
instructional memorandums. We also reviewed the HHS ITIRB meeting
materials, including quarterly status reports, meeting minutes, and
records of decisions. We did not assess progress in establishing the
capabilities found in Stages 4 and 5 because the department acknowledged
that it had not executed any of the key practices in higher maturity
stages. In addition, we conducted interviews with officials from the
Office of the CIO, whose main responsibility is to oversee and ensure that
HHS's IT investment management process is implemented and followed to
determine the level of oversight and guidance the department is providing
to its component agencies. We also interviewed the Centers for Medicare &
Medicaid's Director for Investment Tracking and Assessment to determine
the level of investment management guidance and oversight that is provided
by the department.
As part of our analysis, we selected two HHS enterprisewide and two
component agency IT projects as case studies to verify that the critical
processes and key practices were being applied. The projects selected (1)
are recognized as major systems, (2) were in different life cycle phases,
(3) represent a mix of headquarters and component agency investments, (4)
support different functional areas, and (5) required different levels of
funding. The four projects are described below:
o HHS Public Key Infrastructure-This project supports digital signatures
and other public key-enabled security services; it is intended to be the
underlying architecture to support secure transmissions of electronic
communication, such as encrypted email, by linking a digital key to a
specific person, and issues and manages digital certificates. The intent
of the project is to provide an identity proofing process that is both
fast and certificate authority neutral. It is an agencywide strategic
initiative that provides security services. The project is a major
enterprisewide
Appendix I
Objectives, Scope, and Methodology
investment and is in the operations and maintenance phase. The project has
a planned completion date of July 2011 and is estimated to spend $7.7
million for fiscal year 2006.
o HHS Enterprise Architecture Initiative-This initiative is to provide
the overall framework for planning and managing the technology-supported
information assets of HHS and give the department the ability to identify
data and process redundancies and inefficiencies in its information
systems. The program's objectives focus on development of operational
policies and support that enable identification, analysis and ongoing
management of the business, and information and related technology
architectures. It is to provide leadership, direction, and support to
HHS's component agencies in planning and implementing information systems
to support required business processes. As of fiscal year 2005, the
initiative is a major enterprisewide program investment and is estimated
to spend $15.0 million for fiscal year 2006.
o National Institutes of Health's Electronic Research Administration-
This initiative is the National Institutes of Health's infrastructure for
conducting interactive electronic transactions for the receipt, review,
monitoring, and administration of grant awards to biomedical investigators
worldwide. It is to provide the technology capabilities for the agency to
efficiently and effectively perform grants administration functions. The
system is to provide end-to-end support of the grants administration
process, including receipt of applications, review and selection of
grantees, financial and progress reporting, issuance of final reports and
grant dole-out, invention reporting, and interface with accounting
systems. It is a major component agency investment and is expected to have
a useful life of 13 years. The project is estimated to spend $42.1 million
for fiscal year 2006.
o Food and Drug Administration's Mission Accomplishment and Regulatory
Compliance Services-This program is a comprehensive redesign and
reengineering of core mission-critical systems at the agency, including
the Field Accomplishments and Compliance Tracking System and the Operation
and Administration Support System. The first of these systems is to
support the investigation, tracking of compliance, and laboratory
operations related to domestic operations under the agency's purview; the
second is to primarily support the review and decision-making process of
products imported into the United States. Both are legacy systems that
execute on client-server platforms; while currently viable, the current
systems cannot address many of the
Appendix I
Objectives, Scope, and Methodology
business needs due to the exponential growth in functionality on a rigid
platform that was not designed to support the extent of change that has
been required. The Mission Accomplishment and Regulatory Compliance
Services is a major component agency investment and is expected to move to
production in September 2007 and have a useful life of 10 years. The
project is estimated to spend $10.2 million for fiscal year 2006.
For these projects, we reviewed project management documentation, such as
business cases, status reports, and meeting minutes. We also interviewed
officials from the Office of the CIO for the two component agency
investments and the project managers for the two HHS enterprisewide
projects.
We compared the evidence collected from our document reviews and
interviews to the key practices in ITIM. We rated the key practices as
"executed" on the basis of whether the agency demonstrated (by providing
evidence of performance) that it had met the criteria of the key practice.
A key practice was rated as "not executed" when we found insufficient
evidence of a practice during the review or when we determined that there
were significant weaknesses in HHS's execution of the key practice. In
addition, HHS was provided the opportunity to produce evidence for key
practices rated as "not executed."
To address our second objective, we obtained and evaluated documents
showing what management actions had been taken and what initiatives had
been planned by the agency. This documentation included the Policy
Advisory Board charter, draft investment management policies and
procedures, as well as procedures and guidance for control and evaluate
functionalities within HHS's portfolio management tool. We also
interviewed officials from the Office of the CIO to determine efforts
undertaken to improve IT investment management processes.
We conducted our work at HHS headquarters in Washington, D.C., from
January through September 2005, in accordance with generally accepted
government auditing standards.
Appendix II
Comments from the Department of Health and Human Services
Appendix II
Comments from the Department of Health
and Human Services
Appendix II
Comments from the Department of Health
and Human Services
Appendix II
Comments from the Department of Health
and Human Services
Appendix III
GAO Contact and Staff Acknowledgments
GAO Contact David A. Powner (202) 512-9286, [email protected]
Staff In addition to the person named above, Neil Doherty, Joanne
Fiorino, Sabine Paul, Nik Rapelje, Niti Tandon, and Amos Tevelow made key
Acknowledgments contributions to this report.
GAO's Mission The Government Accountability Office, the audit, evaluation
and investigative arm of Congress, exists to support Congress in meeting
its constitutional responsibilities and to help improve the performance
and accountability of the federal government for the American people. GAO
examines the use of public funds; evaluates federal programs and policies;
and provides analyses, recommendations, and other assistance to help
Congress make informed oversight, policy, and funding decisions. GAO's
commitment to good government is reflected in its core values of
accountability, integrity, and reliability.
Obtaining Copies of The fastest and easiest way to obtain copies of GAO
documents at no cost
is through GAO's Web site (www.gao.gov). Each weekday, GAO postsGAO
Reports and newly released reports, testimony, and correspondence on its
Web site. To Testimony have GAO e-mail you a list of newly posted products
every afternoon, go to
www.gao.gov and select "Subscribe to Updates."
Order by Mail or Phone The first copy of each printed report is free.
Additional copies are $2 each. A check or money order should be made out
to the Superintendent of Documents. GAO also accepts VISA and Mastercard.
Orders for 100 or more copies mailed to a single address are discounted 25
percent. Orders should be sent to:
U.S. Government Accountability Office 441 G Street NW, Room LM Washington,
D.C. 20548
To order by Phone: Voice: (202) 512-6000 TDD: (202) 512-2537 Fax: (202)
512-6061
To Report Fraud, Contact:
Waste, and Abuse in Web site: www.gao.gov/fraudnet/fraudnet.htm
E-mail: [email protected] Programs Automated answering system: (800)
424-5454 or (202) 512-7470
Congressional Gloria Jarmon, Managing Director, [email protected] (202)
512-4400 U.S. Government Accountability Office, 441 G Street NW, Room 7125
Relations Washington, D.C. 20548
Public Affairs Paul Anderson, Managing Director, [email protected] (202)
512-4800 U.S. Government Accountability Office, 441 G Street NW, Room 7149
Washington, D.C. 20548
*** End of document. ***