Business Systems Modernization: Department of the Navy Needs to  
Establish Management Structure and Fully Define Policies and	 
Procedures for Institutionally Managing Investments (31-OCT-07,  
GAO-08-53).							 
                                                                 
In 1995, GAO first designated the Department of Defense's (DOD)  
business systems modernization program as "high-risk," and	 
continues to do so today. In 2004, Congress passed legislation	 
reflecting prior GAO recommendations that DOD adopt a corporate  
approach to information technology (IT) business systems	 
investment management, including tiered accountability for	 
business systems at the department and component levels. To	 
support GAO's legislative mandate to review DOD's efforts, GAO	 
assessed whether the investment management approach of one of	 
DOD's components--the Department of the Navy--is consistent with 
leading investment management best practices. In doing so, GAO	 
applied its IT Investment Management (ITIM) framework and	 
associated methodology, focusing on the stages related to the	 
investment management provisions of the Clinger-Cohen Act of	 
1996.								 
-------------------------Indexing Terms------------------------- 
REPORTNUM:   GAO-08-53						        
    ACCNO:   A77869						        
  TITLE:     Business Systems Modernization: Department of the Navy   
Needs to Establish Management Structure and Fully Define Policies
and Procedures for Institutionally Managing Investments 	 
     DATE:   10/31/2007 
  SUBJECT:   Accountability					 
	     Best practices					 
	     Data collection					 
	     Information management				 
	     Information systems investments			 
	     Information technology				 
	     Investment planning				 
	     Investment Review Board				 
	     IT investment management				 
	     Policies and procedures				 
	     GAO High Risk Series				 
	     GAO Information Technology Investment		 
	     Management Framework				 
                                                                 

******************************************************************
** This file contains an ASCII representation of the text of a  **
** GAO Product.                                                 **
**                                                              **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced.  Tables are included, but    **
** may not resemble those in the printed version.               **
**                                                              **
** Please see the PDF (Portable Document Format) file, when     **
** available, for a complete electronic file of the printed     **
** document's contents.                                         **
**                                                              **
******************************************************************
GAO-08-53

   

This is a work of the U.S. government and is not subject to copyright 
protection in the United States. It may be reproduced and distributed 
in its entirety without further permission from GAO. Because this work 
may contain copyrighted images or other material, permission from the 
copyright holder may be necessary if you wish to reproduce this 
material separately.  

United States Government Accountability Office: 
GAO:  

Report to Congressional Committees:  

October 2007:  

Business Systems Modernization: 

Department of the Navy Needs to Establish Management Structure and 
Fully Define Policies and Procedures for Institutionally Managing 
Investments:  

GAO-08-53:  

GAO Highlights:  

Highlights of GAO-08-53, a report to congressional committees.  

Why GAO Did This Study:  

In 1995, GAO first designated the Department of Defenseï¿½s (DOD) 
business systems modernization program as ï¿½high-risk,ï¿½ and continues to 
do so today. In 2004, Congress passed legislation reflecting prior GAO 
recommendations that DOD adopt a corporate approach to information 
technology (IT) business systems investment management, including 
tiered accountability for business systems at the department and 
component levels. To support GAOï¿½s legislative mandate to review DODï¿½s 
efforts, GAO assessed whether the investment management approach of one 
of DODï¿½s componentsï¿½the Department of the Navyï¿½is consistent with 
leading investment management best practices. In doing so, GAO applied 
its IT Investment Management (ITIM) framework and associated 
methodology, focusing on the stages related to the investment 
management provisions of the Clinger-Cohen Act of 1996.  

What GAO Found:  

The Department of the Navy has yet to establish the management 
structures needed to effectively manage its business systems 
investments or to fully develop many of the related policies and 
procedures outlined in GAOï¿½s ITIM framework (see table below). The 
department has implemented two of the nine key practices that call for 
project-level management structures, policies, and procedures, and none 
of the five practices that call for portfolio-level policies and 
procedures. Specifically, it has developed procedures for identifying 
and collecting information about its business systems to support 
investment selection and control, and assigned responsibility for 
ensuring that the information collected during project identification 
meets the needs of the investment management process. However, the 
department has not established the management structures needed to 
support effective investment oversight. It also has not fully 
documented business system investment policies and procedures for 
directing Investment Review Board operations, selecting new 
investments, reselecting ongoing investments, integrating the 
investment funding and investment selection processes, and developing 
and maintaining complete business system investment portfolio(s).  

Department officials stated that they are aware of the lack of an 
Investment Review Board and the absence of documented policies and 
procedures in certain areas of project and portfolio-level management, 
and are currently working on new guidance to address these areas. 
According to these officials, the new policies and procedures are 
expected to be approved by March 2008. However, until the department 
assigns responsibility for overseeing project-level management and 
portfolio management to a departmentwide review board and fully defines 
policies and procedures for both individual projects and portfolios of 
projects, it risks selecting and controlling these business system 
investments in a way that is inconsistent, incomplete, and ad hoc, 
which in turn reduces the chances that these investments will meet 
mission needs in the most effective manner.  

Status of the Departmentï¿½s Project- and Portfolio-Level Management 
Capabilities:  

Stage 2: Building the investment foundation: Instituting the investment 
board; 
Key practices executed: 0/2.  

Stage 2: Building the investment foundation: Meeting business needs; 
Key practices executed: 0/1.  

Stage 2: Building the investment foundation: Selecting an investment; 
Key practices executed: 0/3.  

Stage 2: Building the investment foundation: Providing investment 
oversight; 
Key practices executed: 0/1.  

Stage 2: Building the investment foundation: Capturing investment 
information; 
Key practices executed: 2/2.  

Stage 2: Building the investment foundation: Overall; 
Key practices executed: 2/9.  

Stage 3: Developing a complete investment portfolio: Defining the 
portfolio criteria; 
Key practices executed: 0/2.  

Stage 3: Developing a complete investment portfolio: Creating the 
portfolio; 
Key practices executed: 0/1.  

Stage 3: Developing a complete investment portfolio: Evaluating the 
portfolio; 
Key practices executed: 0/1.  

Stage 3: Developing a complete investment portfolio: Conducting post 
implementation reviews; 
Key practices executed: 0/1.  

Stage 3: Developing a complete investment portfolio: Overall;
Key practices executed: 0/5. 

Source: GAO.  

What GAO Recommends:  

GAO recommends that the Department of the Navy establish the management 
structures and fully define project and portfolio management policies 
and procedures discussed in GAOï¿½s ITIM framework. In comments on a 
draft of this report, DOD stated that the Department of the Navy was 
developing policies that should address the investment and portfolio 
management deficiencies GAO identified.  

To view the full product, including the scope and methodology, click on 
[hyperlink, http://www.GAO-08-53]. For more information, contact 
Valerie Melvin at (202) 512-6304 or [email protected]. 

[End of section]  

Contents:  

Letter:  

Results in Brief:  

Background:  

Department of the Navy Has Not Yet Established the Management 
Structures Needed to Effectively Manage Business System Investments and 
Has Not Fully Defined Many of the Related Policies and Procedures:  

Conclusions:  

Recommendations for Executive Action:  

Agency Comments and Our Evaluation:  

Appendix I: Objective, Scope, and Methodology:  

Appendix II: Comments from the Department of Defense:  

Appendix III: GAO Contact and Staff Acknowledgments:  

Tables:  

Table 1: DOD and Department of the Navy Business System Investment 
Tiers:  

Table 2: Department of the Navy Investment Management Governance 
Entities and Responsibilities:  

Table 3: Stage 2 Critical Processesï¿½Building the Investment 
Foundation:  

Table 4: Summary of Policies and Procedures for Stage 2 Critical 
Processesï¿½Building the Investment Foundation:  

Table 5: Stage 3 Critical Processesï¿½Developing a Complete Investment 
Portfolio:  

Table 6: Summary of Policies and Procedures for Stage 3 Critical 
Processesï¿½Developing a Complete Investment Portfolio:  

Figures:  

Figure 1: Simplified DOD Organizational Structure:  

Figure 2: Department of the Navy CIO Organizational Structure:  

Figure 3: The Five ITIM Stages of Maturity with Critical Processes:  

Figure 4: Working Relationships among DOD Business Investment 
Management System Governance Entities:  

Figure 5: Department of the Navy Precertification Review and Approval 
Process:  

Abbreviations:  

CIO: chief information officer:  

DAS: Defense Acquisition System:  

DOD: Department of Defense:  

IT: information technology:  

ITIM: Information Technology Investment Management:  

JCIDS: Joint Capabilities Integration and Development System:  

OMB: Office of Management and Budget:  

PPBE: Planning, Programming, Budgeting, and Execution:  

[End of section]  

United States Government Accountability Office: 
Washington, DC 20548:  

October 31, 2007:  

Congressional Committees:  

For decades, the Department of Defense (DOD) has been challenged in 
modernizing its timeworn business systems. [Footnote 1] In 1995, we 
designated DODï¿½s business systems modernization program as high risk, 
and we continue to designate it as such today.[Footnote 2] Our research 
on public and private sector organizations shows that an essential 
ingredient to a successful systems modernization program is having an 
effective institutional approach to managing information technology 
(IT) investments.  

In May 2001, we recommended that DOD establish a corporate approach to 
investment control and decision making.[Footnote 3] Between 2001 and 
2005, we reported that DODï¿½s business systems modernization program was 
still not being effectively managed,[Footnote 4] and we made additional 
investment-related recommendations. Congress subsequently included 
provisions in the Ronald W. Reagan National Defense Authorization Act 
for Fiscal Year 2005[Footnote 5] that reflected our recommendations, 
including those for establishing and implementing effective business 
system investment management structures and processes.  

Between 2005 and 2007,[Footnote 6] we reported that DOD had made 
important progress in establishing and implementing these structures 
and processes; however, much remained to be accomplished. Most 
recently,[Footnote 7] we reported that, according to DOD officials, 
investment management practices are performed at the component level, 
and policies and procedures established for overseeing componentsï¿½ 
execution of these practices are sufficient. However, DOD had not fully 
defined many of the related policies and procedures outlined in GAOï¿½s 
IT Investment Management framework.  

The Fiscal Year 2005 National Defense Authorization Act directs DOD to, 
among other things, establish and implement effective IT business 
system investment management structures and processes. As agreed with 
your offices and to support the legislative mandate that GAO assess 
DODï¿½s actions to comply with this requirement, the objective of our 
review was to determine whether the investment management approach of 
the Department of the Navy is consistent with leading investment 
management best practices. To accomplish our objective, we analyzed 
documents and interviewed agency officials to determine whether the 
department has developed the structures, policies, and procedures 
associated with executing those key practices in our IT Investment 
Management (ITIM) framework[Footnote 8] that assist departments and 
agencies in complying with the investment management provisions of the 
Clinger-Cohen Act of 1996.[Footnote 9] 

We performed our work at Department of the Navy offices in Arlington, 
Virginia, from February 2007 through September 2007 in accordance with 
generally accepted government auditing standards. Details on our 
objective, scope, and methodology are contained in appendix I.  

Results in Brief:  

The Department of the Navy has not yet established the management 
structures needed to effectively manage its business system investments 
nor has it fully developed many of the related policies and procedures 
that our ITIM framework outlines. The department has implemented two of 
the nine key practices that call for project-level management 
structures, policies, and procedures, and none of the five practices 
that call for portfolio-level policies and procedures. Specifically, 
regarding project-level investments, the department has (1) developed 
procedures for identifying and collecting information about its 
business systems to support investment selection and control and (2) 
assigned responsibility for ensuring that the information collected 
during project identification meets the needs of the investment 
management process.  

However, the department has not established the necessary management 
structures needed to support effective investment oversight and has not 
fully developed business system investment policies and procedures 
related to seven key project-level management practices. For example, 
it has not created an Investment Review Board, composed of senior 
executives from across the agency, to govern business system 
investments. In addition, policies and procedures do not (1) fully 
explain the departmentï¿½s IT investment management process (by which it 
selects, controls, and evaluates IT investments); (2) define how 
ongoing IT investments are periodically reviewed and verified with 
respect to the departmentï¿½s business needs; (3) specify how the full 
range of cost, schedule, and performance data accessible to the 
department is to be used in making selection decisions; (4) specify 
processes for identifying, evaluating, and prioritizing reselection of 
ongoing IT investments; (5) describe how funding decisions are 
integrated with the process of selecting an investment; and (6) specify 
the processes for decision making during project oversight and describe 
a process for how corrective actions should be taken when the project 
deviates or varies from the project management plan. Further, regarding 
portfolio management, the department does not have documented policies 
and procedures for (1) defining the portfolio criteria, (2) creating 
the portfolio, (3) evaluating the portfolio, and (4) conducting post-
implementation reviews of business systems. In addition, the department 
has not assigned responsibility for managing the portfolio criteria. As 
discussed in our ITIM guidance, adequately documenting both the 
policies and associated procedures that govern how an organization 
manages its IT projects and investment portfolios is important because 
doing so provides the basis for having rigor, discipline, and 
repeatability in how investments are selected and controlled across the 
entire organization.  

Department officials stated that they are aware of the lack of an 
Investment Review Board and the absence of documented policies and 
procedures in certain areas of project-level and portfolio-level 
management; officials also stated that they are currently working on 
guidance to address these weaknesses. For example, these officials 
stated that they are drafting new portfolio-level policies and 
procedures and are developing guidance that is intended to assign IT 
management roles and responsibilities to new or existing boards. The 
new policies, procedures, and guidance are expected to be approved by 
March 2008. Until the department assigns responsibility for overseeing 
project-level management and portfolio-level management to a 
departmentwide review board and fully defines policies and procedures 
for both individual projects and portfolios of projects, it risks not 
being able to select and control these business system investments in a 
way that is consistent and complete, which in turn reduces the chances 
that these investments will meet mission needs in the most effective 
manner.  

To strengthen its business system management capability, we are 
recommending that the Department of the Navy establish a departmentwide 
Investment Review Board and fully define the policies and procedures 
associated with project-level and portfolio-level investment management 
as discussed in our guidance for IT investment management.[Footnote 
10]  

In written comments on a draft of this report, signed by the Deputy 
Under Secretary of Defense (Business Transformation) and reprinted in 
appendix II, the department partially concurred with the reportï¿½s 
recommendations. It stated that the Department of the Navy was 
developing policies that should address the investment and portfolio 
management deficiencies we identified. However, DOD also stated that, 
based on the Department of the Navyï¿½s pending instruction, it is the 
departmentï¿½s position that a Secretary of Defense directive on the 
matter will not be required. Our recommendations did not state that DOD 
should develop a directive; rather, we emphasized the need for the 
Department of the Navy to develop policies and procedures.  

Background:  

DOD is a massive and complex organization. To illustrate, it reported 
that its fiscal year 2006 operations involved approximately $1.4 
trillion in assets and $2.0 trillion in liabilities, more than 2.9 
million military and civilian personnel, and $581 billion in net cost 
of operations. Organizationally, DOD includes the Office of the 
Secretary of Defense, the Chairman of the Joint Chiefs of Staff, the 
military departments, numerous defense agencies and field activities, 
and various unified combatant commands that are responsible for either 
specific geographic regions or specific functions. Figure 1 provides a 
simplified depiction of DODï¿½s organizational structure.  

Figure 1: Simplified DOD Organizational Structure:  

[See PDF for image]  

This figure is an organizational chart, depicting the following 
hierarchy: 

Secretary of Defense/Deputy Secretary of Defense:
* Department of the Army;
* Department of the Navy; 
* Department of the Air Force; 
* Office of the Secretary of Defense:
- DOD Field Activities; 
- Defense Agencies;
* Inspector General; 
* Joint Chiefs of Staff:
- Combatant Commands[a]; 
* Combatant Commands[a].  

Source: GAO, based on DOD documentation.  

[a] The Chairman of the Joint Chiefs of Staff serves as the spokesman 
for the commanders of the combatant commands, particularly for the 
administrative requirements of their commands.  

[End of figure]  

In support of its military operations, DOD performs an assortment of 
interrelated and interdependent business functions, including logistics 
management, procurement, health care management, and financial 
management. As we have previously reported,[Footnote 11] the systems 
environment that supports these business functions is overly complex 
and error prone, and is characterized by (1) little standardization 
across DOD, (2) multiple systems performing the same tasks, (3) the 
same data stored in multiple systems, and (4) the need for data to be 
entered manually into multiple systems.  

Department of the Navyï¿½s Mission, Organizational Structure, and Use of 
IT:  

The Department of the Navy is a major component of DOD, consisting of 
two uniformed services: the Navy and the Marine Corps. The departmentï¿½s 
mission is to maintain, train, and equip combat-ready naval forces 
capable of winning wars, deterring aggression, and maintaining freedom 
of the seas. To support this mission, the department performs a variety 
of interrelated and interdependent business functions, such as 
logistics and financial management, relying extensively on IT to carry 
out its operations. In fiscal year 2006, the departmentï¿½s budget for IT 
was $4.3 billion, of which $3.9 billion (90.3 percent) was allocated to 
operations and maintenance of existing systems and $424 million (9.7 
percent) was allocated to systems in development and modernization. The 
department was appropriated about $4.2 billion in fiscal year 2007 and 
requested about $4 billion in fiscal year 2008 to operate, maintain, 
and modernize business systems and associated infrastructures.  

The Chief Information Officer (CIO) for the department is accountable 
for all IT business system investments for both the Navy and Marine 
Corps. The CIOï¿½s office is organized to align and integrate information 
management and IT programs across the two services and focus 
departmentwide efforts in support of warfighter priorities. The CIO is 
supported by Deputy CIOs for the Navy and Marine Corps and a Deputy CIO 
for Policy and Integration, who directs the operations of the CIO 
functional teams. The functional teams are led by team leaders who are 
subject matter experts in their areas of responsibility and are 
responsible for implementing the goals and objectives outlined in the 
departmentï¿½s information management and IT strategic plan, which 
includes, among other things, ensuring that investments are effectively 
selected, resourced, and acquired. Figure 2 outlines the department CIO 
organizational structure.  

Figure 2: Department of the Navy CIO Organizational Structure:  

[See PDF for image]  

This figure is an organizational chart, depicting the following 
hierarchy:  

Chief Information Officer (CIO):
* Director of Operations (reports to CIO); 
* Deputy CIO for Policy and Integration (reports to CIO); 
- Investment Management (reports to Deputy CIO for Policy and 
Integration);
- Knowledge Management (reports to Deputy CIO for Policy and 
Integration); 
- Performance Leadership and Management (reports to Deputy CIO for 
Policy and Integration); 
- Enterprise IM/IT Planning (reports to Deputy CIO for Policy and 
Integration); 
- Enterprise Architecture Standards and Infrastructure (reports to 
Deputy CIO for Policy and Integration); 
- IM/IT Workforce Management (reports to Deputy CIO for Policy and 
Integration); 
- Spectrum/Telecom/Wireless Management (reports to Deputy CIO for 
Policy and Integration); 
- Enterprise Transformation (reports to Deputy CIO for Policy and 
Integration); 
- Mission Assurance (reports to Deputy CIO for Policy and Integration); 
- Critical Infrastructure Protection (reports to Mission Assurance); 
- Information Assurance/Identity Management/Privacy (reports to Mission 
Assurance); 
* Deputy CIO (Navy) (communicates with CIO); 
* Deputy CIO (Marine Corps) (communicates with CIO);  

Source: GAO based on Department of Navy documentation.  

[End of figure]  

IT Investment Management Is Critical to Achieving Successful Systems 
Modernization:  

A corporate approach to IT investment management is characteristic of 
successful public and private organizations. Recognizing this, Congress 
enacted the Clinger-Cohen Act of 1996, [Footnote 12] which requires the 
Office of Management and Budget (OMB) to establish processes to 
analyze, track, and evaluate the risks and results of major capital 
investments in IT systems made by executive agencies.[Footnote 13] In 
response to the Clinger-Cohen Act and other statutes, OMB has developed 
policy and issued guidance for the planning, budgeting, acquisition, 
and management of federal capital assets. [Footnote 14] We have also 
issued guidance in this area[Footnote 15] that defines institutional 
structures, such as Investment Review Boards; processes for developing 
information on investments (such as costs and benefits); and practices 
to inform management decisions (such as whether a given investment is 
aligned with an enterprise architecture). 

IT Investment Management: A Brief Description:  

IT investment management is a process for linking IT investment 
decisions to an organizationï¿½s strategic objectives and business plans. 
Consistent with this, the federal approach to IT investment management 
focuses on selecting, controlling, and evaluating investments in a 
manner that minimizes risks while maximizing the return on 
investment.[Footnote 16]  

* During the selection phase, the organization (1) identifies and 
analyzes each projectï¿½s risks and returns before committing significant 
funds to any project and (2) selects those IT projects that will best 
support its mission needs.  

* During the control phase, the organization ensures that projects, as 
they develop and investment expenditures continue, meet mission needs 
at the expected levels of cost and risk. If the project is not meeting 
expectations or if problems arise, steps are quickly taken to address 
the deficiencies.  

* During the evaluation phase, expected results are compared with 
actual results after a project has been fully implemented. This 
comparison is done to (1) assess the projectï¿½s impact on mission 
performance, (2) identify any changes or modifications to the project 
that may be needed, and (3) revise the investment management process 
based on lessons learned.  

Overview of GAOï¿½s ITIM Maturity Framework:  

Our ITIM framework consists of five progressive stages of maturity for 
any given agency relative to selecting, controlling, and evaluating its 
investment management capabilities.[Footnote 17] (See fig. 3 for the 
five ITIM stages of maturity.) This framework is grounded in our 
research of IT investment management practices of leading private and 
public sector organizations. The framework can be used to assess the 
maturity of an agencyï¿½s investment management processes and as a tool 
for organizational improvement. The overriding purpose of the framework 
is to encourage investment processes that increase business value and 
mission performance, reduce risk, and increase accountability and 
transparency in the decision process. We have used the framework in 
several of our evaluations,[Footnote 18] and a number of agencies have 
adopted it.  

ITIMï¿½s five maturity stages represent steps toward achieving stable and 
mature processes for managing IT investments. Each stage builds on the 
lower stages; the successful attainment of each stage leads to 
improvement in the organizationï¿½s ability to manage its investments. 
With the exception of the first stage, each maturity stage is composed 
of ï¿½critical processesï¿½ that must be implemented and institutionalized 
in order for the organization to achieve that stage. These critical 
processes are further broken down into key practices that describe the 
types of activities that an organization should be performing to 
successfully implement each critical process. It is not unusual for an 
organization to be performing key practices from more than one maturity 
stage at the same time. However, our research has shown that agency 
efforts to improve investment management capabilities should focus on 
implementing all lower stage practices before addressing the higher 
stage practices.  

Figure 3: The Five ITIM Stages of Maturity with Critical Processes:  

[See PDF for image]  

This figure illustrates the five ITIM Stages of Maturity with critical 
processes. The following data is depicted: 

Maturity Stage:
Stage 1: Creating Investment awareness; 
Critical processes: 
* IT spending without disciplined investment processes. 

Maturity Stage: 
Stage 2: Building the investment foundation;
Critical processes: 
* Instituting the investment board; 
* Meeting business needs; 
* Selecting investment; 
* Providing investment oversight; 
* Capturing investment information.  

Maturity Stage:
Stage 3: Developing a complete investment portfolio; 
Critical processes: 
* Defining the portfolio criteria; 
* Creating the portfolio; 
* Evaluating the portfolio; 
* Conducting postimplementation reviews. 

Maturity Stage:
Stage 4: Improving the investment process; 
Critical processes: 
* Improving the portfolio's performance; 
* Managing the succession of information systems.  

Maturity Stage:
Stage 5: Leveraging IT for strategic outcomes; 
Critical processes: 
* Optimizing the investment process; 
* Using IT to drive strategic business change.  

Source: GAO. 

[End of figure]  

In the ITIM framework, Stage 2 critical processes lay the foundation 
for sound IT investment management by helping the agency to attain 
successful, predictable, and repeatable investment management processes 
at the project level. Specifically, Stage 2 encompasses building a 
sound investment management foundation by establishing basic 
capabilities for selecting new IT projects. This stage also involves 
developing the capability to control projects so that they finish 
predictably within established cost and schedule expectations and 
developing the capability to identify potential exposures to risk and 
put in place strategies to mitigate that risk. Further, it involves 
evaluating completed projects to ensure they meet business needs and 
collecting lessons learned to improve the IT investment management 
process. The basic management processes established in Stage 2 lay the 
foundation for more mature management capabilities in Stage 3, which 
represents a major step forward in maturity, in which the agency moves 
from project-centric processes to a portfolio approach, evaluating 
potential investments by how well they support the agencyï¿½s missions, 
strategies, and goals.  

Stage 3 requires that an organization continually assess both proposed 
and ongoing projects as parts of a complete investment portfolioï¿½an 
integrated and competing set of investment options. It focuses on 
establishing a consistent, well-defined perspective on the IT 
investment portfolio and maintaining mature, integrated selection (and 
reselection), control, and post-implementation evaluation processes. 
This portfolio perspective allows decision makers to consider the 
interaction among investments and the contributions to organizational 
mission goals and strategies that could be made by alternative 
portfolio selections, rather than focusing exclusively on the balance 
between the costs and benefits of individual investments. Organizations 
that have implemented Stages 2 and 3 practices have capabilities in 
place that assist in establishing selection; control; and evaluation 
structures, policies, procedures, and practices that are required by 
the investment management provisions of the Clinger-Cohen Act.[Footnote 
19]  

Stages 4 and 5 require the use of evaluation techniques to continuously 
improve both the investment portfolio and the investment processes in 
order to better achieve strategic outcomes. At Stage 4, an organization 
has the capacity to conduct IT succession activities and, therefore, 
can plan and implement the deselection of obsolete, high-risk, or low-
value IT investments. An organization with Stage 5 maturity conducts 
proactive monitoring for breakthrough information technologies that 
will enable it to change and improve its business performance. 

DOD and Department of the Navy Approach for Identifying, Funding, and 
Acquiring System Investments:  

DODï¿½s major system investments (i.e., weapons and business systems) are 
governed by three management systems that focus on defining needs, 
budgeting for, and acquiring investments to support the missionï¿½the 
Joint Capabilities Integration and Development System (JCIDS); the 
Planning, Programming, Budgeting, and Execution (PPBE) system; and the 
Defense Acquisition System (DAS). In addition, DODï¿½s business systems 
are subject to a fourth management system, which, for purposes of this 
report, we refer to as the Business Investment Management System. For 
each of these systems, DOD relies on its components to execute the 
underlying policies and procedures. According to DOD, the four 
management systems, collectively, are the means by which DODï¿½and its 
componentsï¿½selects, controls, and evaluates its business systems 
investments.  

Joint Capabilities Integration and Development System:  

JCIDS is a needs-driven, capabilities-based approach to identify 
mission needs and meet future joint forces challenges. It is intended 
to identify future capabilities for DOD; address capability gaps and 
mission needs recognized by the Joint Chiefs of Staff or derived from 
strategic guidance, such as the National Security Strategy Report 
[Footnote 20] or Quadrennial Defense Review;[Footnote 21] and identify 
alternative solutions by considering a range of doctrine, organization, 
training, materiel, leadership and education, personnel, and facilities 
solutions. According to DOD, the Joint Chiefs of Staffï¿½through the 
Joint Requirements Oversight Councilï¿½has primary responsibility for 
defining and implementing JCIDS. All JCIDS documents are submitted to 
the Joint Chiefs of Staff, which determines whether the proposed system 
has joint implications or is component-unique. If it is designated as 
joint interest, then the Joint Requirements Oversight Council is 
responsible for approving and validating the documents. If it is not 
designated as having joint interests, the sponsoring component is 
responsible for validation and approval.  

Planning, Programming, Budgeting, and Execution System:  

PPBE is a calendar-driven approach that is composed of four phases that 
occur over a moving 2-year cycle. The four phasesï¿½planning, 
programming, budgeting, and executingï¿½define how budgets for each 
component and DOD as a whole are created, vetted, and executed. As 
recently reported,[Footnote 22] the components start programming and 
budgeting for addressing a JCIDS-identified capability gap or mission 
need several years before actual product development begins and before 
the Office of the Secretary of Defense formally reviews the componentsï¿½ 
programming and budgeting proposals (i.e., Program Objective 
Memorandums). Once reviewed and approved, the financial details in the 
Program Objective Memorandums become part of the Presidentï¿½s budget 
request to Congress. During budget execution, components may submit 
program change proposals or budget change proposals, or both (e.g., 
program cost increases or schedule delays). According to DOD, the Under 
Secretary of Defense (Policy), the Director for Program Analysis and 
Evaluation,[Footnote 23] and the Under Secretary of Defense 
(Comptroller) have primary responsibility for defining and implementing 
the PPBE system.  

Defense Acquisition System:  

DAS[Footnote 24] is a framework-based approach that is intended to 
translate mission needs and requirements into stable, affordable, and 
well-managed acquisition programs, and it consists of five key program 
life-cycle phases. These five phases are as follows:  

Concept Refinement: Intended to refine the initial JCIDS-validated 
system solution (concept) and create a strategy for acquiring the 
investment solution. A decision is made at the end of this phase 
(Milestone A decision) regarding whether to move to the next phase 
(Technology Development).  

Technology Development: Intended to determine the appropriate set of 
technologies to be integrated into the investment solution by 
iteratively assessing the viability of various technologies while 
simultaneously refining user requirements. Once the technology has been 
demonstrated in a relevant environment, a decision is made (Milestone B 
decision) regarding whether to move to the next phase (System 
Development and Demonstration).  

System Development and Demonstration: Intended to develop a system or a 
system increment and demonstrate through developer testing that the 
system or system increment can function in its target environment. A 
decision is made at the end of this phase (Milestone C decision) 
regarding whether to move to the next phase (Production and 
Deployment).  

Production and Deployment: Intended to achieve an operational 
capability that satisfies the mission needs, as verified through 
independent operational test and evaluation, and ensures that the 
system is implemented at all applicable locations.  

Operations and Support: Intended to operationally sustain the system in 
the most effective manner over its life cycle. A key principle of DAS 
is that investments are assigned a category, where programs of 
increasing dollar value and management interest are subject to more 
stringent oversight. For example, Major Defense Acquisition Programs 
[Footnote 25] and Major Automated Information Systems[Footnote 26] are 
large, expensive programs subject to the most extensive statutory and 
regulatory reporting requirements and, unless delegated, are reviewed 
by acquisition boards at the DOD level. Smaller and less risky 
acquisitions are generally reviewed at the component executive or lower 
levels. Another key principle is that DAS requires acquisition 
management under the direction of a milestone decision authority. 
[Footnote 27] The Milestone Decision Authorityï¿½with support from the 
Program Manager and advisory boards, such as the Defense Acquisition 
Board[Footnote 28] and the IT Acquisition Board[Footnote 29]ï¿½determines 
the projectï¿½s baseline cost, schedule, and performance commitments. The 
Under Secretary of Defense for Acquisition, Technology, and Logistics 
has primary responsibility for defining and implementing DAS.  

DOD relies on its components to execute these investment management 
policies and procedures. To implement DODï¿½s JCIDS process, the 
Department of the Navy has developed service-level processesï¿½the Naval 
Capabilities Development Process and the Marine Corps Expeditionary 
Force Development Systemï¿½to support the requirements generation process 
of JCIDS. To implement the PPBE process, department officials stated 
that they use their budget guidance manual. Finally, to implement the 
DAS process, the department has developed guidance that outlines a 
systematic acquisition framework that mirrors the framework defined by 
DOD and includes the same three event-based milestones and associated 
five program life-cycle phases.  

Business Investment Management System:  

The Business Investment Management System is a calendar-driven approach 
that is described in terms of governance entities, tiered 
accountability, and certification reviews and approvals. This system 
was initiated in 2005, when DOD reassigned responsibility for providing 
executive leadership for the direction, oversight, and execution of its 
business systems modernization efforts to several entities. These 
entities and their responsibilities include the following:  

* The Defense Business Systems Management Committee serves as the 
highest-ranking governance body for business systems modernization 
activities.  

* The Principal Staff Assistants serve as the certification authorities 
for business system modernizations in their respective core business 
missions.  

* The Investment Review Boards are chartered by the principal staff 
assistants and are the review and decision-making bodies for business 
system investments in their respective areas of 
responsibility.[Footnote 30] The boards are also responsible for 
recommending certification for all business system investments costing 
more than $1 million.  

* The component precertification authority is accountable for the 
componentï¿½s business system investments and acts as the componentï¿½s 
principal point of contact for communication with the Investment Review 
Boards. The Department of the Navy has designated its CIO to be the 
Precertification Authority.  

* The Business Transformation Agency is responsible for leading and 
coordinating business transformation efforts across DOD. The agency is 
organized into seven directorates, one of which is the Defense Business 
Systems Acquisition Executiveï¿½the component acquisition executive for 
DOD-wide business systems and initiatives. This directorate is 
responsible for developing, coordinating, and integrating enterprise-
level projects, programs, systems, and initiativesï¿½including managing 
resources such as fiscal, personnel, and contracts for assigned systems 
and programs. Figure 4 provides a simplified illustration of the 
relationships among these entities. 

Figure 4: Working Relationships among DOD Business Investment 
Management System Governance Entities:  

[See PDF for images} 

This figure illustrates the working relationships among DOD Business 
Investment Management System Governance Entities. There are five tiers 
depicted as follows: 

First Tier: Defense Business Systems Management Committee:  

Second Tier: Principal Staff Assistant Certification Authorities: 
* Under Secretary of Defense (Comptroller); 
* Under Secretary of Defense (Acquisition, Technology,and Logistics); 
* Under Secretary of Defense (Personnel and Readiness);  

Third Tier: Investment Review Boards: 
* Financial Management; 
* Weapon Systems Lifecycle Management and Materiel Supply and Services 
Management; 
* Real Property and Installations Lifecycle Management; 
* Human Resources Management;  

Fourth Tier: Business Transformation Agency;  

Fifth Tier: DOD Components. 

There is a direct working relationship between entities on each tier 
with the entities both above and below them. 

Source: GAO, based on DOD documentation.  

[End of figure]  

According to DOD, in 2005 it also adopted a tiered accountability 
approach to business transformation. Under this approach, 
responsibility and accountability for business system investment 
management is allocated among DOD (i.e., Office of the Secretary of 
Defense) and the components, based on the amount of 
development/modernization funding involved and the investmentï¿½s ï¿½tier.ï¿½ 
DOD is responsible for ensuring that all business systems with a 
development/modernization investment in excess of $1 million are 
reviewed by the Investment Review Boards for compliance with the 
business enterprise architecture, certified by the principal staff 
assistants, and approved by the Defense Business Systems Management 
Committee. Components are responsible for certifying 
development/modernization investments with total costs of $1 million or 
less. All DOD development and modernization efforts are assigned a tier 
on the basis of the acquisition category or the size of the financial 
investment, or both. According to DOD, a system is given a tier 
designation when it passes through the certification process. Table 1 
describes the investment tiers and identifies the associated reviewing 
and approving entities for DOD and the Department of the Navy.  

Table 1: DOD and Department of the Navy Business System Investment 
Tiers:  

Tier: Tier 1; 
Description: Major Automated Information Systems and Major Defense 
Acquisition Programs; 
Reviewing/Approving entities: Certified by Investment Review Boards and 
Defense Business Systems Management Committee; precertified by 
Department of the Navy CIO.  

Tier: Tier 2; 
Description: Systems exceeding $10 million in total 
development/modernization costs, but not designated Major Automated 
Information Systems or Major Defense Acquisition Programs; 
Reviewing/Approving entities: Certified by Investment Review Boards and 
Defense Business Systems Management Committee; precertified by 
Department of the Navy CIO.  

Tier: Tier 3; 
Description: Systems exceeding $1 million and up to $10 million in 
total development/modernization costs; 
Reviewing/Approving entities: Certified by Investment Review Boards and 
Defense Business Systems Management Committee; precertified by 
Department of the Navy CIO.  

Tier: Tier 4; 
Description: All other business systems (i.e., those systems with 
development/modernization costs of $1 million or less); 
Reviewing/Approving entities: Certified by Department of the Navy CIO.  

Tier: Non-Tier; 
Description: Those systems that have no development or modernization 
costs that are in sustainment or steady state; 
Reviewing/Approving entities: Reviewed by Functional Area Managers and 
Department of the Navy Deputy CIOs for Navy and Marine Corps.  

Source: DOD and Department of the Navy.  

[End of table]  

DODï¿½s business investment management system includes two types of 
reviews for business systems: certification and annual reviews. 
Certification reviews apply to new modernization projects with total 
costs over $1 million. These reviews focus on program alignment with 
the business enterprise architecture and must be completed before 
components obligate funds for programs. The annual reviews apply to all 
business programs and are intended to determine whether the system 
development effort is meeting its milestones and addressing its 
Investment Review Board certification conditions.  

Certification reviews and approvals: Tier 1 through 3 business system 
investments in development and modernization are certified at two 
levelsï¿½components precertify and DOD certifies and approves these 
system investments. At the component level, program managers prepare, 
enter, maintain, and update information about their investments in 
their data repository, such as regulatory compliance reporting, an 
architectural profile, and requirements for investment certification 
and annual reviews. The component precertification authority validates 
that the system information is complete and accessible on the 
repository, reviews system compliance with the business enterprise 
architecture and enterprise transition plan, and verifies the economic 
viability analysis. This information is then transferred to DODï¿½s IT 
Portfolio Repository.[Footnote 31] The precertification authority 
asserts the status and validity of the investment information by 
submitting a component precertification letter to the appropriate 
Investment Review Board for its review.  

Annual reviews: Tier 1 through 4 business system investments are 
annually reviewed at the component and DOD-levels. At the component 
level, program managers annually review and update information on all 
tiers of system investments that are identified in their data 
repository. For Tier 1 through 3 systems that are in development or 
being modernized, information is updated on cost, milestones, and risk 
variances and actions or issues related to certification conditions. 
The precertification authority then verifies and submits the 
information for these business system investments for the DOD 
Investment Review Boardï¿½s review in an annual review assertion letter. 
The letter addresses system compliance with the DOD business enterprise 
architecture and the enterprise transition plan and includes investment 
cost, schedule, and performance information.[Footnote 32]  

At the DOD level, the Investment Review Boards annually review 
investments for certified Tier 1 through 3 business systems that are in 
development or modernization. These reviews focus on program compliance 
with the business enterprise architecture, program cost and performance 
milestones, and progress in meeting certification conditions. The 
Investment Review Boards can revoke an investmentï¿½s certification when 
the system has significantly failed to achieve performance commitments 
(i.e., capabilities and costs). When this occurs, the component must 
address the Investment Review Boardï¿½s concerns and resubmit the 
investment for certification.  

Department of the Navy Precertification Process:  

As stated earlier, DOD relies on its components to execute investment 
management policies and procedures. The Department of the Navy has 
developed a precertification process for its business systems, which is 
intended to ensure that new or existing systems that are being 
modernized undergo proper scrutiny prior to being precertified by the 
departmentï¿½s Precertification Authority. The precertification process 
is initiated by the Program Manager, who is responsible for completing 
all data elements required for a specific tier, including entering data 
and attachments into the departmentï¿½s repository and entering funding 
information into the DOD budgeting database.  

After the precertification package has been completed by the Program 
Manager, it is to be reviewed by both Functional Area Managers and the 
Deputy CIOs for the Navy and Marine Corps. The Functional Area 
Managersï¿½ primary responsibilities are to functionally review data for 
each defense business system for which they are the lead or stakeholder 
and ensure that IT and business processes are aligned. The primary 
responsibilities of the Deputy CIOs are to technically review each 
defense business system within their service and verify that the 
systemï¿½s architecture complies with the departmentï¿½s enterprise 
architecture and the DOD business enterprise architecture. The final 
task of the Deputy CIO and the Functional Area Managers is to provide a 
recommendation to the department Precertification Authority as to 
whether or not the business system should be certified. The reviews of 
the Deputy CIOs and Functional Area Managers may occur concurrently.  

Following the Functional Area Manager and Deputy CIO reviews, a 
business system is to be sent to the departmentï¿½s CIO for final 
approval. The CIO is responsible for reviewing Tier 1 through 4 
submissions, precertifying Tier 1 through 3 defense business system 
investments, and certifying Tier 4 investments. The CIO is also 
responsible for monitoring the activities of the Functional Area 
Managers and the Deputy CIOs, and for ensuring that functional area 
manager coordination is effective and sufficient for identifying 
redundant investments. Once a Tier 1 through 3 investment has been 
precertified, the CIO is to complete, among other things, a 
precertification letter and send the certification package to DOD for 
review by the applicable DOD Investment Review Board and Defense 
Business Systems Management Committee.  

Table 2 lists decision-making personnel involved in the departmentï¿½s 
investment management process and provides a description of their key 
responsibilities.  

Table 2: Department of the Navy Investment Management Governance 
Entities and Responsibilities:  

Entity: Precertification Authority; 
Roles and responsibilities: 
* Precertify all Tier 1-3 systems and submit certification packages to 
DOD Investment Review Board; 
* Certify all Tier 4 systems; 
Composition: Department of the Navy Chief Information Officer.  

Entity: Department of the Navy Deputy CIOï¿½Navy and Marine Corps; 
Roles and responsibilities: 
* Technically review certification packages; 
* Verify compliance with department and business enterprise 
architecture; 
* Endorse system information; 
* Recommend to the department CIO whether to approve system; 
Composition: Department of the Navy Deputy CIO for Navy; Department of 
the Navy Deputy CIO for Marine Corps.  

Entity: Functional Area Managers; 
Roles and responsibilities: 
* Functionally review certification packages; 
* Ensure IT/business process alignment; 
* Validate system information; 
* Recommend to the department CIO whether to approve system; 
Composition: Comprised of 32 Functional Area Managers: 16 Navy, 12 
Marine Corps, and 4 Secretariat-level. Functional Area Managers are 
divided into the five core business mission areas.[a] 

Entity: Program Manager; 
Roles and responsibilities: 
* Prepare certification packages for their systems; 
* Enter and maintain system information in departmentï¿½s repository; 
Composition: System owner/manager.  

Source: GAO analysis of Department of the Navy data.  

[a] DOD has five core business mission areas: human resources 
management, financial management, materiel supply and services 
management, weapon system life-cycle management, and real property and 
installations life-cycle management.  

[End of table]  

Figure 5 shows a simplified overview of the process flow of 
precertification reviews and approvals for the Department of the Navy.  

Figure 5: Department of the Navy Precertification Review and Approval 
Process: 

{See PDF for image]  

This figure depicts the Department of the Navy precertification review 
and approval process, as follows:  

Program Manager: Enters and maintains Business System Investments 
information in the department repository for Tier 1-4, completes 
certification package requirements for Tier 1-3. Submits to: 

Functional Area Manager: Functionally reviews and validates program 
information and certification packages for Tier 1-4. Makes 
recommendations to: 

Department of the Navy Deputy CIOs (Navy & Marine Corps): Technically 
reviews and endorses program information and certification packages for 
Tier 1-4. Makes recommendations to:  

Department of the Navy CIO/Precertification Authority: Precertifies 
Tier 1-3 and approves Tier 4, loads certification packages to DOD 
repository for Tier 1-3. Submits precertified Tier 1-3 to:  

DOD: Reviews Tier 1-3 certification packages.  

Source: GAO, based on Department of the Navy documentation.  

[End of figure]  

Department of the Navy Has Not Yet Established the Management 
Structures Needed to Effectively Manage Business System Investments and 
Has Not Fully Defined Many of the Related Policies and Procedures:  

Although DOD relies on its components to execute investment management 
policies and procedures,[Footnote 33] the Department of the Navy has 
not yet established the management structures needed to effectively 
manage its business system investments or fully developed many of the 
related policies and procedures outlined in our ITIM framework. 
Relative to its business system investments, the department has 
implemented two of the nine key practices that call for project-level 
management structures, policies, and procedures and none of the five 
key practices that call for portfolio-level policies and procedures. 
Department officials stated that they are currently working on guidance 
to address these weaknesses. For example, the officials stated that 
they are drafting new portfolio-level policies and procedures and are 
developing guidance that is intended to assign IT management roles and 
responsibilities to new or existing boards. The new policies and 
procedures and guidance are expected to be approved by March 2008. 
According to our ITIM framework, adequately documenting both the 
policies and the associated procedures that govern how an organization 
manages IT projects and investment portfolios is important because 
doing so provides the basis for having rigor, discipline, and 
repeatability in how investments are selected and controlled across the 
entire organization.  

Until the department establishes the necessary management structure and 
fully defines policies and procedures for both individual projects and 
the portfolios of projects, it risks not being able to select and 
control these business system investments in a consistent and complete 
manner, which in turn reduces the chances that these investments will 
meet mission needs in the most effective manner.  

Department of the Navy Has Yet to Build a Foundation for Project-Level 
Investment Management:  

At ITIM Stage 2, an organization has attained a repeatable and 
successful IT project-level investment control process and basic 
selection processes. Through these processes, the organization can 
identify project expectation gaps early and take the appropriate steps 
to address them. ITIM Stage 2 critical processes include (1) defining 
investment board operations, (2) identifying the business needs for 
each investment, (3) developing a basic process for selecting new 
proposals and reselecting ongoing investments, (4) developing project-
level investment control processes, and (5) collecting information 
about existing investments to inform investment management decisions.  

Table 3 describes the purpose of each of these Stage 2 critical 
processes.  

Table 3: Stage 2 Critical Processesï¿½Building the Investment 
Foundation:  

Critical process: Instituting the investment board; 
Purpose: To define and establish an appropriate IT investment 
management structure and the processes for selecting, controlling, and 
evaluating IT investments. 

Critical process: Meeting business needs; 
Purpose: To ensure that IT projects and systems support the 
organizationï¿½s business needs and meet user needs.  

Critical process: Selecting an investment; 
Purpose: To ensure that a well-defined and disciplined process is used 
to select new IT proposals and reselect ongoing investments.  

Critical process: Providing investment oversight; 
Purpose: To review the progress of IT projects and systems, using 
predefined criteria and checkpoints, in meeting cost, schedule, risk, 
and benefit expectations and to take corrective action when these 
expectations are not being met.  

Critical process: Capturing investment information; 
Purpose: To make information available to decision makers to evaluate 
the impacts and opportunities created by proposed (or continuing) IT 
investments.  

Source: GAO.  

[End of table]  

Within these five critical processes are nine key practices that call 
for policies and procedures associated with effective project-level 
management. The department has fully defined the policies and 
procedures for two of these nine processes. Specifically, it has 
policies and procedures for capturing investment information by 
submitting, updating, and maintaining investment information in its 
repository and loading information to the DOD repository. Further, the 
department has assigned its CIO the responsibility of ensuring that 
information contained in its repository is accurate and complete.  

However, the management structures and policies and procedures 
associated with the remaining seven project-level management practices 
are missing critical elements needed to effectively carry out essential 
investment management activities. For example:  

* The department has not yet established an Investment Review Board, 
composed of senior executives from its IT and business units, to define 
and implement the organizationï¿½s IT investment governance process. 
Without an Investment Review Board, the departmentï¿½s ability to ensure 
that investment decisions are consistent and reflect the needs of the 
organization is limited.  

* The department does not have a documented IT investment management 
process that completely explains the agencyï¿½s selection, control, and 
evaluation of IT investments. Without such an investment management 
process, the department may not make consistent decisions regarding its 
IT investments.  

* The departmentï¿½s policies and procedures do not explain how ongoing 
IT investments are periodically reviewed and verified relative to 
meeting the business needs of its organization and users. Without 
documenting how officials are to ensure that IT business system 
investments maintain alignment with the organizationï¿½s strategic plans 
and business goals and objectives, the department cannot ensure a 
consistent selection of investments that best meet its needs and 
priorities.  

* The departmentï¿½s procedures for selecting new investments do not 
specify how the full range of cost, schedule, and benefit data are used 
by department officials (CIO, Deputy CIOs, and Functional Area 
Managers) in making selection decisions. Without documenting how these 
officials are to consider factors such as cost, schedule, and benefits 
when making selection decisions, the department cannot ensure that it 
can consistently and objectively select system investments to best meet 
its needs and priorities.  

* Policies and procedures do not specify how reselection decisions 
(i.e., annual review decisions) consider investments that are in 
operations and maintenance. Without policies and procedures, its 
ability to make informed and consistent reselection and termination 
decisions is limited.  

* Policies and procedures do not specify how funding decisions are 
integrated into the process of selecting an investment. Without 
considering its budget constraints and opportunities, the department 
risks making investment decisions that do not effectively consider the 
relative merits of various projects and systems when funding 
limitations exist.  

* Policies and procedures for providing oversight into the departmentï¿½s 
investment management activities do not specify the processes for 
decision making during project oversight and do not describe how 
corrective actions should be taken when the project deviates or varies 
from the project management plan. Without such policies and procedures, 
the department risks investing in systems that are duplicative, 
stovepiped, nonintegrated, and unnecessarily costly to manage, 
maintain, and operate.  

Table 4 summarizes our findings relative to the departmentï¿½s execution 
of the nine key practices for policies and procedures needed to manage 
IT investments at the project level.  

Table 4: Summary of Policies and Procedures for Stage 2 Critical 
Processesï¿½Building the Investment Foundation:  

Critical process: Instituting the investment board; 
Key practice: 1. An enterprisewide IT investment board composed of 
senior executives from IT and business units is responsible for 
defining and implementing the organizationï¿½s IT investment governance 
process. 
Rating: Not executed. 
Summary of evidence: The department has not yet established an IT 
investment board composed of senior executives from across the 
department that has responsibility for defining and implementing its IT 
investment governance process. Department officials stated that they 
are currently developing guidance that is intended to assign IT 
management roles and responsibilities to new or existing boards. This 
new guidance is expected to be completed by March 2008. 

Critical process: Instituting the investment board; 
Key practice: 2. The organization has a documented IT investment 
process directing each investment boardï¿½s operations. 
Rating: Not executed. 
Summary of evidence: Although the department has developed certain 
guidance that describes its precertification of defense business 
systems and the specific roles and responsibilities of individuals 
involved in the review of these business systems, the department does 
not have a documented IT investment management process that fully 
explains its selection, control, and evaluation of IT investments. 
Also, the department has yet to establish an investment board that 
oversees its IT investment management process. According to department 
officials, it is currently developing new guidance that is intended to 
explain how JCIDS, PPBE, and DAS are used to select, control, and 
evaluate IT investments; they expect this new guidance to be completed 
by March 2008.  

Critical process: Meeting business needs;
Key practice: 1. The organization has documented policies and 
procedures for identifying IT projects or systems that support the 
organizationï¿½s ongoing and future business needs. 
Rating: Not executed.
Summary of evidence: The department has defined a process intended to 
ensure that proposed IT business system investments support its ongoing 
and future business needs by requiring Tier 1 through 4 systems going 
through the precertification process to comply with the departmentï¿½s 
enterprise architecture and the DOD business enterprise architecture. 
Although department officials stated that Functional Area Managers and 
Deputy CIOs conduct annual reviews of ongoing IT investments, this 
process is not currently documented. According to officials, the 
department intends to revise the Precertification Workflow Guidance to 
include the annual review of investments in operations and maintenance 
by March 2008.  

Critical process: Selecting an investment; 
Key practice: 1. The organization has documented policies and 
procedures for selecting new IT proposals. 
Rating: Not executed.
Summary of evidence: The department has not defined a structured method 
for identifying, evaluating, prioritizing, and selecting new business 
system investments that addresses all needed aspects of selecting such 
systems. According to department officials, selection of new business 
system investments occurs in the JCIDS, PPBE, and DAS processes. 
However, the departmentï¿½s processes do not specify how cost, schedule, 
and benefit data are to be used in making selection decisions.  

Critical process: Selecting an investment; 
Key practice: 2. The organization has documented policies and 
procedures for reselecting ongoing IT investments. 
Rating: Not executed.
Summary of evidence: The department does not have documented policies 
and procedures for reselecting ongoing IT investments that specify 
processes for identifying, evaluating, and prioritizing these 
investments. According to department officials, the Precertification 
Workflow Guidance will be revised to include the annual review of IT 
investments in operations and maintenance by March 2008.  

Critical process: Selecting an investment; 
Key practice: 3. The organization has documented policies and 
procedures for integrating funding with the process of selecting an 
investment. 
Rating: Not executed.
Summary of evidence: The department does not have policies and 
procedures for integrating funding with the process of selecting an 
investment. Specifically, it does not specify how funding decisions are 
integrated with the process of selecting an investment and does not 
specify how officials use this information in carrying out decisions on 
system certification and approvals.  

Critical process: Providing investment oversight; 
Key practice: 1. The organization has documented policies and 
procedures for management oversight of IT projects and systems. 
Rating: Not executed.
Summary of evidence: The department does not have well-defined policies 
and procedures for overseeing the management of IT projects and 
systems. For example, although it has assigned roles and 
responsibilities for overseeing business system investments and states 
that its management oversight is accomplished through the acquisition 
process, the department has not specified the processes for decision 
making during project oversight and has not described how corrective 
actions should be taken when the project deviates or varies from the 
project management plan.  

Critical process: Capturing investment information; 
Key practice: 1. The organization has documented policies and 
procedures for identifying and collecting information about IT projects 
and systems to support the investment management process. 
Rating: Executed; 
Summary of evidence:  

Critical process: Capturing investment information; 
Key practice: 
Rating: Executed; 
Summary of evidence: The department has assigned responsibility to the 
CIO for ensuring that the information collected during project and 
systems identification meets the needs of the investment management 
process. Specifically, the CIO is responsible for ensuring that 
investment information contained in the department repository and the 
DOD repository is accurate and complete.  

Source: GAO.  

[End of table]  

According to department officials, they are aware of the absence of 
documented policies and procedures in certain areas of project-level 
management, and plan to issue new policies and procedures addressing 
these areas by March 2008. However, until the department has documented 
IT investment management policies and procedures that include fully 
defined Stage 2 activities, specify the linkages between the various 
related processes, and describe how investments are to be governed in 
the operations and maintenance phase, it risks not being able to carry 
out investment management activities in a consistent and disciplined 
manner. Moreover, the department risks selecting investments that will 
not effectively meet its mission needs.  

Department of the Navy Has Not Yet Defined the Policies and Procedures 
Associated with Effective Portfolio-Level Management:  

At Stage 3, an organization has defined the critical processes for 
managing its investment as a portfolio or set of portfolios.[Footnote 
34] Portfolio management is a conscious, continuous, and proactive 
approach to allocating limited resources among competing initiatives in 
light of the investmentsï¿½ relative benefits. Taking an agencywide 
perspective enables an organization to consider its investments 
comprehensively, so that collectively the investments optimally address 
the organizationï¿½s missions, strategic goals, and objectives. Managing 
IT investments as portfolios also allows an organization to determine 
its priorities and make decisions about which projects to fund based on 
analyses of the relative organizational value and risks of all 
projects, including projects that are proposed, under development, and 
in operation. Although investments may initially be organized into 
subordinate portfoliosï¿½based on, for example, business lines or life-
cycle stagesï¿½and managed by subordinate Investment Review Boards, they 
should ultimately be aggregated into enterprise-level portfolios.  

According to ITIM, Stage 3 involves four critical processes (1) 
defining the portfolio criteria; (2) creating the portfolio; (3) 
evaluating (i.e., overseeing) the portfolio; and (4) conducting post-
implementation reviews. Within these critical processes are five key 
practices that call for policies and procedures to ensure effective 
portfolio management. Table 5 summarizes the purpose of each of these 
critical processes.  

Table 5: Stage 3 Critical Processesï¿½Developing a Complete Investment 
Portfolio:  

Critical process: Defining the portfolio criteria; 
Purpose: To ensure that the organization develops and maintains IT 
portfolio selection criteria that support its mission, organizational 
strategies, and business priorities.  

Critical process: Creating the portfolio; 
Purpose: To ensure that IT investments are analyzed according to the 
organizationï¿½s portfolio selection criteria and to ensure that an 
optimal IT investment portfolio with manageable risks and returns is 
selected and funded.  

Critical process: Evaluating the portfolio; 
Purpose: To review the performance of the organizationï¿½s investment 
portfolios at agreed-upon intervals and to adjust the allocation of 
resources among investments as necessary.  

Critical process: Conducting post-implementation reviews. 
Purpose: To compare the results of recently implemented investments 
with the expectations that were set for them and to develop a set of 
lessons learned from these reviews.  

Source: GAO.  

[End of table]  

The department has not fully defined the policies and procedures needed 
to effectively execute the five portfolio management practices. 
Specifically, it does not have policies and procedures for defining the 
portfolio criteria or assigning responsibility for managing the 
portfolio criteria. In addition, the department does not have policies 
and procedures for creating and evaluating the portfolio. Further, it 
does not have component-level policies and procedures for conducting 
post-implementation reviews.  

Table 6 summarizes the rating for each critical process required to 
manage IT investments as a portfolio and summarizes the evidence that 
supports these ratings.  

Table 6: Summary of Policies and Procedures for Stage 3 Critical 
Processesï¿½Developing a Complete Investment Portfolio:  

Critical process: Defining the portfolio criteria; 
Key practice: 1. The organization has documented policies and 
procedures for creating and modifying IT portfolio selection criteria. 
Rating: Not executed. 
Summary of evidence: While the department is currently developing new 
guidance for IT portfolio management, it has not completed and issued 
policies and procedures for creating and modifying the portfolio 
selection criteria.  

Critical process: Defining the portfolio criteria; 
Key practice: 2. Responsibility is assigned to an individual or group 
for managing the development and modification of the IT portfolio 
selection criteria. 
Rating: Not executed. 
Summary of evidence: While the department is currently developing new 
guidance for IT portfolio management, which is intended to assign 
responsibility to an individual or group for managing the development 
and modification of portfolio selection criteria, the guidance has not 
been finalized and approved. According to department officials, the 
guidance is expected to be completed by March 2008.  

Critical process: Creating the portfolio; 
Key practice: 1. The organization has documented policies and 
procedures for analyzing, selecting, and maintaining the investment 
portfolios. 
Rating: Not executed. 
Summary of evidence: While the department is currently developing new 
guidance for IT portfolio management, which is intended to include a 
description of its analysis, selection, control, and evaluation 
processes, the guidance has not been finalized and approved. According 
to department officials, the guidance is expected to be completed by 
March 2008.  

Critical process: Evaluating the portfolio; 
Key practice: 1. The organization has documented policies and 
procedures for reviewing, evaluating, and improving the performance of 
its portfolios. 
Rating: Not executed. 
Summary of evidence: While the department is currently developing new 
guidance for IT portfolio management, it does not have documented 
policies and procedures for reviewing, evaluating, and improving the 
performance of its portfolios. According to department officials, the 
guidance is expected to be completed by March 2008.  

Critical process: Conducting post-implementation reviews; 
Key practice: 1. The organization has documented policies and 
procedures for conducting post-implementation reviews. 
Rating: Not executed. 
Summary of evidence: While DOD and the department require post-
implementation reviews for Tier 1 systems as part of DAS, there are no 
documented policies or procedures for conducting such reviews for 
systems in the remaining tiers.  

Source: GAO.  

[End of table]  

Department officials agreed that portfolio management is primarily a 
component responsibility and are aware that they are required to 
develop and implement a portfolio management capability. Currently, 
they are developing policy and associated procedures that are intended 
to address these areas and plan to complete them by March 2008. In the 
absence of policies and procedures for managing business system 
investment portfolios, the department is at risk of not consistently 
selecting the mix of investments that best supports the mission needs 
and not being able to ensure that investment-related lessons learned 
are shared and applied departmentwide.  

Conclusions:  

Given the importance of business systems modernization to the 
Department of the Navyï¿½s mission, performance, and outcomes, it is 
vital for the department to adopt and employ an effective institutional 
approach to managing business system investments. However, although 
department officials acknowledged shortcomings and the importance of 
addressing them, the department has not yet established the management 
structures needed to effectively manage its business system 
investments. The department is also missing other important elements, 
such as specific policies and procedures that are needed for project-
level and portfolio-level investment management. In the absence of 
these essential elements, the department lacks an institutional 
capability to ensure that it is investing in business systems that best 
support its strategic needs and that ongoing projects meet cost, 
schedule, and performance expectations. Until the department develops 
this capability, it will be impaired in its ability to optimize 
business mission area performance and accountability.  

Recommendations for Executive Action:  

To strengthen the Department of the Navyï¿½s business system investment 
management capability and address the weaknesses discussed in this 
report, we recommend that the Secretary of Defense direct the Secretary 
of the Navy to ensure that well-defined and disciplined business system 
investment management policies and procedures are developed and issued. 
At a minimum, this should include instituting project-and portfolio-
level policies and procedures that address seven key practices:  

* Establishing an enterprisewide IT Investment Review Board composed of 
senior executives from IT and business units, including assigning the 
investment board responsibility, authority, and accountability for 
programs throughout the investment life cycle.  

* Documenting an investment management process that includes how it is 
coordinated with JCIDS, PPBE, DAS, and the precertification process.  

* Ensuring that systems in operations and maintenance are aligned with 
ongoing and future business needs.  

* Selecting new investments, including specifying how cost, schedule, 
and benefit data are to be used in making decisions and specifying the 
criteria and steps for prioritizing and selecting these investments.  

* Documenting an annual review process that includes the reselection of 
ongoing IT investments.  

* Integrating funding with the process of selecting an investment, 
including specifying how department officials are using funding 
information in carrying out decisions.  

* Overseeing IT projects and systems, including specifying the 
processes for the investment boardsï¿½ operations and decision making 
during project oversight.  

These well-defined and disciplined business system investment 
management policies and procedures should also include portfolio-level 
management policies and procedures that address the following five 
areas:  

* Creating and modifying IT portfolio selection criteria for business 
system investments.  

* Defining the roles and responsibilities for managing the development 
and modification of the IT portfolio selection criteria.  

* Analyzing, selecting, and maintaining business system investment 
portfolios.  

* Reviewing, evaluating, and improving the performance of its 
portfolios by using project indicators, such as cost, schedule, and 
risk.  

* Conducting post-implementation reviews for all investment tiers and 
specifying how conclusions, lessons learned, and recommended management 
actions are to be shared with executives and others.  

Agency Comments and Our Evaluation:  

In written comments on a draft of this report, signed by the Deputy 
Under Secretary of Defense (Business Transformation) and reprinted in 
appendix II, DOD partially concurred with our recommendations. It 
stated that the Department of the Navy has drafted Instruction 8115.02, 
Information Technology Portfolio Management Implementation, which when 
finalized, will address our recommendations. According to DOD, the 
instruction is scheduled to be signed in March 2008. DOD added that it 
would provide assistance, where appropriate, to the Navy to ensure 
alignment with enterprise-level portfolio management policies and 
procedures as they are matured. However, DOD also stated that, based on 
this pending document from the Department of the Navy, it is the 
departmentï¿½s position that a Secretary of Defense directive on the 
matter will not be required. Our recommendations did not state that DOD 
should develop a directive; rather, we emphasized the need for the 
Department of the Navy to develop policies and procedures.  

We are sending copies of this report to interested congressional 
committees; the Director, Office of Management and Budget; the 
Secretary of Defense; the Deputy Secretary of Defense; the Secretary of 
the Navy; the Department of the Navy Chief Information Officer; the 
Commandant of Marine Corps; and the Under Secretary of Defense for 
Acquisition, Technology, and Logistics. Copies of this report will be 
made available to other interested parties on request. This report will 
also be made available at no charge on our Web site at [hyperlink, 
http://www.gao.gov].  

Should you or your staffs have any questions on matters discussed in 
this report, please contact me at (202) 512-6304 or [email protected]. 
Contact points for our Offices of Congressional Relations and Public 
Affairs may be found on the last page of this report. GAO staff who 
made major contributions to this report are listed in appendix III.  

Signed by:  

Valerie C. Melvin: 
Director: 
Human Capital and Management Information Systems Issues:  

List of Committees:  

The Honorable Carl Levin: 
Chairman: 
The Honorable John McCain: 
Ranking Member: 
Committee on Armed Services: 
United States Senate:  

The Honorable Daniel Inouye: 
Chairman: 
The Honorable Ted Stevens: 
Ranking Member: 
Subcommittee on Defense: 
Committee on Appropriations: 
United States Senate:  

The Honorable Ike Skelton: 
Chairman: 
The Honorable Duncan Hunter: 
Ranking Member: 
Committee on Armed Services: 
House of Representatives:  

The Honorable John P. Murtha: 
Chairman: 
The Honorable C.W. Bill Young: 
Ranking Member: 
Subcommittee on Defense: 
Committee on Appropriations: 
House of Representatives:  

[End of section]  

Appendix I: Objective, Scope, and Methodology:  

Our objective was to determine whether the investment management 
approach of the Department of the Navy (a major Department of Defense 
(DOD) component) is consistent with leading investment management best 
practices. Our analysis was based on the best practices contained in 
GAOï¿½s Information Technology Investment Management (ITIM) framework and 
the frameworkï¿½s associated evaluation methodology, and focused on the 
departmentï¿½s establishment of policies and procedures for business 
system investments needed to assist organizations in complying with the 
Clinger-Cohen Act of 1996 (Stages 2 and 3).  

To address our objective, we asked the department to complete a self-
assessment of its investment management process and provide the 
supporting documentation. We then reviewed the results of the 
departmentï¿½s self-assessment of Stages 2 and 3 organizational 
commitment practicesï¿½those practices related to structures, policies, 
and proceduresï¿½and compared them against our ITIM framework. We focused 
on Stages 2 and 3 because these stages represent the processes needed 
to meet the standards of the Clinger-Cohen Act, and they establish the 
foundation for effective acquisition management. We also validated and 
updated the results of the self-assessment through document reviews and 
interviews with officials, such as the Director of the Investment 
Management Team and other staff in the department Chief Information 
Officerï¿½s office. In doing so, we reviewed written policies, 
procedures, and guidance and other documentation providing evidence of 
executed practices, including the Department of the Navyï¿½s Business 
Information Technology System Precertification Workflow Guidance, 
Secretary of Navy Instruction 5000.2C, and the Budget Guidance Manual.  

We compared the evidence collected from our document reviews and 
interviews with the key practices in ITIM. We rated the key practices 
as ï¿½executedï¿½ on the basis of whether the agency demonstrated (by 
providing evidence of performance) that it had met all of the criteria 
of the key practice. A key practice was rated as ï¿½not executedï¿½ when we 
did not find sufficient evidence of all elements of a practice being 
fully performed or when we determined that there were significant 
weaknesses in the departmentï¿½s execution of the key practice. In 
addition, we provided the agency the opportunity to produce evidence 
for the key practices rated as ï¿½not executed.ï¿½  

We conducted our work at Department of the Navy offices in Arlington, 
Virginia, from February 2007 through September 2007 in accordance with 
generally accepted government auditing standards.  

Appendix II: Comments from the Department of Defense:  

Office Of The Under Secretary Of Defense: 
Acquisition, Technology And Logistics: 
3000 Defense Pentagon: 
Washington, DC 20301-3000:  

October 18, 2007:  

Ms. Valerie C. Melvin: 
Director, Human Capital and Management Information Systems Issues: 
U.S. Government Accountability Office: 
441 G Street, N.W.: 
Washington, DC 20548:  

Dear Ms. Melvin:  

This is the Department of Defense (DOD) response to the GAO Draft 
Report, GAO-08-53, "Business Systems Modernization: Department 
of the Navy Needs to Establish Management Structure and Fully Define 
Policies and Procedures for Institutionally Managing Investments," 
dated September 17, 2007 (GAO Code 310638).  

The Department partially concurs with the GAO's recommendations. The 
Department of the Navy has been proactively seeking opportunities to 
improve upon its existing investment management processes for its 
business systems, as evidenced by its decision in 2006 to draft the 
Secretary of the Navy Instruction 8115, 02, Information Technology 
Portfolio Management Implementation. The instruction is scheduled to be 
signed by March 15, 2008 and when finalized, it will address the GAO's 
recommendations. In accordance with the Department's system of multi-
layered accountability, it is DOD's position that a Secretary of 
Defense directive, in addition to the Secretary of the Navy's pending 
document, is not required. However, where appropriate, DoD will provide 
assistance and support to the Navy to ensure alignment with enterprise-
level portfolio management policies and procedures as they are matured. 

DoD appreciates GAO's recommendations, and strongly values our 
relationship. Information technology investment management continues to 
be a top priority throughout the entire DoD, and we remain committed to 
establishing the appropriate management structures and project and 
portfolio-level processes and procedures that will provide leadership 
the ability to make sound investment decisions. As the Department 
continues to move forward, we welcome the GAO's insight and 
participation in our on-going business transformation efforts. 

Signed by:  

Paul A. Brinkley: 
Deputy Under Secretary of Defense: 
(Business Transformation):  

[End of letter]  

GAO Draft Report Dated September 17, 2007: 
GAO-08-53 (GAO CODE 310638):  

"Business Systems Modernization: Department Of The Navy Needs To 
Establish Management Structure And Fully Define Policies And Procedures 
For Institutionally Managing Investments"  

Department Of Defense Comments To The GAO Recommendation:  

Recommendation 1: The GAO recommended that the Secretary of Defense 
direct the Secretary of the Navy to ensure that well-defined and 
disciplined business system investment management policies and 
procedures are developed and issued. At a minimum, these should include 
instituting project-and portfolio-level policies and 
procedures that address: 

* Establishing an enterprisewide Information Technology (IT) Investment 
Review Board composed of senior executives from IT and business units, 
including assigning the investment board responsibility, authority, and 
accountability for programs throughout the investment life cycle.  

* Documenting an investment management process that includes how it is 
coordinated with Joint Capabilities Integration and Development System, 
Planning, Programming, Budgeting and Execution, Defense Acquisition 
System, and the pre-certification process.  

* Ensuring that systems in operations and maintenance are aligned with 
ongoing and future business needs.  

* Selecting new investments, including specifying how cost, schedule, 
and benefit data are to be used in making decisions and specifying the 
criteria and steps for prioritizing and selecting these investments.  

* Documenting an annual review process that includes the reselection of 
ongoing IT investments.  

* Integrating funding with the process of selecting an investment, 
including specifying how department officials are using funding 
information in carrying out decisions.  

* Overseeing IT projects and systems, including specifying the 
processes for investment boards' operations and decision making during 
project oversight.  

(p. 36/GAO Draft Report): 

Recommendation 2: The GAO recommended that the Secretary of Defense 
direct the Secretary of the Navy to ensure that the above well-defined 
and disciplined business system investment management policies and 
procedures also include portfolio-level management policies and 
procedures that address: 

* Creating and modifying IT portfolio selection criteria for business 
system investments.  

* Defining roles and responsibilities for managing the development and 
modification of the IT portfolio selection criteria.  

* Analyzing, selecting, and maintaining business system investment 
portfolios.  

* Reviewing, evaluating, and improving the performance of its 
portfolio(s) by using project indicators, such as cost, schedule, and 
risk.  

* Conduct post-implementation reviews for all investment tiers and 
specifying how conclusions, lessons learned, and recommended management 
actions are to be shared with executives and others.  

(p. 36/GAO Draft Report) 

DOD Response (Recommendations 1 And 2): Partially Concur. The 
Department of Navy (DON) has recognized the need for a single policy 
document or suite of documents to define its information technology 
portfolio management roles and responsibilities and information system 
investment practices. As such, DON initiated action in 2006 to draft 
the Secretary of the Navy Instruction 8115.02, Information Technology 
Portfolio Management Implementation. The draft instruction, now 
undergoing internal review and comment, should be signed by March 15, 
2008. Based on this pending document from the DON and under the tiered 
accountability concept, it is DoD's position that a Secretary of 
Defense directive on the matter will not be required.  

[End of section]  

Appendix III: GAO Contact and Staff Acknowledgments:  

GAO Contact:  

Valerie C. Melvin, (202) 512-6304 or [email protected]:  

Staff Acknowledgments:  

In addition to the contact person named above, key contributors to this 
report were Tonia Johnson, Assistant Director; Jacqueline Bauer; Elena 
Epps; Nancy Glover; and Jeanne Sung.  

[End of section]  

Footnotes:  

[1] Business systems are information systems that include financial and 
nonfinancial systems and support DODï¿½s business operations, such as 
civilian personnel, finance, health, logistics, military personnel, 
procurement, and transportation.  

[2] GAO, High-Risk Series: An Update, GAO-07-310 (Washington, D.C.: 
January 2007).  

[3] GAO, Information Technology: Architecture Needed to Guide 
Modernization of DODï¿½s Financial Operations, GAO-01-525 (Washington, 
D.C.: May 17, 2001).  

[4] See, for example, GAO, DOD Business Systems Modernization: Long-
standing Weaknesses in Enterprise Architecture Development Need to Be 
Addressed, GAO-05-702 (Washington, D.C.: July 22, 2005); DOD Business 
Systems Modernization: Billions Being Invested without Adequate 
Oversight, GAO-05-381 (Washington, D.C.: Apr. 29, 2005); DOD Business 
Systems Modernization: Limited Progress in Development of Business 
Enterprise Architecture and Oversight of Information Technology 
Investments, GAO-04-731R (Washington, D.C.: May 17, 2004); DOD Business 
Systems Modernization: Important Progress Made to Develop Business 
Enterprise Architecture, but Much Work Remains, GAO-03-1018 
(Washington, D.C.: Sept. 19, 2003); and GAO-01-525.  

[5] Ronald W. Reagan National Defense Authorization Act for Fiscal Year 
2005, Pub. L. No. 108-375, ï¿½ 332, 118 Stat. 1811, 1851-1856 (Oct. 28, 
2004) (codified in part at 10 U.S.C. ï¿½2222).  

[6] GAO, Business Systems Modernization: DOD Needs to Fully Define 
Policies and Procedures for Institutionally Managing Investments, GAO-
07-538 (Washington, D.C.: May 11, 2007); Defense Business 
Transformation: A Comprehensive Plan, Integrated Efforts, and Sustained 
Leadership Are Needed to Assure Success, GAO-07-229T (Washington, D.C.: 
Nov. 16, 2006); Business Systems Modernization: DOD Continues to 
Improve Institutional Approach, but Further Steps Needed, GAO-06-658 
(Washington, D.C.: May 15, 2006); and DOD Business Systems 
Modernization: Important Progress Made in Establishing Foundational 
Architecture Products and Investment Management Practices, but Much 
Work Remains, GAO-06-219 (Washington, D.C.: Nov. 23, 2005).  

[7] GAO-07-538.  

[8] We rated the key practices as ï¿½executedï¿½ on the basis of whether 
the agency demonstrated (by providing evidence of performance) that it 
had met all of the criteria of the key practice. A key practice was 
rated as ï¿½not executedï¿½ when we found insufficient evidence of any 
elements of a practice being fully performed or when we determined that 
there were significant weaknesses in the departmentï¿½s execution of the 
key practice.  

[9] GAO, Information Technology Investment Management: A Framework for 
Assessing and Improving Process Maturity, GAO-04-394G (Washington, 
D.C.: March 2004).  

[10] GAO-04-394G.  

[11] GAO-06-658.  

[12] The Clinger-Cohen Act of 1996, 40 U.S.C. ï¿½ï¿½ 11101-11704. This act 
expanded the responsibilities of OMB and the agencies that had been set 
under the Paperwork Reduction Act with regard to IT management. See 44 
U.S.C. 3504(a)(1)(B)(vi) (OMB); and 44 U.S.C. 3506(h)(5) (agencies).  

[13] We have made recommendations to improve OMBï¿½s process for 
monitoring high-risk IT investments; see GAO, Information Technology: 
OMB Can Make More Effective Use of Its Investment Reviews, GAO-05-276 
(Washington, D.C.: Apr. 15, 2005). 

[14] This policy is set forth and guidance is provided in OMB Circular 
A-11 (Nov. 2, 2005) (section 300) and in OMBï¿½s Capital Programming 
Guide, which directs agencies to develop, implement, and use a capital 
programming process to build their capital asset portfolios.  

[15] See, for example, GAO-04-394G; GAO, Information Technology: A 
Framework for Assessing and Improving Enterprise Architecture 
Management (Version 1.1), GAO-03-584G (Washington, D.C.: April 2003); 
and Assessing Risks and Returns: A Guide for Evaluating Federal 
Agenciesï¿½ IT Investment Decision-making, GAO/AIMD-10.1.13 (Washington, 
D.C.: February 1997).  

[16] GAO-04-394G; GAO/AIMD-10.1.13; GAO, Executive Guide: Improving 
Mission Performance Through Strategic Information Management and 
Technology, GAO/AIMD-94-115 (Washington, D.C.: May 1994); and Office of 
Management and Budget, Evaluating Information Technology Investments, A 
Practical Guide (Washington, D.C.: November 1995).  

[17] GAO-04-394G.  

[18] GAO, Information Technology: Centers for Medicare and Medicaid 
Services Needs to Establish Critical Investment Management 
Capabilities, GAO-06-12 (Washington, D.C.: Oct. 28, 2005); Information 
Technology: HHS Has Several Investment Management Capabilities in 
Place, but Needs to Address Key Weaknesses, GAO-06-11 (Washington, 
D.C.: Oct. 28, 2005); Information Technology: FAA Has Many Investment 
Management Capabilities in Place, but More Oversight of Operational 
Systems Is Needed, GAO-04-822 (Washington, D.C.: Aug. 20, 2004); Bureau 
of Land Management: Plan Needed to Sustain Progress in Establishing IT 
Investment Management Capabilities, GAO-03-1025 (Washington, D.C.: 
Sept. 12, 2003); Information Technology: Departmental Leadership 
Crucial to Success of Investment Reforms at Interior, GAO-03-1028 
(Washington, D.C.: Sept. 12, 2003); United States Postal Service: 
Opportunities to Strengthen IT Investment Management Capabilities, GAO-
03-3 (Washington, D.C.: Oct. 15, 2002); and Information Technology: DLA 
Needs to Strengthen Its Investment Management Capability, GAO-02-314 
(Washington, D.C.: Mar. 15, 2002).  

[19] The Clinger-Cohen Act of 1996, 40 U.S.C. ï¿½ï¿½ 11311-11313.  

[20] The National Security Strategy Report required by 50 U.S.C. 404a 
is a comprehensive report on the national security strategy of the 
United States submitted by the President to Congress.  

[21] See 10 U.S.C. 118. The Quadrennial Defense Review is a 
comprehensive examination of the national defense strategy, force 
structure, force modernization plans, infrastructure, budget plan, and 
other elements of the defense program and policies of the United States 
with a view toward determining and expressing the defense strategy of 
the United States and establishing a defense program for the next 20 
years.  

[22] GAO, Best Practices: An Integrated Portfolio Management Approach 
to Weapon System Investments Could Improve DODï¿½s Acquisition Outcomes, 
GAO-07-388 (Washington, D.C.: Mar. 30, 2007).  

[23] The Director for Program Analysis and Evaluation is the principal 
staff assistant who conducts independent analysis for, and provides 
independent advice on, all DOD program and evaluation matters to the 
Secretary and Deputy Secretary of Defense.  

[24] As described in DOD Directive 5000.1, May 12, 2003, and DOD 
Instruction 5000.2, May 12, 2003.  

[25] A Major Defense Acquisition Program is an acquisition program that 
is estimated by the Under Secretary of Defense for Acquisition, 
Technology, and Logistics to require an eventual total expenditure for 
research, development, and test and evaluation of more than $365 
million (fiscal year 2000 constant dollars) or, for procurement, of 
more than $2 billion (fiscal year 2000 constant dollars).  

[26] A Major Automated Information System is a program or initiative 
that is so designated by the Assistant Secretary of Defense (Networks 
and Information Integration)/Chief Information Officer or that is 
estimated to require program costs in any single year in excess of $32 
million (fiscal year 2000 constant dollars), total program costs in 
excess of $126 million (fiscal year 2000 constant dollars), or total 
life-cycle costs in excess of $378 million (fiscal year 2000 constant 
dollars).  

[27] According to DOD, the milestone decision authority is the 
designated individual who has overall responsibility for an investment. 
This person has the authority to approve an investmentï¿½s progression in 
the acquisition process and is responsible for reporting cost, 
schedule, and performance results. For example, the milestone decision 
authority for a Major Defense Acquisition Program when not delegated to 
the component level, is the Under Secretary of Defense for Acquisition, 
Technology, and Logistics, and the milestone decision authority for a 
Major Automated Information System is the Assistant Secretary of 
Defense (Networks and Information Integration)/Chief Information 
Officer or a designee.  

[28] The Defense Acquisition Boardï¿½chaired by the Under Secretary of 
Defense for Acquisition, Technology, and Logisticsï¿½conducts reviews for 
major defense acquisition programs at major program milestones and 
documents the decisions resulting from the review in an Acquisition 
Decision Memorandum.  

[29] The IT Acquisition Boardï¿½chaired by the Assistant Secretary of 
Defense (Networks and Information Integration)/Chief Information 
Officerï¿½conducts reviews for Major Automated Information System at 
major program milestones and documents the decision(s) resulting from 
the review in an Acquisition Decision Memorandum.  

[30] The four Investment Review Boards are (1) financial management, 
established by the Deputy Under Secretary of Defense for Financial 
Management; (2) weapon systems life-cycle management and materiel 
supply and services management; (3) real property and installations 
life-cycle management, both established by the Under Secretary of 
Defense (Acquisition, Technology, and Logistics); and (4) human 
resources management, established by the Under Secretary of Defense for 
Personnel and Readiness.  

[31] DODï¿½s IT portfolio repository is the authoritative repository for 
certain information about DODï¿½s business systems, such as system names 
and the responsible DOD components that are required for the 
certification, approval, and annual reviews of these business system 
investments.  

[32] In addition, each component precertification authority submits a 
list of system names to the Investment Review Boards on a semiannual 
basis, to include Tier 4 systems and systems in operations and 
maintenance that have been reviewed at the component level.  

[33] These investment management policies and procedures include 
precertifying Tier 1 through 3 business system investments by the 
component. These systems are then reviewed and certified by DOD. Tier 4 
systems are certified by the components.  

[34] Investment portfolios are integrated agencywide collections of 
investments that are assessed and managed collectively on the basis of 
common criteria.  

[End of section]  

GAO's Mission:  

The Government Accountability Office, the audit, evaluation and 
investigative arm of Congress, exists to support Congress in meeting 
its constitutional responsibilities and to help improve the performance 
and accountability of the federal government for the American people. 
GAO examines the use of public funds; evaluates federal programs and 
policies; and provides analyses, recommendations, and other assistance 
to help Congress make informed oversight, policy, and funding 
decisions. GAO's commitment to good government is reflected in its core 
values of accountability, integrity, and reliability.  

Obtaining Copies of GAO Reports and Testimony:  

The fastest and easiest way to obtain copies of GAO documents at no 
cost is through GAO's Web site [hyperlink, http://www.gao.gov]. Each 
weekday, GAO posts newly released reports, testimony, and 
correspondence on its Web site. To have GAO e-mail you a list of newly 
posted products every afternoon, go to [hyperlink, http://www.gao.gov] 
and select "Subscribe to Updates."  

Order by Mail or Phone:  

The first copy of each printed report is free. Additional copies are $2 
each. A check or money order should be made out to the Superintendent 
of Documents. GAO also accepts VISA and Mastercard. Orders for 100 or 
more copies mailed to a single address are discounted 25 percent. 
Orders should be sent to:  

U.S. Government Accountability Office: 
441 G Street NW, Room LM: 
Washington, D.C. 20548:  

To order by Phone: 
Voice: (202) 512-6000: 
TDD: (202) 512-2537: 
Fax: (202) 512-6061:  

To Report Fraud, Waste, and Abuse in Federal Programs:  

Contact:  

Web site: [hyperlink, http://www.gao.gov/fraudnet/fraudnet.htm]: 
E-mail: [email protected]: 
Automated answering system: (800) 424-5454 or (202) 512-7470:  

Congressional Relations:  

Gloria Jarmon, Managing Director, [email protected]: 
(202) 512-4400: 
U.S. Government Accountability Office: 
441 G Street NW, Room 7125: 
Washington, D.C. 20548:  

Public Affairs: 

Chuck Young, Managing Director, [email protected]: 
(202) 512-4800: 
U.S. Government Accountability Office: 
441 G Street NW, Room 7149: 
Washington, D.C. 20548: 

*** End of document. ***