Defense Management: Additional Actions Needed to Enhance DOD's	 
Risk-Based Approach for Making Resource Decisions (15-NOV-05,	 
GAO-06-13).							 
                                                                 
The Department of Defense (DOD) is simultaneously conducting	 
costly military operations and transforming its forces and	 
business practices while it is also competing for resources in an
increasingly constrained fiscal environment. As a result, GAO has
advocated that DOD adopt a comprehensive threat or risk 	 
management approach as a framework for decision making. In its	 
2001 strategic plan, the Quadrennial Defense Review (QDR), DOD	 
stated its intent to establish an approach--the risk management  
framework--to balance priorities against risk over time and	 
monitor results against its strategic goals. GAO was asked to (1)
assess the extent to which DOD has implemented the framework,	 
including using it to make investment decisions, and (2) identify
the most significant challenges DOD faces in implementing the	 
framework, or a similar approach.				 
-------------------------Indexing Terms------------------------- 
REPORTNUM:   GAO-06-13						        
    ACCNO:   A41497						        
  TITLE:     Defense Management: Additional Actions Needed to Enhance 
DOD's Risk-Based Approach for Making Resource Decisions 	 
     DATE:   11/15/2005 
  SUBJECT:   Accountability					 
	     Decision making					 
	     Defense budgets					 
	     Defense capabilities				 
	     Internal controls					 
	     Performance measures				 
	     Policy evaluation					 
	     Risk management					 
	     Strategic planning 				 
	     Transparency					 
	     DOD Quadrennial Defense Review			 
	     OMB Program Assessment Rating Tool 		 

******************************************************************
** This file contains an ASCII representation of the text of a  **
** GAO Product.                                                 **
**                                                              **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced.  Tables are included, but    **
** may not resemble those in the printed version.               **
**                                                              **
** Please see the PDF (Portable Document Format) file, when     **
** available, for a complete electronic file of the printed     **
** document's contents.                                         **
**                                                              **
******************************************************************
GAO-06-13

Defense Management Defense Management Defense Management Defense
Management Defense Management Defense Management Defense Management
Defense Management Defense Management Defense Management Defense
Management Defense Management Defense Management Defense Management
Defense Management Defense Management Defense Management Defense
Management Defense Management Defense Management Defense Management
Defense Management Defense Management Defense Management Defense
Management Defense Management Defense Management Defense Management
Defense Management Defense Management Defense Management Defense
Management Defense Management Defense Management Defense Management
Defense Management

Contents

Letter 1

Results in Brief 3
Background 5
Despite Positive Steps, Additional Actions Needed to Fully Implement the
Risk Management Framework 9
Cultural Resistance, Combined with the Lack of Leadership, Implementation
Goals, and Process Integration, Affects DOD's Implementation of the Risk
Management Framework 18
Conclusions 24
Recommendations for Executive Action 25
Agency Comments and Our Evaluation 25
Appendix I Scope and Methodology 29
Appendix II Comments from the Department of Defense 31
Appendix III GAO Contact and Staff Acknowledgments 34

Tables

Table 1: Definitions and Examples of DOD Department-Level Measures (as of
November 2004) 11
Table 2: The Number of Activity and Performance Measures for Each Quadrant
12
Table 3: Military Service and Defense-Wide Percentage of the 2005 and 2006
Future Years Defense Programs 16
Table 4: Select Initiatives to Improve Investment Decision Making 22

Figures

Figure 1: The Risk Management Cycle 6
Figure 2: Comparison of the Balanced Scorecard and the Risk Management
Framework 9

This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed in
its entirety without further permission from GAO. However, because this
work may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this material
separately.

Abbreviations

CBO Congressional Budget Office

CMO chief management official

DOD Department of Defense

FYDP Future Years Defense Program

GAO Government Accountability Office

GPRA Government Performance and Results Act

JCIDS Joint Capabilities Integration and Development System

OSD Office of the Secretary of Defense

PA&E Program Analysis and Evaluation (PA&E)

PPBE Planning, Programming, Budgeting, and Execution

P&R Personnel and Readiness

PART Program Assessment Rating Tool

QDR Quadrennial Defense Review

United States Government Accountability Office

Washington, DC 20548

November 15, 2005

The Honorable John Ensign Chairman The Honorable Daniel K. Akaka Ranking
Minority Member Subcommittee on Readiness and Management Support Committee
on Armed Services United States Senate

Among the 21st century challenges facing the Department of Defense (DOD)
and the nation as a whole are difficult decisions concerning how to strike
an affordable balance between current and future national security needs
and between national security and domestic needs.1 For example, DOD is
simultaneously maintaining a high pace of military operations for
combating terrorism and transforming its military forces and business
operations for the 21st century while it is also competing for federal
resources in an increasingly fiscally constrained environment. We have
advocated that DOD-as well as the rest of the federal government-adopt a
comprehensive threat or risk management approach as a framework for
decision making.2 This approach would fully link strategic goals to plans
and budgets; assess the values and risks of various courses of actions as
a tool for reexamining defense programs, setting priorities, and
allocating resources; and use performance measures to assess outcomes.

To its credit, DOD introduced a balanced scorecard for risk management,
commonly known as the risk management framework, in its strategic plan,
the 2001 Quadrennial Defense Review (QDR) report. The 2001 strategic plan
articulated the new administration's emphasis on transforming military
forces and defense business practices to meet the emerging challenges
facing our nation. DOD intended the framework to be used as a management
tool to focus DOD's efforts on implementing the defense program as
outlined in the strategic plan. In particular, DOD's senior leadership
intended the risk management framework to assist decision makers in
formulating top-down strategy, balancing investment priorities against
risk over time, measuring near- and midterm outputs against strategic
goals, and focusing on actual performance results. According to DOD
officials, the risk management framework also was intended to increase
transparency within the department over the decision-making process.
During the ongoing 2005 QDR, DOD plans to refine the risk management
framework.

1See GAO, 21st Century Challenges: Reexamining the Base of the Federal
Government, GAO-05-325SP (Washington, D.C.: February 2005) for a
comprehensive compendium of areas throughout the federal government that
could be considered for reexamination and review by Congress.

2GAO, High-Risk Series: An Update, GAO-05-207 (Washington, D.C.: January
2005).

You asked us to examine the status of DOD's efforts to adopt a risk-based
approach to decision making, given the emphasis that DOD was placing on
the risk management framework. In response, we (1) assessed the extent to
which DOD has implemented its risk management framework, including the
extent to which DOD has used the framework to make investment decisions;
and (2) identified the most significant challenges DOD faced in
implementing the risk management framework or a similar risk-based and
results-oriented management approach.

To assess the extent to which DOD has implemented the risk management
framework, we analyzed key documents, policy guidance, data, and interview
results, and compared the analysis to the principles for managing risk and
results identified in prior GAO reports. In addition, we conducted
interviews with DOD and service officials, and members of the Joint Staff.
We discussed the department's progress in implementing the risk management
framework with members of the Defense Business Board. We also analyzed
DOD's department-level performance goals and measures that are associated
with the risk management framework and assessed how DOD reported that
information externally. We did not validate the appropriateness of the
risk management framework's risk quadrants or the procedures that DOD has
in place to ascertain the reliability of performance data and we also did
not assess the basis for DOD's investment decisions. To identify the most
significant challenges DOD faced in implementing the risk management
framework, we analyzed documents, data, and interview results, and
compared the results of this analysis to the key practices to assist
mergers and organizational transformation identified in prior GAO
reports.3 A more detailed discussion of our scope and methodology is
presented in appendix I.

Our work was performed from October 2004 through September 2005 in
accordance with generally accepted government auditing standards.

                                Results in Brief

DOD has taken positive steps toward implementing the risk management
framework; however, additional actions are needed before the framework is
fully implemented and DOD can demonstrate real and sustainable progress in
using a risk-based and results-oriented approach to strategically allocate
resources across the spectrum of its investment priorities. For example,
while DOD established four risk areas, or quadrants, and developed
performance goals and measures of two types-activity measures (measures to
track initiatives) and performance measures-the majority of these measures
do not provide sufficient information to monitor performance against the
risk quadrants' goals. Specifically, and contrary to results-oriented
management principles, the risk management framework's measures (1) do not
clearly demonstrate results, (2) do not provide a well-rounded depiction
of performance across the department, and (3) are not being systemically
monitored across all quadrants, except for the force management quadrant.
In addition, the framework's performance goals and measures are not
clearly linked to DOD's current strategic plan and strategic goals.
Lacking measures that follow results-oriented management principles and
clear linkages to strategic goals, DOD may be unable to provide a clear
roadmap of how its activities at all levels contribute to meeting DOD's
strategic goals. Finally, although DOD officials stated that risk was
considered in the fiscal year 2006 budget cycle, the fiscal year 2006
budget submission does not include any specific information on how DOD
systematically identified or assessed departmental risks to establish
DOD-wide investment priorities. Therefore, the linkages between the risk
management framework and the budget are unclear. Without better measures,
clear linkages, and greater transparency, DOD will be unable to fully
measure progress in achieving strategic goals or demonstrate to Congress
and others how it considered risks and made trade-off decisions, balancing
needs and costs for weapon system programs and other investment
priorities.

3GAO, Highlights of a GAO Forum: Mergers and Transformation: Lessons
Learned for a Department of Homeland Security and Other Federal Agencies,
GAO-03-293SP (Washington, D.C.: Nov. 14, 2002), and Results-Oriented
Cultures: Implementation Steps to Assist Mergers and Organizational
Transformations, GAO-03-669 (Washington, D.C.: July 2, 2003).

DOD faces four key challenges that affect its ability to fully implement
the risk management framework, or a similar risk-based and
results-oriented management approach: (1) overcoming cultural resistance
to the transformational change represented by such an approach in a
department as massive, complex, and decentralized as DOD; (2) maintaining
sustained leadership and clear accountability for this cultural
transformation; (3) providing implementation goals and timelines to gauge
progress in transforming the culture; and (4) integrating the risk
management framework with decision support processes and related reform
initiatives into a coherent, unified management approach for the
department. Our prior work on results-oriented management and
organizational transformation and mergers has shown that addressing these
challenges is at the center of successful change management efforts in
leading organizations. DOD is having difficulties implementing the
framework because it has not addressed these four challenges. With respect
to the first challenge, DOD's size and complexity result in a culture that
makes developing department-level approaches to priority setting and
investment decision making difficult. For example, the allocation of
budgets on a proportional, rather than a strategic, basis among the
services is a long-standing budgetary problem that we have reported about
for years. Second, the lack of sustained leadership and clear
accountability for the framework's implementation has resulted in a lack
of emphasis and understanding of its status and purpose within the
department. Because of the lack of sustained leadership for other
management reform efforts, we have supported legislation to create a chief
management official (CMO) at DOD to provide this leadership.4 Third, DOD
did not establish implementation goals or timelines with which to
establish accountability, measure progress, and build momentum. Finally,
integrating the risk management framework with other decision support
processes and related reform initiatives into a coherent, unified
management approach is a challenge that DOD intends to address in the
ongoing 2005 QDR. Illustrating this challenge, DOD is attempting to
implement the risk management framework while it is also shifting to
biennial budgeting and reforming defense planning. Our work has shown that
if risk-based and results-oriented management approaches are to be
successfully implemented, they must be integrated into the usual cycle of
agency decision making. Unless DOD addresses these challenges and
successfully implements the risk management framework, or a similar
approach, it may continue to experience (1) a mismatch between programs
and budgets, and (2) the proportional, rather than strategic, allocation
of resources to the services. Therefore, Congress may have insufficient
transparency into how DOD has identified and assessed risks and made
trade-offs in its investment decision making.

4S. 780, 109th Cong. S:1 (2005).

In this report, we recommend that DOD take various actions to increase its
chances of successfully implementing a risk-based approach for investment
decision making. This includes developing results-oriented measures and
assigning clear leadership with appropriate accountability and authority
to implement and sustain the risk management framework, or a similar
approach. In written comments on a draft of this report, DOD partially
concurred with our recommendations. DOD's comments and our evaluation of
them are on page 25 of this report.

                                   Background

In our report, High-Risk Series: An Update,5 we identified agencies' lack
of comprehensive risk management strategies as an emerging challenge for
the federal government. Increasingly limited fiscal resources across the
federal government, coupled with the emerging requirements from the
changing security environment, emphasize the need for DOD to develop a
risk-based strategic investment approach. For this reason, we have
advocated that DOD adopt a comprehensive risk management approach for
decision making.6 Furthermore, DOD and other federal agencies are required
by statute to develop a results-oriented management approach to
strategically allocate resources on the basis of performance.7 The
balanced scorecard-a concept to balance an organization's focus across
financial, customer, internal business, and learning and growth management
areas-is one approach for developing results-oriented management that
government agencies have recently started to adopt.8 At the direction of
the Secretary of Defense, DOD developed a risk management framework that
DOD later aligned with its results-oriented management activities through
a DOD balanced scorecard.

5GAO-05-207.

6GAO-05-207.

7The Government Performance and Results Act of 1993 (Pub. L. No. 103-62).

8The balanced scorecard approach was advocated by Professor Robert Kaplan
and Dr. David Norton in the November/December 1992 Harvard Business
Review.

Risk Management Is an Emerging 21st Century Challenge

An emerging challenge for the federal government involves the need for the
completion of comprehensive national threat and risk assessments in a
variety of areas. For example, emerging requirements from the changing
security environment, coupled with increasingly limited fiscal resources
across the federal government, emphasize the need for agencies to adopt a
sound approach to establishing resource decisions.9 We have advocated that
the federal government, including DOD, adopt a comprehensive threat or
risk management approach as a framework for decision making that fully
links strategic goals to plans and budgets, assesses values and risks of
various courses of actions as a tool for setting priorities and allocating
resources, and provides for the use of performance measures to assess
outcomes. Based on our review of the literature,10 as shown in figure 1,
the goal of risk management is to integrate systematic concern for risk
into the usual cycle of agency decision making and implementation.

Figure 1: The Risk Management Cycle

9GAO-05-325SP.

10See for example, Committee of Sponsoring Organizations of the Treadway
Commission, Enterprise Risk Management-Integrated Framework: Executive
Summary (New York, N.Y.: September 2004).

A risk management cycle represents a series of analytical and managerial
steps, basically sequential, that can be used to assess risk, evaluate
alternatives for reducing risks, choose among those alternatives,
implement the alternatives, monitor their implementation, and continually
use new information to adjust and revise the assessments and actions, as
needed. Adoption of a risk management cycle such as this can aid in
assessing risk by determining which vulnerabilities should be addressed,
and how they should be addressed, within available resources. For the
purposes of this report, we focused on the stages of the risk management
cycle that involve DOD's actions to set strategic goals and objectives,
establish investment priorities based on risk assessments, and
implementation and monitoring.

Risk management's objectives are essentially the same as those of good
management, and they are consistent with the broad economy and efficiency
objectives of good government-namely, to provide better outcomes for the
same amount of money, or to provide the same outcomes with less money.
Therefore, risk management's objectives are also compatible with those of
the federal government's results-oriented management approach, which was
enacted in the Government Performance and Results Act (GPRA) of 1993,11
and the balanced scorecard approach. Congress enacted GPRA to focus the
federal government on achieving results through the creation of clear
links between the process of allocating scarce resources and an agency's
strategic goals, or the expected results to be achieved with those
resources. Building on GPRA's foundation, the current administration has
taken steps to strengthen the integration of budget, cost, and performance
information by including budget and performance integration as one of its
management initiatives under the umbrella of the President's Management
Agenda.12 The Budget and Performance Integration initiative includes
efforts such as the Program Assessment Rating Tool (PART), improving
outcome measures, and improving monitoring of program performance.13 The
balanced scorecard approach is a management tool that some federal
agencies have adopted to help them translate the strategy set forth in a
results-oriented management approach into the operational objectives that
drive both behavior and performance. The balanced scorecard consists of
four management areas that organizations should focus on-financial,
customer, internal business, and learning and growth.

11Pub. L. No. 103-62 (1993).

12The President's Management Agenda, by focusing on a number of targeted
areas, seeks to improve the performance management of the federal
government.

DOD's 2001 Strategic Plan Outlines a New Risk Management Framework

DOD introduced the risk management framework in its strategic plan, the
2001 QDR report. The 2001 strategic plan articulated the new
administration's emphasis on transforming military forces and defense
business practices to meet the changing threats facing our nation. In his
guidance to the department for the 2001 QDR strategic planning process,
the Secretary of Defense stated the need for DOD to use a risk mitigation
approach for balancing force, resource, and modernization requirements
across defense planning timelines. This guidance also stated that DOD must
include the identification of output-based measures to reduce
inefficiencies through the department in any approach to risk management.
Building on the guidance, the 2001 QDR outlined DOD's risk management
framework. According to the QDR, the framework would enable DOD to address
the tension between preparing for future threats and meeting the demands
of the present with finite resources. It was also intended to ensure that
DOD was sized, shaped, postured, committed, and managed with a view toward
accomplishing the strategic plan's defense policy goals.

DOD adapted the balanced scorecard concept to the risk management
framework by substituting the four dimensions of risk-force management,
operational, future challenges, and institutional-for the scorecard's four
management areas. The risk management framework was to be a
transformational tool that would provide a balanced perspective of the
organization's execution of strategy and ensure a top-down approach. The
2002 policy guidance also designated four preliminary performance goals
for each of the four risk quadrants. In addition, the guidance required
that performance goals and measures were to be cascaded to the services
and defense agencies. Figure 2 shows a comparison, as provided by DOD.

13For further information see: GAO, Performance Budgeting: PART Focuses
Attention on Program Performance, but More Can Be Done to Engage Congress,
GAO-06-28 (Washington, D.C.: Oct. 28, 2005); Management Reform: Assessing
the President's Management Agenda, GAO-05-574T (Washington, D.C.: Apr. 21,
2005); Results-Oriented Government: GPRA Has Established a Solid
Foundation for Achieving Greater Results, GAO-04-38 (Washington, D.C.:
Mar. 10, 2004); and Performance Budgeting: Observations on the Use of
OMB's Program Assessment Rating Tool for the Fiscal Year 2004 Budget,
GAO-04-174 (Washington, D.C.: Jan. 30, 2004).

Figure 2: Comparison of the Balanced Scorecard and the Risk Management
Framework

 Despite Positive Steps, Additional Actions Needed to Fully Implement the Risk
                              Management Framework

Despite positive steps, DOD needs to take additional actions before the
risk management framework is fully implemented and DOD can demonstrate
real and sustainable progress in using a risk-based and results-oriented
approach to strategically allocate resources across the spectrum of its
investment priorities. For example, DOD is still in the process of
developing department-level measures for the framework that address
results-based management principles, such as linking performance
information to strategic goals so that this information can be used to
monitor performance results and determine how well the department is doing
in achieving its strategy. Without more results-oriented performance
measures, DOD may be unable to provide the services and other defense
components with clear roadmaps of how their activities contribute to
meeting DOD's strategic goals. In addition, the framework's performance
goals and measures are not clearly linked to DOD's current strategic plan
and strategic goals. Furthermore, the extent to which the risk management
framework is linked to the budget cycle is unclear. Without better
measures, clear linkages, and greater transparency, DOD will be unable to
fully measure progress in achieving strategic goals or demonstrate to
Congress and others how it considered risks and made trade-offs in making
investment decisions.

Developing a Set of Measures That Can Be Used to Monitor Performance Is a Work
in Progress

DOD has taken positive steps toward developing measures for each of the
performance goals under the framework's four risk quadrants; however,
developing a set of measures that can be used to monitor performance
results is still a work in progress. Based on GAO's prior work on
results-based management principles, we found that leading organizations'
performance measures are: (1) designed to demonstrate results, or provide
information on how well the organization is achieving its goals; (2)
limited to a vital few, and balanced across priorities; and (3) used by
management to improve performance.14 However, the set of measures DOD has
developed for the risk management framework do not adequately address
these principles. While DOD established four risk quadrants and developed
performance goals and measures of two types-activity measures (measures to
track initiatives) and performance measures-the majority of its measures
do not provide sufficient information to monitor performance against the
risk quadrants' goals.

First, DOD officials acknowledge that establishing department-level
measures for the framework that demonstrate results is still a work in
progress, as the majority of the risk management framework's measures
require further development or refinement. In fact, as shown in table 1,
44 of the 77 department-level measures for all four quadrants, or over 50
percent, are activity measures. According to DOD sources, activity
measures are to result in a new performance measure, a new baseline or
benchmark, or define a new capability, rather than monitor a specific
annual performance target. Once these activities are completed, DOD
officials stated that the department will be better able to monitor
department-level performance against strategic goals. However, our
analysis found that the activity measures, as defined in DOD's external
reports, typically do not provide sufficient information to monitor the
department's progress in achieving the stated goal they are to measure,
such as developing a new performance measure or baseline. The desired
outcomes for activity measures generally state that a task was or will be
completed by a certain date but they do not provide sufficient information
on whether the activity is on schedule, the interdependencies among tasks,
or the contribution toward enhancing the department's performance.
Therefore, Congress and other external stakeholders lack information and
adequate assurances that DOD is making progress in implementing a
risk-based and results-oriented management approach to making investment
decisions.

14GAO, Executive Guide: Effectively Implementing the Government
Performance and Results Act, GAO/GGD-96-118 (Washington, D.C.: June 1996)
and Managing for Results: Enhancing Agency Use of Performance Information
for Management Decision Making, GAO-05-927 (Washington, D.C.: Sept. 9,
2005).

Table 1: Definitions and Examples of DOD Department-Level Measures (as of
November 2004)

                                                              Description of  
                                                              desired outcome 
                                                              monitored by    
Type        Number Definition               Examples       measure         
Activity       44a Activity measures track  Deny enemy     Roadmap will be 
measures           developmental            advantages and complete by the 
                      activities, are usually  exploit        end of fiscal   
                      qualitative, and track   weaknesses     year 2005       
                      key milestones or events Enhance        Strategy will   
                      in lieu of a specific    homeland       be complete by  
                      annual performance       defense and    the first       
                      target                   consequence    quarter of      
                                               management     fiscal year     
                                                              2005            
Performance    33a Performance measures     Reserve        Target > 90% of 
measures           track current outputs    component      recruits        
                      and set quantitative     enlisted       holding high    
                      annual targets for       recruiting     school diplomas 
                      performance that are     quality                        
                      measurable                              Actual 88% of   
                                                              recruits        
                                                              holding high    
                                                              school diplomas 
                                               Reduce         Target 15 days  
                                               customer wait  from order to   
                                               time (in days) receipt for     
                                                              material goods  
                                                                              
                                                              Actual 24 days  
                                                              from order to   
                                                              receipt for     
                                                              material goods  

Source: GAO analysis of the Risk Management Framework's performance
measures.

a We have recoded five performance measures as activity measures as these
measures tracked milestones and events, which corresponds with DOD's
definition of an activity measure.

Second, DOD's department-level performance measures are still a work in
progress in that these measures do not provide a well-rounded depiction of
DOD's performance. In our previous work, we have found that performance
measurement efforts that are not balanced across priorities may skew an
agency's performance and keep its senior leadership from seeing the whole
picture.15 For example, in developing department-level measures for the
risk management framework, DOD appears to have overemphasized its force
management priorities at the expense of operational risk. As illustrated
in table 2, the operational risk quadrant has no performance measures,
while the force management risk quadrant has a total of 36 measures,
including 15 activity measures and 21 performance measures.

Table 2: The Number of Activity and Performance Measures for Each Quadrant

                             Activity                                     
                             measures Performance measures Total measures
Force Management                15                   21            36a 
Operational                      9                    0              9 
Institutional                   11                   10            21b 
Future Challenges                9                    2            11c 

Source: GAO analysis of DOD data.

a We have recoded two performance measures as activity measures as these
measures tracked milestones and events, which corresponds with DOD's
definition of an activity measure.

b We have recoded one performance measure as an activity measure as this
measure tracked milestones and events, which corresponds with DOD's
definition of an activity measure.

c We have recoded two performance measures as activity measures as these
measures tracked milestones and events, which corresponds with DOD's
definition of an activity measure.

In providing technical comments to a draft of this report, DOD objected to
our recoding of five department-level performance measures as activity
measures. We recoded these measures because they tracked milestones and
events, which corresponded to DOD's definition of an activity measure. The
measures we recoded addressed the following:

           o  a civilian human resources strategic plan,
           o  a military human resources strategic plan,
           o  monitor the status of defense technology objectives,
           o  strategic transformation appraisal, and
           o  support acquisition excellence goals.

15GAO/GGD-96-118.

Finally, DOD officials indicated that DOD is systematically using
performance measures to monitor progress and improve performance for only
one risk quadrant, although individual measures under the other three risk
quadrants may be monitored. We have found that leading organizations use
performance information to improve organizational performance and identify
performance gaps, and to provide incentives that reinforce a
results-oriented management approach.16 According to DOD officials, the
force management quadrant is the only quadrant that is managed by one
individual and one office-the Under Secretary of Defense for Personnel and
Readiness and his office. These officials stated that this situation is a
critical factor in the progress DOD has made in systematically monitoring
performance across the force management quadrant on a routine basis. For
example, officials stated that the Under Secretary of Defense personally
leads quarterly monitoring sessions on the force management quadrant's
performance. DOD officials also told us that the Under Secretary of
Defense for Personnel and Readiness has greatly facilitated this
monitoring by developing a centralized database to capture the performance
data used to track DOD's performance in meeting the quadrant's goals.
Unless all of the risk management framework's quadrants are systematically
monitored, implementation of the framework may be hindered and the
framework risks becoming a paper-driven, compliance exercise. Indeed, one
DOD official told us that he views the risk management framework and its
measures as a "reporting drill" and, in addition, his office would not
change its processes if DOD was to no longer use the framework.

Cascading the Risk Management Framework's Goals and Measures Is an Ongoing
Effort

DOD is still in the process of cascading the risk management framework's
goals and measures to the services. We have found that leading
organizations seek to establish clear hierarchies of goals and measures
that cascade down so that subordinate units have straightforward roadmaps
to demonstrate how their activities contribute to meeting the
organization's strategy.17 According to DOD officials, all of the services
are attempting to align their existing performance measures with the
department-level performance goals and measures. However, service
officials said that it is challenging to cascade the department-level
activity measures, because these measures represent very broad initiatives
that may not be applicable at all DOD levels. Officials from one service
said they have had to develop new measures to align with the
department-level measures, because they had been assessing performance
with fewer measures than the Office of the Secretary of Defense had
developed.

16GAO/GGD-96-118 and GAO-05-927.

17GAO/GGD-96-118.

Developing a Strategic Plan with Clear Linkages between the Risk Management
Framework and Strategic Goals Is a Critical Next Step

The risk management framework's performance goals and measures are not
clearly linked-a key principle of results-oriented management-to a
coherent strategic plan.18 The development of such a strategic plan is a
critical next step in using a risk-based and results-oriented approach to
making investment decisions. Without these linkages, DOD cannot easily
demonstrate how achievement of a performance goal or measure contributes
to the achievement of strategic goals and ultimately the organization's
mission. Our previous work indicated that DOD's strategic plan, the 2001
QDR, did not provide a sound foundation for the risk management
framework.19 We reported that the usefulness of the 2001 QDR was limited
by the lack of focus on longer-term threats and requirements for critical
support capabilities, and provided few insights into how future threats
and planned technical advances could affect future force requirements. In
turn, this lack of focus and insight limited the QDR's usefulness as a
foundation for fundamentally reassessing U.S. defense plans and programs
and for balancing resources across near- and midterm risks.

DOD officials indicated that DOD has not yet defined the linkages between
the risk management framework's performance goals and the strategic goals
in the 2001 QDR. Furthermore, the Defense Business Board's official
minutes for its July 28, 2005, meeting contained a recommendation that the
Secretary of Defense define department-level objectives, which should then
be cascaded down the department.20 In discussing the ongoing 2005 QDR, DOD
stated that although the department would continue its efforts to do so,
establishing these linkages was very challenging because of the size and
scope of DOD's operations. However, as suggested by the Defense Business
Board and our previous work, if DOD's strategic plan is to drive the
department's operations, a straightforward linkage is needed among
strategic goals, annual performance goals, and day-to-day activities.21
The ongoing 2005 QDR offers DOD the opportunity to strengthen its
strategic planning.

18GAO/GGD-96-118.

19GAO, Quadrennial Defense Review: Future Reviews Can Benefit from Better
Analysis and Changes in Timing and Scope, GAO-03-13 (Washington, D.C.:
Nov. 4, 2002).

20The Defense Business Board was established in 2001 by the Secretary of
Defense to provide DOD's senior leadership with leading-edge, actionable
advice on management improvements.

Although Risk Considered, Linkages Between the Risk Management Framework and
Budget Are Unclear

According to DOD officials, the department has begun to consider risk in
its investment decision making; however, the full extent to which the
framework's risk-based and results-oriented approach has been linked to
the fiscal year 2006 budget cycle is unclear. Our work indicates that
leading organizations link strategy to the budget process through
results-oriented management to evaluate potential investments or
initiatives.22

DOD sources indicated that the department has begun to consider risk
during its usual cycle of investment decision making. For example,
according to DOD sources, the Secretary of Defense articulated broad areas
for increasing or decreasing risk under each quadrant in the fiscal years
2006-2011 planning guidance, leaving it up to the defense components to
decide how to structure their investment decisions within those broad
areas consistent with the Secretary's risk guidance. In addition, DOD
officials stated that the framework has increased awareness within the
department on the need to balance risk over time. For example, when DOD
reduced the fiscal years 2006-2011 defense program by $30 billion, DOD
officials stated that the department did not take the traditional
budgetary approach of cutting each defense component's budget by a certain
percentage. Instead, DOD officials stated that the Secretary of Defense
used a collaborative approach with service participation to discuss where
to take the budget reductions and how these cuts would affect risk,
although DOD officials offered various views on how extensively the
framework was used to make those decisions.

Second, DOD required that the services and other defense components offset
any funding increase in one area with a funding decrease in another area
for the fiscal years 2006-2007 budget submission. According to DOD
officials, risk-whether on the basis of "professional judgment" or
analysis-was considered in these deliberations. For example, the Army's
plan for fiscal years 2006-2023 articulated areas for increasing risks so
that it could decrease risk in the operational risk dimension by investing
in current capacity.

21GAO, Managing for Results: Critical Issues for Improving Agencies'
Strategic Plans, GAO/GGD-97-180 (Washington, D.C.: Sept. 16, 1997).

22See GAO/GGD-96-118.

However, the fiscal year 2006 budget submission does not include any
specific information on how DOD systematically identified or assessed
departmental risks to establish DOD-wide investment priorities. For
example, the military services' share of the Future Years Defense Program
(FYDP) remained relatively unchanged from fiscal year 2005 to fiscal year
2006 (see table 3),23 providing one indication that the risk management
framework may not yet be a useful tool for balancing departmental risks
across the services.

Table 3: Military Service and Defense-Wide Percentage of the 2005 and 2006
Future Years Defense Programs

                     2005 Percentage 2006 Percentage     Percentage change by 
                             of FYDP         of FYDP               department 
Department of the           24.23           24.63                     0.40 
Army                                              
Department of the           29.75           29.47                    -0.28 
Navy                                              
Department of the           29.80           29.82                     0.02 
Air Force                                         
Defense-wide                16.22           16.08                    -0.14 
Total                      100.00          100.00 

Source: GAO analysis of DOD FYDP data.

Note: Totals may not add due to rounding.

DOD has reported on the risk management framework in the department's GPRA
and other reporting requirements. For example, the fiscal year 2004
Performance and Accountability Report describes what DOD is doing, or
plans to do, to define, measure, and monitor performance goals in the four
risk quadrants but does not discuss the implementation status of the risk
management framework. Furthermore, the fiscal year 2004 report, the most
recent available, provided insufficient information to assist Congress in
overseeing how DOD plans to prioritize investment decisions within or
across the risk quadrants. Without more detailed information, Congress may
have insufficient transparency into how DOD has identified and assessed
risks and made trade-offs in its investment decision making. In addition,
we reported in May 2004 that congressional visibility over investment
decision making also was limited by the absence of linkages between the
risk management framework and military capabilities planning and the
FYDP.24 Because the FYDP lacked these linkages, we concluded that decision
makers could not use it to determine how a proposed increase in capability
would affect the risk management framework.

23The Future Years Defense Program provides information on DOD's current
and planned outyear budget requests.

Our work also has shown that the FYDP may understate the costs of weapon
system programs; therefore, DOD may be starting more programs than it can
afford. For example, our assessment of 54 major programs, representing an
investment of over $800 billion, found that the majority of these programs
were costing more and taking longer to develop than planned.25 Problems
occurred because of DOD's overly optimistic planning assumptions about the
long-term costs of weapon system programs and its failure to capture early
on the requisite knowledge that is needed to efficiently and effectively
manage program risks. When DOD has too many programs competing for funding
and approves programs with low levels of knowledge, it is accepting the
attendant likely adverse cost and schedule risks. As a result, it will
probably get fewer quantities for the same investment or face difficult
choices on which investments it cannot afford to pursue. The findings of
our work suggest that having a departmentwide investment strategy for
weapon systems, to allocate resources across investment priorities, would
help reduce these risks.

24GAO, Future Years Defense Program: Actions Needed to Improve
Transparency of DOD's Projected Resource Needs, GAO-04-514 (Washington,
D.C.: May 7, 2004).

25GAO, Defense Acquisitions: Assessments of Selected Major Weapon
Programs, GAO-05-301 (Washington, D.C.: Mar. 31, 2005).

Cultural Resistance, Combined with the Lack of Leadership, Implementation Goals,
  and Process Integration, Affects DOD's Implementation of the Risk Management
                                   Framework

Four key challenges impede DOD's progress toward implementing the risk
management framework. The first implementation challenge facing DOD is
overcoming cultural resistance to change in a department as massive,
complex, and decentralized as DOD. The second challenge is the lack of
sustained leadership, and the third challenge is the absence of
implementation goals and timelines. These challenges relate to DOD's
failure to follow crucial transformational steps. The fourth
challenge-integrating the risk management framework with decision support
processes and related reform initiatives, into a coherent, unified
management approach for the department-relates to key results-oriented
management practices. Unless DOD addresses these challenges and
successfully implements the risk management framework, or a similar
approach, it may continue to experience (1) a mismatch between programs
and budgets, and (2) the proportional, rather than strategic, allocation
of resources to the services.

Transforming DOD's Organizational Culture Is a Significant Challenge

Transforming DOD's organizational culture-from a focus on inputs and
programs to strategically balancing investment risks and monitoring
outcomes across the department-through the implementation of the risk
management framework is a significant challenge for the department for
several reasons. First, as we noted in our 21st Century Challenges report,
to successfully transform, DOD needs to overcome the inertia of various
organizations, policies, and practices that became rooted in the Cold War
era.26 The department's expense, size, and complexity, however, make
overcoming this resistance and inertia difficult. In fiscal year 2004, DOD
reported that its operations involved $1.2 trillion in assets, $1.7
trillion in liabilities, over $605 billion in net cost of operations, and
over 3.3 million military and civilian personnel. For fiscal year 2005,
DOD received appropriations of about $417 billion. Moreover, execution of
its operations spans a wide range of defense organizations, including the
military services and their respective major commands and functional
activities, numerous large defense agencies and field activities, and
various combatant and joint operation commands, which are responsible for
military operations for specific geographic regions or theaters of
operations.

Second, DOD's highly decentralized management structure is another
contributing factor that makes cultural change difficult. Although under
the authority, direction, and control of the Secretary of Defense, the
military services have the legislative authority to organize, equip, and
train the nation's armed forces for combat under Title 10 of the U.S.
Code. Furthermore, Congress directly appropriates funds to the services
for programs and activities that support these purposes. In the opinion of
knowledgeable DOD officials, this legislative authority has resulted in a
culture that makes it difficult to develop department-level, or joint,
management approaches. For example, the allocation of budgets on a
proportional, rather than a strategic basis, among the military services
is a long-standing budgetary problem that we have identified as a major
management challenge for the department.27 In addition, the Joint Defense
Capabilities Study, chartered by the Secretary of Defense in March 2003,
made the following observations on how DOD's organizational culture does
not reinforce a departmental or joint approach to investment decision
making and results management:28

26GAO-05-325SP.

           o  DOD's bottom-up strategic planning process did not support
           early senior leadership involvement and did not provide integrated
           departmentwide objectives, priorities, and roles as a framework
           for planning joint capabilities.
           o  Service-centric focus on programs and weapons platforms
           resulted in a process that did not provide an accurate picture of
           joint needs, nor did it provide a consistent view of priorities
           and acceptable risks across the department.
           o  The resulting budget did not optimize capabilities at either
           the department or the service level.
           o  Accountability and feedback focused on monetary input rather
           than output; therefore, much of the information provided did not
           support the senior leaders' decision making as it did not tell how
           well the department was being resourced to meet current and future
           mission requirements.

           The lack of sustained leadership attention and appropriate
           accountability has challenged DOD's progress in implementing the
           risk management framework. Our work has indicated that sustained
           leadership is a key transformational, or change management,
           practice.29 However, knowledgeable DOD officials indicated that
           DOD's senior leadership did not provide sustained attention to the
           framework's implementation. For example, a DOD official actively
           involved in the framework's implementation stated that meetings
           with senior leadership that were to provide oversight of the
           framework's implementation have not been regularly scheduled. DOD
           officials indicated that as a result of this lack of sustained
           leadership, DOD has not placed much emphasis on implementing the
           risk management framework at the department level. In addition,
           other DOD officials stated that changes in leadership have made it
           difficult to implement the risk management framework or develop
           performance measures. For example, since October 2004, DOD has
           experienced turnover in the following senior level positions,
           including the Deputy Secretary of Defense; the Under Secretary of
           Defense for Acquisition, Technology and Logistics; and the
           Director of Program Analysis and Evaluation (PA&E). Lacking
           sustained leadership attention, DOD officials offered conflicting
           perspectives on the status of the risk management framework with
           some officials suggesting that the framework had been overtaken by
           other performance-based or risk-based management initiatives while
           another suggested that the framework was primarily a compliance
           exercise. DOD officials also held differing perspectives on the
           purpose of the framework, including the beliefs that it was
           developed to monitor the Secretary of Defense's priority areas or
           that it was a programming and budgeting tool.

           Implementation of the risk management framework has also been
           challenged by the lack of clear lines of authority and appropriate
           accountability. No single individual or organization has been
           given overarching leadership responsibilities, authority, or the
           accountability for achieving the framework's implementation.
           Instead, the responsibility for various tasks and performance
           measures have been spread among several organizations, including
           the Director, PA&E the Under Secretary of Defense for Personnel
           and Readiness (P&R); and the Under Secretary of Defense,
           Comptroller/Chief Financial Officer.

           We testified in April 2005 that as DOD embarks on large-scale
           change initiatives, the complexity and long-term nature of these
           initiatives require the development of an executive position
           capable of providing strong and sustained leadership-over a number
           of years and various administrations.30 For this reason, we have
           supported legislation to create a CMO at DOD to provide such
           sustained leadership.31 A CMO could also provide the leadership
           needed to successfully develop a risk-based and results-oriented
           management approach at DOD, such as the risk management framework.

           Accountability for implementation of the risk management framework
           also has been hindered by the absence of implementation goals and
           timelines with which to gauge progress. As we have previously
           reported, successful change management efforts use implementation
           goals and timelines to pinpoint performance shortfalls and gaps,
           suggest midcourse corrections, and build momentum by demonstrating
           progress.32 However, DOD's limited guidance on the risk management
           framework did not establish implementation goals and timelines,
           nor did it require that implementation goals and timelines be
           developed. According to knowledgeable DOD officials, DOD did not
           see the need for implementation goals or timelines because the
           framework was not meant to change processes or create new ones,
           but rather was a management tool to improve upon investment
           decision-making processes. Regardless of how DOD classifies the
           risk management framework, we have found that implementation goals
           and timelines are essential to any transformational change, such
           as that envisioned by the Secretary of Defense with the risk
           management framework, because of the number of years it can take
           to complete the change.33 Moreover, the absence of implementation
           goals and timelines makes it difficult to determine whether
           progress has been made in implementing the framework over the last
           2  1/2 years, and whether DOD's revisiting of the framework during
           the 2005 QDR represents an evolutionary progression or
           implementation delays.

           DOD faces a significant challenge integrating the risk management
           framework with decision support processes for planning,
           programming, and budgeting and with related reform initiatives
           into a coherent, unified management approach. The goal of both
           risk management and results-oriented management is to integrate
           the systematic concern for risk and performance into the usual
           cycle of agency decision making and implementation. DOD's
           challenge in meeting these goals is demonstrated by the number of
           initiatives, as shown in table 4, that DOD has put in place to
           improve investment decision making and manage performance results.
           For example, both capabilities planning and the risk management
           framework are to define risks and develop performance measures
           but, according to DOD officials, the department is still
           determining how to align capabilities planning with the risk
           management framework. Other initiatives, including GPRA and PART,
           are also to develop performance measures and DOD is still working
           on integrating these initiatives with the risk management
           framework and individual performance monitoring approaches of the
           services and other defense components into a single, integrated
           system. In December 2002, the Deputy Secretary of Defense issued a
           memorandum to correct this situation by requiring the alignment of
           the risk management framework and the President's Management
           Agenda with DOD's results-oriented management activities,
           including those associated with GPRA.

27GAO-05-325SP.

28Joint Defense Capabilities Study Team, Joint Defense Capabilities Study:
Final Report (Washington, D.C.: December 2003).

Lack of Sustained Leadership and Appropriate Accountability Has Challenged DOD's
Implementation of the Risk Management Framework

29GAO, Management Reform: Elements of Successful Improvement Initiatives,
GAO/T-GGD-00-26 (Washington, D.C.: Oct. 15, 1999).

Lack of Implementation Goals and Timelines Further Challenges DOD's
Implementation of Risk Management Framework

30GAO-05-520T and GAO-05-629T.

31S. 780, 109th Cong. S:1 (2005).

32GAO-03-669.

33GAO-03-669.

Integrating the Risk Management Framework with Decision Support Processes and
Related Reform Initiatives Is a Significant Challenge

Table 4: Select Initiatives to Improve Investment Decision Making

Initiative              Description                                        
Two-Year Planning,      In 2003, DOD implemented a 2-year cycle for its    
Programming, Budgeting, strategic planning, program development, and       
and Execution Process   resource determination process. DOD stated that    
(PPBE)                  this change was needed to integrate DOD's          
                           processes for strategic planning, identification   
                           of needs for military capabilities, systems        
                           development and acquisition, and program and       
                           budget development. During the second year of the  
                           biennial budget, DOD is to focus on budget         
                           execution and program performance.                 
Enhanced Planning       In fiscal year 2004, DOD initiated a reform of     
Process                 defense planning to make it more responsive and    
                           adaptive to the needs of senior decision makers.   
                           The process is to result in fiscally constrained   
                           guidance and priorities-for military forces,       
                           modernization, readiness and sustainability, and   
                           supporting business processes and infrastructure   
                           activities-for program development. The enhanced   
                           planning process is to integrate the outcomes of   
                           operational, enterprise, and capabilities planning 
                           efforts in a document called the Joint Programming 
                           Guidance. The Joint Programming Guidance is to     
                           provide a link between planning and programming,   
                           and it is to provide guidance to the DOD           
                           components for the development of their program    
                           proposals.                                         
Capabilities Planning   The 2001 QDR announced a defense strategy built    
                           around the concept of shifting to a                
                           "capabilities-based" approach to defense.          
                           According to the 2001 QDR, while DOD cannot know   
                           with confidence what nation, group of nations, or  
                           nonstate actor might pose a threat to U.S. vital   
                           interests, it is possible to anticipate the        
                           capabilities an adversary might employ.            
                           Capabilities planning is to provide a top-down,    
                           competitive approach to weigh options against      
                           resource constraints across a spectrum of          
                           challenges and to apportion risk against those     
                           challenges. It is also to enable risk assessments  
                           and trade-off decisions across DOD organizational  
                           stovepipes. The new concept stresses joint         
                           solutions to problems, requires the identification 
                           of risk trade-offs within and across mission       
                           areas, and treats uncertainty explicitly.          
Program/Budget          As part of the financial management enterprise     
Framework Initiative    initiatives of the Business Management             
                           Modernization Program, this initiative is to       
                           provide a foundation for a new program and budget  
                           data structure using a common language that        
                           enables senior level DOD decision makers to weigh  
                           options versus resource constraints across a       
                           spectrum of challenges. The framework is to        
                           consist of a number of related data transparency   
                           initiatives that span across all portions of the   
                           PPBE process, including creating department-level  
                           definitions for the four risk quadrants. One of    
                           the stated benefits is establishing an ability to  
                           view programs and resources based on the risk      
                           management framework.                              
Joint Capabilities      A system for the Joint Staff to assess gaps in     
Integration and         military joint warfighting capabilities and        
Development System      recommend solutions to resolve those gaps. This    
(JCIDS)                 system is replacing DOD's requirements-generation  
                           process for major acquisitions in an effort to     
                           shift the focus to a more capabilities-based       
                           approach for determining joint warfighting needs   
                           rather than a threat-based approach focused on     
                           individual systems and platforms. Under this       
                           system, boards comprised of high-level DOD         
                           civilians and military officials are to identify   
                           future capabilities needed around key functional   
                           concepts and areas, such as command and control,   
                           force application, and battlespace awareness, and  
                           to make trade-offs among air, space, land, and sea 
                           platforms in doing so.                             
President's Management  The President's Management Agenda contains five    
Agenda                  initiatives aimed at improving federal agency      
                           management and performance: (1) strategic human    
                           capital management, (2) competitive sourcing, (3)  
                           improved financial performance, (4) expand         
                           electronic government, and (5) budget and          
                           performance integration. The President cited our   
                           work on high-risk areas and major management       
                           challenges in developing his initiatives, and      
                           implementation of the agenda has reinforced the    
                           need to focus agencies' efforts on achieving key   
                           management and performance improvements.           
Budget and Performance  The budget and performance integration initiatives 
Integration             of the President's Management Agenda include       
                           elements such as the PART used to review programs, 
                           an emphasis on improving outcome measures, and     
                           improving monitoring of program performance. PART  
                           is the central element in the performance          
                           budgeting piece of the President's Management      
                           Agenda. PART builds on GPRA by actively promoting  
                           the use of results-oriented information to assess  
                           programs in the budget.                            

Source: GAO analysis.

We note that these reform initiatives address key business processes
within the department and that we have placed DOD's overall business
transformation on our list of federal programs and activities at high risk
of waste, fraud, abuse, and mismanagement.34

34GAO-05-207.

The Under Secretary of Defense for Acquisition, Technology and Logistics
indicated that DOD plans to address the challenge associated with the
integration of DOD's planning, resourcing, and execution processes and
initiatives, including the risk management framework. The Under Secretary
stated that one task of the ongoing 2005 QDR was "strategic process
integration." The Under Secretary also stated that the department is
planning to provide a roadmap with performance goals and timelines on how
it will implement initiatives to improve strategic process integration.
This roadmap is to be submitted with the 2005 QDR report to Congress in
early 2006 with the fiscal year 2007 budget.

                                  Conclusions

DOD has made some progress in implementing the risk management framework,
including establishing risk quadrants and performance goals. However, more
work will be required for DOD to be able to put in place a management
tool, such as the risk management framework, to strategically balance the
allocation of resources across the spectrum of its investment priorities
against risk over time and to monitor performance. The development of
performance measures that clearly demonstrate results and that are
cascaded down throughout the department would enable DOD to provide a
clear roadmap of how its activities at all levels contribute to meeting
its strategic goals and would assist the department in aligning the core
processes and resources of its four military services and multiple defense
agencies to better support a departmental or joint approach to national
security. Furthermore, the risk management framework cannot be fully
implemented until its performance goals are clearly linked to DOD's
strategic planning goals. Unless a cause and effect relationship can be
demonstrated between the department's performance measures and strategic
goals, the framework's usefulness as a tool for monitoring DOD's execution
of its strategic plan and identifying performance goals will be severely
restricted, if not eliminated. Furthermore, the fiscal year 2006 budget
submission does not provide sufficient information on how DOD identified
or assessed departmental risks to establish DOD-wide investment
priorities; thus, the linkages between the framework and the budget are
unclear. Without better measures, clear linkages, and greater
transparency, DOD will be unable to fully measure progress in achieving
strategic goals or demonstrate to Congress and others how it considered
risks and made trade-off decisions, balancing needs and costs for weapon
programs and other investment priorities.

The efforts of DOD's senior leadership to establish a risk-based and
results-oriented management approach have been impeded by some key
challenges. The lack of sustained leadership and clear lines of
accountability has hampered implementation of the risk management
framework and the establishment and achievement of implementation goals
and timelines. Strong and sustained leadership could enable DOD to
overcome resistance to change that exists in a department as massive and
complex as DOD. In addition, the establishment of implementation goals and
timelines could enable DOD to determine what progress has been made in
implementing the risk management framework. Furthermore, the successful
integration of the risk management framework into DOD's investment
decision-making processes, including recent reform initiatives, could
assist DOD in its overall transformation efforts. Until DOD develops a
risk-based and results-oriented management approach for making investment
decisions, it will likely continue to experience a mismatch between
programs and budgets, and the proportional, rather than strategic,
allocation of resources to the services.

                      Recommendations for Executive Action

To address the challenges associated with implementing the risk management
framework, or a similar risk-based management approach, we recommend that
the Secretary of Defense take the following four actions:

           o  develop or refine department-level performance measures so that
           they clearly demonstrate performance results and cascade those
           measures down throughout the department,
           o  assign clear leadership with accountability and authority to
           implement and sustain the risk management framework,
           o  develop implementation goals and timelines, and
           o  demonstrate the integration of the risk management framework
           with DOD's decision support processes and related reform
           initiatives to improve investment decision making and manage
           performance results.

           In written comments on a draft of this report, DOD partially
           concurred with our four recommendations. DOD's written comments
           are reprinted in their entirety in appendix II. DOD also provided
           technical comments, which we incorporated as appropriate.

           DOD partially concurred with our first recommendation. DOD stated
           that it concurred with our recommendation that the Secretary of
           Defense refine department-level performance measures so that they
           clearly demonstrate results, but that it did not concur with the
           notion that effectively cascading the risk management framework
           has been inhibited by the current suite of performance measures.
           DOD noted that that a number of defense components-including the
           Army, DOD Comptroller, the Defense Logistics Agency, and the
           Defense Information Systems Agency-have successfully cascaded
           departmentwide strategic goals and implemented frameworks to
           measure their organization's performance. DOD also believes that
           empowering the leadership at the component level to develop
           measures, while ensuring strategic alignment, is the most
           effective way of encouraging performance management and increasing
           its utility. In our report, we acknowledge that DOD has taken
           positive steps toward developing a performance monitoring system
           and cascading the framework's goals and measures to defense
           components. However, our recommendation addresses limitations in
           those measures that currently hinder DOD's ability to use the risk
           management framework as a management tool for aligning the
           components' performance goals and measures with the risk
           management framework, or for strategic balancing investment
           decisions across the risk quadrants. For example, the majority of
           the risk management framework's measures are activity measures, or
           initiatives, that do not monitor a specific annual performance
           target, nor do these measures provide sufficient information to
           determine whether the activity is on schedule or contributes to
           enhancing the department's overall performance. Finally, our
           recommendation is not intended to suggest that DOD not empower the
           components to develop performance measures, but rather that DOD
           establish a clear hierarchy of goals and measures that provide
           straightforward roadmaps to demonstrate how the components'
           activities contribute to meeting DOD's strategic goals.

           DOD partially concurred with our second recommendation that the
           Secretary of Defense assign clear leadership with accountability
           and authority to implement and sustain the risk management
           framework. DOD stated that, although it agrees that such
           leadership is key to any successful performance management system,
           the department's senior executives provide sufficient leadership
           and accountability for implementing and sustaining the risk
           management framework. DOD also stated that it did not agree that a
           new organization or bureaucratic structure is needed to ensure
           successful implementation and sustainment of the risk management
           framework. We agree that DOD has assigned specific roles and
           responsibilities for goals and measures associated with the risk
           management framework to various high-level DOD officials. However,
           we based our recommendation on the fact that no single individual,
           with appropriate authority, was held responsible for ensuring that
           the risk management framework was implemented across the
           department. Further, our recommendation does not propose that DOD
           set up a new organization or bureaucratic structure, but, as
           stated in this report, we continue to believe that one way to
           provide strong and sustained leadership for change initiatives,
           such as the risk management framework, over a number of years and
           various administrations is to legislatively establish a CMO.

           In partially concurring with our third recommendation to develop
           implementation goals and timelines, DOD agreed that tracking
           progress in implementing the risk management framework is a good
           management practice. DOD stated that it has established goals and
           timelines for the risk management framework that are unique to the
           individual metrics, or measures, and that because the risk
           management framework continually evolves over time, new metrics
           will be developed while others may be retired. As we stated in the
           report, successful change management efforts use implementation
           goals-such as, for example, linking the risk management framework
           to the budget-and timelines for meeting those goals, to pinpoint
           shortfalls and gaps, suggest midcourse corrections, and build
           momentum by demonstrating progress. Therefore, while DOD may
           continually refine the individual goals and measures associated
           with the framework's risk quadrants, we believe that goals and
           timelines for the overall implementation of the framework across
           the department are essential for keeping this reform initiative on
           track.

           DOD partially concurred with our fourth recommendation that the
           Secretary of Defense demonstrate the integration of the risk
           management framework with DOD's decision support processes and
           related reform initiatives to improve investment decision making
           and manage performance results. DOD stated that the department is
           currently studying ways to further integrate the risk management
           framework with other decision support processes, but no single
           framework or decision model can provide all the necessary
           information or flexibility needed by the Secretary of Defense and
           his senior leadership team. We recognize that DOD's senior
           leadership needs reliable information from a variety of sources
           and flexibility to make decisions among alternative actions or
           solutions. However, if the risk management framework is to
           successfully serve as a management tool to assist decision makers
           in formulating top-down strategy, balancing investment priorities
           against risk over time, measuring near- and midterm outputs
           against strategic goals, and focusing on actual performance
           results-as intended by DOD's senior leadership-it is crucial that
           it be successfully integrated with DOD's investment
           decision-making processes, including recent reform initiatives.

           We are sending copies of this report to interested congressional
           committees; the Secretaries of Defense, Army, Navy, and Air Force;
           the Commandant of the Marine Corps; and the Director, Office of
           Management and Budget. We will also make copies available to
           others upon request. In addition, the report will be available at
           no charge on GAO's Web site at http://www/gao.gov.

           If you or your staff have any questions about this report, please
           contact me at (202) 512-9619 or [email protected]. Contact points
           for our offices of Congressional Relations and Public Affairs may
           be found on the last page of this report. GAO staff who made major
           contributions to this report are listed in appendix III.

           Sharon L. Pickup Director, Defense Capabilities and Management

           To assess to what extent the Department of Defense (DOD) has
           implemented the risk management framework, we obtained and
           analyzed DOD directives, briefings, and other documents that
           described the risk management framework's purpose, implementation
           status, and performance measures. We also obtained and analyzed
           DOD's 2001 Quadrennial Defense Review and annual strategic
           planning and budget documents. Moreover, we interviewed
           knowledgeable DOD and service officials involved with the
           implementation of the risk management framework. Specifically, we
           obtained testimonial evidence from officials representing the
           Office of the Secretary of Defense (OSD) offices-such as Program
           Analysis and Evaluation; Comptroller; Policy; Acquisition,
           Technology and Logistics; and Personnel and Readiness-the Joint
           Staff, the military services, and the Defense Business Board. To
           identify key risk-based and results-oriented management
           principles, we reviewed our prior reports and other relevant
           literature, including information on the balanced scorecard
           concept. For example, we identified characteristics of
           results-oriented performance measures. These characteristics
           focused on performance measures that are (1) designed to
           demonstrate results by providing information on how well the
           organization is achieving its goals; (2) limited to a vital few,
           and balanced across priorities; and (3) used by management to
           improve performance. As another example, risk-based and
           results-oriented management principles indicate that leading
           organizations seek to establish clear hierarchies of goals and
           measures that cascade down so that subordinate units have
           straightforward roadmaps to demonstrate how their activities
           contribute to meeting the organization's strategy. We
           systematically analyzed and compared the risk management
           framework's department-level performance measures with these
           characteristics. However, we did not validate the procedures that
           DOD has in place to ascertain the reliability of the data used to
           support the performance measures. Regarding strategic planning,
           these principles focused on (1) establishing clear linkages among
           strategic planning goals, resources, performance goals and
           measures and (2) integrating the consideration of risk into the
           usual cycle of agency decision making and implementation. While
           these principles do not cover all attributes associated with
           risk-based and results-oriented management approaches, we believe
           that they are the most important ones for assessing DOD's progress
           in implementing the risk management framework.

           To identify the most significant challenges, we reviewed our
           previous work on change management principles. We then compared
           DOD's implementation of the risk management framework to sound
           change management principles and interviewed knowledgeable DOD
           officials about the challenges that faced the department in
           implementing the risk management framework. In addition, we
           reviewed our previous work to determine to what extent
           deficiencies in DOD's overall business transformation efforts
           might influence the implementation of the risk management
           framework.

           Our work was performed from October 2004 through September 2005 in
           accordance with generally accepted government auditing standards.

           Sharon Pickup, (202) 512-9619 or [email protected]

           In addition to the contact named above, David Moser, Assistant
           Director; Donna Byers; Gina Flacco; and Renee S. Brown made key
           contributions to this report.

           The Government Accountability Office, the audit, evaluation and
           investigative arm of Congress, exists to support Congress in
           meeting its constitutional responsibilities and to help improve
           the performance and accountability of the federal government for
           the American people. GAO examines the use of public funds;
           evaluates federal programs and policies; and provides analyses,
           recommendations, and other assistance to help Congress make
           informed oversight, policy, and funding decisions. GAO's
           commitment to good government is reflected in its core values of
           accountability, integrity, and reliability.

           The fastest and easiest way to obtain copies of GAO documents at
           no cost is through GAO's Web site (www.gao.gov). Each weekday, GAO
           posts newly released reports, testimony, and correspondence on its
           Web site. To have GAO e-mail you a list of newly posted products
           every afternoon, go to www.gao.gov and select "Subscribe to
           Updates."

           The first copy of each printed report is free. Additional copies
           are $2 each. A check or money order should be made out to the
           Superintendent of Documents. GAO also accepts VISA and Mastercard.
           Orders for 100 or more copies mailed to a single address are
           discounted 25 percent. Orders should be sent to:

           U.S. Government Accountability Office 441 G Street NW, Room LM
           Washington, D.C. 20548

           To order by Phone: Voice: (202) 512-6000 TDD: (202) 512-2537 Fax:
           (202) 512-6061

           Contact:

           Web site: www.gao.gov/fraudnet/fraudnet.htm E-mail:
           [email protected] Automated answering system: (800) 424-5454 or
           (202) 512-7470

           Gloria Jarmon, Managing Director, [email protected] (202) 512-4400
           U.S. Government Accountability Office, 441 G Street NW, Room 7125
           Washington, D.C. 20548

           Paul Anderson, Managing Director, [email protected] (202)
           512-4800 U.S. Government Accountability Office, 441 G Street NW,
           Room 7149 Washington, D.C. 20548

                       Agency Comments and Our Evaluation

Appendix I: Scope and Methodology  Appendix I: Scope and Methodology

Appendix II: Comments from the Department of Defense  Appendix II:
Comments from the Department of Defense

Appendix III: GAOA  Appendix III: GAO Contact and Staff Acknowledgments

                                  GAO Contact

                                Acknowledgments

(350611)

GAO's Mission

Obtaining Copies of GAO Reports and Testimony

Order by Mail or Phone

To Report Fraud, Waste, and Abuse in Federal Programs

Congressional Relations

Public Affairs

www.gao.gov/cgi-bin/getrpt?GAO-06-13.

To view the full product, including the scope

and methodology, click on the link above.

For more information, contact Sharon Pickup at (202) 512-9619 or
[email protected].

Highlights of GAO-06-13, a report to the Subcommittee on Readiness and
Management Support, Committee on Armed Services, U.S. Senate

November 2005

DEFENSE MANAGEMENT

Additional Actions Needed to Enhance DOD's Risk-Based Approach for Making
Resource Decisions

The Department of Defense (DOD) is simultaneously conducting costly
military operations and transforming its forces and business practices
while it is also competing for resources in an increasingly constrained
fiscal environment. As a result, GAO has advocated that DOD adopt a
comprehensive threat or risk management approach as a framework for
decision making. In its 2001 strategic plan, the Quadrennial Defense
Review (QDR), DOD stated its intent to establish an approach-the risk
management framework-to balance priorities against risk over time and
monitor results against its strategic goals.

GAO was asked to (1) assess the extent to which DOD has implemented the
framework, including using it to make investment decisions, and

(2) identify the most significant challenges DOD faces in implementing the
framework, or a similar approach.

What GAO Recommends

GAO recommends that DOD take various actions to increase its chances of
successfully implementing a risk-based approach for investment decision
making, such as developing results-oriented measures and assigning clear
leadership with appropriate accountability and authority to implement the
framework. DOD partially concurred with our recommendations.

DOD has taken some positive steps to implement the framework, but
additional actions are needed before DOD can show real and sustainable
progress in using a risk-based and results-oriented approach to
strategically allocate resources across the spectrum of its investment
priorities. For example, DOD defined four risk areas, and developed
performance goals and department-level measures, but it needs to, among
other things, further develop and refine the measures so that they clearly
demonstrate results and provide a well-rounded depiction of departmental
performance. DOD's current strategic plan and goals also are not clearly
linked to the framework's performance goals and measures, and linkages
between the framework and budget are also unclear. While DOD officials
stated that risk was considered during the fiscal year 2006 budget cycle,
DOD's budget submission does not specifically discuss how DOD identified
or assessed risks to establish DOD-wide investment priorities. Without
better measures, clear linkages, and greater transparency, DOD will be
unable to fully measure progress in achieving strategic goals or
demonstrate to Congress and others how it considered risks, and made
trade-off decisions, balancing needs and costs for weapon programs and
other investment priorities.

DOD's Risk Management Framework

Force Management Risk                   Operational Risk                   
                                                                              
Definition: Challenge of sustaining     Definition: Challenge of deterring 
personnel, infrastructure, and          or defeating near-term threats     
equipment                               
Future Challenges Risk                  Institutional Risk                 
                                                                              
Definition: Challenge of dissuading,    Definition: Challenge of improving 
deterring, defeating longer-term        efficiency (includes financial     
threats                                 management)                        

Source: DOD.

DOD faces four challenges that have affected the implementation of the
framework. First, DOD's organizational culture resists department-level
approaches to priority setting and investment decisions. Second, sustained
leadership, adequate transparency, and appropriate accountability are
lacking. Further, no one individual or office has been assigned overall
responsibility or sufficient authority for the framework's implementation.
DOD also has not developed implementation goals or timelines with which to
establish accountability, or measure progress. Finally, integrating the
risk management framework with decision support processes and related
reform initiatives into a coherent, unified management approach for the
department is a challenge that DOD plans to address during the 2005 QDR.
However, GAO has concerns about DOD's ability to follow through on this
integration, because of its limited success in implementing other
management reforms. Unless DOD successfully addresses these challenges and
effectively implements the framework, or a similar approach, it will
likely continue to experience (1) a mismatch between programs and budgets,
and (2) a proportional, rather than strategic, allocation of resources to
the services.
*** End of document. ***