-------------------------Indexing Terms------------------------- 
REPORTNUM:   GAO-02-856		

TITLE:     Military Transformation: Actions Needed to Better Manage 
DOD's Joint Experimentation Program

DATE:   08/29/2002
				                                                                         
----------------------------------------------------------------- 

******************************************************************
** This file contains an ASCII representation of the text of a  **
** GAO Product.                                                 **
**                                                              **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced.  Tables are included, but    **
** may not resemble those in the printed version.               **
**                                                              **
** Please see the PDF (Portable Document Format) file, when     **
** available, for a complete electronic file of the printed     **
** document's contents.                                         **
**                                                              **
******************************************************************
GAO-02-856

   Report to Congressional Committees

   United States General Accounting Office

   GAO

   August 2002 MILITARY TRANSFORMATION

   Actions Needed to Better Manage DOD*s Joint Experimentation Program

   GAO- 02- 856

   Page i GAO- 02- 856 Military Transformation Letter 1

   Results in Brief 2 Background 4 Participation in Joint Experimentation
   Continues to Evolve 7 No Recommendations for Change Have Been Approved or

   Implemented 16 Some Key Management Elements Are Missing 19 Conclusions 24
   Recommendations for Executive Action 25 Agency Comments and Our Evaluation
   26 Scope and Methodology 26

   Appendix I Chronology of Key Events Important to Joint Experimentation 29

   Appendix II Additional Information on Joint Concept Development and
   Experimentation Activities 31

   Appendix III Comments from the Department of Defense 33

   Appendix IV Organizations and Offices Contacted 35

   Appendix V GAO Contacts and Staff Acknowledgments 38

   Tables

   Table 1: Organizations Involved in Joint Experimentation 8 Table 2: First
   Three Recommendations Submitted by Joint Forces

   Command 17 Table 3: Types of Activities and Experiments Associated with
   Joint

   Experimentation 32 Contents

   Page ii GAO- 02- 856 Military Transformation Figures

   Figure 1: Joint Concept Development and Experimentation Process 6 Figure
   2: Involvement of Organizations in Preparing Joint

   Experimentation Guidance and Campaign Plan 12 Figure 3: Joint Concept
   Development and Experimentation

   Activities 31

   Page 1 GAO- 02- 856 Military Transformation

   August 29, 2002 The Honorable Carl Levin Chairman The Honorable John W.
   Warner Ranking Minority Member Committee on Armed Services United States
   Senate

   The Honorable Bob Stump Chairman The Honorable Ike Skelton Ranking
   Minority Member Committee on Armed Services House of Representatives

   The Department of Defense (DOD) considers the transformation 1 of the U.
   S. military a strategic imperative to meet the security challenges of the
   new century. In October 1998, DOD established a joint concept development
   and experimentation program to provide the engine of change for this
   transformation. The U. S. Joint Forces Command, the executive agent of the
   program, carries out experiments* with support from the military services,
   the combatant commands, and other DOD organizations* to explore and
   analyze new joint operational and organizational concepts. The experiments
   can include studies and analyses, conferences, workshops, seminars, war
   games, and simulated or live field experiments.

   On the basis of these experiments, the Joint Forces Command recommends
   changes aimed at achieving significant improvements in joint

   1 DOD states that transformation of the military *results from the
   exploitation of new approaches to operational concepts and capabilities,
   the use of old and new technologies, and new forms of organization that
   more effectively anticipate new or still emerging strategic and
   operational challenges and opportunities and that render previous methods
   of conducting war obsolete or subordinate.*

   United States General Accounting Office Washington, DC 20548

   Page 2 GAO- 02- 856 Military Transformation

   capabilities. The Joint Requirements Oversight Council 2 reviews the
   Command*s recommendations, and the Chairman of the Joint Chiefs of Staff
   is responsible for approving them. The Council also oversees the
   implementation of approved recommendations. The cumulative changes are
   expected to contribute to the way U. S. military forces are transformed.
   In November 2001, DOD also created an Office of Force Transformation to
   foster innovation and experimentation and assist the Secretary of Defense
   to develop guidance and identify strategies and policies for
   transformation.

   Because of the central role DOD*s joint experimentation program plays in
   military transformation, we reviewed related activities to determine
   whether changes are needed to improve the program*s ability to focus on
   future joint operations, military transformation, and the rapid
   integration of results. Specifically, we assessed the extent to which (1)
   the Joint Forces Command obtains participation from all relevant sources
   in developing and conducting its experimentation activities; (2) the
   recommendations flowing from the joint experimentation process have been
   approved and implemented; and (3) the necessary management elements are in
   place to foster program success. We are providing this report to you
   because of your oversight responsibilities and interest in military
   transformation issues.

   In the nearly 4 years since becoming the executive agent for joint concept
   development and experimentation, the Joint Forces Command has increased
   the participation of key DOD stakeholders* the military services, the
   combatant commands, and other organizations and agencies* in its
   experimentation activities. The Command has also expanded the
   participation of federal agencies and departments, academia, the private
   sector, and some foreign allies. The Command uses many different ways to
   obtain input* including meetings, seminars, workshops, video conferencing,
   E- mail, and the Internet* and integrates the results of military
   operations, technology efforts, and other DOD organizations* experiments
   into its activities. Although stakeholders

   2 The Joint Requirements Oversight Council is an advisory council that
   assists the Chairman of the Joint Chiefs of Staff in identifying and
   assessing the priorities for joint military requirements to achieve
   current and future military capabilities. Chaired by the Vice Chairman of
   the Joint Chiefs of Staff, the Council is comprised of a senior officer
   from each of the military services. Representatives from other DOD
   entities, such as the combatant commands and the Joint Staff, serve in an
   advisory role to the Council. Results in Brief

   Page 3 GAO- 02- 856 Military Transformation

   determine the extent of their involvement in joint experimentation
   activities, many more organizations are in fact involved in the planning
   and execution of this year*s major field experiment than there were in a
   similar experiment 2 years ago. Most DOD officials whom we spoke with were
   generally satisfied with the way the Command reached out to obtain their
   input, but many also believe more frequent contacts with the Joint Forces
   Command would improve communications. Some believe that the Command could
   do more to expand the participation of non- DOD agencies and departments
   and allies and coalition partners, and the Command is planning initiatives
   to do so, though this outreach would involve broader agency- resource and
   national- security considerations.

   No recommendations flowing from joint experimentation have been approved
   or implemented. Although the Joint Forces Command issued three
   recommendations nearly a year ago, they were not approved by the Joint
   Requirements Oversight Council because of confusion among the Joint Staff
   and the Joint Forces Command about a proposed change in guidance that
   required additional data (on costs and timelines) be included when
   submitting these recommendations. As a result, it is not clear when these
   recommendations will contribute to military transformation. The Command
   plans to re- submit the recommendations with the required data later this
   year. However, at the time we concluded our review, the official guidance
   on what information should accompany joint experimentation recommendations
   had not been approved and issued. In addition, although it is too early to
   assess how DOD integrates the results of its experimentation into its
   plans, programs, and budgets, several DOD officials expressed concern that
   the resource allocation process may be too slow to provide rapid and
   timely funding for the implementation of new concepts emerging from joint
   experimentation and that other, more rapid resource mechanisms may be
   needed.

   While DOD has been providing more specific and clearer guidance for joint
   experimentation, DOD and the Joint Forces Command are missing some key
   management elements that are generally considered necessary for successful
   program management. DOD and the Joint Forces Command thus lack the
   following very important management tools needed to carry out their joint
   experimentation responsibilities:

    It is not yet clear what role the Office of Force Transformation will
   have in joint experimentation and how it will coordinate its activities
   with other DOD stakeholders. As a result, the Secretary of Defense may not
   be effectively using this new office to interact with stakeholders to
   foster and

   Page 4 GAO- 02- 856 Military Transformation

   monitor joint experimentation as a means to advance DOD*s transformation
   agenda.

    DOD has not developed meaningful performance measures to assess joint
   experimentation*s contribution to the advancement of military
   transformation. Therefore, DOD cannot gauge the progress of joint
   experimentation efforts and of the program*s overall contribution to
   transformation.

    The Joint Forces Command does not have the strategic- planning tools (a
   strategic plan with associated performance plans and reports) that would
   allow it to clearly define what it is seeking to accomplish, identify what
   strategy it will use to achieve desired results, and finally measure how
   well it is succeeding in meeting results- oriented goals and objectives.

   We are making recommendations designed to improve the processes and the
   management oversight and accountability for the joint concept development
   and experimentation program and to clarify the role of the Office of Force
   Transformation. In its comments on a draft of this report, DOD agreed with
   our recommendations and stated that our findings provide a factual summary
   of the history and progress of joint experimentation and identify the
   critical weaknesses currently being addressed by DOD. It expects that a
   forthcoming Transformation Planning Guidance or subsequent guidance
   documents will address the issues and problems with the joint
   experimentation program identified in our report.

   The Joint Forces Command, in coordination with the Joint Staff, 3 the
   services, and other combatant commands and DOD agencies, is responsible
   for creating and exploring new joint war- fighting concepts, as well as
   for planning, designing, conducting, and assessing a program of joint
   experimentation. The Command executed its second large- scale field
   experiment, Millennium Challenge 2002, 4 this year, and it plans another
   one in 2004 and others every third year thereafter. These experiments are
   intended to examine how well the concepts previously explored by the

   3 The staff that assists the Chairman of the Joint Chiefs of Staff in
   executing the Chairman*s responsibilities. 4 Millennium Challenge 2002
   focuses on how the U. S. military can use its current and near- term
   capabilities differently in this decade. It will involve the U. S. Special
   Operations Command and forces from all the services. Another major field
   experiment* Olympic Challenge* is planned for 2004. Background

   Page 5 GAO- 02- 856 Military Transformation

   Command in smaller venues will work when applied with the emerging
   concepts being developed by the services and other combatant commands. For
   example, Millennium Challenge 2002 tested how well U. S. forces fared
   against a regional power with a sizable conventional military force and so
   called *anti- access* capabilities* which can include advanced surface-
   to- air missiles, antiship missiles and mines, and chemical and biological
   weapons* and validated the results of earlier experiments to develop the
   Command*s *rapid decisive* operations concept. The aim of the experiment
   was to come up with changes that can be made during the current decade.
   (App. I provides a chronology of major events important to joint
   experimentation.)

   Over the next several years, the Command*s experimentation will focus
   primarily on two concepts: one to develop a standing joint force
   headquarters to improve joint command and control; another to conduct more
   effective joint operations through *rapid decisive* operations. In
   November 2001, the Chairman of the Joint Chiefs of Staff 5 directed that
   the Command make development of the prototype headquarters its highest
   near- term priority. 6 Additionally, the Command will develop a number of
   other concepts aimed at specialized issues or operational problems to
   support the two primary concepts.

   Joint experimentation is a continuous process that begins with the
   development of new operational and organizational concepts that have the
   potential to improve significantly joint operations (see fig. 1). The
   Joint Forces Command identifies new joint concepts including those
   developed by other DOD organizations (such as the Joint Staff, services,
   and combatant commands) and the private sector and tests them in
   experiments that range from simple (workshops, seminars, war games, and
   simulations) to complex (large- scale virtual simulations and *live*

   field experiments). Appendix II provides additional information on joint
   experimentation program activities.

   5 The Unified Command Plan, April 30, 2002, assigns the Chairman of the
   Joint Chiefs of Staff responsibility for providing overarching guidance on
   joint experimentation. 6 The Command is directed to develop this prototype
   headquarters by 2004 and make it available for implementation by the
   regional combatant commands during fiscal year 2005. The headquarters will
   provide uniform, standard operating procedures, tactics, techniques, and
   technical system requirements, with the ability to move expertise among
   commands.

   Page 6 GAO- 02- 856 Military Transformation

   Figure 1: Joint Concept Development and Experimentation Process

   Source: Joint Forces Command.

   After analyzing experimentation data, the Command prepares and submits
   recommendations to the Joint Requirements Oversight Council for review
   and, ultimately, to the Chairman of the Joint Chiefs of Staff for
   approval. 7 Before submitting them to the Council, however, the Command
   submits its recommendations to the Joint Staff for preliminary review and
   coordination. The recommendations are distributed for review and comment
   to the Joint Staff directorates, the military services, the combatant
   commands, and other DOD and federal government organizations. The Council
   then reviews the recommendations and advises the Chairman of the Joint
   Chiefs of Staff on whether they should be approved. The changes, if
   approved, provide the basis for pursuing the capabilities needed to
   implement a specific operational concept. The Council is also responsible
   for overseeing the implementation of the recommendations, but it can
   designate an executive agent, such as the Joint Forces Command, to do so.
   The Council (or its designated executive

   7 We recently reported on the status of actions taken by the Chairman of
   the Joint Chiefs of Staff to improve the Joint Requirements Oversight
   Council*s oversight and the joint requirements process, including
   establishing procedures for submitting recommendations stemming from joint
   experimentation to the Council. See U. S. General Accounting Office,

   Defense Plans: Status of DOD*s Efforts to Improve Its Joint Warfighting
   Requirements Process, GAO- 02- 100R (Washington, D. C.: Nov. 9, 2001).

   Page 7 GAO- 02- 856 Military Transformation

   agent) is responsible for obtaining the resources needed to implement the
   recommendations through DOD*s Planning, Programming, and Budgeting System.
   8 The Council also assists the Chairman, in coordination with the
   combatant commands, the services, and other DOD organizations, to identify
   and assess joint requirements and priorities for current and future
   military capabilities. The Council considers requirements (and any
   proposed changes) for joint capabilities across doctrine, organizations,
   training, materiel, leadership and education, personnel, and facilities.

   The Department of the Navy*s budget 9 provides funding to the Joint Forces
   Command for joint experimentation and other Command missions. In fiscal
   year 2002, the Command received from the Navy about $103 million for its
   joint concept development and experimentation program, and it planned to
   spend about half of this amount for Millennium Challenge 2002. The Command
   has requested that the Navy provide about $98 million for the program in
   fiscal year 2003. The Command also provides some funds to the services,
   the combatant commands, and other DOD organizations for efforts that
   support its program activities. However, the services fund the operations
   and support costs of forces participating in joint experimentation. Also,
   the individual experimentation efforts of the services and the combatant
   commands are funded from within their own budgets.

   Since it first began joint experimentation, the Joint Forces Command has
   broadened and deepened the inclusion of other DOD organizations, federal
   agencies and departments, the private sector, and allies and coalition
   partners in its process for capturing and identifying new joint ideas and
   innovations. Organizations participating in joint experimentation are
   generally satisfied with current opportunities for their ideas to be
   considered, and many have increased their participation in the program.
   However, the participation of different stakeholders* the extent of which

   8 The Planning, Programming, and Budgeting System is designed to assist
   the Secretary of Defense in making choices about the allocation of
   resources among a number of competing or possible programs and
   alternatives to accomplish specific national security objectives. The
   objective of the system is to provide the optimal mix of forces,
   equipment, and support that can be achieved within fiscal constraints.

   9 DOD Directive 5100. 3, Support of the Headquarters of Combatant and
   Subordinate Joint Commands, Nov. 15, 1999, designates the Navy as the
   executive agent responsible for arranging for the program- funding of the
   Joint Forces Command*s activities, including the joint experimentation
   program. Participation in Joint

   Experimentation Continues to Evolve

   Page 8 GAO- 02- 856 Military Transformation

   is determined by the stakeholder* varies considerably and some would like
   more visits and contacts with the Command. The Command is planning
   initiatives to increase stakeholder participation in the future,
   particularly for federal agencies and departments and key allies, but this
   increase will depend on agency- resource and national- security
   considerations.

   As the program gradually evolved, the Joint Forces Command solidified a
   process to involve the military services, the combatant commands, and
   other DOD organizations in the planning and execution of its joint
   experimentation activities. Because future joint operations will involve
   diplomatic, information, and economic actions, as well as military
   operations, many DOD, federal, and private organizations and governments
   participate and provide input into the joint experimentation program (see
   table 1). The Joint Forces Command functions as a facilitator to solicit
   and coordinate the involvement of these organizations and incorporate
   their input, as appropriate, into concept development and experimentation
   activities.

   Table 1: Organizations Involved in Joint Experimentation Organizational
   area Examples of organizations and government

   entities

   DOD headquarters and support organizations

    Office of the Secretary of Defense

    Chairman of the Joint Chiefs of Staff and Joint Staff

    Office of Force Transformation

    Joint Advanced Warfighting Program

    Defense Intelligence Agency

    National Defense University Military services  Army

    Navy

    Air Force

    Marine Corps Combatant commands  U. S. Central Command

    U. S. European Command

    U. S. Pacific Command

    U. S. Southern Command

    U. S. Special Operations Command

    U. S. Transportation Command

    U. S. Space Command

    U. S. Strategic Command Federal agencies and departments

    Department of State

    Department of Justice

    Central Intelligence Agency Private sector  Academia

    Think tank groups

    Industry

   Participation Has Grown and Continues to Expand

   Page 9 GAO- 02- 856 Military Transformation

   Organizational area Examples of organizations and government entities

   Allies and potential coalition partners

    Some North Atlantic Treaty Organization countries

    Australia Source: GAO interviews and DOD data.

   Because the stakeholders determine the extent of their participation in
   the program, it can vary considerably. However, Joint Forces Command
   officials stated that participation by the services, the combatant
   commands, and other DOD organizations has grown steadily since the program
   was created and continues to grow, as participants become increasingly
   aware of the strong emphasis that DOD leaders are placing on
   experimentation. For example, in contrast to the first field experiment in
   2000, which had limited involvement by the services, this year*s
   Millennium Challenge has seen the services more actively involved in early
   planning, and their individual experiments better coordinated and
   integrated into the field experiment. Our comparison of participation in
   the Command*s major field experiment in 2000 with plans for this year*s
   experiment found a significant increase in the diversity and number of
   participating organizations and in the number of concepts and initiatives
   proposed by these organizations. For example, the total number of
   organizations participating in Millennium Challenge 2002 more than doubled
   from the prior experiment in 2000 (from 12 to 29 organizations), and the
   total number of service initiatives increased from 4 to 29.

   The Command provides several ways for organizations to participate and
   provide inputs: they can review program plans and strategies; attend
   meetings, seminars, and workshops; take part in experimentation
   activities; and use various communication tools such as E- mail, Internet,
   and video conferencing. Additionally, the Command obtains input from the
   various experimentation and research and development organizations of the
   military services and of some combatant commands and DOD organizations. 10
   The Command also considers the results of Advanced

   10 Participating organizations would include each of the services*
   experimentation programs, service battle labs and supporting research and
   development organizations, DOD organizations such as Defense Advanced
   Research Projects Agency and Joint Advanced Warfighting Program, and
   federally funded research and development organizations such as the
   Institute for Defense Analyses, the RAND Corporation, and the Center for
   Naval Analysis.

   Page 10 GAO- 02- 856 Military Transformation

   Concept Technology Demonstrations efforts, 11 innovations, and recent
   military operations in developing its program. For example, as a result of
   its operational experiences in Kosovo, the U. S. European Command
   identified various joint capability shortfalls in its recent list of
   Command priorities as a means of guiding the Joint Forces Command in
   selecting focal areas and activities for experimentation. Further, the
   Command is taking steps to (1) align its experimentation activities with
   the schedules of major service and combatant command exercises and (2)
   adjust its program to allow for earlier consideration of new concepts
   proposed by the services and the combatant commands in the input process.
   These adjustments would improve synchronization of experiments with the
   availability of forces and the training schedules of the services and the
   combatant commands, allow for greater involvement of these entities in the
   process, and increase the likelihood that joint requirements are
   sufficiently considered early in the development of concepts.

   Participating organizations also provide input during the annual
   preparation of two key joint experimentation- program documents: the
   Chairman of the Joint Chiefs of Staff*s guidance on joint experimentation
   and the Joint Forces Command*s Joint Concept Development and
   Experimentation Campaign Plan 12 (see fig. 2). Each year the Chairman
   provides guidance to the Joint Forces Command to use in developing its
   Campaign Plan for joint concept development and experimentation. The basis
   for the Chairman*s guidance is derived from several sources, including
   strategy and planning documents, studies, and other assessments. 13
   Additionally, key DOD stakeholders, including the Chairman*s Joint
   Warfighting Capability Assessment teams 14 and the Joint

   11 DOD initiated the Advanced Concept Technology Demonstration program to
   determine the extent to which a given mature technology will improve
   military capabilities before entering the normal acquisition process and,
   by using mature technology, reduce the length of time to develop and
   acquire weapon systems.

   12 The Campaign Plan provides the structure for executing experimentation
   and outlines the Joint Forces Command*s objectives, methods, and events to
   achieve those objectives. The most recent Campaign Plan is: U. S. Joint
   Forces Command, The Joint Concept Development and Experimentation Campaign
   Plan Fiscal Years 2002- 2007 (Norfolk, Va.: 2002).

   13 Examples of these sources include the President*s National Security
   Strategy, the Secretary of Defense*s annual defense planning guidance, and
   the Chairman of the Joint Chief of Staff*s Joint Vision and National
   Military Strategy.

   14 Teams of DOD war- fighting and functional- area experts appraise
   various aspects of joint war fighting within their respective areas. The
   Chairman of the Joint Chiefs of Staff uses the teams* assessments to
   develop guidance and to support decision making.

   Page 11 GAO- 02- 856 Military Transformation

   Requirements Oversight Council, provide input to the Joint Staff to use in
   developing the Chairman*s guidance. The Joint Forces Command uses this
   guidance, with additional input from DOD stakeholders, in preparing its

   Campaign Plan, which is the primary vehicle for synchronizing its joint
   experimentation activities and coordinating resources. The Command also
   solicits and considers input for the Campaign Plan from some other federal
   agencies and departments, academia, private sector, and allies. After
   review and endorsement by the combatant commands, the services, and the
   Joint Requirements Oversight Council, the Chairman approves the

   Campaign Plan.

   Page 12 GAO- 02- 856 Military Transformation

   Figure 2: Involvement of Organizations in Preparing Joint Experimentation
   Guidance and Campaign Plan

   Source: Joint Staff and Joint Forces Command.

   Officials at the military services, the combatant commands, and other DOD
   organizations we talked with said they were generally satisfied with the
   opportunities for input provided by the Joint Forces Command. At the same
   time, DOD stakeholders have taken various actions to increase their
   participation. Some, however, would like more contacts and communication
   with the Command. The Command is responding with some initiatives.
   Stakeholders Generally

   Satisfied but Point to Some Possible Improvements

   Page 13 GAO- 02- 856 Military Transformation

   Each service, the Joint Staff, the U. S. Special Operations Command, the
   U. S. Space Command, as well as some DOD and federal agencies (such as the
   National Imagery and Mapping Agency and the National Security Agency) have
   assigned liaison officers at the Joint Forces Command. 15 However,
   officials at the Central, Pacific, and Southern Commands stated that their
   staffing levels currently do not allow them to devote personnel in this
   role. Combatant command officials indicated that the frequency and number
   of meetings, conferences, and other events held at the Joint Forces
   Command often make it difficult for their organizations to attend. The
   officials believe that as a result, the views and positions of their
   organizations are not always fully captured in some discussions and
   deliberations. Some of the combatant commands have or are planning to
   establish their own joint experimentation offices. Officials from the
   Pacific and Special Operations Commands stated that although their
   respective joint experimentation offices are largely focused on supporting
   their own experimentation efforts, the offices provide a cadre of staff
   who can better coordinate and participate more consistently in the Joint
   Forces Command*s joint experimentation program. For example, Pacific
   Command officials said that their own experimentation efforts to improve
   the command of joint operations over the past few years have contributed
   to joint experimentation by providing significant insights for the Joint
   Forces Command*s development of the standing joint- force headquarters
   concept. Central Command and Southern Command officials said their
   Commands have plans to establish similar offices soon.

   While satisfied with their participation and their ability to provide
   input into the program, officials at some combatant commands believe that
   a number of things could be done to improve the program, assuming
   resources are available. They believe that the Joint Forces Command could
   increase its visits to and participation in combatant- command activities.
   Some of the officials also believe that if the Joint Forces Command
   assigned liaison officers to their commands, the Command would obtain
   first- hand knowledge and a better appreciation of the various commands*
   individual requirements. These officials believe that such a presence at
   their commands would demonstrate the Joint Forces Command*s commitment to
   joint experimentation and would allow for interaction with staff
   throughout their commands. The Joint Forces Command does not favor doing
   this because of the cost and the difficulty

   15 Additionally, Germany, the United Kingdom, Canada, and Australia have
   each assigned a liaison officer at the Joint Forces Command.

   Page 14 GAO- 02- 856 Military Transformation

   in providing the staff necessary to fulfill this role. Officials at the
   Pacific, Central, and Southern Commands also believe that some level of
   funding should be provided to the combatant commands for their use in
   supporting individual command and the Joint Forces Command experimentation
   efforts. Combatant command officials stated that currently, funds from
   other command activities must be diverted to support these efforts.

   Out of concern about the need to improve communications and participation
   in joint experimentation planning, the Joint Forces Command is planning
   some initiatives such as the following:

    It plans to create a virtual planning- center site for joint
   experimentation on its Intranet to provide DOD stakeholders with easily
   accessible weekly updates to information on planned experiments;
   participants; goals and objectives; and ongoing experimentation by the
   Joint Forces Command, the services, the combatant commands, and DOD
   agencies. It plans to develop the requirements for the site during fall
   2002 and to initiate the project soon after.

    It established Project Alpha* a *think- tank* group* in early 2002 to
   provide another source of input and outputs. The project will interface
   with researchers throughout DOD, Department of Energy national
   laboratories, private industry, and academia to find cutting- edge
   technologies for inclusion in service and joint experimentation. This
   relationship will provide an opportunity for the Joint Forces Command to
   leverage the work of these organizations and similarly, for these
   organizations to gain a better understanding of and include their work in
   the joint experimentation program.

   As the joint experimentation program matured, participation by non- DOD
   federal agencies and departments gradually increased. Participation,
   however, depends upon the agencies* desire to be involved and their
   available resources. Lack of involvement could lead to missed
   opportunities. And participation by allies and coalition partners has been
   limited by security concerns.

   The Joint Forces Command*s input process allows individual federal
   agencies and departments, such as the Departments of State and Justice, to
   participate in joint experimentation events as they choose. Interagency
   participation is improving, according to Command officials. For example,
   federal agencies and departments are participating in Millennium Expanding
   Interagency

   and Foreign Participation in Experimentation Events Is Being Considered

   Page 15 GAO- 02- 856 Military Transformation

   Challenge 2002 to assist the Command in developing its standing jointforce
   headquarters concept.

   However, resource and staffing constraints prevent some agencies and
   departments from taking part in experiments. For example, according to a
   Joint Forces Command official, the Department of Transportation and the
   Central Intelligence Agency decided not to send representatives to
   Millennium Challenge 2002 because of staffing constraints. Not only could
   non- DOD agencies provide important insights and contributions to joint
   operations, but also some important opportunities could be missed if these
   agencies do not consistently participate in joint experimentation events.

   While federal agencies and departments are beginning to increase their
   role in joint experimentation, several service and combatant command
   officials we spoke with believe that greater involvement is needed because
   of the role these organizations are likely to have in future joint
   operations. For example, these non- DOD federal agencies and departments
   would provide support (economic, diplomatic, and information actions) to
   U. S. military forces in their conduct of operations aimed at defeating an
   adversary*s war- making capabilities* support that is critical to
   implementation of the Joint Forces Command*s rapid decisive operations
   concept.

   Several DOD (service, combatant command, Office of the Secretary of
   Defense, and other DOD organizations) officials we spoke with believe that
   the Joint Forces Command should explore ways to boost the participation
   and involvement of allies and coalition partners in joint experimentation.
   Joint Forces Command officials agree and believe that such cooperation
   would foster a better understanding of allied perspectives, allow the
   Command to leverage concept development work, expand available
   capabilities, and facilitate the development of multinational
   capabilities. The Command recently created a multinational concept-
   development and experimentation site on its Intranet to facilitate the
   involvement of allies and coalition partners in joint experimentation.
   However, some DOD officials believe that the Joint Forces Command should
   do more because future U. S. military operations will likely be conducted
   with other countries. The officials stress that other nations* military
   personnel should be included in experiments to develop new operational
   concepts, if these concepts are to be successful. Joint Forces Command
   officials pointed out, however, that the participation and involvement of
   other countries are often constrained by restrictions on access to
   sensitive security information. For example, North Atlantic Treaty
   Organization countries only participated as observers in

   Page 16 GAO- 02- 856 Military Transformation

   Millennium Challenge 2002 because of security information restrictions.
   The Command, however, plans to develop ways to better handle these
   restrictions to allow greater participation by other nations in its next
   major field experiment in 2004.

   Nearly 4 years after the program was established, only three
   recommendations have flowed from the joint experimentation program, and
   none of them have been approved. Confusion about proposed changes in
   guidance regarding the information required for submitting these
   recommendations has partly delayed their approval. At the time we
   concluded our review, official guidance on what information should
   accompany joint experimentation recommendations had not been approved. In
   addition, several DOD officials expressed concern that the process used to
   review and approve recommendations, the same as that used for major
   acquisition programs, may not be the most appropriate for a program whose
   aim is to integrate changes quickly. However, the officials could not
   pinpoint any specific impasses in the approval process. The DOD officials
   are also concerned about potential delays in the integration of new
   concepts because of the lengthy DOD resource allocation process.

   The Joint Forces Command submitted one recommendation to the Chairman of
   the Joint Chiefs of Staff in August 2001 and two more in November 2001
   (see table 2). At the time we ended our review, none of the
   recommendations had been approved. The recommendations to improve the
   planning and decision- making capabilities of joint forces and provide
   better training for personnel conducting theater missile defense
   operations were based on analyses of results of experiments carried out in
   the first 3 years of joint experimentation. Inputs included two major
   experiments: Millennium Challenge 2000 (live field experiment in August-
   September 2000) and the Unified Vision 2001 (virtual simulation experiment
   in May 2001). The first recommendation was submitted for review just 3
   months after the end of the last experiment. No Recommendations

   for Change Have Been Approved or Implemented

   Concerns Raised about the Delays in Approving Recommendations

   Page 17 GAO- 02- 856 Military Transformation

   Table 2: First Three Recommendations Submitted by Joint Forces Command
   Recommendation Description Changes required a Date submitted

   for review

   Collaborative environment and collaborative tools

   Develop deployable Internet- like tools to improve the collaborative
   planning and decision- making capabilities of joint forces.

    Joint doctrine

    Training

    Leadership and education

    Personnel August 18, 2001

   Joint intelligence preparation of the battle space

   Develop objectives to train and exercise joint forces on intelligence
   requirements for preparing the battle space, with a focus on theater
   missile defense.

    Joint doctrine

    Training

    Leadership and education

   November 1, 2001

   Theater missile defense and time critical targeting team training

   Develop a training course to improve the effectiveness of teams involved
   in theater missile defense and time- critical targeting activities.

    Training

    Leadership and education

    Personnel

    Facilities November 1,

   2001 a Recommendations can propose changes be made to doctrine,
   organizations, training, materiel, leadership and education, personnel,
   and/ or facilities. Source: Joint Forces Command.

   According to a Joint Staff official, however, approval of the
   recommendations has been delayed because Joint Forces Command and Joint
   Staff officials were confused about proposed changes in guidance. In May
   2001, the Joint Requirements Oversight Council proposed new guidance,
   which would require that information on costs and timelines be included in
   joint experimentation recommendations. Prior guidance did not require such
   information. Although the recommendations went through preliminary review
   by the Joint Staff, the omission was not caught until the recommendations
   were to be scheduled for review by the Joint Requirements Oversight
   Council. Joint Forces Command officials told us that they were not aware
   of the change in guidance until that time. When we ended our review, Joint
   Forces Command officials were working with the Joint Staff to assess how
   much data could be prepared and when. Command officials said that the
   recommendations will be resubmitted in fall 2002 together with other
   recommendations emerging from Millennium Challenge 2002. As a result, no
   recommendations have yet been reviewed or approved. Also, at the time we
   ended our review, the draft guidance on joint experimentation
   recommendations had not been approved and

   Page 18 GAO- 02- 856 Military Transformation

   issued. This guidance will become especially important because joint
   experimentation is expected to produce new recommendations more rapidly as
   the program matures.

   The requirement for costs and timeline data is consistent with that of
   recommendations for major weapon- system- acquisition programs. However,
   joint experimentation officials at the Joint Forces Command believe that
   requiring this type of information on joint- experimentation
   recommendations may not be appropriate because (1) these recommendations
   are generally intended to convince decision makers to develop particular
   joint capabilities, not specific weapon systems; (2) the new requirement
   may slow the preparation of future recommendations; and (3) it will be
   difficult to provide accurate estimates of costs and timelines for
   recommendations that span further into the future. It is too early to
   determine whether these concerns are valid.

   Some DOD officials were also concerned that the system currently used to
   allocate resources to implement joint- experimentation recommendations*
   DOD*s Planning, Programming, and Budgeting System* may not be the most
   efficient because it usually takes a long time to review, approve, and
   provide funding in future budgets. A recommendation approved in 2002, for
   example, would not be incorporated into DOD*s budget until 2004 or even
   later. This delay could result in missed opportunities for more rapid
   implementation.

   A Joint Staff official told us that the Joint Staff and the Joint Forces
   Command recently adjusted the timing of events to better align the joint
   experimentation process with the Planning, Programming, and Budgeting
   System. Additionally, DOD established a special fund for the Joint Forces
   Command to use as a temporary funding source to speed up the
   implementation of certain critical or time- sensitive recommendations.
   This source will provide early funding for implementation until funding is
   provided through DOD*s Planning, Programming, and Budgeting System.
   However, Joint Forces Command and other DOD officials believe other ways
   to implement new joint capabilities within the framework of existing
   budget and oversight practices may need to be considered. Concerns Raised
   about the

   Process for Funding Implementation of Recommendations

   Page 19 GAO- 02- 856 Military Transformation

   DOD has been providing more specific and clearer guidance on its goals,
   expectations, and priorities for the joint experimentation program.
   Nevertheless, the management of joint experimentation is missing a number
   of key elements that are necessary for program success: some roles and
   responsibilities have not yet been defined; current performance measures
   are not adequate to assess progress; and the Joint Forces Command lacks
   strategic planning tools for the program.

   DOD officials stated that the joint experimentation program had difficulty
   in its first years because guidance was evolving and was not specific:
   DOD*s transformation goals were not adequately linked to transformation
   efforts, and roles and responsibilities were not clearly defined. 16 Over
   time, the Secretary of Defense and the Chairman of the Joint Chiefs of
   Staff have provided more specific guidance on the goals and expectations
   for joint experimentation and its contribution to DOD*s transformation
   efforts. Guidance for joint experimentation has evolved gradually over the
   program*s nearly 4- year life span, partly because of shifting defense
   priorities and lack of clarity about the roles of various DOD
   stakeholders. Roles and responsibilities have also matured with the
   program. 17

   The Secretary of Defense*s 2001 Quadrennial Defense Review Report 18
   established six transformation goals, which include improving U. S.
   capabilities to defend the homeland and other bases of operations, denying
   enemies sanctuary, and conducting effective information operations.
   According to DOD officials, the Secretary of Defense*s most recent
   planning guidance tasked the Joint Forces Command to focus its
   experimentation on developing new joint operational concepts for these
   goals. To begin meeting these goals, the Chairman has also provided the
   Joint Forces Command with clarifying guidance that identified specific
   areas for the Command to include in its experimentation, such as the

   16 We have pointed this difficulty out in prior transformation- related
   reports. See U. S. General Accounting Office, Military Transformation:
   Navy Efforts Should Be More Integrated and Focused, GAO- 01- 853
   (Washington, D. C.: Aug. 2, 2001) and U. S. General Accounting Office,
   Military Transformation: Army Has A Comprehensive Plan for Managing Its
   Transformation but Faces Major Challenges, GAO- 02- 96 (Washington, D. C.:
   Nov. 16, 2001).

   17 The roles and responsibilities for most DOD organizations are broadly
   defined in the Chairman of the Joint Chiefs of Staff Instruction 3010.
   02A, Joint Vision Implementation Master Plan, Apr. 15, 2001.

   18 Department of Defense, Quadrennial Defense Review Report (Washington,
   D. C.: 2001). Some Key

   Management Elements Are Missing

   Guidance More Clearly Defined

   Page 20 GAO- 02- 856 Military Transformation

   development of a standing joint- force headquarters concept and of a
   prototype to strengthen the conduct of joint operations. The Command has
   reflected this new guidance in its latest Joint Concept Development and
   Experimentation Campaign Plan. Additionally, the Secretary of Defense
   reassigned the Command*s geographic responsibilities to focus it more
   clearly on its remaining missions, particularly transformation and joint
   experimentation.

   DOD officials at both headquarters and the field believe that the recent
   guidance begins to provide a better framework for the Joint Forces Command
   to establish and focus its joint experimentation efforts. Some officials,
   however, believe that future guidance should further clarify the link
   between joint experimentation and DOD priorities and the required
   resources necessary to support joint experimentation. DOD, in its comments
   to a draft of this report, stated that it expects the

   Transformation Planning Guidance* currently being prepared by the Office
   of the Secretary of Defense* will establish the requirements necessary to
   link experimentation to changes in the force.

   While roles and responsibilities for DOD organizations are now broadly
   defined, the new DOD Office of Force Transformation*s role in joint
   experimentation and its relationship to other stakeholders have not yet
   been clearly established. The Office*s charter or terms of reference have
   not been released. DOD plans to issue a directive later this year that
   will include a charter and description of the Office*s authorities and
   responsibilities. However, there is still uncertainty about the extent of
   authority and involvement the Office will have in the joint
   experimentation program and the Office*s ability to link the program with
   DOD*s overall transformation efforts. Joint Forces Command and other DOD
   officials consider having a transformation advocate in the Office of the
   Secretary of Defense as a beneficial link between the Joint Forces
   Command*s, the services*, and the combatant commands* joint
   experimentation programs and DOD*s overall transformation agenda.

   According to DOD*s 2001 Quadrennial Defense Review Report, the Office of
   Force Transformation, created in November 2001, is to play a role in
   fostering innovation and experimentation and should have an important
   responsibility for monitoring joint experimentation and for providing the
   Secretary of Defense with policy recommendations.

   An Office of Force Transformation official told us that the Office will be
   an advocate for transformation and will help develop guidance and make
   Role of Office of Force

   Transformation Is Not Clear

   Page 21 GAO- 02- 856 Military Transformation

   recommendations on transformation issues to the Secretary of Defense (the
   Office provided comments on the Secretary*s annual planning guidance and
   developed instructions for the services on preparing their first
   transformation road maps).

   The Office has also decided to take a cautious approach in carrying out
   its mission because of possible resistance from other DOD organizations,
   the same official said. The Office plans to offer its assistance to DOD
   organizations in their transformation efforts and attempt to influence
   their thinking on key issues, rather than asserting itself directly into
   their efforts, for example by funding military use of existing private-
   sector technology to act as a surrogate for evaluating possible concepts,
   uses, and designs.

   Joint Forces Command officials stated that as of May 2002, they had had
   only limited discussions with the Office and had not established any
   working agreements on how the Office would participate in the joint
   experimentation program.

   The Office of Force Transformation has only recently assembled its staff
   and is beginning to plan its work and establish contacts within DOD and
   with other organizations. The Office*s budget for fiscal years 2002 and
   2003 is about $18 million and $35 million, respectively.

   DOD*s performance measures (or metrics) for assessing joint
   experimentation* by measuring only the number of experiments carried out*
   do not provide a meaningful assessment of the program*s contribution
   toward meeting its performance goal for military transformation because
   they are only quantitative. Consistent with good management practices and
   in order to effectuate the purposes of the Government Performance and
   Results Act of 1993, 19 federal agencies devise results- oriented metrics
   that provide an assessment of outcomes or the results of programs as
   measured by the difference they make. In its fiscal year 2000 performance
   report, 20 the most recent it has issued, DOD described the performance
   indicators for the joint experimentation

   19 Pub. L. 103- 62, sec. 2( b), Aug. 3, 1993, sometimes referred to as the
   Results Act. 20 Department of Defense, Government Performance and Results
   Act: Department of Defense Fiscal Year 2000 Performance Report
   (Washington, D. C.: 2001). As of August 2002, DOD did not submit a
   performance report to the Congress for fiscal year 2001. Current
   Performance

   Measures Are Not a Useful Assessment Tool

   Page 22 GAO- 02- 856 Military Transformation

   program in terms of the number of experiments conducted against a target
   goal for the prior, current, and following fiscal years. In fiscal year
   2000, DOD exceeded its target number of experiments and did not project
   any shortfalls in meeting its target in the next fiscal year. Although
   this measure does provide a quantitative assessment of experimental
   activity, it does not provide a meaningful method for assessing how joint
   experimentation is helping to advance military transformation.

   An Office of the Secretary of Defense official stated that DOD recognizes
   that better performance measures are needed for assessing how joint
   experimentation advances transformation and for two other metrics
   currently used to assess its military transformation goal. 21 The official
   stated that developing such measures is a challenge because joint
   experimentation does not easily lend itself to traditional measurement
   methods. For example, most programs consider a failure as a negative
   event, but in joint experimentation, a failure can be considered as a
   success if it provides insights or information that is helpful in
   evaluating new concepts or the use of new technologies.

   An Office of the Secretary of Defense official told us that the RAND
   Corporation and the Institute for Defense Analyses recently completed
   studies to identify possible performance measures for assessing the
   progress of transformation. DOD is evaluating them and is preparing the

   Transformation Planning Guidance to provide more specific information on
   the priorities, roles, and responsibilities for executing its
   transformation strategy. The same official stated that the new guidance
   will include a discussion of the types of performance measures needed for
   assessing transformation progress or will assign an organization to
   determine them. In either case, measures will still need to be developed
   and implemented. DOD plans to issue the new guidance later in 2002 but has
   not determined how new performance measures would be incorporated into its
   annual performance report.

   The Joint Forces Command has not developed the strategic planning tools* a
   strategic plan, an associated performance plan, and performancereporting
   tools* for assessing the performance of the joint

   21 DOD*s fiscal year 2000 performance report included three metrics for
   assessing its transformation of U. S. military forces performance goal:
   joint experimentation, procurement spending, and defense technology
   objectives. Joint Forces Command*s

   Program Lacks Strategic Planning Tools

   Page 23 GAO- 02- 856 Military Transformation

   experimentation program. Strategic planning is essential for this type of
   program, especially considering its magnitude and complexity and its
   potential implications for military transformation. Such planning provides
   an essential foundation for defining what an organization seeks to
   accomplish, identifies the strategy it will use to achieve desired
   results, and then determines* through measurement* how well it is
   succeeding in reaching results- oriented goals and achieving objectives.
   Developing strategic- planning tools for the joint experimentation program
   would also be consistent with the principles set forth in the Government
   Performance and Results Act of 1993, which is the primary legislative
   framework for strategic planning in the federal government.

   The Joint Forces Command prepares an annual Joint Concept Development and
   Experimentation Campaign Plan that broadly describes the key goals of its
   program, the strategy for achieving these goals, and the planned
   activities. However, a February 2002 progress report, prepared by the
   Joint Forces Command*s Joint Experimentation Directorate, on the
   development of the Directorate*s performance management system indicated
   that one- fourth of those organizations providing feedback 22 on the
   Campaign Plan believed that the Plan lacks specificity in terms of the
   program*s goals and objectives and an associated action plan that outlines
   the activities to be carried out in order to achieve those goals.
   Officials we spoke with at the military services, the combatant commands,
   and the Joint Forces Command all cited the need for more specific and
   clearer goals, objectives, and performance measures for the program.

   In the progress report, the Command acknowledged the benefits of strategic
   planning and the use of this management tool to align its organizational
   structure, processes, and budget to support the achievement of missions
   and goals. The report proposed that the Command develop a strategic plan,
   possibly by modifying its annual

   Campaign Plan, and subsequently prepare a performance plan and a
   performance report. Command officials indicated that the basic
   requirements of a strategic plan could be incorporated into the

   Campaign Plan, but they were unsure, if such an approach were taken,
   whether the changes could be made before the annual Campaign Plan is

   22 DOD organizations providing comments to the draft Campaign Plan
   included various offices and activities of the Army, the Navy, the Air
   Force, the Marine Corps, the U. S. Special Operations Command, the U. S.
   Strategic Command, the U. S. Southern Command, the U. S. Pacific Command,
   and the U. S. European Command.

   Page 24 GAO- 02- 856 Military Transformation

   issued later this year. Similarly, the Joint Forces Command has had
   difficulty in developing specific performance measures for joint
   experimentation. A Command official stated that the Command has tried to
   leverage the performance measures developed by other organizations like
   itself, but found that there is widespread awareness throughout the
   research and development community, both within and outside DOD, that such
   measures are needed but do not exist. Additionally, a Joint Forces Command
   official stated that whatever metrics the Command develops must be linked
   to its mission- essential tasks for joint experimentation and that the
   Command is currently developing these tasks. At the time we ended our
   review, the Command had identified six broad areas for which specific
   metrics need to be developed. These included quality of life, customer
   relationships, and experimentation process management.

   After nearly 4 years, the Joint Forces Command*s process for obtaining
   inputs for the development and execution of DOD*s joint experimentation
   program has become more inclusive. However, questions continue about
   whether the program is the successful engine for change envisioned when it
   was established. Since the program*s inception, only three recommendations
   have flowed from experimentation activities and their review, approval,
   and implementation have been delayed from confusion over a change in
   guidance that required additional information be included in the
   recommendations. As a result, no recommendations for change have been
   approved or implemented to date. To the extent that the draft guidance on
   what should be submitted with joint experimentation recommendations can be
   officially approved and issued, future recommendations could be submitted
   for approval and implementation more quickly. Underscoring the need to
   finalize the guidance is the anticipated recommendations to be made after
   this year*s major field experiment, Millennium Challenge 2002.

   The lack of strategic planning for joint experimentation deprives the
   Joint Forces Command of necessary tools to effectively manage its program.
   Implementation of strategic planning at the Joint Forces Command would
   create a recurring and continuous cycle of planning, program execution,
   and reporting and establish a process by which the Command could measure
   the effectiveness of its activities as well as a means to assess the
   contributions of those activities to the operational goals and mission of
   the program. Such planning could also provide a tool* one that is
   currently missing* to identify strengths and weaknesses in the development
   and execution of the program and a reference document for the effective
   oversight and management of the program. Conclusions

   Page 25 GAO- 02- 856 Military Transformation

   Performance measures developed under the Command*s strategic planning
   could provide the standard for assessing other experimentation efforts
   throughout DOD, which are also lacking such metrics.

   The lack of a meaningful performance measure for assessing the
   contribution of the joint experimentation program to advance DOD*s
   transformation agenda limits the usefulness and benefit of this management
   tool to assist congressional and DOD leaders in their decision- making
   responsibilities. Establishing a *meaningful* joint experimentation
   performance measure for its annual performance report would provide
   congressional and DOD leadership a better assessment of the program*s
   contribution and progress toward advancing transformation. Such a metric
   would also be consistent with the intent of the Results Act to improve the
   accountability of federal programs for achieving program results. Because
   the role and relationships of the Secretary of Defense*s new Office of
   Force Transformation have not yet been clarified, the Secretary may not be
   effectively using this office in DOD*s transformation efforts. This
   office, if given sufficient authority, could provide the Secretary with a
   civilian oversight function to foster and monitor the joint
   experimentation program to ensure that it is properly supported and
   provided resources to advance the DOD*s overall transformation agenda.
   Rectifying these shortcomings is critical in view of the importance that
   DOD has placed on joint experimentation to identify the future concepts
   and capabilities for maintaining U. S. military superiority.

   To improve the management of DOD*s joint experimentation program, we
   recommend that the Secretary of Defense direct the Chairman of the Joint
   Chiefs of Staff to

    approve and issue guidance that clearly defines the information required
   to accompany joint experimentation recommendations for the Joint
   Requirements Oversight Council*s review and approval and

    require the Commander in Chief of the U. S. Joint Forces Command to
   develop strategic planning tools to use in managing and periodically
   assessing the progress of its joint experimentation program.

   We further recommend that the Secretary of Defense

    develop both quantitative and qualitative performance measures for joint
   experimentation in DOD*s annual performance report to provide a better
   Recommendations for

   Executive Action

   Page 26 GAO- 02- 856 Military Transformation

   assessment of the program*s contribution to advancing military
   transformation and

    clarify the role of the Office of Force Transformation and its
   relationship to the Chairman of the Joint Chiefs of Staff, the Joint
   Forces Command, and other key DOD stakeholders in DOD*s joint
   experimentation program.

   We received written comments from DOD on a draft of this report, which are
   included in their entirety as appendix III. DOD agreed with our
   recommendations and indicated that it expects that a forthcoming

   Transformation Planning Guidance and subsequent guidance will be
   responsive to them by clarifying roles and missions across DOD,
   implementing recommendations for changes, and establishing clear
   objectives. We believe such strategic guidance from the Secretary of
   Defense could provide a significant mechanism for better linking and
   clarifying the importance of the joint experimentation program with DOD*s
   transformation agenda. DOD also provided technical comments to the draft
   that were incorporated in the report where appropriate.

   To determine the extent to which the Joint Forces Command obtains input
   from stakeholders and other relevant sources in developing and conducting
   its joint experimentation activities, we reviewed an array of documents
   providing information about participants in joint experimentation,
   including guidance and other policy documents, position papers, fact
   sheets, reports, and studies of the military services, the combatant
   commands, the Joint Staff, and other DOD organizations. We also reviewed
   Joint Forces Command plans and reports. Additionally, we made extensive
   use of information available on public and DOD Internet web sites. To
   assess the change in participation by various stakeholders over time, we
   compared the differences in the numbers of participating organizations and
   initiatives provided by these organizations between the Joint Forces
   Command*s first two major field experiments in 2000 and 2002 (Millennium
   Challenge 2000 and Millennium Challenge 2002). We conducted discussions
   with officials at five combatant commands, the Joint Staff, the military
   services, and other DOD organizations, such as the Joint Advanced
   Warfighting Program and the Defense Advanced Research Projects Agency.
   Appendix IV lists the principal organizations and offices where we
   performed work. At the Joint Forces Command, we discussed with joint
   experimentation officials the process for soliciting and incorporating
   inputs for joint experimentation from the military services and the
   combatant commands. We also attended conferences and other sessions hosted
   by the Agency Comments

   and Our Evaluation Scope and Methodology

   Page 27 GAO- 02- 856 Military Transformation

   Joint Forces Command to observe and learn about joint experimentation
   participants and their contributions and coordination. For example, we
   attended sessions for the Command*s preparation of its annual Joint
   Concept Development and Experimentation Campaign Plan and planning for
   this year*s Millennium Challenge experiment. With officials from each of
   the services and the combatant commands, we discussed perceptions of the
   effectiveness of coordination and participation in joint experimentation.
   We also obtained observations about participants* involvement from several
   defense experts who track joint experimentation and military
   transformation. Although we did not include a specific assessment of the
   individual experimentation efforts of the services and combatant commands,
   we did discuss with service and command officials how their efforts were
   coordinated and integrated into joint experimentation. We also did not
   determine the extent that individual inputs obtained from various
   participating organizations were considered and incorporated into the
   joint experimentation program.

   To determine the extent to which recommendations flowing from the joint
   experimentation process have been approved and implemented, we reviewed
   and analyzed data that tracked the progress of the first three joint
   experimentation recommendations submitted by the Joint Forces Command. We
   also obtained and analyzed relevant guidance and held discussions with
   Joint Staff, Joint Forces Command, and Office of the Secretary of Defense
   officials on the Joint Requirements Oversight Council process for
   reviewing and approving joint experimentation recommendations. We also
   discussed issues relating to implementation of joint experimentation
   recommendations through DOD*s Planning, Programming, and Budgeting System.

   To assess whether key management elements, such as policy, organization,
   and resources, were in place for the program, we conducted a comprehensive
   review of current legislative, policy, planning, and guidance documents
   and reports and studies. We used the principles laid out in the Government
   Performance and Results Act of 1993 as an additional benchmark for
   assessing the adequacy of performance measures established for the program
   and of tools used to manage the program. We also discussed the status and
   evolution of joint experimentation oversight and management, including
   office roles and responsibilities and joint experimentation metrics, with
   officials at the Joint Forces Command, the Joint Staff, the services, the
   combatant commands, the Office of the Secretary of Defense, the Office of
   Force Transformation, and other DOD organizations. Several defense experts
   who follow joint experimentation and military transformation discussed

   Page 28 GAO- 02- 856 Military Transformation

   with us joint experimentation oversight and management and gave us their
   impressions regarding current joint experimentation management practices.

   Our review was conducted from October 2001 through May 2002 in accordance
   with generally accepted government auditing standards.

   We are sending copies of this report to interested congressional
   committees, the Secretary of Defense, the Chairman of the Joint Chiefs of
   Staff, and the Commander in Chief, U. S. Joint Forces Command. We will
   also make copies available to others upon request. In addition, this
   report will be available at no charge on the GAO Web site at http:// www.
   gao. gov.

   Please contact Richard G. Payne at (757) 552- 8119 if you or your staff
   have any questions concerning this report. Key contacts and contributors
   to this report are listed in appendix V.

   Henry L. Hinton, Jr. Managing Director Defense Capabilities and Management

   Appendix I: Chronology of Key Events Important to Joint Experimentation

   Page 29 GAO- 02- 856 Military Transformation

   Date Event Relevance to joint experimentation

   May 1996 Chairman of the Joint Chiefs of Staff issued Joint Vision 2010.
   This vision of future war fighting provides a conceptual template for

   the Department of Defense*s (DOD) transformation efforts across all
   elements of the armed forces. May 1997 DOD*s Report of the Quadrennial

   Defense Review issued. Report discussed the importance of preparing for
   future national security challenges. It concluded that DOD needed to
   institutionalize innovative investigations, such as war- fighting
   experiments, to ensure future concepts and capabilities are successfully
   integrated into the forces in a timely manner. May 1998 Secretary of
   Defense designated

   Commander in Chief, U. S. Joint Forces Command, a as executive agent for
   joint experimentation.

   The Secretary of Defense tasked the Joint Forces Command to design and
   conduct joint war- fighting experimentation to explore, demonstrate, and
   evaluate joint war- fighting concepts and capabilities. April 1998 Joint
   Advanced Warfighting Program

   established. DOD established the program at the Institute for Defense
   Analyses to serve as a catalyst for achieving the objectives of Joint
   Vision 2010

   (and later Joint Vision 2020). To that end, the program is to develop and
   explore breakthrough operational concepts and capabilities that support
   DOD*s transformation goals. October 1998 Joint Forces Command assumed

   responsibility as the executive agent for joint experimentation.

   Joint concept development and experimentation program initiated. June-
   August 1999 Joint Advanced Warfighting Program

   conducted the first joint experiment for Joint Forces Command.

   An experiment* J9901* that investigated approaches for attacking critical
   mobile targets. Experiment allowed the Joint Forces Command to begin its
   learning process on how to conduct joint experimentation. September 1999
   Report of the Defense Science Board

   Task Force on DOD Warfighting Transformation issued.

   Report proposed several recommendations to promote military
   transformation.

   June 2000 Chairman of the Joint Chiefs of Staff issued Joint Vision 2020.
   Updated vision statement described the joint war- fighting capabilities

   required through 2020. August- September 2000 Millennium Challenge 2000
   major field

   experiment conducted. The first major field experiment coordinated by the
   Joint Forces Command among the services and other stakeholders. April 2001
   Chairman of the Joint Chiefs of Staff

   issued updated Joint Vision Implementation Master Plan.

   Guidance described the process for generation, coordination, approval, and
   implementation of recommendations emerging from joint experimentation and
   defined the roles and responsibilities of DOD stakeholders. April 2001
   Transformation Study Report:

   Transforming Military Operational Capabilities issued.

   Study conducted for the Secretary of Defense to identify capabilities
   needed by U. S. forces to meet the twenty- first century security
   environment. Made several recommendations directed at improving joint
   experimentation. May 2001 Joint Forces Command conducted

   Unified Vision 2001 experiment. A major joint experiment* largely modeling
   and simulation* conducted to refine and explore several war- fighting
   concepts, such as *rapid decisive* operations. August 2001 Secretary of
   Defense*s planning

   guidance issued. Required studies by defense agencies and the Joint Staff
   to develop transformation road maps and a standing- joint- force
   headquarters prototype. September 2001 DOD*s Quadrennial Defense Review

   Report issued. The report established priorities and identified major
   goals for transforming the Armed Forces to meet future challenges. It
   called for new operational concepts, advanced technological capabilities,
   and an increased emphasis on joint organizations, experimentation, and
   training.

   Appendix I: Chronology of Key Events Important to Joint Experimentation

   Appendix I: Chronology of Key Events Important to Joint Experimentation

   Page 30 GAO- 02- 856 Military Transformation

   Date Event Relevance to joint experimentation

   November 2001 Chairman of the Joint Chiefs of Staff issued joint
   experimentation guidance. The guidance directed the Joint Forces Command
   to focus its nearterm experimentation on developing a standing joint force

   headquarters prototype. November 2001 Office of Force Transformation

   established. Office assists the Secretary of Defense in identifying
   strategy and policy, and developing guidance for transformation. April
   2002 Unified Command Plan 2002 issued. Plan reduced the number of missions
   assigned to the Joint Forces

   Command to allow the Command to devote more attention to its remaining
   missions such as joint experimentation. May 2002 Secretary of Defense*s
   planning

   guidance issued. The guidance directed the Joint Forces Command to develop
   new joint concepts that focus on the six transformation goals set forth in
   the 2001 Quadrennial Defense Review Report.

   July- August 2002 Joint Forces Command conducted Millennium Challenge
   2002. Second major field experiment conducted to culminate a series of

   experiments to assess *how* to do rapid decisive operations in this
   decade. a In October 1999, the U. S. Atlantic Command was redesignated as
   the U. S. Joint Forces Command.

   Appendix II: Additional Information on Joint Concept Development and
   Experimentation Activities

   Page 31 GAO- 02- 856 Military Transformation

   The Joint Forces Command uses various types of assessment activities to
   develop, refine, and validate joint concepts and associated capabilities.
   As shown in figure 3, the Command begins to move through the five joint
   concept development phases by conducting workshops, seminars, and war
   games to develop information and identify possible areas to explore in
   developing new concepts and associated capabilities and then uses
   simulated or live experiment events to confirm, refute, or modify them.
   These activities vary in scale and frequency, but each activity becomes
   larger and more complex. They can involve a small group of retired flag
   officers and academics, up to 100 planners, operators, and technology
   experts, or several thousand in the field. Near the end of the process,
   the Command will conduct a large- scale simulation experiment (such as
   Unified Vision 2001), followed by a major field experiment (such as
   Millennium Challenge 2002). The process continuously repeats itself to
   identify additional new concepts and capabilities.

   Figure 3: Joint Concept Development and Experimentation Activities

   Source: Joint Forces Command.

   Appendix II: Additional Information on Joint Concept Development and
   Experimentation Activities

   Appendix II: Additional Information on Joint Concept Development and
   Experimentation Activities

   Page 32 GAO- 02- 856 Military Transformation

   Table 3 provides additional information about the characteristics, scale,
   and frequency of these and other associated activities and experiments.

   Table 3: Types of Activities and Experiments Associated with Joint
   Experimentation Type of activities and experiments Characteristics Scale
   and frequency

   Workshop Brief, intense experimentation program used to discover and
   generate concepts. Participants identify focus areas and research topics.
   Workshops are conducted live and may have virtual components. Subject
   matter experts participate.

   Small scale Many each year

   Seminar Discussion forums used to discover and define problem boundaries,
   pose solutions, and exchange information. Seminars are conducted live, and
   may have virtual components. Subject matter experts participate.

   Small scale Many each year

   War game An event, using simulated and emulated systems, centering on the
   interaction of two or more opposing forces depicting a possible or actual
   scenario. Identifies key variables in an experiment; refines concepts and
   assesses alternatives.

   Scale varies One or more each year

   Limited objective experiment An event, limited in scope, that permits
   participants to see how specific, focused ideas or concepts will work in a
   given context and works with supporting elements of the integrating
   concept. It confirms or refines experiments and is primarily virtual or
   simulated. May include workshops, seminars, and war games.

   Small scale Varies (6 in 2001)

   Leveraged event Service or other organization*s event in which the Joint
   Forces Command and its partners share lessons learned. The event may
   involve the use of live forces in a theater scenario.

   Scale varies Frequency varies

   Spiral A group of smaller events and meetings requiring live participants;
   concepts are refined and plans are made for larger live Joint Forces
   Command experiments.

   Small to medium scale A few each year

   Major simulation experiment A constructive, virtual, or live experiment
   that makes extensive use of models and simulation to provide detailed
   information on the expected outcomes of an event or sequence of events. It
   may use live and/ or simulated participants.

   Large scale Not more than one every 2 years

   Major field experiment Integrates several smaller, previous events.
   Determines how effectively the concepts previously explored in smaller
   experiments will operate when integrated with the concepts of the other
   services and combatant commands. It validates concepts and requires live
   participants as well as simulated opponents.

   Large scale Not more than one each year

   Appendix III: Comments from the Department of Defense

   Page 33 GAO- 02- 856 Military Transformation

   Appendix III: Comments from the Department of Defense

   Report number is now: GAO- 02- 856.

   Appendix III: Comments from the Department of Defense

   Page 34 GAO- 02- 856 Military Transformation

   Now on p. 26. Now on p. 25.

   Now on p. 25.

   Appendix IV: Organizations and Offices Contacted

   Page 35 GAO- 02- 856 Military Transformation

   Department of Defense

    Office of the Secretary of Defense, Program Analysis and Evaluation

    Office of the Under Secretary of Defense for Policy

    Office of the Under Secretary of Defense for Acquisition, Technology,
   and Logistics

    Joint Advanced Warfighting Program

    Defense Advanced Research Project Agency

    Office of Force Transformation The Joint Staff

    Operational Plans and Interoperability Directorate

    Joint Vision and Transformation Division

    Command, Control, Communications, and Computers Directorate

    Force Structure, Resources, and Assessment Directorate Department of the
   Army

    Directorate of Training

    Directorate of Integration

    Directorate for Strategy, Concepts, and Doctrine Department of the Navy

    Office of the Deputy Chief of Naval Operations for Warfare Requirements
   and Programs

    Marine Corps Combat Development Command Department of the Air Force

    Wargaming and Experimentation Division Other Organizations

    Booz Allen Hamilton

    The Carlyle Group

    Center for Strategic and Budgetary Assessments

    Hicks & Associates, Inc. Appendix IV: Organizations and Offices

   Contacted Washington, D. C., Area

   Appendix IV: Organizations and Offices Contacted

   Page 36 GAO- 02- 856 Military Transformation

   U. S. Pacific Command

    Joint Experimentation Division

    Intelligence Architecture and Systems Division

    Exercise Division

    Regional Strategy and Policy Division U. S. Pacific Fleet

    Joint Experimentation Office U. S. Army Pacific

    Internal Review Office U. S. Pacific Air Forces

    Operational Requirements Marine Forces Pacific

   U. S. Southern Command

    Future Operations

    Programs and Resources Directorate

    Office of Analysis and Simulations Navy Warfare Development Command

   U. S. Joint Forces Command

    Joint Experimentation Directorate

    Joint Integration and Interoperability Directorate Honolulu, Hawaii,

   Area Miami, Fla., Area Newport, R. I., Area Norfolk, Va., Area

   Appendix IV: Organizations and Offices Contacted

   Page 37 GAO- 02- 856 Military Transformation

   Department of the Army

    Training and Doctrine Command

    Army Joint Venture Directorate

    Army Audit Agency Department of the Air Force

    Air Combat Command

    Air Force Experimentation Office Joint Forces Staff College

   U. S. Central Command

    Futures Division

    Exercise Division U. S. Special Operations Command

   Special Operations Requirements and Resources

    Joint Experimentation Division

    Special Operations Acquisition and Logistics

    Intelligence and Information Operations Tampa, Fla., Area

   Appendix V: GAO Contacts and Staff Acknowledgments

   Page 38 GAO- 02- 856 Military Transformation

   Sharon L. Pickup (202) 512- 9619 Richard G. Payne (757) 552- 8119

   In addition to the individuals named above, Carol R. Schuster, Mark J.
   Wielgoszynski, John R. Beauchamp, Kimberley A. Ebner, Lauren S. Johnson,
   and Stefano Petrucci made key contributions to this report. Appendix V:
   GAO Contacts and Staff

   Acknowledgments GAO Contacts Acknowledgments

   (350120)

   The General Accounting Office, the investigative arm of Congress, exists
   to support Congress in meeting its constitutional responsibilities and to
   help improve the performance and accountability of the federal government
   for the American people. GAO examines the use of public funds; evaluates
   federal programs and policies; and provides analyses, recommendations, and
   other assistance to help Congress make informed oversight, policy, and
   funding decisions. GAO*s commitment to good government is reflected in its
   core values of accountability, integrity, and reliability.

   The fastest and easiest way to obtain copies of GAO documents at no cost
   is through the Internet. GAO*s Web site (www. gao. gov) contains abstracts
   and fulltext files of current reports and testimony and an expanding
   archive of older products. The Web site features a search engine to help
   you locate documents using key words and phrases. You can print these
   documents in their entirety, including charts and other graphics.

   Each day, GAO issues a list of newly released reports, testimony, and
   correspondence. GAO posts this list, known as *Today*s Reports,* on its
   Web site daily. The list contains links to the full- text document files.
   To have GAO e- mail this list to you every afternoon, go to www. gao. gov
   and select *Subscribe to daily E- mail alert for newly released products*
   under the GAO Reports heading.

   The first copy of each printed report is free. Additional copies are $2
   each. A check or money order should be made out to the Superintendent of
   Documents. GAO also accepts VISA and Mastercard. Orders for 100 or more
   copies mailed to a single address are discounted 25 percent. Orders should
   be sent to:

   U. S. General Accounting Office 441 G Street NW, Room LM Washington, D. C.
   20548

   To order by Phone: Voice: (202) 512- 6000 TDD: (202) 512- 2537 Fax: (202)
   512- 6061

   Contact: Web site: www. gao. gov/ fraudnet/ fraudnet. htm E- mail:
   fraudnet@ gao. gov Automated answering system: (800) 424- 5454 or (202)
   512- 7470

   Jeff Nelligan, managing director, NelliganJ@ gao. gov (202) 512- 4800 U.
   S. General Accounting Office, 441 G Street NW, Room 7149 Washington, D. C.
   20548 GAO*s Mission

   Obtaining Copies of GAO Reports and Testimony

   Order by Mail or Phone To Report Fraud, Waste, and Abuse in Federal
   Programs

   Public Affairs
*** End of document. ***