Best Practices: Stronger Practices Needed to Improve DOD	 
Technology Transition Processes (14-SEP-06, GAO-06-883).	 
                                                                 
The Department of Defense (DOD) relies on its science and	 
technology community to develop innovative technologies for	 
weapon systems, spending $13 billion on basic, applied, and	 
advanced technology research. Several GAO reports have addressed 
problems in transitioning technologies to the acquisition	 
community. This report, which was prepared under the Comptroller 
General's authority to conduct evaluations, compares DOD's	 
technology transition processes with commercial best practices.  
Specifically, GAO identifies technology transition techniques	 
used by leading companies and assesses the extent to which DOD	 
uses the techniques						 
-------------------------Indexing Terms------------------------- 
REPORTNUM:   GAO-06-883 					        
    ACCNO:   A60909						        
  TITLE:     Best Practices: Stronger Practices Needed to Improve DOD 
Technology Transition Processes 				 
     DATE:   09/14/2006 
  SUBJECT:   Best practices					 
	     Comparative analysis				 
	     Cost analysis					 
	     Cost overruns					 
	     Defense cost control				 
	     Defense procurement				 
	     Evaluation criteria				 
	     Military research and development			 
	     Military technology				 
	     Program evaluation 				 
	     Risk assessment					 
	     Risk management					 
	     Schedule slippages 				 
	     Strategic planning 				 
	     Technology transfer				 
	     Weapons research and development			 
	     Weapons systems					 

******************************************************************
** This file contains an ASCII representation of the text of a  **
** GAO Product.                                                 **
**                                                              **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced.  Tables are included, but    **
** may not resemble those in the printed version.               **
**                                                              **
** Please see the PDF (Portable Document Format) file, when     **
** available, for a complete electronic file of the printed     **
** document's contents.                                         **
**                                                              **
******************************************************************
GAO-06-883

     

     * Results in Brief
     * Background
     * Leading Companies Rely on Strategic Planning, a Gated Proces
          * Strategic Planning Is a Critical Precursor to Successful Tra
          * Leading Companies Use Gated Reviews to Validate Readiness fo
          * Technology Transition Is Bolstered by Formal Agreements, Rel
               * Technology Transition Agreements
               * Relationship Managers
               * Metrics
     * DOD Lacks Breadth and Depth of Techniques That Leading Compa
          * Strategic Planning Affects DOD's Ability to Meet Warfighters
          * DOD Does Not Use a Disciplined, Gated Process to Test for Re
          * DOD Is Adopting Transition Tools, but Use Is Not Widespread
               * Technology Transition Agreements
               * Relationship Managers
               * DOD Programs to Aid Transition
               * Metrics
     * Differences in Environment and Incentives Contribute to Diff
     * Conclusions
     * Recommendations for Executive Action
     * Agency Comments and Our Evaluation
          * Boeing
          * 3M
          * IBM
          * Motorola
     * GAO's Mission
     * Obtaining Copies of GAO Reports and Testimony
          * Order by Mail or Phone
     * To Report Fraud, Waste, and Abuse in Federal Programs
     * Congressional Relations
     * Public Affairs

Report to Congressional Committees

United States Government Accountability Office

GAO

September 2006

BEST PRACTICES

Stronger Practices Needed to Improve DOD Technology Transition Processes

GAO-06-883

Contents

Letter 1

Results in Brief 3
Background 5
Leading Companies Rely on Strategic Planning, a Gated Process, and Tools
to Transition Needed Technologies 9
DOD Lacks Breadth and Depth of Techniques That Leading Companies Use to
Effectively Transition Technologies 22
Differences in Environment and Incentives Contribute to Different
Practices and Outcomes 36
Conclusions 39
Recommendations for Executive Action 40
Agency Comments and Our Evaluation 41
Appendix I Objectives, Scope, and Methodology 46
Appendix II Comments from the Department of Defense 50
Appendix III DOD Research, Development, Technology, and Engineering Budget
54
Appendix IV DOD Technology Readiness Levels 56
Related GAO Products 57

Tables

Table 1: DOD's Two Development Phases with Technology-Related Activities 6
Table 2: Metrics Used by Leading Companies to Assess Lab Projects and
Processes 20

Figures

Figure 1: Fiscal Year 2006 Categories for DOD Research, Development, Test,
and Evaluation Budget (dollars in billions) 7
Figure 2: Average Program Research, Development, Test, and Evaluation Cost
Growth from First Full Estimate (sample of 52 DOD weapon programs) 8
Figure 3: General Flow of Process Leading Up to Technology Transition 10
Figure 4: Generalized Depiction of Deliverables and Funding under Gated
Process 12
Figure 5: Linkage between Technology Development and Product Development
14
Figure 6: Notional Boeing Technology Maturity Scorecard for a Hypothetical
Technology 16
Figure 7: Questions That Should Be Answered in Technology Transition
Agreements 17
Figure 8: Communication Flow for Motorola's Three Levels of Relationship
Managers 18
Figure 9: Path that DOD Routinely Follows for Technology Development and
Product Development 23
Figure 10: Criteria for DOD Technology Development Phase 26
Figure 11: Accountability for Management and Funding of Technology 29
Figure 12: Excerpts from a Defense Science Board Task Force Report on the
Manufacturing Technology Program 31
Figure 13: DARPA, a Success Story 33

Abbreviations

DARPA Defense Advanced Research Projects Agency DDRE Director, Defense
Research and Engineering DOD Department of Defense RDT&E research,
development, test, and evaluation S&T science and technology TRL
technology readiness level

This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed in
its entirety without further permission from GAO. However, because this
work may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this material
separately.

United States Government Accountability Office

Washington, DC 20548

September 14, 2006

The Honorable John W. Warner

Chairman

The Honorable Carl Levin

Ranking Minority Member

Committee on Armed Services United States Senate

The Honorable Duncan Hunter

Chairman

The Honorable Ike Skelton

Ranking Minority Member

Committee on Armed Services House of Representatives

The Department of Defense (DOD) relies on the technological superiority of
its weapon systems and armed forces to protect U.S. interests at home and
abroad. The DOD science and technology (S&T) community is tasked with
ensuring that technologies are mature when DOD's acquisition community
takes over and integrates the technologies into weapon systems. In fiscal
year 2006, DOD plans to spend approximately $13 billion1 in its science
and technology efforts to develop technologies that are as innovative as
stealth and global positioning were when they were first developed. In
doing its work, the S&T community must strike a balance between meeting
the short-term needs of today's warfighters and the long-term needs of
future years' warfighters. Achieving this balance is made more challenging
because of the need to keep pace with or exceed the pace of innovation and
to counter technologies developed by potential U.S. adversaries.

Although the United States has produced the best weapons in the world, its
acquisition programs often incur cost overruns, schedule delays, and
performance shortfalls that undermine DOD's buying power. This dilemma is
due in part to DOD's difficulty transitioning technologies from a
technology development environment to an acquisition program. Because the
acquisition community frequently pulls technologies too early, it takes on
the additional task of maturing the technologies-an activity that is the
primary responsibility of technology developers-at the start of an
acquisition program.2 The start of a program ushers in a high-cost,
delivery-oriented phase in which the acquisition community is supposed to
be focused on integrating subsystems and working on system development and
demonstration. DOD has continued to allow the acquisition community to
take over this task before the S&T community considers the technologies
ready for transition. Numerous GAO reports have addressed the problems of
proceeding with immature technologies and have explained what leading
commercial companies do to ward off such problems.3

1This represents the amount of money Congress authorized DOD for basic
research, applied research, and advanced technology development.

This report examines DOD's technology transition processes through the
prism of best practices. Specifically, our objectives are to (1) identify
techniques that commercial companies use to transition mature technologies
before the start of product development and (2) assess the extent to which
DOD is using these techniques. We also describe how the environments at
private companies and DOD differ with regard to technology development.
The overall effort to hand off and integrate mature technologies is
referred to in this report as technology transition. Also in this report,
the private sector acquisition community is generally referred to as a
product line or business unit. GAO has prepared this report under the
Comptroller General's authority to conduct evaluations on his own
initiative as part of a continued effort to assist Congress in overseeing
DOD's technology and acquisition investments.

In conducting our work, we interviewed lab and product line managers at
four leading commercial companies-Boeing, IBM, Motorola, and 3M-to
identify processes for successfully transitioning mature technologies.
These companies develop a variety of products such as the 787 airplane,
network servers, integrated radio communications and information
solutions, and fuel cell technology for cars, buses, trains, homes, and
businesses. We also conducted interviews and collected pertinent documents
from each of DOD's military services and the Defense Advanced Research
Projects Agency (DARPA), which is the central science and technology
organization for DOD. We compared commercial and DOD practices to identify
potential areas for improvement. Appendix I includes additional details
about how we performed our review, which was conducted from June 2005 to
July 2006 in accordance with generally accepted government auditing
standards.

2GAO, Defense Acquisitions: Assessments of Selected Major Weapon Programs.
GAO-06-391 (Washington, D.C.: March 31, 2006).

3GAO, Best Practices: Better Management of Technology Development Can
Improve Weapon System Outcomes. GAO/NSIAD-99-162 (Washington, D.C.: July
30, 1999) and Best Practices: Better Matching of Needs and Resources Will
Lead to Better Weapon System Outcomes GAO-01-288 ( Washington, D.C.: March
8, 2001). Other best practices reports are listed in the Related GAO
Products section at the back of this report.

                                Results in Brief

Successful transition in leading companies starts with strong strategic
planning followed by a structured technology development process led by
research labs and supported by tools that pave the way for a smooth
handoff to the product line. Strategic planning is considered a precursor
to transition and allows managers to identify market needs so the company
can quickly adapt its technology portfolios to meet those needs. A gated
technology development process continually tests for relevancy and
feasibility of technologies and gauges the commitment of product line
managers to accept them. Once a technology is ready to transition,
management and funding responsibilities gradually shift from the lab to
the product line. By the end of transition, but before product development
starts, the technology is validated as mature enough for use in the
intended product. Companies highlighted three tools to facilitate
transition: technology transition agreements, relationship managers, and
metrics. The agreements put in writing the technology and business-related
expectations, such as specific cost, schedule, and performance
characteristics that labs must demonstrate. The agreements also may
require documenting manufacturing costs or specifying whether certain lab
scientists will be loaned to the product line to provide continuity in
technical knowledge. Relationship managers communicate across the labs and
product lines to address transition issues. Last, metrics gauge the
effectiveness of the technologies and the process itself. These tools
require the active involvement of the labs and product lines to ensure
successful transition of technologies to new products.

Over the past several years, DOD has taken steps to improve its transition
processes, but it lacks many of the techniques that are hallmarks of
leading companies' ability to transition technology smoothly onto new
products. From a strategic perspective, the department lacks strong
influence at the corporate level to guide the department's technology
investments. In addition, DOD does not use a gated process with criteria
that would allow lab and program managers to know when a technology is
ready to transition. Consequently, technologies are often not ready when
needed and acquisition programs pull the technologies into their programs
too early, leading to inefficiency during product development and cost and
schedule increases. We found that DOD has taken some positive steps to aid
technology transition. They hold promise, but must be accepted, improved,
and replicated significantly more than currently to have a positive
impact. For example, each of the military services has established boards
to select and oversee some of their technology projects and has elevated
the importance of meaningful metrics. They are also using technology
transition agreements. However, use of these agreements thus far has been
low. With regard to improving communication, DARPA is using relationship
managers to address transition issues. And the Office of the Secretary of
Defense has initiated a number of new programs, including the Joint
Capability Technology Demonstration program, which requires the S&T and
acquisition communities to work more collaboratively earlier in the
acquisition process.

The environment and incentives of private world-class companies differ
dramatically from DOD's. Despite these differences, the practices used by
commercial companies can help DOD make better progress in transitioning
technologies to weapon programs. Private companies operate in a
competitive environment that demands speedy delivery of innovative,
high-quality products to satisfy market needs or the company will go out
of business. DOD has a variety of "customers" and complex relationships
that often hinder the chief technology officer at the corporate level from
providing the type of strategic leadership found at successful companies.
DOD puts pressure on itself to develop many new technologies. And because
competition for funding is fierce, the technologies described with many
superlatives for speed and lethality tend to get more attention than
others do. We previously reported that to secure funding, DOD program
managers frequently make overly optimistic promises to the warfighter
about technologies' cost, feasibility, risk, and delivery schedule.4 The
challenge for DOD and congressional decision makers lies not only in the
"how to" aspects of technology transition but also in creating stronger
and more uniform incentives that encourage the S&T and acquisition
communities to work together to deliver mature technologies to programs.

We are making recommendations that DOD strengthen its technology
development and transition processes by developing a gated process that
includes a transition phase and identifies criteria that can be used to
support funding decisions. In addition, we are recommending that DOD
expand the use of transition agreements, relationship managers, and
metrics. We also recommend that DOD set aside a portion of research and
development (advanced component development and prototypes) funds for the
S&T community to manage the transition of technologies to acquisition
programs.

4GAO, Best Practices: Better Support of Weapon System Program Managers
Needed to Improve Outcomes. GAO-06-110 (Washington, D.C.: Nov. 1, 2005).

DOD concurred with our recommendations to develop a gated process for
developing and transitioning technology and to expand the use of
technology transition agreements, although we do not see its plans as
fully responsive. DOD partially concurred with the recommendation to
include additional metrics in technology transition agreements, stating
that it recently developed metrics to gauge manufacturing readiness and
that other metrics should be applied on a project-by-project basis. DOD
also partially concurred with expanding the use of relationship managers,
stating it already uses written documents to facilitate communication.
However, DOD agreed that more time should be devoted by staff at the
execution level on transition activities. We continue to believe that
additional person-to-person communication needs to take place at all
levels within the S&T and acquisition communities to supplement the
written agreements. DOD did not concur with our recommendation to adopt
more process oriented metrics because it believes that other processes it
plans to mature over the next 4-5 years will capture these metrics. We
believe the metrics DOD wants to use will not allow the department to
analyze its investment portfolio and make adjustments appropriately.
Finally, DOD did not agree with our recommendation to set aside a portion
of its 6.4 budget activity funding, known as advanced component
development and prototypes funds, for the S&T community to manage for the
transition of technology. DOD does not believe the S&T community should
have additional resources to mature technologies to the point of
successful transition to programs. We found otherwise in the commercial
world and believe DOD's approach to funding technology development and
transition is flawed.

                                   Background

Mature technologies are pivotal in developing new products. Without mature
technologies at the outset, a product development program will almost
certainly incur cost and schedule problems. Effectively managing
technology as a separate process from product development can improve the
potential for on-time product delivery at reduced cost. Leading companies
know this and have established disciplined processes to prioritize their
technology investments based on market needs, eliminate technologies that
are not relevant or feasible, balance technology push and customer pull,
and then transition technologies to product development efficiently.
Overall, effective management of technology facilitates the delivery of
new, innovative products to the user in less time. While DOD is very aware
of the need for new advanced technology in its weapon systems, it has not
always been effective in transitioning mature and relevant technologies to
product development.

Development of DOD's new weapon systems depends on two distinct phases:
technology development and product development. DOD relies on its S&T
community to identify, pursue, and develop new technologies that improve
and enhance military operations and ensure technological superiority over
adversaries. This includes the development of technologies for new weapons
programs as well as those that will be inserted into existing systems. The
acquisition community takes these new technologies, develops weapon system
programs, and delivers the products-that is, the weapon systems-to the
warfighter. Table 1 shows technology-related activities that take place
during technology development and product development, who performs the
work, fiscal year 2006 funding, and overall goals.

Table 1: DOD's Two Development Phases with Technology-Related Activities

Technology development phase         Product development phase             
Activities: identify, fund,          Activities: integrate, demonstrate,   
develop, and manage new technology   support, and upgrade technologies on  
projects that warfighter needs       weapon systems                        
                                                                              
Who does the work: government,       Who does the work: prime developer,   
industry, and academic               industry supplier, and acquisition    
technologists                        community's development lab engineers 
                                                                              
2006 funding: about $13 billion      2006 funding: about $58 billion       
                                                                              
Goal: demonstrate a technology or    Goal: develop and deliver weapon      
subsystem in a relevant environment  systems to the warfighter             

Source: DOD (data); GAO (presentation and analysis).

Note: Activities listed under the product development phase focus solely
on technology maturation and do not include other activities that the
acquisition community is responsible for, such as overall design and
manufacturing.

DOD uses research, development, test, and evaluation appropriations to
fund its technology activities. The budget is divided into categories that
generally follow a sequential path for developing technologies. Figure 1
shows the amount of money DOD spends in each category. (App. III contains
additional details about the budget.)

Figure 1: Fiscal Year 2006 Categories for DOD Research, Development, Test,
and Evaluation Budget (dollars in billions)

The S&T community controls the budget for basic research, applied
research, and advanced technology development. The budget category of
advanced component development and prototypes involves testing and
evaluating prototypes of systems or subsystems in a high-fidelity or
realistic environment before product development starts. DOD officials say
it is assumed that either the S&T community or the acquisition community
could carry out this work, but traditionally the acquisition side
prevails. After this point, additional technology-related activities are
completed as part of product development under the authority of the
acquisition community, namely, the program manager for a weapon system.

Prior GAO reports have said that DOD launches new weapon programs with
immature technology. We found this inability to mature the technology
before the start of product development to be a major contributor to
weapon system cost and schedule growth. In our March 2006 review of 52
major DOD weapon programs, we found that 90 percent of the programs
started with immature technologies5. More than half of the programs were
working with immature technologies at design review, when DOD acquisition
policy expects the design to be stable. And by the time production began,
one-third of programs still did not have mature technologies.

Not surprisingly, we found that DOD research, development, test, and
evaluation cost estimates increased dramatically for programs having
immature technologies at program start. Figure 2 shows the average cost
growth of DOD programs we reviewed when technologies were mature and
immature at program start.

Figure 2: Average Program Research, Development, Test, and Evaluation Cost
Growth from First Full Estimate (sample of 52 DOD weapon programs)

Programs that started with mature technologies have averaged a modest 4.8
percent cost growth above the first full estimate, whereas programs that
started with immature technologies have averaged about 35 percent cost
growth. Some programs experienced significantly greater cost growth. A
consequence of this cost growth is that the services typically deliver
weapon systems late, have to reduce quantities to stay within cost
estimates, shift funds away from other projects to make up for added
costs, or some combination of the three. For example, the Air Force has
incurred a 189 percent growth in the cost per aircraft for its F/A-22
tactical aircraft program, in large part because of technology maturation
issues. In response, the Air Force has reduced the quantity of F/A-22
aircraft it plans to procure by 72 percent from 648 to 183 to offset
escalating costs. In the case of the Space-Based Infrared System High
satellite, technology and design components matured late in development,
contributing to research, development, test, and evaluation cost growth
and four Nunn-McCurdy unit cost breaches6. Instead of purchasing five
satellites, the Air Force now plans to buy three at a program acquisition
unit cost of about $3.4 billion, a 315 percent increase.

5 GAO-06-391

  Leading Companies Rely on Strategic Planning, a Gated Process, and Tools to
                         Transition Needed Technologies

To successfully develop and transition technologies from their labs to
their product lines, leading commercial companies depend on three key
techniques:

           o  strategic planning at the corporate level;

           o  a gated management review process that ensures a technology's
           relevancy, feasibility, and transition readiness; and

           o  effective tools to solidify commitment, address transition
           issues, and gauge project progress and process effectiveness.

           Overall, corporations are incentivized to follow these critical
           techniques because the opportunity cost of not meeting customer
           demand is late delivery and lost revenues and market share for the
           company. Through rigorous adherence to these practices, leading
           companies increase chances of eventual success because the
           strategy for developing the technology fits the company's
           objectives, commitment is strong for incorporating the technology,
           and only after the technology is considered mature enough for use
           in a certain product does product development begin. Because the
           cost of developing new and breakthrough technologies can be high,
           funding typically comes from the corporate level rather than from
           a single product line unit, enabling the company's product lines
           to manage only product development risk, not technology risk as
           well.

           The central focus of this report is on how the lab and the product
           line communities work together to solidify the final steps of
           technology transition. Figure 3 depicts the general flow of
           technology development and where technology transition resides in
           the process.

610 U.S.C S: 2433 establishes the requirement for unit cost reports if
certain thresholds for program costs are exceeded (known as unit cost or
Nunn-McCurdy breaches). DOD is required to report to Congress and, if
applicable, certify the program to Congress.

Figure 3: General Flow of Process Leading Up to Technology Transition

This report touches briefly on strategic planning, which precedes
technology development. As the lab community identifies and develops
high-priority technologies during the technology development phase,
product line managers develop the business case for a new product by
identifying the market potential of the new product and using systems
engineering to set product requirements. This report does not focus on
these product line activities or on product development.7

Strategic Planning Is a Critical Precursor to Successful Transition

Strategic planning-the effort to identify desirable technologies and
prioritize resources-is an important early step in a company's ability to
eventually deliver the highest-priority technologies to various product
lines. Leading companies organize their research and development
activities into technological "thrusts", or competencies that represent
the core markets of their businesses. 3M and the other companies we
visited undergo strategic planning at least annually, and this process
enables corporate management to conduct portfolio analysis, identify
long-term market needs, and match the projects in each thrust area to
market needs. Eventually, managers determine which projects appear to be
relevant and feasible, which ones are applicable for which products,
whether the right projects are getting the right resources, whether the
company wants to be first to market, and whether the final products should
be released to the marketplace as soon as possible or several years down
the road. Managers may decide to establish new thrust areas as new ideas
come to light, rely on outside suppliers for some technology, or hire new
people to fill technology voids. Projects that no longer are relevant or
feasible are eventually terminated. This type of strategic planning is
critical to ensuring that the right technologies ultimately transition to
the right product line in an economical and timely way.

7Numerous GAO reports address issues surrounding product development. See
Related GAO Products at the end of this report.

Each company we visited funds technology differently, however, all ensure
that funding for early technology projects is protected at the corporate
level and not immediately beholden to product lines. 3M, for example, has
17 technological thrust areas, and the projects are directly funded with
seed money from the corporate level. IBM receives the majority of its
funding from the corporate level. IBM labs also receive and manage funding
from product lines for specific projects. Both Motorola and Boeing require
business units to fund a portion of lab costs each year based on their
historical usage of lab resources, but the labs have freedom to use those
funds in line with corporate strategy. Decisions are made at least
annually on the composition of projects in the portfolio so it includes
the most relevant and feasible projects.

Leading Companies Use Gated Reviews to Validate Readiness for Technology
Transition

After technology projects are selected for funding, they enter the
development "pipeline" where a gated process is used to manage and oversee
technology exploration, development, and transition. At each gated review,
lab managers assess technology progress and ensure that certain criteria
are met before technologies can enter the next stage of development. The
final phase is dedicated to technology transition activities. The number
and names of the gates vary by company, but the type of information
collected and knowledge obtained are similar. For example, 3M uses a
three-gate process, Boeing has a four-gate process, and Motorola has a
five-gate process. Regardless of where the funds came from-a corporate or
a product line unit-labs are responsible for managing, overseeing, and
using the money to fund technology projects until the projects have
transitioned to a product line. Figure 4 generally depicts the technology
development gates that companies use, lists types of deliverables expected
during each gate, and shows who provides funding.

Figure 4: Generalized Depiction of Deliverables and Funding under Gated
Process

Explore gate: Lab technologists turn technology proposals into viable,
executable projects that ideally will meet future market needs, while lab
managers determine the amount of resources they need to invest in the
projects. Funding goes to projects that are deemed relevant. Prior to the
gated review, technologists deliver to lab managers a preliminary
assessment of the competitiveness of the technology and a road map for
completion. Technologists must address key deliverables related to such
areas as manufacturing, intellectual property, marketing, life cycle
management, and plans for the next gate.

Develop gate: Technologists develop prototypes of the project and measure
performance for relevancy and feasibility on an eventual product.
Technologists deepen their understanding of the technology, refine the
technology solution, identify the most attractive market segments for
introducing the technology, and select a product line that will
incorporate the technology into a new product. 3M and Motorola require
product line commitment to transition a technology before the gated review
draws near. This prevents labs from wasting valuable time and resources
developing technologies that product lines do not want. Projects that do
not gain product line commitment will either be terminated or go through a
different process to demonstrate their relevancy and feasibility. Motorola
leadership, for example, has a special program called an early stage
accelerator under which selected projects can obtain funds to build
additional prototypes to demonstrate the value of technology to product
line managers or potential customers. Senior lab managers carefully
monitor these projects through quarterly reviews to determine how they are
progressing. If the project is not making sufficient progress, the project
is quickly terminated so resources can be spent on other projects.

Transition gate: Technologists demonstrate a prototype in an operational
environment. The product line maintains a "customer" role until managers
are confident that the technology will work in the intended product. Then
gradually, the responsibility for funding and further developing the
technology shifts from the lab to the product line. This shift in
responsibility requires up-front and continuous planning by both the lab
and product line communities to ensure a smooth transition. As shown in
figure 5, companies we visited plan for technology transition to occur
prior to program start. This provides product line managers sufficient
time to gain additional knowledge about a technology's attributes as they
are maturing it to its final form, fit, and function. Product line
managers, who concurrently have been working on the business case for the
product, are able to validate that the technology can be integrated into
the preliminary product design. In addition, they can develop more
reliable product development cost and revenue estimates before they are
locked into needing this technology as a product feature.

Figure 5: Linkage between Technology Development and Product Development

During the transition phase, labs and product lines must complete a number
of activities for transition to go smoothly. For example, labs must
demonstrate that the technology meets product line cost, schedule, and
performance requirements. In addition, production costs must be identified
and acceptable to the product line. According to 3M officials, past
experience has shown that costly manufacturing is a major reason for
product line managers deciding not to transition a technology. Labs must
also address intellectual property concerns, a step that is crucial to the
company's ability to be a market leader. Product lines must address any
people or organizational issues, such as the transition of jobs and
training requirements that may result from using the new technology. They
must also work out any agreements with the labs for on-going support. At
the end of the phase, technical documentation related to the technology is
transitioned from the labs to the product line.

Boeing Commercial Airplanes8 assesses the extent to which these
activities, as well as others, have been completed to determine whether a
particular technology is ready to transition. The information is then
summarized on a scorecard that lab and product line managers can use to
quickly identify areas that need additional attention. The scorecard is
updated continually and measures a technology's readiness from a business
and production standpoint as well as its technological readiness. Boeing's
labs have three phases for technology development, which they call
discovery, feasibility, and practicality, plus a technology transition
phase. Each technology is evaluated in 10 readiness categories, and bars
are plotted horizontally across the scorecard to indicate how much
objective evidence exists for technology readiness. Once a technology has
demonstrated full readiness in all 10 categories, it is ready to be
transitioned. Figure 6 shows a notional picture of a technology's
readiness at one point in time. Because the hypothetical technology has
reached full readiness in only five categories and is halfway through
transition in one category, the technology is not ready for transition.
The lab and product line would have to address issues involving readiness
categories 3, 4, 6 and 9 before transition can begin or the technology is
in jeopardy of not being applied to a product.

8Boeing has two primary businesses: Boeing Commercial Airplanes and Boeing
Integrated Defense systems. Both businesses are supported by the same
science and technology lab. However, the two businesses have different
tools for transitioning technology.

Figure 6: Notional Boeing Technology Maturity Scorecard for a Hypothetical
Technology

Technology Transition Is Bolstered by Formal Agreements, Relationship Managers,
and Metrics

Companies we visited use three tools to aid transition, including
technology transition agreements, relationship managers, and metrics.
Technology transition agreements are formal documents that detail the
specific cost, schedule, and performance attributes of the technology that
labs must demonstrate before transition can occur. Relationship managers
address the details and issues surrounding transition, and metrics allow
managers to identify the effectiveness of their technology development and
transition processes and make adjustments when necessary. These tools
require the active involvement of the labs and product lines to ensure
successful transition of technologies to new products.

  Technology Transition Agreements

Labs and product lines use technology transition agreements to facilitate
each technology's transition to product development. What the agreements
specify varies by company but typically consists of specific technology
and business readiness metrics, such as cost, schedule, and performance
characteristics that labs must demonstrate for product line managers to
agree to accept the technology. As figure 7 shows, the agreements should
answer some basic questions.

Figure 7: Questions That Should Be Answered in Technology Transition
Agreements

Is it real?                      
                                    
Is it relevant?                  
                                    
Is it marketable?                
                                    
Where will it transition?        
                                    
Do we have product line support? 

Source: GAO.

These formal agreements may also include identification of product line
resources needed to transition the technology, such as nonrecurring costs
to further mature the technology, as well as consideration of recurring
manufacturing costs associated with integrating the new technology on the
product. Metrics collected in the agreements are used by product line
managers to manage technology risk. For example, if the labs cannot
develop needed technology within cost and schedule parameters or with
specified performance characteristics, the product lines can terminate the
agreement and go with an alternative technology.

At Motorola, agreements enable technology project leaders to customize
their work for a particular product line's needs. Product lines may
request that the labs identify alternative technologies for a particular
product, produce a detailed report about a specific technology, or conduct
various tests to demonstrate the relevancy or feasibility of a technology.
Each of these items becomes a "deliverable" in the agreement. 3M uses
transition agreements only for technologies that are expected to go into a
product. 3M's agreements incorporate many of the same metrics used by
Motorola, including feasibility, relevancy, and application of the
technology. In addition, agreements include an assessment of the product
line's ability and commitment to transition technology. For example, they
evaluate a product line's resource constraints that could inhibit a
technology from transitioning, such as asking if the technology is too
expensive or if manufacturing it would be too costly. 3M officials told us
that some agreements include loaning key lab technologists to the product
line for a period of time so product line teams can maintain momentum
after the lab has signed off and moved on to developing technology for
other parts of the company.

  Relationship Managers

3M, IBM, and Motorola use lab and product line relationship managers to
smooth transition. Relationship managers foster effective transition
practices by preventing the labs from pushing technologies that product
line managers do not want and by preventing product line managers from
pulling immature technologies from the labs. A Motorola official referred
to relationship managers as the most important communication tool because
communication occurs more frequently, thus allowing problems to be
identified and addressed more quickly.

Motorola designates key people at three different levels in both the labs
and the product line to serve as relationship managers. As a technology is
matured and demonstrated, communication intensifies and additional levels
of relationship managers become involved. Figure 8 depicts how Motorola's
relationship managers communicate with each other within the lab or
product line and with their designated counterparts in the other
community.

Figure 8: Communication Flow for Motorola's Three Levels of Relationship
Managers

Executive managers in Motorola's labs and product lines are responsible
for interfacing with each other on a periodic basis. Altogether, Motorola
has eight executive managers-four in the labs and four on the product line
side. Motorola officials told us that this number provides a one-to-one
match between the labs and the product lines. Lab executive managers are
responsible for ensuring product line needs are identified and new
technology projects are started or existing projects are reprioritized to
meet those needs. Their counterparts have the final word on what priority
each technology project has with respect to the needs of the product line.
Lab and product line executive managers are required to sign off on key
technology projects and milestones for the current year. In addition, they
try to remove any roadblocks within their own units and work with their
counterparts to address any roadblocks between the two communities.

Motorola considers liaisons the most critical in the process. Again, there
are four liaisons in the labs and four on the product line side. They are
the primary interface for coordination, collaboration, and communication
during technology development and transition. Liaisons in the lab have
broad oversight of technologies being developed and share information
about technology breakthroughs with their counterparts on the product line
side, and remove roadblocks for lab technologists. These lab liaisons work
intimately with their product line counterparts to approve technology
transition agreements and assess technology readiness. The lab liaisons
oversee 40 to 60 technology transition agreements at any one time.
Liaisons on the product line side are responsible for providing
information about the changing needs of the product lines on a more
real-time basis. They determine product line needs and priorities during
the annual planning process and remove roadblocks for technologists on the
product line side. Lab and product line liaisons are incentivized through
their annual performance assessments and pay increases to work together to
ensure the successful transition of technologies.

The most direct and constant communication is between the lab's
technologist, who developed the technology, and the product line's
technologist, who is responsible for maturing the technology for inclusion
into the product. These two technologists stay in continuous contact
because they are the ones who have the most working knowledge about the
technology. The lab technologist is expected to spend as much time as
needed to make sure that transition happens as smoothly as possible. While
the lab technologist does not become an official asset of a product line,
he or she might have to spend a period of time working in the product
line's development lab.

  Metrics

Leading companies use product metrics-such as weight, power, and
reliability measurements-to assess the readiness of transitioning
technologies and process metrics-such as profit growth and cycle time-to
gauge the impact of their technology development processes and identify
areas for improvement. The companies analyze data gained from the use of
these metrics to evaluate how well they deliver on what is promised,
better understand the value of their respective science and technology
activities, and identify cases of inefficient investment in technologies
or underutilization of lab technologies. Table 2 provides a composite
example of metrics used at the companies we visited and the value of the
metrics to management.

Table 2: Metrics Used by Leading Companies to Assess Lab Projects and
Processes

Categories of metrics Examples of metrics         Use of the metrics       
Project               
Technology-specific   Nonrecurring development    Allows lab and product   
                         costs Scheduled delivery    line managers to assess  
                         Recurring manufacturing     the status of technology 
                         costs Performance           development and whether  
                         characteristics             the technology meets the 
                                                     needs of the product. If 
                            o  Size                  technologies do not meet 
                            o  Weight                agreed-upon goals, the   
                            o  Power                 technology project may   
                            o  Reliability           be terminated or the     
                                                     product line manager may 
                         Head count                  decide to include it on  
                                                     a future product.        
Process               
Status                Number of ongoing projects  Provides lab managers    
                         Number of projects with a   information on how many  
                         technology transition       technology projects      
                         agreement Number of         transitioned, were       
                         projects completed by labs  terminated, and are      
                         Number of technologies      still ongoing. Companies 
                         transitioned Number of      expect that almost all   
                         projects terminated         technologies that make   
                                                     it to the final stages   
                                                     of technology            
                                                     development will         
                                                     transition. If they      
                                                     experience a lower       
                                                     transition rate than     
                                                     expected, officials will 
                                                     examine their processes  
                                                     to determine what        
                                                     changes are necessary to 
                                                     improve transition or    
                                                     determine why the        
                                                     project was not          
                                                     terminated earlier.      
Timeliness            Development cycle time      Measures the amount of   
                         Percentage of tasks on time time it takes labs to    
                         Task slippage Time to       develop technologies.    
                         market                      The metrics allow lab    
                                                     managers to identify and 
                                                     focus on projects that   
                                                     are moving slower than   
                                                     expected. A lab project  
                                                     may be terminated or     
                                                     additional resources may 
                                                     be allocated to speed up 
                                                     development. Use of the  
                                                     metrics allows product   
                                                     line managers to decide  
                                                     whether a technology     
                                                     will be ready in time to 
                                                     include on a given       
                                                     product.                 
Impact                Number of technologies      Provides lab and         
                         commercialized Number of    corporate managers       
                         multiple transitions Return feedback on the market   
                         on investment Profit growth impact of their          
                         Market share growth Orders  technology investments   
                         captured Cost reduction     in terms of revenue and  
                         Number of                   market share. The        
                         patents/influential papers  metrics also provide     
                         Customer satisfaction       information on product   
                         People rotation             line satisfaction of lab 
                                                     performance.             
                                                     Satisfaction survey      
                                                     results are a useful     
                                                     tool for identifying     
                                                     development and          
                                                     transition problem areas 
                                                     that need management     
                                                     attention.               

Source: Interviews with leading companies (data); GAO (presentation and
analysis).

Motorola's labs use a suite of metrics, many of which are shown above, to
monitor technology projects and lab performance. For example, the company
tracks the estimated and actual development time and costs for each
project to determine estimation accuracy. In addition, it identifies and
tracks the number of lab and product line people (headcount) needed for
development and transition. These metrics are not treated with the same
rigor as metrics that are used after product development begins. Corporate
managers understand that technology development must be managed with more
latitude to accommodate calculated risk taking. In addition, Motorola asks
each product line to fill out satisfaction surveys twice a year to assess
lab performance. Because relationships are at the heart of the process,
the product line's perceptions are of paramount importance, making
internal customer satisfaction a key metric. A Motorola official indicated
that satisfaction surveys are a ubiquitous part of the Motorola culture.
Not only do they help the labs improve, but they help the company improve
overall, the official said. The surveys also provide compelling anecdotal
information about performance problems that need to be addressed by
certain labs.

In addition to technology-specific metrics, Boeing tracks the number of
technologies that were actually integrated into products. While the intent
is that all completed technologies will make it into a product, this does
not always happen because of changes in market requirements, timing, or
funding constraints. Boeing also has metrics that assess the technology's
impact on orders for new airplanes and on its ability to reduce
manufacturing costs. The labs, for example, have a responsibility for
improving the way Boeing designs and builds its products. As such, they
have goals for reducing recurring and nonrecurring costs and cycle time.
Finally, Boeing uses a metric that encourages the transfer of people,
along with the technologies, to the product line side. The company tracks
the rotation of people from its labs to its product lines specifically to
improve its transition processes and to refresh staff with "program
people." As a result, company officials believe they can better align
their labs to their product lines strategies.

Company officials also told us that, similar to DOD, they have struggled
to assess return on investment for technologies they develop. This is
because technology can undergo a metamorphosis after it leaves the labs
and can be applied to multiple products or altered before going into the
intended product. Company officials said they are actively trying to track
technology utilization and the impact their technology investments have
had on company revenues. 3M has a technology database it uses to track
each project it funds. The database contains cost, schedule, and
performance data and summarizes other information, such as how long a
project took to complete and lab and product line funding used to mature
the technology. 3M then compares the cost information with worldwide
sales, tracked through bar codes, to determine its return on investment
for a particular technology. Company officials indicated that there are
still problems with this methodology for estimating return on investment
and that they are continuing to refine their approach.

    DOD Lacks Breadth and Depth of Techniques That Leading Companies Use to
                      Effectively Transition Technologies

DOD has taken some steps over the past few years to improve its technology
transition processes, but the practice of accepting high levels of
technology risk at the start of major weapon system acquisition programs
continues to be the norm. This shortcoming is a major contributor to DOD's
poor cost and schedule outcomes. Many of DOD's problems can be attributed
to deficiencies in strategic planning for critical technologies, processes
for technology development and transition, and tools that support
transition. A recent Defense Science Board report states that DOD is not
as well positioned as it should be from a strategic standpoint to meet the
challenges and exploit the opportunities offered by technology. The report
identified several opportunities for improvement at the Director, Defense
Research and Engineering level (DDRE)9 to help the department jointly
identify, prioritize, develop, and deliver the technologies most relevant
and critical to meeting weapon system requirements in a timely manner.
DOD's technology development process is undisciplined and lacks criteria
for maintaining a close connection between fledgling technologies and the
products that will need them to meet the warfighter's future needs. As a
result, technologies are often not ready when they are needed, and
acquisition programs pull the technologies into their programs too early,
leading to inefficiency during product development and cost and schedule
increases. DOD's process for transitioning technologies to product
development is funded and managed by its acquisition community, the
opposite approach to that taken by the companies we visited.

We identified some initiatives within DOD's S&T community that emulate
some of the tools we found in the commercial world. They hold promise, but
must be accepted, improved, and replicated significantly more than
currently to have a positive impact. For example, the military services
are using technology transition agreements and have established boards to
select and oversee some of its technology portfolio. Also, DARPA uses
relationship managers to expedite the efficient transition of technologies
to products and DOD has established other programs that institutionalize
some best practices. The collection and use of meaningful metrics,
however, remain a problem.

9DDRE is the principal staff adviser to the Under Secretary of Defense for
Acquisition, Technology, and Logistics and the Secretary and Deputy
Secretary of Defense for research and engineering matters. DDRE serves as
the chief technology officer for the department.

On the basis of our previous review of 52 major weapons programs, we found
that DOD's typical path involves starting programs with immature
technologies and concurrently working on technology development and
product development. Figure 9 is a depiction of this path.

Figure 9: Path that DOD Routinely Follows for Technology Development and
Product Development

Strategic Planning Affects DOD's Ability to Meet Warfighters' Needs

DOD has an annual strategic planning process that involves the development
of several plans at the corporate level and within the services and
defense agencies for determining how to invest its S&T funding, which
amounted to about $13 billion in fiscal year 2006. Despite these efforts,
a recent Defense Science Board report10 stated that DOD is not as well
positioned as it should be at the DDRE level (the corporate level) to meet
the challenges and exploit the opportunities offered by technology. The
report identified several opportunities for improvement. Until
improvements are made, it is likely that the there will be strategic gaps
and overlaps in technology coverage as the services and agencies develop
their own approaches to meet critical warfighter needs.

10DOD. Defense Science Board Task Force on The Roles and Authorities of
the Director of Defense Research and Engineering, (Washington, D.C.: Oct.
28, 2005).

Similar to private industry, the department goes through an annual
strategic planning process led by its corporate level. As part of its
responsibilities, DDRE develops the Defense Science and Technology
Strategy, which serves as the foundation for DOD's science and technology
strategic planning process. The strategy identifies five generic
technology thrust areas-information assurance, battlespace awareness,
force protection, reduced cost of ownership, and maintaining basic
research-that have high priority in the department, and is supported by
four other documents, including the

           o  Basic Research Plan, which presents the DOD objectives and
           investment strategy for DOD-sponsored basic research performed by
           universities, industry, and service laboratories;

           o  Defense Technology Area Plan, which presents the DOD objectives
           and the strategies for applied research and advanced technology
           development investments;

           o  Joint Warfighting Science and Technology Plan, which takes a
           joint perspective across the applied research and advanced
           technology development plans to ensure that the science and
           technology program supports priority future joint warfighting
           capabilities; and

           o  Defense Technology Objectives, which identify specific
           technology advancements that will be developed or demonstrated,
           the anticipated date of availability, and specific expected
           benefits.

           Together, these documents, as well as the supporting S&T plans of
           the military services and defense agencies, are supposed to
           provide the framework for decisionmaking throughout the science
           and technology community.

           However, a recent Defense Science Board report states over time
           there has been a relative decline in the influence of DDRE at the
           corporate level on strategic matters. In the 1960s and 1970s, the
           corporate level was more proactive and provided high-level
           direction that drove several decisive technological developments,
           including stealth; standoff precision strike; and tactical
           intelligence, surveillance, and reconnaissance systems that have
           transformed U.S. military capabilities. Since the 1980's DDRE has
           used a decentralized management approach, relying on the services
           and agencies to determine and prioritize the most needed
           technology projects. The report states that as a result, the
           department is not wellpositioned to implement new operational
           capabilities needed for future warfighters, including

           o  identifying and tracking terrorists,

           o  addressing commercialization and globalization issues,

           o  rapidly evolving technologies such as bio- and
           nanotechnologies, and

           o  identifying the proper mix of short- and long-term projects to
           work on.

           One of the recommendations included in the report is for the
           Undersecretary of Defense (Acquisition, Technology, and Logistics)
           to develop a strategic technology plan and that DDRE be tasked
           with assuring that all research and development organizations are
           implementing DDRE's strategic technology guidance. The board also
           believes DDRE needs to be more involved in strategic challenges
           related to (1) gathering and nurturing technology from a variety
           of sources, (2) developing and exploiting technology to enable new
           disruptive capabilities; (3) identifying and countering disruptive
           capabilities developed by adversaries using readily available or
           advanced technology, and (4) ensuring an adequate level of
           long-term research for DOD needs. Without effective corporate
           level leadership, warfighter needs are addressed in a
           decentralized, uncoordinated manner, thereby increasing the risk
           that the department will not fulfill these needs.

           DOD Does Not Use a Disciplined, Gated Process to Test for Relevancy,
           Feasibility, and Transition Readiness

           DOD's 5000 series acquisition policy specifies that technology
           development should be separated from product development and that
           a project shall not exit the technology development phase until
           the technology has been demonstrated in a relevant environment.
           This is in line with the practices we found at leading companies,
           with the exception that the companies require technologies to be
           demonstrated in an operational environment before program start,
           which is a higher degree of readiness. However, the department
           lacks a structured, gated process for managing technology
           development and transition, as well as criteria that would allow
           decision makers to know when technology is ready to progress from
           the technology development environment into an acquisition program
           to begin product development. As a result, the services continue
           to launch new programs with immature technologies, and acquisition
           programs take on technology development responsibilities, an
           activity that is considered too risky for commercial companies we
           visited. The following figure shows where the technology
           development phase occurs in DOD's acquisition process and the
           entrance and exit criteria for that phase.

DOD Does Not Use a Disciplined, Gated Process to Test for Relevancy,
Feasibility, and Transition Readiness

Figure 10: Criteria for DOD Technology Development Phase

As shown in the figure above, DOD has two phases prior to program start
(also referred to in DOD nomenclature as milestone B). The first is the
concept refinement phase, when acquisition programs refine the initial
concept and develop a technology development strategy that is supposed to
guide activities during the technology development phase. According to the
acquisition policy, the strategy should include

           o  a discussion of the planned acquisition approach, including a
           summary of the considerations and rationale concerning the
           approach;

           o  a discussion of the planned strategy to manage research and
           development;

           o  a complete description of the first technology demonstration;
           and

           o  a test plan.

           The policy, however, does not address the role of the S&T
           community in designing or supporting a technology development
           strategy. And according to S&T officials, their role in this phase
           is minimal. We recently reported that DOD skipped a formal
           milestone meeting that should take place at the end of the concept
           refinement phase, referred to as milestone A, for 80 percent of
           the programs we reviewed.11 This inhibits the S&T and acquisition
           communities' ability to create a realistic plan for developing and
           maturing needed technologies prior to program start and executing
           a plan once the program begins.

           During the technology development phase, the acquisition policy
           calls for DOD to focus on the development, maturation, and
           demonstration of the technologies needed for the capability under
           consideration. To exit this phase, technologies should be
           demonstrated to be sufficiently mature. DOD defines this as
           demonstrating technologies in a relevant environment. (See app. IV
           for definitions of technology readiness.)

           When comparing the DOD process with that used by commercial
           companies we visited, we identified two major differences. First,
           although DOD policy states that the S&T community "shall enable
           rapid, successful transition ... to useful military products,"
           there is no defined transition phase with criteria to facilitate
           assessment of a technology's readiness to transition. Instead, the
           services have senior-level boards that review projects on an
           annual basis to determine if they are on track relative to cost,
           schedule, and performance goals set out for the program and rely
           on a technology readiness tool to gauge development progress. The
           reviews do not include a formal assessment of many of the
           technical and business criteria contained in the Boeing scorecard,
           such as determining if the costs, benefits, and risk are well
           understood and technology is affordable. As a result, program
           managers often pull a technology from technology developers before
           the program manager has the opportunity to validate that the
           technology can be integrated into the preliminary product design
           and to develop good product development estimates prior to the
           start of system development. As stated earlier, commercial
           companies we visited had a dedicated phase at the end of the
           technology development process for shifting technology
           responsibilities to their product lines. We recently reported that
           almost three-fourths of the programs started since the acquisition
           policy was revised in 2000 began with critical technologies that
           were not ready for product development.12 Seven of the nine
           programs we reviewed more indepth, about 80 percent, were approved
           to begin development even though program officials reported that
           technologies were below readiness levels required for entering
           product development. This included programs like the Future Combat
           System and the Joint Strike Fighter.

           We found DOD generally views immature critical technologies at the
           beginning of development as an acceptable risk as long as there is
           a plan to mature the technologies by the time the program reaches
           its design readiness review. In effect, the department views risk
           management plans as an acceptable substitute for demonstrated
           knowledge. For example, the Navy's Multi-Mission Maritime Aircraft
           program had none of its critical technologies mature at program
           initiation. Instead of holding the program to the acquisition
           policy criteria for entering development, the decision maker
           simply directed the Navy to work with the Office of the Secretary
           of Defense to implement risk mitigation and technology maturation
           plans during the integration phase of system development.

           Second, under DOD's current funding structure, the transition of
           technology-which should occur prior to the beginning of product
           development and milestone B-is funded and managed by acquisition
           programs. This contrasts with the approach used by commercial
           companies, where the lab manages and funds these activities.
           Commercial firms find that holding their labs accountable for the
           management and funding of technology as it transitions to a new
           product forces them to deliver more quickly and efficiently and
           allows the product lines to focus on product development and the
           risks associated with it. Figure 11 illustrates the difference in
           responsibility.

11GAO, Defense Acquisitions: Major Weapon Systems Continue to Experience
Cost and Schedule Problems under DOD's Revised Policy. GAO-06-368
(Washington, D.C.: April 13, 2006).

12 GAO-06-368 .

Figure 11: Accountability for Management and Funding of Technology

As shown above, in the commercial companies we visited, product developers
are allowed to act like customers for emerging technologies; they are not
required to accept, manage, and fund technology risk. This significantly
improves the chances of their products reaching the market quickly, at
predictable cost, and with high quality. In DOD, major weapon system
acquisition programs pull technologies that are not yet ready for a
product in order to meet that product's requirements. The S&T community,
although still called upon to review technology readiness, has no funds at
stake and is no longer responsible for the risk from technology at that
point. This is a major contributor to the significant cost overruns and
late deliveries of major weapon systems to the warfighter in recent years.

DOD Is Adopting Transition Tools, but Use Is Not Widespread

DOD is using several tools we found were also being used by commercial
companies to facilitate the transition of technology to weapons programs.
For example, the services are using technology transition agreements for
some of their technology projects, DARPA is using relationship managers to
address transition issues, the services have established boards to oversee
and manage a portion of their technology portfolios, and DOD has several
new programs, including the Joint Capability Technology Demonstration
program to speed a technology's transition to a weapon system. However,
because of the newness or small amount of funding associated with these
tools, widespread use has yet to occur. We also found that DOD is using
technology-specific and status metrics, but does not have sufficient
measures in place to assess the impact of their technology investments or
technology transition processes.

  Technology Transition Agreements

The services have recently begun using technology transition agreements to
formalize technical expectations labs must demonstrate in order for
acquisition programs to transition technologies into their programs. DOD
science and technology officials believe the agreements are a useful way
to hold the technology developers accountable and gauge lab progress
toward meeting technology specifications. We are encouraged by their use
of agreements, particularly in the Navy. Agreements we reviewed, however,
did not contain some of the information or metrics commercial companies
believe is valuable to track, and use thus far has been limited.

The agreements we reviewed contained some of the same elements included in
agreements used by leading companies we visited, such as a description of
the technology project, key technology developer and weapon system
personnel associated with the project, and specific performance
characteristics that the lab must meet for transition to occur. In
addition, they identify the amount of funding acquisition programs will
have for transition and when that funding will be available. Unlike the
leading companies we visited, however, these agreements did not typically
require the technology developer to demonstrate cost metrics for the
technology to be included in a weapons program. Commercial companies, for
example, include an assessment of manufacturing costs that could be
expected if a technology were to be included on a new product and they
usually demand prototypes of the technology as demonstration of readiness
to transition. A recent Defense Science Board Report highlighted the
importance of addressing manufacturing concerns during technology
development. Excerpts are shown in figure 12.

Figure 12: Excerpts from a Defense Science Board Task Force Report on the
Manufacturing Technology Program

"Immature technologies and manufacturing challenges have a significant     
impact on DOD's ability to rapidly and affordably transition technology to 
the war fighter."                                                          
                                                                              
"S&T program managers often believe that affordability and manufacturing   
issues are not relevant concerns in 6.3 programs, focusing instead on      
fabrication of test and evaluation and prototype articles. But this line   
of thinking leads to higher costs later in a program, when manufacturing   
concerns are addressed after technical designs are considered `ready.' "   
                                                                              
"In order to achieve the objective of lower cost equipment, manufacturing  
concerns must be addressed earlier in the program life cycle. Production   
and support costs need to become a component of key technical design       
requirements, before the final stages of development when technologies are 
released for prototyping."                                                 

Source: Defense Science Board study (data); GAO (presentation and
analysis).

We also found that the services' use of technology transition agreements
has been very limited. For example, we found that the services have a
combined total of 224 applied and advanced technology projects that they
have identified as candidates for transition, of which 146-or about 65
percent-have technology transition agreements. The Navy accounted for the
greatest percentage, with 90 of its 115 projects having an agreement,
almost 80 percent. However, there are hundreds of other military services
applied and advanced technology research projects that were not selected
as candidates where agreements may also be useful. As stated earlier,
commercial companies we visited had agreements for nearly all technology
projects. The agreements were updated at least annually or sooner if
particular tasks were completed. Lab officials use metrics included in the
agreements, as well as other criteria, to determine if the project should
be continued.

It should also be pointed out that while technology transition agreements
are useful tools for firming weapons program commitment to transition,
they do not guarantee transition success. For example, a Navy official
told us that about one-third of the projects that had been expected to
transition in fiscal year 2005 did not transition for the following
reasons:

           o  The technology did not meet the cost parameters required for
           transition to occur.

           o  Weapon system requirements changed, forcing changes to the
           original agreement.

           o  Program offices did not identify dollars in its budget to
           transition the technology.
           o  A program office decided the technology was too risky compared
           with current hardware in production.

           Air Force and Army officials told us they do not track information
           on why their technologies do not transition. However, they have
           experienced similar problems as the Navy. They also stated that
           some lab technologies do not transition because prime contractors
           decide to use technologies developed in-house, even though DOD
           technology developers met the metrics included in the transition
           agreements. Because agreements are usually established between the
           technology developers and program offices prior to the selection
           of prime contractors, there is no assurance that the technologies
           will transition. Even in cases where a prime contractor is known,
           such as for ongoing development programs, the prime contractor is
           not a party in the agreement.

           Relationship Managers
			  
			  With the exception of DARPA, DOD does not use relationship
           managers in the same manner as leading private companies.
           According to an Air Force lab official, relationship managers
           market technologies being developed by the labs or gather data
           about ongoing projects for senior lab management. Most
           communication about technology transition in DOD takes place
           through integrated product teams or during annual reviews of
           technology projects by the senior-level oversight boards for each
           of the services. Use of relationship managers for these purposes
           are helpful, but the managers do not necessarily serve as points
           of contact within the labs and acquisition communities, do not
           devote time toward efficiently transitioning technologies to
           multiple weapon system programs, and do not help identify and
           address systemic transition problems.

           Within DARPA, senior officers, called operational liaisons, focus
           on marketing and transitioning DARPA-sponsored technologies.
           According to the DARPA director, the liaisons have been very
           helpful with transitioning technologies because they are well
           practiced at using the command chain of their respective services
           and finding the right service contact at the right time. The
           liaisons

           o  provide operational advice for planning and strategy
           development;

           o  provide an understanding of service perspectives, issues and
           needs so that potential customers can be identified and effective
           agreements can be written;

           o  draft and coordinate agreements between DARPA and the services;
           and

           o  direct technology insertion in the services.

           Figure 13 describes the impact that operational liaisons have had
           on transitioning a DARPA-sponsored program called Boomerang. We
           believe that DARPA's approach could serve as a model for how the
           military services might establish more formal roles for
           communication between the S&T and acquisition communities.

           Figure 13: DARPA, a Success Story
			  
			  The DARPA director credits operational liaisons for the quick transition   
of the Boomerang, an acoustic shot-detection system, from the lab to       
troops in Iraq. DARPA developed the system in response to feedback from    
Iraq that convoys were being engaged by snipers yet remained unaware of    
sniper attacks until a windshield was broken, a soldier was hit, or a      
vehicle was visibly damaged upon inspection at the end of the convoy       
mission.                                                                   
                                                                              
Within 60 days of an urgent Army request, DARPA fielded the first          
Boomerang system. But DARPA's director said the system did not hold up     
well in the extreme weather conditions and under wartime conditions. As    
Boomerang II was being prepared for fielding, the director said the        
operational liaisons helped craft a more realistic concept of operations,  
training package, and logistical support package to ensure that Boomerang  
II not only was technologically ready for combat but was properly          
supported with spare parts, maintenance facilities, maintenance personnel, 
training, and lessons-learned feedback to DARPA and the Army. The liaisons 
also ensured that the Army acquisition community was alerted to Boomerang  
II's deployment so product developers would be ready to evaluate the final 
product for movement into the more traditional acquisition process.       

           Source: DARPA (data); GAO (presentation and analysis).

           DOD Programs to Aid Transition

           The military services and the Office of the Secretary of Defense
           have a variety of programs to help transition technologies to
           weapons systems or directly to the warfighter. These programs,
           some of which are relatively new and others that have been around
           for several years, have met with some success. However, these
           programs represent a small portion of $13 billion DOD spends on
           science and technology development. As such, they cannot
           single-handedly overcome transition problems, but rather
           demonstrate various ways to ease transition.

           In recent years, each of the military services has established
           senior-level boards to oversee technology programs that include
           advanced and in some cases applied research projects. This
           includes the Air Force's Advanced Technology Demonstration
           program, the Army Technology Objectives program, and the Navy's
           Future Naval Capabilities program. The boards, which are comprised
           of representatives from the lab, acquisition, and warfighter
           communities, are responsible for selecting projects to be
           developed, allocating funding to those projects, and reviewing the
           projects' progress on an annual basis. The boards emphasize
           transition planning and can address obstacles to successful
           transition. According to officials, these programs represent about
           $1.3 billion of DOD's S&T budget.

           The Advanced Concept Technology Demonstration program and the
           newly established Joint Capability Technology Demonstration
           program were initiated by DOD as a way to get technologies that
           meet critical military needs into the hands of users faster and at
           less cost than by the traditional acquisition process. Under these
           programs, military operators test prototypes that have already
           been developed and matured in realistic settings. If they find the
           items to have military utility, DOD may choose to buy additional
           quantities or just use the items remaining after the
           demonstration. Fiscal year 2006 was the first year of a 3- to
           5-year period where the current Advanced Concept Technology
           Demonstration program will be phased out in favor of the new Joint
           Capability Technology Demonstration program. Department officials
           believe the new joint program offers improvements over the earlier
           program in that capabilities will be demonstrated 1-2 years
           earlier, there is a greater focus on combatant command needs
           during the selection process, and the Office of the Secretary of
           Defense provides significantly more funding during the first 2
           years of the demonstration project.

           A big difference between the two programs is that the new joint
           program is expected to provide a better path for the transition of
           technologies because it includes funding for both advanced
           technology development and advanced component development and
           prototypes. The Advanced Concept Technology Demonstration program
           only includes funding for advanced technology development. It must
           rely on the acquisition community to identify advanced component
           development and prototype funding for transitioning the
           technologies. An additional benefit is that the Office of the
           Secretary of Defense is partnering with the Naval Postgraduate
           School to conduct business case analysis for completed
           demonstration projects. This type of analysis is expected to aid
           decision makers in evaluating alternative approaches to the
           allocation of scarce resources competing for transition funds.

           The Manufacturing Technology Program is aimed at quickly
           identifying and solving production problems associated with
           technology transition. It focuses on the needs of weapon system
           programs for affordable, low-risk development and production. For
           example, one project, which was given $730,000, was geared toward
           improving the reliability and strength of large-diameter fasteners
           used to attach various components on the Seawolf and Virginia
           Class submarines. Corrosion concerns required that the existing
           fasteners be replaced at periodic intervals to preclude
           catastrophic failure. Navy and commercial companies evaluated a
           wide variety of materials for strength and corrosion resistance to
           determine the best candidate material and then adjusted the
           material manufacturing process to obtain the required strength and
           toughness levels to ensure the fasteners could be produced without
           extraordinary measures or a deterioration of the material
           properties. Among other things, program officials expect the
           project to result in cost avoidance of $1.1 million per fielded
           submarine per year and improved reliability.

           There are other, smaller programs that also focus on transition.
           Two of these are the Foreign Comparative Testing Program, which
           focuses on identifying, evaluating, and procuring technologies
           that have already been developed and tested in other countries,
           and the Technology Transition Initiative, which focuses on
           speeding the transition of technologies developed by DOD's S&T
           programs into acquisition programs. These programs received about
           $37 million and $29 million in fiscal 2006, respectively. The
           Foreign Comparative Testing Program, for example, successfully
           evaluated a South African mine-protected clearance vehicle that
           will protect soldiers from the effects of landmine explosions
           during route clearance operations. As of June 2006, 61 of these
           vehicles have been delivered to the Army, Navy, and Marine Corps.

           Metrics
			  
			  For the most part, the military services track and use only a few
           of the metrics we found at commercial companies. As stated
           earlier, they have some technology-specific metrics in their
           technology transition agreements. In addition, they have status
           metrics to track some of the on-going, completed, and transitioned
           projects that are funded by the Air Force, Army, and Navy S&T
           communities as well as the number of projects with technology
           transition agreements. However, the services have few metrics that
           would enable them to gauge the impact of their investments and the
           effectiveness of their processes for developing and transitioning
           technology.

           DOD officials told us that establishing good, quantifiable metrics
           to measure transition success is difficult to do. Some of the
           challenges they face include determining

           o  how long to track a technology after it has transitioned,

           o  how to track and distinguish transition success when the same
           technology is used on multiple weapons programs, and

           o  how much credit to take when only portions of a technology
           transition to a weapon program.

           Further, DOD S&T officials do not believe that they have enough
           resources to track technology once technology developers finish
           working on a project. They say that it would be very
           labor-intensive to track long-term measures, such as the impact of
           transitioned technology in terms of cost savings and improved
           performance. Instead, the services rely heavily on projected cost,
           schedule, and performance improvements that are required by S&T
           program management as part of the project selection process.
           Although progress in meeting technology expectations is monitored
           throughout technology development, once technology developers are
           finished working on a technology, little is done to determine if
           it actually went onto a weapon system and if it is being used in
           the field.

           Last year, we reported that the Office of the Secretary of Defense
           had difficulty tracking the impact of three small technology
           transition programs it oversees-the Technology Transition
           Initiative, the Defense Acquisition Challenge Program, and the
           Quick Reaction Fund.13 Nevertheless, we pointed out that there may
           already be readily available starting points within the department
           to capture more information regarding return on investments. For
           example, we pointed out that the DOD Foreign Comparative Testing
           Program has established metrics to measure the health, success,
           and cost-effectiveness of the program and has a database to
           facilitate return on investment analyses. Further, a study by the
           Naval Postgraduate School to identify metrics for the Advanced
           Concept Technology Demonstration program may also be useful to the
           military services for assessing transition success and impact.

           Differences in Environment and Incentives Contribute to Different 
			  Practices and Outcomes

           There are critical differences between the environments and
           cultures of private world-class companies and DOD that must be
           recognized before tangible progress can be made in establishing
           more efficient practices for transitioning technologies to major
           weapon system acquisition programs. Examples from past initiatives
           serve as reminders that just changing the mechanics of technology
           transition processes, without changing the environment that
           determines incentives, may not produce better outcomes. Private
           companies operate in a competitive environment that demands speedy
           delivery of innovative, high-quality technologies and products to
           satisfy market needs. If the company cannot meet those criteria,
           it will cease to exist in that environment. On the other hand, DOD
           operates in an environment with a complex variety of "customers."
           These complex relationships often hinder the ability of DDRE, its
           "corporate" component, to provide strategic leadership similar to
           that of the private company. Success in this environment is often
           based on a single service's ability to launch a program to address
           critical needs and to secure annual funding for the program. The
           challenge for DOD and congressional decision makers may not lie so
           much in the "how to" aspects of technology transition as in
           creating stronger and more uniform incentives that encourage the
           S&T and acquisition communities to work together to deliver mature
           technologies to programs.

           Private firms must continually deliver innovative, reliable
           products to market very quickly to prosper. The delivery-oriented
           environment that private firms live in creates a need for
           strategic simplicity and directness from leadership and forces
           product lines and labs to measure success in terms of lower costs
           and increased revenue and market share. The ability to deliver
           new, innovative, reliable products of high quality to market as
           soon as possible drives revenue and market share. This environment
           creates incentives to maintain an efficient technology base that
           is focused on market needs and can efficiently transition feasible
           technologies into new products, keeping costs down while achieving
           greater revenues. Because commercial firms understand that they do
           not succeed until a new, innovative product is delivered to the
           customer, they view successful transition of technologies to
           products as a critical part of their value chain. The corporation,
           therefore, insists on accountability from and supports its
           technology developers with a strong strategic process for ensuring
           that its product portfolio is balanced. It does not want too many
           programs vying for scarce investment dollars.

           For example, IBM officials stated that poor strategic planning was
           one of the problems the company had to address after it incurred
           an $8 billion loss in 1992-1993. According to company officials,
           technologies no longer could be developed for the sake of
           development; they had to be aligned with a product line and have
           market potential to be funded. IBM labs now conduct a situation
           analysis that takes an internal and external look at technologies
           being developed and a gap analysis to determine where IBM might be
           behind in technology development.

           At 3M, meeting market imperatives is critical and senior
           leadership has set a goal to increase sales 5 to 8 percent
           annually through new products. In this environment, the company
           has established a companywide initiative referred to as 2X/3X.
           Generally, the goal is to double the number of new ideas going
           into the product development pipeline and to triple the market
           success of the products coming out of the pipeline. Lab and
           product line managers are evaluated and held accountable for the
           success of the initiative based upon their ability to meet sales
           goals and properly resource their projects. According to 3M's 2004
           annual report, the initiative is producing strong results. 3M
           officials believe their culture is well aligned with this
           competitive environment. For example, technologists participate in
           companywide technology forums to discuss problems and share
           information. A technology council, comprised of about 80 lab and
           technical directors, meets monthly to share ideas, experiences,
           and best practices among the labs.

           At Motorola, the corporate vision is known as "seamless mobility."
           Senior leaders have aligned the company's global
           resources-including its $3.1 billion annual research and
           development budget and a worldwide research network-to help
           achieve this goal. According to Motorola officials, the seamless
           mobility vision has been instrumental in uniting Motorola labs and
           product lines over the past 3 years toward a common goal.

           On the other hand, DOD operates in an environment that often
           hinders the ability of its corporate component-DDRE-to provide
           strategic leadership similar to the private company. Not only must
           the department worry about national security imperatives, it must
           also answer to Congress and its oversight agencies; manage a
           stove-piped, parochial culture; ensure the public's trust that tax
           dollars are wisely and fairly spent; and incentivize an industrial
           base to deliver cutting-edge, often very risky technologies and
           products. This is a complex set of deliverables by any measure. It
           creates an environment that makes it difficult for DDRE to lead
           and has contributed to an undisciplined strategic planning
           process. Success in this environment is often based on a service's
           ability to launch a program to address critical needs and to
           secure annual funding for the program. It is characterized by
           fierce competition among the services to win scarce funding to
           begin major weapon system acquisition programs, regardless of the
           readiness of technologies to meet far-reaching requirements. As a
           result, programs are pressured to distinguish themselves from
           other programs previously developed by promising new, enhanced
           features such as being faster and/or more lethal than anything
           else. It is very common for these new programs to include
           requirements for new technologies with overly optimistic
           assessments of technological feasibility, risk, and delivery
           schedules. This environment and these incentives breed uncertainty
           and risk and, as a result, cost overruns and delivery delays.

           Conclusions
			  
			  In DOD, delivering mature technologies to weapon system
           acquisition programs at the right time continues to be a
           challenge. Rather than addressing technology issues in a science
           and technology environment before product development starts,
           acquisition programs carry significant technology risk into
           product development. This brings with it a high risk that costs
           will rise and deliveries to the warfighter will be delayed. In
           fact, there is strong evidence that acquisition programs that
           start with immature technologies encounter significantly poorer
           acquisition outcomes than others. This approach is in sharp
           contrast to the approach taken in the commercial world.
           High-performing companies solve technology challenges in the S&T
           environment. To do this, they have put in place processes and
           adopted techniques that are pertinent to DOD. Strong strategic
           planning defines critical investment priorities, and a structured
           process defines the path towards a technology's transition to
           product development. This transition is supported by technology
           transition agreements that hold the research labs accountable for
           what they must deliver. They also include clarification of the
           responsibilities the product developer has in accepting new
           technologies. Metrics are used to force demonstration of relevancy
           and feasibility at key points in the process and gauge the success
           of individual projects and the process itself. Funding for
           technology development largely comes from the corporate level,
           with the research labs having responsibility for technology
           development until technology is matured and transitioned to the
           product line. This is critical to success, because it allows
           product developers to play a customer role. They are allowed to
           say no to technologies that are not ready for their programs and
           can focus on their job at hand-product integration, supplier
           management, and quality.

           DOD has adopted some of these practices. The department has begun
           using technology transition agreements and relationship managers
           and has initiated programs that place greater emphasis on
           technology transition planning. However, the reach of these
           initiatives is limited, and there is no unified, corporate
           approach to using them at this point. We recognize that the
           environment and incentives for DOD are very different than those
           of commercial firms we visited. For commercial practices to work
           on a broad scale, the DOD environment must be conducive for
           applying such practices. DOD senior leadership is a critical
           factor in providing this direction and vision as well as in
           maintaining the culture of the organization. The department must
           devote greater attention to strategic planning and technology
           development processes so that resources are spent on technologies
           that can and will be transformed into capabilities for the
           warfighter. In addition, the S&T and acquisition communities must
           work together to expand the use of technology transition
           agreements and relationship managers. Finally, the department
           should examine the way it currently funds technology transition.
           It may benefit from an examination of commercial companies'
           methods in this regard. They hold their technology developers
           accountable for delivering relevant technologies that are ready to
           be integrated into new products, but they empower them to succeed
           by providing adequate funding to get the job done.

           Recommendations for Executive Action

           DOD should take steps to improve its transition of technologies to
           more efficiently deliver capabilities to its warfighters. The
           Defense Science Board has recommended strengthening of DOD's
           strategic planning function, and we believe this would move the
           department in the right direction. We believe a disciplined, gated
           approach for technology development, supported by technical and
           business criteria, would provide adequate knowledge to acquisition
           program managers about the risk of including particular
           technologies on specific weapons programs. It would also provide a
           more systematic way for the S&T community to continually assess
           the relevancy, feasibility, and potential transition commitment of
           its technologies and make decisions about future investments.
           DOD's current process lacks specific decision points with "go/no
           go" decisions. As such, we recommend that the Secretary of Defense
           take the following actions:

           o  develop a gated process for developing and transitioning
           technologies that establishes a transition phase and defines
           activities that should occur during this phase, and

           o  include specific criteria to support continued funding of
           specific projects in that process.

           We also believe greater use of tools, such as technology
           transition agreements, relationship managers, and metrics, could
           help the department improve its ability to deliver mature
           technologies when needed, address transition issues more quickly,
           and gauge the impact of their science and technology investments
           and lab processes. Therefore, we recommend that the Secretary of
           Defense:

           o  expand the use of technology transition agreements to applied
           and advanced development projects;

           o  include additional metrics in technology transition agreements
           to provide S&T and acquisition program managers demonstrated
           knowledge about the manufacturing readiness, producibility, other
           benefits, and risks of including the technology on a weapons
           program;

           o  expand the use of relationship managers by designating people
           at various levels in both the S&T and acquisition communities to
           address systemic transition issues and those related to specific
           weapon system programs. Also, define responsibilities for each
           level of relationship manager;

           o  adopt additional process-oriented metrics, such as the
           percentage of advanced technologies that-once past milestone A of
           the acquisition process-transitioned into a weapons program or
           were fielded and the cycle time from milestone A to milestone B as
           a way to measure the effectiveness of S&T processes and the impact
           of science and technology investments;

           Commercial companies fund technology development and transition
           activities in their labs and hold the labs accountable for
           delivering mature technology to their product lines. As such, we
           are recommending that the Secretary of Defense

           o  Set aside a portion of advanced component development and
           prototype funds for the S&T community to manage the transition of
           technologies to acquisition programs. For this funding to be used
           effectively, it will require the discipline provided by corporate
           leadership in defining priorities, processes, and metrics.

           Agency Comments and Our Evaluation
			  
			  DOD provided us with written comments on a draft of this report.
           DOD generally concurred with the recommendations in our report to
           improve its ability to transition technology to our warfighters.
           In doing so, it emphasized that that commercial industry program
           managers operate in an environment driven by profit and market
           opportunities, while DOD is organized to support our warfighters.
           We understand the basic differences in the environments, but
           believe there are many lessons that can be gleaned from our
           commercial visits. While it is true that private companies are
           motivated by profit, they can not achieve a profit without being
           focused on the timely delivery of products to the market. We note
           in this and other best practices reports that commercial firms
           have significant pressures on them to deliver cutting edge
           technologies quickly. They often bet their very existence on
           delivering new products and technologies before anyone else does
           so. In some cases, such as medical systems, their customers have
           urgent, life-or-death needs, similar to those of the warfighters,
           that demand very complex technological solutions. The real
           difference between these companies and the department is that they
           understand the futility of promising more than can be delivered to
           the market at any given point. They understand that, in the final
           analysis, they only succeed-or survive-only by delivering needed
           capability. On the other hand, the department's acquisition
           programs consistently accept immature technologies into product
           development that become a major cause of cost increases, schedule
           delays, and ultimately very late delivery to the warfighter who
           needs the equipment. The list of programs that have delivered late
           is extensive.

           Rather than focus on the different environment in the commercial
           world, DOD should focus on the practices used in the commercial
           world that allow it to deliver mature technology to product
           development (then, to the user) quickly and efficiently, an
           outcome woefully lacking in the department.

           Specially, of the six recommendations we made, DOD concurred with
           two recommendations, partially concurred with two others, and did
           not concur with the final two. DOD's comments appear in appendix
           II.

           DOD concurred with the recommendation that called for establishing
           a gated process for developing and transitioning technologies. In
           its response, the department stated that it has a gated process
           for programs that have a formal milestone A decision point and
           that technology maturity and technology readiness assessments are
           used in this process. DOD explained that the technology maturity
           assessments, in particular, often lead to risk mitigation plans,
           which include explicit gates that immature technologies must pass
           through to become qualified for adoption.

           We are not confident that DOD's implementation plans are fully
           responsive to this recommendation. First, the department does not
           have a process similar to those that we found at commercial
           companies to actively manage and make investment decisions across
           all technologies and determine if an individual technology is
           ready to transition. In its comments, DOD refers to a process for
           programs beginning at milestone A. However, we have reported that
           80 percent of the programs we reviewed that passed milestone B
           since 2000 did not have a milestone A.

           Second, we found that even when technology readiness assessments
           indicated that technologies were not ready to be included in an
           acquisition program, DOD often decided to use them anyway. Third,
           risk mitigation plans used by DOD are subjective engineering
           judgments and are of limited value when evaluating the transition
           readiness of technologies.

           The department also concurred with our recommendation that it
           expand the use of technology transition agreements to applied and
           advanced development projects. However, in its response, DOD did
           not identify specific actions it plans to take to implement this
           recommendation.

           DOD partially concurred with the recommendation that it include
           additional metrics in technology transition agreements. In its
           response, the department indicated that it has taken steps to
           incorporate manufacturing readiness assessments as a means to
           improve technology readiness and that any additional metrics
           should be project-specific and included as exit criteria as part
           of the technology transition agreement. We are encouraged by DOD's
           recent development of manufacturing readiness levels and believe
           they should be a mandatory tool for the S&T and acquisition
           program managers to use to assess the manufacturability of a new
           technology. While we agree that the department should develop
           additional metrics on a project-by-project basis, we also believe
           that a more holistic approach, as depicted earlier in the report,
           is needed to determine the transition readiness of a particular
           technology. Transition readiness is not solely dependent upon the
           technical maturity achieved by S&T, but is also dependent upon the
           readiness of an acquisition program to accept the new technology
           and therefore should include an assessment of other factors such
           as cost, benefit, risk, scalability, and acquisition program
           endorsement.

           DOD also partially concurred with the recommendation to expand the
           use of relationship managers to address systemic transition issues
           and those related to specific weapon system programs. In its
           response, DOD indicated that it relies on written documents-the
           technology maturity assessments and technology transition
           agreements-to facilitate communication between the S&T and
           acquisition communities, particularly at the executive level. As
           stated earlier, these documents do not address many of the factors
           that could hinder transition. As DOD states in its comments,
           technology transition is referred to as a "body-contact sport." We
           agree, and this is why our recommendation is aimed at more direct
           person-to-person communication. Leading companies believe
           relationship managers play a key role in their ability to
           successfully transition technology. While DOD did acknowledge that
           its S&T investment would benefit from expanded emphasis by staff
           at the execution levels on technology transition issues, it did
           not directly address the need for midlevel relationship managers.
           We continue to believe DOD would benefit from the use of midlevel
           relationship managers. These managers provide the back and forth
           communications between the S&T and acquisition communities that
           help ensure that the right technology is being developed in the
           time frame needed. These managers are also uniquely positioned to
           ensure that technology is applied to multiple platforms when
           applicable and can identify systemic transition problems that
           should be addressed.

           DOD did not concur with the recommendation that it should develop
           additional process-oriented metrics, even though it stated that it
           is installing a process to do this through the use of technology
           maturity and technology readiness assessments. DOD stated that it
           does not want to commit to overly burdensome metrics that may be
           more oriented to measuring the process for the sake of
           measurement. Like DOD, we do not believe the department should
           develop metrics just for the sake of measurement. Rather, we
           believe DOD should develop and use metrics that allow DOD,
           Congress, and taxpayers to gauge the effectiveness of DOD S&T
           investments, which are expected to reach about $13 billion this
           year. The metrics that DOD references-those derived from
           technology maturity and technology readiness assessments-are
           project-level metrics rather than portfolio and process metrics
           that would allow the department to analyze its investment and make
           adjustments appropriately.

           DOD also did not concur with the recommendation to set aside a
           portion of advanced component development and prototype funds for
           the S&T community to manage the transition of technologies to
           acquisition programs. In its response, DOD stated that acquisition
           programs are best suited for transitioning technology because they
           have the training and discipline to field systems and have the
           responsibility to ensure a stable design, identify a responsive
           and responsible contractor, manage execution, and plan for life
           cycle support of the system. While DOD discusses transition to the
           warfighter, our recommendation is aimed at transitioning
           technology from S&T to acquisition programs. We found that in the
           commercial world, the training and discipline for transitioning
           technologies to product development is managed by the technology
           development community and is adequately funded. DOD believes its
           current approach of setting aside a small portion of S&T funds for
           transition through programs such as the Advanced Concept
           Technology Demonstration program, Joint Capability Technology
           Demonstration program, the Technology Transition Initiative,
           Manufacturing Technology program, and the Foreign Comparative
           Testing program is appropriate. We continue to believe DOD's
           approach to funding transition is flawed and that small pots of
           money for specific transition activities offer a piecemeal
           solution to a more systemic problem. DOD currently uses some of
           its advanced component development and prototype funds for S&T
           activities, including transition. However, acquisition programs
           typically carry out and fund these activities. We believe the S&T
           community should be responsible for these activities and as such,
           should be given the appropriate level of funding to carry them
           out. We believe DOD would be better positioned to develop and
           deliver weapon systems more quickly to its warfighters if the S&T
           community was responsible for developing, maturing, and
           transitioning mature technologies to the acquisition community and
           if the acquisition community focused solely on product development
           activities and delivering weapon systems to the warfighter That
           being said, we recognize that the acquisition and warfighting
           communities play critical roles in this process and therefore must
           continue to work toward setting realistic program requirements and
           establishing an evolutionary approach for developing new weapon
           programs.

           We are sending copies of this report to the Secretary of Defense,
           the Director of the Office of Management and Budget, and
           interested congressional committees. We will also make copies
           available at no charge on the GAO Web site at http://www.gao.gov .

           If you have any questions about this report or need additional
           information, please contact me at (202) 512-4841 or
           [email protected]. Contact points for our Offices of Congressional
           Relations and Public Affairs may be found on the last page of this
           report. Key contributors to this report were Karen Zuckerstein,
           Assistant Director; Cheryl Andrew; Lily Chin; Sameena Ismailjee;
           and Sean Merrill.

           Michael Sullivan Director Acquisition and Sourcing Management

           Appendix I: Objectives, Scope, and Methodology
			  
			  This report examines the Department of Defense's (DOD) efforts to
           improve its technology transition processes, with a focus on
           identifying specific management, funding, and organizational
           practices that could improve technology transition and weapon
           system outcomes. Specifically, our objectives were to (1) identify
           techniques that commercial companies use to transition mature
           technologies before the start of product development and (2)
           assess the extent to which DOD is using these techniques.

           We used a case study approach to compare and contrast DOD and
           leading commercial companies' practices. Companies were selected
           on the basis of such factors as the amount of money spent on
           research and development activities over the past several years
           and the percentage of change in research and development spending.
           For the most part, we selected Fortune 500 companies that were in
           the top 100 for research and development spending and had not
           experienced major cutbacks in this funding. We also took into
           account the type of products each company develops and selected
           companies representing several different business sectors. Below
           are descriptions of the four companies featured in this report.
			  
			  Boeing

           Boeing is a leading aerospace company and the largest manufacturer
           of commercial jetliners and military aircraft combined, with
           capabilities in rotorcraft, electronic and defense systems,
           missiles, satellites, launch vehicles, and advanced information
           and communication systems. Boeing Phantom Works develops advanced
           systems solutions, such as advanced homeland security and air
           traffic management, as well as breakthrough technologies, such as
           advanced avionics and composite materials that are intended to
           significantly improve the performance, quality, and affordability
           of aerospace products and services. We met with research
           officials, as well as product line officials representing both its
           commercial and defense sectors in Seattle, Washington.

           3M
			  
			  3M is a diversified technology company with a worldwide presence
           in various markets, including consumer and office; display and
           graphics; electronics and communications; health care; industrial;
           safety, security, and protection services; and transportation.
           With more than 55,000 products, 3M invests more than $1 billion
           annually in research and development and related activities
           associated with 30-plus core technologies. The company was awarded
           nearly 500 U.S. patents in 2005. We met with research, business
           unit, and government programs officials at 3M headquarters in St.
           Paul, Minnesota.

           IBM
			  
			  IBM is one of the world's largest technological companies,
           spending about $3 billion annually on research and development
           activities. It is the largest supplier of hardware, software, and
           information technology services and pioneered the development and
           implementation of on-demand business. With 3,248 U.S. patents, IBM
           earned more patents than any other company for the 12th
           consecutive year in 2004. In the past 4 years, IBM inventors
           received more than 13,000 patents-approximately 5,400 more than
           any other patent recipient. We met with research and product
           development officials at the Watson Research Center in Hawthorne,
           New York.

           Motorola
			  
			  Motorola is a Fortune 500 global communications leader that
           provides seamless mobility products and solutions across
           broadband, embedded systems and wireless networks for products in
           homes, automobiles, and workplaces. Motorola spent about $3.5
           billion in 2005 on research and development activities.
           Approximately 22,000 professional employees were engaged in
           research activities during 2005. We met with research officials
           and product development officials, as well as partnership
           development managers at its offices in Schaumburg, Illinois.

           For each of the companies, we interviewed senior management
           officials knowledgeable about research and development activities
           to gather consistent information about processes, practices, and
           metrics the companies use to transition technology smoothly. In
           particular, we discussed their (1) strategic planning process for
           identifying and prioritizing customer needs, (2) technology
           development process used to fund and mature technologies required
           to meet customer needs, (3) tools used to facilitate communication
           between labs and product lines to transition technology, and (4)
           technology transition process, including when transition occurs,
           the organizations involved, and how technology is funded
           throughout the transition phase. We synthesized information from
           GAO's past best practices work about technology and product
           development.

           To determine DOD's practices for transitioning technology, we met
           with science and technology (S&T) and acquisition officials to
           discuss the same categories of questions listed above that we
           asked leading commercial companies. The following is a list of the
           organizations we met with:

           o  Director, Defense Research and Engineering, Alexandria,
           Virginia

                        o  Director, Plans and Programs
                        o  Deputy Under Secretary of Defense, Advanced
                        Systems and Concepts
                        o  Director, Defense Advanced Research Projects
                        Agency

           o  Air Force

                        o  Office of the Assistant Secretary of the Air Force
                        for Acquisition (Science, Technology, and
                        Engineering), Alexandria, Virginia
                        o  Air Force Material Command, Wright Patterson Air
                        Force Base, Ohio
                        o  Air Force Research Labs, Wright Patterson Air
                        Force Base, Ohio
                        o  Joint Unmanned Combat Air Systems Program Office,
                        Wright Patterson Air Force Base, Ohio

           o  Army

                        o  Assistant Secretary of the Army Acquisition,
                        Logistics and Technology, Alexandria, Virginia
                        o  Deputy Assistant Secretary for Research and
                        Technology Chief Scientist, Alexandria, Virginia
                        o  Joint Tactical Radio System Cluster 5 Program
                        Office, Alexandria, Virginia

           o  Navy

                        o  Deputy Assistant Secretary of Navy for Research,
                        Development, Test and Evaluation, Alexandria,
                        Virginia
                        o  Office of Naval Research, Alexandria, Virginia
                        o  Naval Air Systems Command, Patuxent River Naval
                        Air Station, Maryland
                        o  Naval Sea Systems Command, Washington Navy Yard,
                        D.C.
                        o  CVN-21 Program Office, Washington Navy Yard, D.C.
                        o  Multi-mission Maritime Aircraft Program Office,
                        Patuxent River Naval Air Station, Maryland
                        o  Aircraft Launch and Recovery Equipment Program
                        Office, Patuxent River Naval Air Station, Maryland

           At each of these locations, we collected appropriate documents
           that describe the various programs, organizations,
           responsibilities, and funding. We obtained examples of key
           documents, such as technology transition agreements or memorandums
           of agreement that are used to solidify agreements made between
           labs and weapon programs. We reviewed DOD and military service
           strategic plans and research, development, test, and evaluation
           funding documents. In addition, we reviewed documents required as
           part of the selection and oversight process for the Air Force
           Advanced Technology Development program, the Army Technology
           Objectives program, and the Future Naval Capabilities program. We
           relied on previous GAO best practices and weapon system reports
           that highlight cost and schedule impacts of launching new weapon
           programs with immature technology. A list of these reports can be
           found at the end of this report.

           Appendix II: Comments from the Department of Defense

Appendix III: DOD Research, Development, Technology, and Engineering
Budget 

Dollars in billions                                                
                                                          Which DOD   Budget  
                                                          community   (fiscal 
                    Budget                                controls    year    
Name             activity Description                  spending    2006)   
Basic research          1 Basic research is systematic S&T         $1.5    
                             study directed toward                    
                             greater knowledge or                     
                             understanding of the                     
                             fundamental aspects of                   
                             phenomena and of observable              
                             facts without specific                   
                             applications towards                     
                             processes or products in                 
                             mind. It includes all                    
                             scientific study and                     
                             experimentation directed                 
                             towards increasing                       
                             fundamental knowledge and                
                             understanding in those                   
                             fields of the physical,                  
                             engineering, environmental,              
                             and life sciences related to             
                             long-term national security              
                             needs. It is farsighted                  
                             high-payoff research that                
                             provides the basis for                   
                             technological progress.                  
Applied research        2 Applied research is systemic S&T         $5.2    
                             study to understand the                  
                             means to meet a recognized               
                             and specific need. It is a               
                             systematic expression and                
                             application of knowledge to              
                             develop useful materials,                
                             devices, and systems or                  
                             methods. Applied research                
                             may translate promising                  
                             basic research into                      
                             solutions for broadly                    
                             defined military needs,                  
                             short of system development.             
                             Applied research precedes                
                             system-specific technology               
                             investigations or                        
                             development.                             
Advanced                3 Advanced technology          S&T         $6.6    
technology                development includes                     
development               development of subsystems                
                             and components and efforts               
                             to integrate them into                   
                             system prototypes for field              
                             experiments and/or tests in              
                             a simulated environment. The             
                             results of this type of                  
                             effort are proof of                      
                             technological feasibility                
                             and assessment of subsystem              
                             and component operability                
                             and producibility rather                 
                             than the development of                  
                             hardware for service use.                
                             Projects in this category                
                             have a direct relevance to               
                             identified military needs.               
                             Program elements in this                 
                             category involve                         
                             pre-acquisition efforts,                 
                             such as system concept                   
                             demonstration, joint and                 
                             service-specific                         
                             experiments, or technology               
                             demonstrations, and                      
                             generally have technology                
                             readiness levels (TRLs) of               
                             4, 5, or 6. Projects in this             
                             category do not necessarily              
                             lead to subsequent                       
                             development or procurement               
                             phases, but should have the              
                             goal of moving out of                    
                             science and technology and               
                             into the acquisition process             
                             within years defense                     
                             program. Upon successful                 
                             completion of projects that              
                             have military utility, the               
                             technology should be                     
                             available for transition.                
Advanced                4 Advanced component           Acquisition $13.9   
component                 development and prototypes               
development and           consists of efforts                      
prototypes                necessary to evaluate                    
                             integrated technologies or               
                             prototype systems in a high              
                             fidelity and realistic                   
                             operating environment. These             
                             activities include                       
                             system-specific efforts that             
                             help expedite technology                 
                             transition from the                      
                             laboratory to operational                
                             use. Emphasis is on proving              
                             component and subsystem                  
                             maturity prior to                        
                             integration in major and                 
                             complex system sand may                  
                             involve risk reduction                   
                             initiatives. Advanced                    
                             component development and                
                             prototypes efforts are to                
                             occur before an acquisition              
                             program starts product                   
System                  5 System development and       Acquisition $19.3   
development and           demonstration consists of                
demonstration             newly initiated acquisition              
                             programs and includes                    
                             engineering and                          
                             manufacturing development                
                             tasks aimed at meeting                   
                             validated requirements prior             
                             to full-rate production.                 
                             Characteristics of this                  
                             activity involve mature                  
                             system development,                      
                             integration, and                         
                             demonstration to support a               
                             production decision.                     
Research,               6 RDT&E management support     Acquisition $4.0    
development,              includes efforts to sustain              
test and                  and/or modernize the                     
evaluation                installations or operations              
management                required for general RDT&E.              
support                   Such efforts may related to              
                             test ranges, military                    
                             construction, maintenance                
                             support of laboratories,                 
                             operation and maintenance of             
                             test aircraft and ships, and             
                             studies and analyses I                   
                             support of the RDT&E                     
                             program.                                 
Operational             7 Operational system           Acquisition $20.6   
system                    development includes                     
development               development efforts to                   
                             upgrade systems that have                
                             been fielded or have                     
                             received approval for                    
                             full-rate production and                 
                             anticipate production                    
                             funding in the current or                
                             subsequent fiscal year.  
									  
Source: DOD (data); GAO (presentation and analysis).                

Appendix IV: DOD Technology Readiness Levels

Technology readiness level        Description                              
      1. Basic principles observed   Lowest level of technology readiness.    
      and reported.                  Scientific research begins to be         
                                     translated into applied research and     
                                     development. Examples might include      
                                     paper studies of a technology's basic    
                                     properties.                              
      2. Technology concept and/or   Invention begins. Once basic principles  
      application formulated.        are observed, practical applications can 
                                     be invented. Applications are            
                                     speculative and there may be no proof or 
                                     detailed analysis to support the         
                                     assumption. Examples are still limited   
                                     to analytic studies.                     
      3. Analytical and experimental Active research and development is       
      critical function and/or       initiated. This includes analytical      
      characteristic proof of        studies and laboratory studies to        
      concept.                       physically validate analytical           
                                     predictions of separate elements of the  
                                     technology. Examples include components  
                                     that are not yet integrated or           
                                     representative.                          
      4. Component and/or            Basic technological components are       
      breadboard. Validation in      integrated to establish that they will   
      laboratory environment.        work together. This is relatively "low   
                                     fidelity" compared to the eventual       
                                     system. Examples include integration of  
                                     "ad hoc" hardware in a laboratory.       
      5. Component and/or breadboard Fidelity of breadboard technology        
      validation in a relevant       increases significantly. The basic       
      environment.                   technological components are integrated  
                                     with reasonably realistic supporting     
                                     elements so it can be tested in a        
                                     simulated environment. Examples include  
                                     "high fidelity" laboratory integration   
                                     of components.                           
      6. System/subsystem model or   Representative model or prototype        
      prototype demonstration in a   system, which is well beyond that of TRL 
      relevant environment.          5, is tested in a relevant environment.  
                                     Represents a major step up in a          
                                     technology's demonstrated readiness.     
                                     Examples include testing a prototype in  
                                     a high-fidelity laboratory environment   
                                     or in simulated operational environment. 
      7. System prototype            Prototype near, or at, planned           
      demonstration in an            operational system. Represents a major   
      operational environment.       step up from TRL 6, requiring            
                                     demonstration of an actual system        
                                     prototype in an operational environment  
                                     such as in an aircraft, vehicle, or      
                                     space. Examples include testing the      
                                     prototype in a test bed aircraft.        
      8. Actual system completed and Technology has been proven to work in    
      qualified through test and     its final form and under expected        
      demonstration.                 conditions. In almost all cases, this    
                                     TRL represents the end of true system    
                                     development. Examples include            
                                     developmental test and evaluation of the 
                                     system in its intended weapon system to  
                                     determine if it meets design             
                                     specifications.                          
      9. Actual system proven        Actual application of the technology in  
      through successful mission     its final form and under mission         
      operations.                    conditions, such as those encountered in 
                                     operational test and evaluation.         
                                     Examples include using the system under  
                                     operational mission conditions.          



Source: DOD (data); GAO (presentation and analysis).

           
			  
			  Related GAO Products Related GAO Products

           Defense Acquisitions: Assessments of Selected Major Weapon
           Programs. GAO-06-391 . Washington, D.C.: March 31, 2006.

           Best Practices: Better Support of Weapon System Program Managers
           Needed to Improve Outcomes. GAO-06-110 . Washington, D.C.:
           November 1, 2005.

           DOD Acquisition Outcomes: A Case for Change. GAO-06-257T .
           Washington, D.C.: November 15, 2005.

           Defense Acquisitions: Stronger Management Practices Are Needed to
           Improve DOD's Software-Intensive Weapon Acquisitions. GAO-04-393 .
           Washington, D.C.: March 1, 2004.

           Best Practices: Setting Requirements Differently Could Reduce
           Weapon Systems' Total Ownership Costs. GAO-03-57 . Washington,
           D.C.: February 11, 2003.

           Defense Acquisitions: Factors Affecting Outcomes of Advanced
           Concept Technology Demonstration. GAO-03-52 . Washington, D.C.:
           December 2, 2002.

           Best Practices: Capturing Design and Manufacturing Knowledge Early
           Improves Acquisition Outcomes. GAO-02-701 . Washington, D.C.: July
           15, 2002.

           Defense Acquisitions: DOD Faces Challenges in Implementing Best
           Practices. GAO-02-469T . Washington, D.C.: February 27, 2002.

           Best Practices: Better Matching of Needs and Resources Will Lead
           to Better Weapon System Outcomes. GAO-01-288 . Washington, D.C.:
           March 8, 2001.

           Best Practices: A More Constructive Test Approach Is Key to Better
           Weapon System Outcomes. GAO/NSIAD-00-199 . Washington, D.C.: July
           31, 2000.

           Defense Acquisition: Employing Best Practices Can Shape Better
           Weapon System Decisions. GAO/T-NSIAD-00-137 . Washington, D.C.:
           April 26, 2000.

           Best Practices: DOD Training Can Do More to Help Weapon System
           Programs Implement Best Practices. GAO/NSIAD-99-206 . Washington,
           D.C.: August16, 1999.

           Best Practices: Better Management of Technology Development Can
           Improve Weapon System Outcomes. GAO/NSIAD-99-162 . Washington,
           D.C.: July 30, 1999.

           Defense Acquisitions: Best Commercial Practices Can Improve
           Program Outcomes. GAO/T-NSIAD-99-116 . Washington, D.C.: March 17,
           1999.

           Defense Acquisition: Improved Program Outcomes Are Possible.
           GAO/T-NSIAD-98-123 . Washington, D.C.: March 17, 1998.

           Best Practices: DOD Can Help Suppliers Contribute More to Weapon
           System Programs. GAO/NSIAD-98-87 . Washington, D.C.: March 17,
           1998.

           Best Practices: Successful Application to Weapon Acquisition
           Requires Changes in DOD's Environment. GAO/NSIAD-98-56 .
           Washington, D.C.: February 24, 1998.

           Best Practices: Commercial Quality Assurance Practices Offer
           Improvements for DOD. GAO/NSIAD-96-162 . Washington, D.C.: August
           26, 1996.

           GAOï¿½s Mission
			  
			  The Government Accountability Office, the audit, evaluation and
           investigative arm of Congress, exists to support Congress in
           meeting its constitutional responsibilities and to help improve
           the performance and accountability of the federal government for
           the American people. GAO examines the use of public funds;
           evaluates federal programs and policies; and provides analyses,
           recommendations, and other assistance to help Congress make
           informed oversight, policy, and funding decisions. GAO's
           commitment to good government is reflected in its core values of
           accountability, integrity, and reliability.

           Obtaining Copies of GAO Reports and Testimony

           The fastest and easiest way to obtain copies of GAO documents at
           no cost is through GAO's Web site ( www.gao.gov ). Each weekday,
           GAO posts newly released reports, testimony, and correspondence on
           its Web site. To have GAO e-mail you a list of newly posted
           products every afternoon, go to www.gao.gov and select "Subscribe
           to Updates."

           Order by Mail or Phone
			  
			  The first copy of each printed report is free. Additional copies
           are $2 each. A check or money order should be made out to the
           Superintendent of Documents. GAO also accepts VISA and Mastercard.
           Orders for 100 or more copies mailed to a single address are
           discounted 25 percent. Orders should be sent to:

           U.S. Government Accountability Office 441 G Street NW, Room LM
           Washington, D.C. 20548

           To order by Phone: Voice: (202) 512-6000 TDD: (202) 512-2537 Fax:
           (202) 512-6061

           To Report Fraud, Waste, and Abuse in Federal Programs
			  
			  Contact:

           Web site: www.gao.gov/fraudnet/fraudnet.htm E-mail:
           [email protected] Automated answering system: (800) 424-5454 or
           (202) 512-7470

           Congressional Relations
			  
			  Gloria Jarmon, Managing Director, [email protected] (202) 512-4400
           U.S. Government Accountability Office, 441 G Street NW, Room 7125
           Washington, D.C. 20548

           Public Affairs
			  
			  Paul Anderson, Managing Director, [email protected] (202)
           512-4800 U.S. Government Accountability Office, 441 G Street NW,
           Room 7149 Washington, D.C. 20548

13GAO, Defense Technology Development: Management Process Can Be
Strengthened for New Technology Transition Programs. GAO-05-480
(Washington, D.C.: June 17, 2005).

(120454)

www.gao.gov/cgi-bin/getrpt? GAO-06-883 .

To view the full product, including the scope 
and methodology, click on the link above.

For more information, contact Michael J. Sullivan at (202) 512-4841 or
[email protected].

Highlights of GAO-06-883 , a report to congressional committees

September 2006

BEST PRACTICES

Stronger Practices Needed to Improve DOD Technology Transition Processes

The Department of Defense (DOD) relies on its science and technology
community to develop innovative technologies for weapon systems, spending
$13 billion on basic, applied, and advanced technology research. Several
GAO reports have addressed problems in transitioning technologies to the
acquisition community. This report, which was prepared under the
Comptroller General's authority to conduct evaluations, compares DOD's
technology transition processes with commercial best practices.
Specifically, GAO identifies technology transition techniques used by
leading companies and assesses the extent to which DOD uses the
techniques.

What GAO Recommends

GAO recommends that DOD strengthen its technology transition processes by
developing a gated process with criteria to support funding decisions;
expanding the use of transition agreements, relationship managers, and
metrics; and setting aside funding for transition activities. DOD
generally agreed with GAO's recommendations with the exception of adopting
process-oriented metrics and setting aside funding for transition. It
cited ongoing initiatives it believes address several of the
recommendations. GAO believes DOD's actions to date are incomplete and all
recommendations warrant further attention.

Leading commercial companies use three key techniques for successfully
developing and transitioning technologies, with the basic premise being
that technologies must be mature before transitioning to the product line
side.

           o  Strategic planning at the corporate level: Strategic planning
           precedes technology development so managers can gauge market
           needs, identify the most desirable technologies, and prioritize
           resources.
           o  Gated management reviews: A rigorous process is used to ensure
           a technology's relevancy and feasibility and enlist product line
           commitment to use the technologies once the labs are finished
           maturing them.
           o  Corroborating tools: To secure commitment, technology
           transition agreements solidify and document specific cost,
           schedule, and performance metrics labs need to meet for transition
           to occur. Relationship managers address transition issues within
           the labs and product line teams and across both communities.
           Meaningful metrics gauge project progress and process
           effectiveness.

Not only does DOD lack the breadth and depth of these techniques, the
department routinely accepts high levels of technology risk at the start
of major weapon acquisition programs. The acquisition community works with
technologies before they are ready to be transitioned and takes on
responsibility for technology development and product development
concurrently, as shown in the following figure. A defined phase for
technology transition is not evident. These shortcomings contribute
significantly to DOD's poor cost and schedule outcomes.

Path That DOD Follows for Technology Development and Product Development

A stark contrast exists between DOD's and private industry's environments
for developing technology. The numerous examples of DOD programs that have
incurred cost overruns, schedule delays, and reduced performance serve as
reminders that inserting a few best practices and changing the mechanics
of technology transition processes without changing the environment that
determines incentives may not produce better outcomes.
*** End of document. ***