Defense Technology Development: Management Process Can Be
Strengthened for New Technology Transition Programs (17-JUN-05,
GAO-05-480).
The Department of Defense (DOD) and Congress both recognize that
Defense technology innovations sometimes move too slowly from the
lab to the field. Three new programs have been recently created
in DOD to help speed and enhance the transition of new
technologies. A report accompanying the fiscal year 2003 National
Defense Authorization Act required GAO to review two of these
programs--the Technology Transition Initiative (TTI) and Defense
Acquisition Challenge Program (DACP). The first is designed to
speed transition of technologies from DOD labs to acquisition
programs and the second is designed to introduce cost-saving
technologies from inside and outside DOD. We were also asked to
review the Quick Reaction Fund, which is focused on rapidly field
testing promising new technology prototypes. We assessed the
impact the programs had on technology transition and the
programs' selection, management and oversight, and assessment
practices.
-------------------------Indexing Terms-------------------------
REPORTNUM: GAO-05-480
ACCNO: A26957
TITLE: Defense Technology Development: Management Process Can Be
Strengthened for New Technology Transition Programs
DATE: 06/17/2005
SUBJECT: Combat readiness
Defense capabilities
Defense procurement
Military research and development
Program evaluation
Research program management
Research programs
Technology assessment
Defense cost control
DOD Defense Acquisition Challenge
Program
DOD Quick Reaction Fund
DOD Technology Transition Initiative
T-58-16A Helicopter
******************************************************************
** This file contains an ASCII representation of the text of a **
** GAO Product. **
** **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced. Tables are included, but **
** may not resemble those in the printed version. **
** **
** Please see the PDF (Portable Document Format) file, when **
** available, for a complete electronic file of the printed **
** document's contents. **
** **
******************************************************************
GAO-05-480
United States Government Accountability Office
GAO
Report to Congressional Committees
June 2005
DEFENSE TECHNOLOGY DEVELOPMENT
Management Process Can Be Strengthened for New Technology Transition Programs
GAO-05-480
[IMG]
June 2005
DEFENSE TECHNOLOGY DEVELOPMENT
Management Process Can Be Strengthened for New Technology Transition Programs
What GAO Found
The ability to spur and leverage technological advances is vital to
sustaining DOD's ability to maintain its superiority over others and to
improve and even transform how military operations are conducted. The
three new transition programs we reviewed are all appropriately targeted
on what has been a critical problem in this regard-quickly moving
promising technologies from the laboratory and commercial environment into
actual use. Moreover, by tailoring processes and criteria to focus on
different objectives, whether that may be saving time or money or
broadening the industrial base, DOD has had an opportunity to experiment
with a variety of management approaches and criteria that can be used to
help solve transition problems affecting the approximately $69 billion
spent over the past 3 years on later stages of technology development.
However, it is too soon for us to determine the impact the three new DOD
technology transition programs are having. At the time of our review, the
programs-the TTI, DACP, and Quick Reaction Fund-had completed only 11 of
68 projects funded in fiscal years 2003 and 2004; of those, only 4 were
providing full capability to users. Additionally, the programs have
limited measures to gauge success of individual projects and return on
investment. Nonetheless, reports from the programs have pointed to an
array of benefits, including quicker fielding of technological
improvements, cost savings, and the opportunity for DOD to tap into
innovative technologies from firms that are new to defense work. Some
sponsored technologies are bringing benefits to warfighters, such as a
small, unmanned aircraft that can detect chemical and biological agents,
and a device the size of an ink pen that can be used to purify water on
the battlefield or in disaster areas. Furthermore, DOD officials credit
the programs with giving senior leaders the flexibility to rapidly address
current warfighter needs and for highlighting smaller technology projects
that might otherwise be ignored.
Long-term success for the programs likely will depend on how well the
programs are managed and overseen. The programs must have effective
processes for selecting the best projects, and management and oversight
processes that will catch potential problems early. Thus far, of the three
programs, the DACP has adopted the most disciplined and structured process
for selecting and managing projects, and has encountered few problems
managing projects. However, the program has had some difficulties
processing the large number of proposals it receives. The TTI has also
established selection criteria and processes, but it is unclear the extent
to which it is reaching its intended audience and has had less success in
tracking its projects. The Quick Reaction Fund has the least structured
processes of the three programs-a deliberate approach seen as providing
the flexibility needed to field innovations rapidly. It has had some
difficulty selecting, managing and tracking projects.
United States Government Accountability Office
Contents
Letter
Results in Brief
Background
Technology Transition Programs Offering Benefits, but It Is too
Early to Determine Their Impact Selection, Management and Oversight, and
Assessment Processes
Could Be Improved by Adopting Additional Practices Conclusions
Recommendations for Executive Action Agency Comments and Our Evaluation
1
3 4
9
13 23 23 24
Appendix I Comments from the Department of Defense
Tables
Table 1: The Office of the Secretary of Defense Sponsored
Technology Transition Programs Table 2: Technology Transition Programs
Table 3: Examples of Projects Table 4: Projects Completed Table 5: Details
of Selection Process Table 6: Details on Management and Oversight
6 8 8 12 16 20
Figures
Figure 1: Funding for TTI, DACP, and Quick Reaction Fund 7 Figure 2: RAM
Optic Assembly and Missile Launch 10 Figure 3: DARPA Water Purification
System 11
Abbreviations
ACTD Advanced Concept Technology Demonstration
DACP Defense Acquisition Challenge Program
DARPA Defense Advanced Research Projects Agency
DOD Department of Defense
FCT Foreign Comparative Testing
MANTECH Manufacturing Technology Program
S&T science and technology
TRL technology readiness level
TTI Technology Transition Initiative
This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed in
its entirety without further permission from GAO. However, because this
work may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this material
separately.
United States Government Accountability Office Washington, DC 20548
June 17, 2005
The Honorable John Warner
Chairman
The Honorable Carl Levin
Ranking Minority Member
Committee on Armed Services
United States Senate
The Honorable Duncan L. Hunter
Chairman
The Honorable Ike Skelton
Ranking Minority Member
Committee on Armed Services
House of Representatives
Over the past 3 years, the Department of Defense (DOD) has invested
almost $69 billion1 on a wide range of advanced technology development
projects-from detecting and neutralizing deeply buried facilities that
protect weapons of mass destruction, to miniaturizing power and energy
technologies to reduce the weight of gear soldiers carry into battle, to
improving access to space. As we have previously reported, the majority of
these dollars are spent within large weapons programs that have taken too
long to get to the warfighter, in large part because these programs often
attempt to incorporate technology advances that have not been proven.
Invariably, this practice has resulted not only in large cost and schedule
increases for large programs, but less available funding and commitment
for small-scale development.
Both DOD and Congress recognize these as well as other problems in
transitioning technology and have initiated a number of programs over the
past decade aimed at spurring and fielding smaller-scale technology
advances. Recently, the fiscal year 2003 National Defense Authorization
Act2 required DOD to establish two new programs, the Technology
1 This represents funding for applied research, advanced technology
development projects, and advanced component and development prototypes.
It does not include basic research.
2 Bob Stump National Defense Authorization Act for Fiscal Year 2003, P.L.
107-314, S:S: 242 and 243, Dec. 2, 2002.
DRAFT
Transition Initiative (TTI) and the Defense Acquisition Challenge Program
(DACP)-the first designed to speed transition of technologies from DOD's
science and technology (S&T) programs to acquisition programs and the
second designed to introduce innovative and cost-saving technologies from
inside and outside DOD. In addition, DOD established the Quick Reaction
Fund to rapidly field-test promising new technology prototypes. All three
programs target relatively small-scale projects and their total combined
annual budget is about $64 million in fiscal year 2005-a very small
portion of the overall dollars dedicated to technology research and
development. We were required by the conference report accompanying the
act to assess the implementation of the first two programs and were later
asked by your offices to also review the Quick Reaction Fund. As discussed
with your offices, our specific objectives were to assess (1) the impact
the programs have had on technology transition and (2) practices that have
helped or hindered the selection, management and oversight, and assessment
of the programs.
In conducting our work, we interviewed the managers of the three programs
we reviewed and analyzed documents that detailed how the programs have
been implemented, expected project benefits, and the types of performance
metrics they used to gauge individual project and overall program success.
We met with several high-level DOD officials who were appointed to a
council to oversee the Technology Transition Initiative, and we observed
meetings of a working group in charge of supporting the same initiative.
In addition, we judgmentally selected 24 of 68 projects funded by the
three transition programs for more in-depth review, targeting those that
had already been completed or were in the completion stages as well as
assuring we had projects representing all of the military services and key
Defense agencies. We conducted interviews with the managers of these 24
projects to discuss implementation, management, and oversight activities.
We analyzed pertinent project documents, including original proposals and
quarterly status reports. We also analyzed documents relating to the
transition programs as well as broader transition issues and conducted
interviews with officials in DOD, the Army, the Navy, the Air Force, the
Marine Corps, the Defense Advanced Research Projects Agency, the Missile
Defense Agency, the Defense Threat Reduction Agency, Defense Information
Systems Agency, Special Operations Command, Joint Chiefs of Staff, and
military department research laboratories. We relied on our previous best
practices work, other transition program guidance, and general management
practices as criteria when assessing the program offices' efforts to
select, manage and oversee, and assess projects. We also analyzed DOD
budget documents to identify program funding, as well as
Results in Brief
applied research and advanced technology development funding for fiscal
years 2003 to 2009 and confirmed with DOD officials responsible for
maintaining this information that our analysis was correct. Our review was
conducted from June 2004 to April 2005 in accordance with generally
accepted government auditing standards.
It is too soon for us to determine the impact the three programs are
having on technology transition within DOD because, at the time we
selected projects for review, only 11of the 68 projects that were funded
in fiscal years 2003 and 2004 had been completed, and, of those, only 4
were actually providing full capability to users. Moreover, the three
transition programs have limited measures to gauge individual project
success and return on investment. However, the Technology Transition
Initiative and the Defense Acquisition Challenge Program, which are being
implemented consistent with congressional intent and the Quick Reaction
Fund are expected to result in several benefits. For example, each tout
benefits such as quicker fielding of new or improved technologies, cost
savings, and in the case of the Defense Acquisition Challenge Program, the
opportunity for DOD to use new, innovative technologies from smaller firms
and companies that have not done business with DOD in the past. In
addition, some sponsored technologies are providing valued capabilities,
such as the ability to detect chemical and biological agents via small
unmanned aircraft and new ways of purifying water on the battlefield or
disaster zones. DOD officials also believe the programs are worthwhile
from the standpoint of giving senior leaders the flexibility to rapidly
address current warfighter needs and by providing visibility for smaller
technology projects that may not be high enough on an acquisition
program's priority list to receive funding. As a result, DOD plans to
increase funding collectively for these programs from about $24 million in
fiscal year 2003 to about $94 million by fiscal year 2009.
Whether the three programs will be successful over the long run will
likely hinge on how well they are managed and overseen, particularly if
DOD's investment increases as planned. Specifically, it will be important
for the programs to have effective processes for selecting projects, to
ensure that the best possible candidates are chosen and that the
technologies themselves are ready for final testing and evaluation stages.
It will also be vital that they instill effective management and oversight
processes, so that they can identify and correct problems before they
throw projects off track and so that they can sustain acquisition program
commitment. In addition, given the importance of enhancing technology
transition within DOD and the expectation that the investment in these
programs will grow,
it will be important for all three programs to demonstrate to others that
they are providing a worthwhile return on investment. To date, the DACP
has adopted a fairly disciplined and structured process for selecting and
managing projects. While its selection process has been slowed due to an
overwhelming response and it has not yet completed a project, the program
has encountered few problems in managing projects. The TTI has also
established criteria and processes, but had less success in marketing the
program and in tracking progress of individual projects. The Quick
Reaction Fund had the least structured process and criteria, believing
that a high degree of flexibility is needed in order to get technology
prototypes quickly out to the field, where they can immediately impact
military operations. At the same time, it had some difficulty in
selecting, managing, and tracking the status of projects.
All three programs are continuing to strengthen their management
processes. This report recommends that DOD develop data and measures that
can be used to assess short-and long-term impacts of the programs and take
other actions, as appropriate, to further strengthen selection,
management, and oversight as investments increase. DOD agreed with our
recommendations as they related to the DACP and TTI programs. However,
since the Quick Reaction Fund is meant to quickly test a new technology,
DOD did not believe the recommendations should apply to that program.
Background DOD relies on its research laboratories and test facilities as
well as industry and academia to develop new technologies and systems that
improve and enhance military operations and ensure technological
superiority over adversaries. Yet, historically, DOD has experienced
problems in bringing technologies out of the lab environment and into real
use. At times, technologies do not leave the lab because their potential
has not been adequately demonstrated or recognized. In other cases,
acquisition programs-which receive the bulk of DOD's funding in research,
development, testing and evaluation of technology-are simply unwilling to
fund final stages of development of a promising technology, preferring to
invest in other aspects of the program that are viewed as more vital to
success. Other times, they choose to develop the technologies themselves,
rather than rely on DOD labs to do so-a practice that brings cost and
schedule risk since programs may well find themselves addressing problems
related to technology immaturity that hamper other aspects of the
acquisition process. And often, DOD's budgeting process, which requires
investments to be targeted at least 2 years in advance of their
activation, makes it difficult for DOD to seize
opportunities to introduce technological advances into acquisition
programs. In addition, it is challenging just to identify and pursue
technologies that could be used to enhance military operations given the
very wide range of organizations inside and outside of DOD that are
focused on technology development and the wide range of capabilities that
DOD is interested in advancing.
In recognizing this array of challenges, DOD and Congress have established
a number of "technology transition" programs, each with a particular
focus. (See table 1.) The Advanced Concept Technology Demonstration (ACTD)
program, for example, was initiated by DOD in 1994 as a way to get
technologies that meet critical military needs into the hands of users
faster and at less cost than the traditional acquisition process. Under
this program, military operators test prototypes that have already been
developed and matured in realistic settings. If they find the items to
have military utility, DOD may choose to buy additional quantities or just
use the items remaining after the demonstration. In 1980, DOD established
the Foreign Comparative Testing (FCT) Program to identify, evaluate, and
procure technologies that have already been developed and tested in other
countries-saving DOD the costly burden of maturing the technology itself.
Other programs include those that seek to quickly identify and solve
production problems associated with technology transition (the
Manufacturing Technology Program-MANTECH) and to partner with the
commercial sector in completing projects that are useful to both military
and industry (the Dual Use Science and Technology program). Even taken
together, however, these programs represent a very small portion of DOD
dollars spent on applied research and advanced technology
development-about $9 billion annually-and considerably less of total money
spent on the later stages of technology development, which includes an
additional $60 billion spent on advanced component development and
prototypes, largely within weapons acquisition programs. As such, they
cannot single-handedly overcome transition problems, but rather
demonstrate various ways to ease transition and broaden participation from
the industrial base.
Table 1: The Office of the Secretary of Defense Sponsored Technology
Transition Programs
Fiscal year 2004
Program Purpose funding
Mature and validate emerging
manufacturing technologies to $256 million
Manufacturing Technology facilitate
(MANTECH) Program production of new affordable and
sustainable technologies
Expedite the transition of
Advanced Concept maturing technologies from the 223 million
Technology developers to the
users by having military
operators and users test
Demonstration (ACTD) technology prototypes
Defense Production Act Create, maintain, modernize, or expand 78 million
Title III the productive capacities of domestic
sources for critical components,
Program technology items and industrial
resources
essential for national defense.
Foreign Comparative Identify, test, and evaluate mature 34 million
Testing foreign technologies that DOD can adopt
(FCT) Program rather than having to sponsor
development itself
Defense Acquisition Identify and introduce innovative and 17 million
Challenge cost-saving technology or products from
Program (DACP) within DOD's science and technology
community as well as externally into
existing DOD acquisition programs
Quick Reaction Fund Identify and rapidly field-test 15 million
promising new technologies within DOD's
budget execution years
Technology Transition Facilitate the rapid transition of new 13 million
Initiative technologies from DOD science and
(TTI) technology programs into acquisition
programs
Joint Advanced Assist in developing the capabilities 10 million
Warfighting envisioned in Joint Vision 2010 by
Program leveraging advanced technology,
innovative operational concepts, and new
organizational structures
Dual Use Science and Partner with industry to jointly fund 0a
the development of dual use technologies
needed to maintain technological
Technology superiority on the battlefield
Tech Link Mission is threefold: (1) integrate advanced 3 million
commercial-sector technologies
into DOD systems, particularly from nontraditional
defense contractors; (2)
spin-off DOD-developed technologies to the commercial
sector to make these
technologies more affordable for military acquisition;
and (3) establish
collaborative research and development projects with
the private sector for
cost-sharing of new dual-use technology development
Independent Research and Increase efficiency and productivity of
contractor independent research and 0b Development development activities
Source: DOD.
aThe Air Force provided $10 million for Dual Use Science and Technology
projects in fiscal year 2004.
bOSD does not spend any money on this program. Rather, DOD compiles data
on the amount of independent research and development spending by major
defense contractors and reports this information annually.
Three of the more recent initiatives include the TTI and DACP, both
established by Congress in fiscal year 2003, and the Quick Reaction Fund,
established by DOD the same year. TTI is focused on speeding the
transition of technologies developed by DOD's S&T programs into
acquisition programs, while DACP is focused on introducing innovative and
cost-saving technologies developed inside and outside DOD. The Quick
Reaction Fund is focused on field testing technology prototypes.
All three programs are managed by DOD's Office of Defense Research and
Engineering, which reports to the Under Secretary of Defense for
Acquisition, Technology and Logistics.
Together, these three programs received about $64 million in fiscal year
2005-a fraction of the $9.2 billion DOD invested in applied research and
advanced technology development the same year and a relatively small
budget compared to some of the other transition programs. Nevertheless,
DOD has been increasing its investment in these programs and plans to
further increase it over the next few years. (See figure 1.)
Figure 1: Funding for TTI, DACP, and Quick Reaction Fund
Dollars in millions
2003 2004 2005 2006 2007 2008 2009
Planned funding
Fiscal year Source: DOD.
Table 2 highlights similarities and differences between DACP, TTI, and
Quick Reaction Fund. Table 3 provides examples of projects that have
already been funded.
Table 2: Technology Transition Programs
Total fiscal year Completion time 2003 and 2004 Projects funded to Program
Objective Scope frames funding date
DACP Introduce innovative and cost-saving technologies or products into
existing acquisition programs
Other objectives of the program include expansion of opportunities for
emerging defense suppliers and widening the U.S. defense industrial base
Any person or Up to 3 years; 12 $29.83 million 30 projects ranging
activity inside or to 24 months is in value from outside DOD preferred
$100,000 to $2.54
million
TTI Speed transition of DOD science Up to 4 $19.27 21 projects
and years; up million ranging
technologies from DOD technology to 24 months from
is $323,000 to
science and technology preferred $5.33
programs million
programs to acquisition
programs
Quick Field and test DOD science 6 to 12 $21.54 17 projects
Reaction prototypes and months million ranging
Fund
that respond to technology from
immediate $240,000 to
and emerging warfighter $3.5
programs million
needs
Source: DOD.
Table 3: Examples of Projects DACP TTI Quick Reaction Fund
o A collapsible stock for the M-4 rifle that makes it more effective in
an urban environment and ergonomic
o An enhanced optics system for the Rolling Airframe Missile that will
permit allweather operational capability
o A new process for repairing titanium cracks in the B-2 aircraft that
will improve mission readiness rates and has the potential to save DOD
millions of dollars in replacement costs
o A new welding process for naval propellers that will improve weld
repair techniques and increase the strength and quality of the casting in
repaired areas o An ink-pen size device that purifies water
o A custom-design earplug that provides blast noise protection and
increased communication capabilities
o A low-cost, flame resistant coverall for infantry soldiers
o A new coating for H-46 helicopter engine blades that reduces the rate
of premature engine removals related to desert operations
o Extending the field of view for a sensor targeting system that is
capable of detecting, classifying, and locating weapons firing in real
time
o A greaseless M-4 rifle that is more suitable for desert operations
o A special material that strengthens the physical structure of an
unmanned aerial vehicle
o A chemical and biological detection capability for small unmanned
aerial vehicle
o An accuracy enhancement for unguided rockets
Source: DOD.
Technology Transition Programs Offering Benefits, but It Is too Early to
Determine Their Impact
The three transition programs, which are being implemented consistent with
congressional intent, reported that benefits can already be seen in many
projects, including improvements to performance, affordability,
manufacturability, and operational capability for the warfighter. While
such benefits may have eventually been achieved through normal processes,
program officials believe the three transition programs enabled DOD to
realize them sooner due to the immediate funding that was provided to
complete testing and evaluation as well as attention received from senior
managers. DOD officials also emphasized that these programs are calling
attention to emerging technologies that have the potential to offer
important performance gains and cost savings but, due to their size and
relative obscurity, may otherwise be overlooked when competing against
other, larger-scaled technologies and/or technologies already deemed as
vital to a particular acquisition program's success. Another benefit cited
with the DACP is an expansion of the Defense industrial base, because the
program invites participation from companies and individuals that have not
been traditional business partners with DOD. Nevertheless, it is too early
for us to determine the impact that these programs have had on technology
transition. At the time we selected projects to review, few projects had
been completed. In addition, the programs had limited performance measures
to gauge success of individual projects or track return on investment over
time.
The following examples highlight some of the reported benefits of
individual projects.
o Host Weapons Shock Profile Database-DOD spends a significant amount of
time and resources to test new accessories (e.g., night vision scopes) for
Special Operations Forces weapons. Currently, when new accessories are
added, they must go through live fire testing to determine if they work
properly and will meet reliability standards. This process could take
several months to complete as the acquisition office must schedule time at
a test range to complete the testing. Program officials must also identify
and pay for an expert to conduct the testing and pay for ammunition that
will be used in the test. The DACP is funding the test and evaluation of a
database that will simulate the vibration or shock of various machine guns
in order to test new accessories for that gun. This will eliminate almost
all of the testing costs mentioned above and greatly reduce the amount of
time needed for testing. The project office estimates that it will save
almost $780,000 per year in ammunition costs alone.
o Enhanced Optics for the Rolling Airframe Missile-The Rolling Airframe
Missile is part of the Navy's ship self-defense system to counter attacks
from missiles and aircraft. However, the missile experiences operational
deficiencies in certain weather conditions, and the program has had
problems producing components for the optics. The DACP is providing
funding to a small business to test and evaluate a new sapphire dome and
optics for the missile to resolve these problems. Program officials
estimate that program funding will accelerate the development of a
solution 1 to 2 years earlier than anticipated. If the DACP project is
successful, an added benefit will be that the dome material will be
readily available from manufacturers in the United States instead of a
single overseas supplier, as is currently the case.
Figure 2: RAM Optic Assembly and Missile Launch
Source: Raytheon Missile Systems. Source: Raytheon Missile Systems.
o Water Purification System- For tactical situations in which deployed
troops do not have quick and easy access to potable water, the pen will
allow soldiers to treat up to 300 liters of any available, non-brackish
water source on one set of lithium camera batteries and common table salt.
The pen eliminates the risk of the soldiers' exposure to diseases and
bio-chemical pollutants. TTI funding was used to purchase approximately
6,600 water pens for distribution to the military services. In addition,
TTI funding enabled this item to be placed on a General
Services Administration schedule, where approximately 8,600 additional
water pens have been purchased by DOD customers. DOD and the company that
produces the pen donated hundreds of these systems to the tsunami relief
effort in Southeast Asia.
Figure 3: DARPA Water Purification System
Source: Defense Advanced Research Projects Agency.
o Dragon Eye-The Dragon Eye is a small, unmanned aerial vehicle with
video surveillance capabilities used by the marines. To address the
concerns over a chemical and biological threat to troops in Iraq, the
Quick Reaction Fund funded the integration of a small chemical detection
and biological collection device on the Dragon Eye. The lowflying Dragon
Eye can tell troops in real time where and when it is collecting samples,
and in cases where a plume is detected, it can determine the direction the
plume is moving. According to program officials, Quick Reaction funding
allowed the chemical and biological detection capability to be developed 2
years ahead of schedule. The technology was available to a limited number
of Special Operations Forces at the beginning of the Iraqi conflict.
Despite the evident benefits of certain projects, it is too early to
determine the programs' impact on technology transition. At the time we
selected projects for review, only 11 of 68 projects started in fiscal
years 2003 and 2004 had been completed, and, of those, only 4 were
currently available to warfighters. These include one TTI project-a
miniaturized water purification system that is now being offered through a
General Services Administration schedule to the warfighter-and three
projects under the Quick Reaction Fund, including the Dragon Eye chemical
and biological sensor, planning software used by Combatant Commanders
dealing with weapons of mass destruction targets, and special materials
that strengthen unmanned aerial vehicles.
Since the time we selected projects, 20 have been reported as completed
and 13 have been reported as available to warfighters. The latest project
completion information by program is shown in Table 4.
Table 4: Projects Completed
Projects funded since Projects completed
Program the program began as of 2/28/2005
TTI 21
DACP 30
Quick Reaction Program 17
Total 68
Source: DOD.
It is important to note that, even though 20 TTI and Quick Reaction Fund
projects are considered to be complete, not all of the capabilities have
reached the warfighter. For example:
o The T58 Titanium Nitride Erosion Protection is a TTI project that has
transitioned to an acquisition program but has not yet reached the
warfighter. The project is being developed to improve the reliability of
T-58-16A helicopter engines used in Iraq. While the compressor blades are
designed for 3000 operating hours, the Marine Corps has had to remove
engines with fewer than 150 operational hours due to sand ingestion. The
project received funding from the TTI in fiscal years 2003 and 2004 to
develop a titanium nitride coating for engine blades that would
significantly mitigate erosion problems in a desert environment. According
to program documents, blades with the new coating will be included in
future production lots beginning in July 2005. Modification kits will also
be developed for retrofitting engines already produced. Program officials
expect the project will double the
compressor life of the engine in a sand environment and save about $12
million in life-cycle costs through fiscal year 2012.
o The Ping project, funded by the Quick Reaction Fund, is an example of
a project that is considered complete, but a prototype was never field
tested by the warfighter. The Air Force had hoped to broaden the
capability of the microwave technology it used to identify large objects
such as tanks or cars to also detect concealed weapons or explosives- such
as suicide vests. However, the project was cancelled after some initial
testing revealed that the technology was not accurate enough to determine
the microwave signatures of small arms or suicide vests that could have
numerous configurations and materials. DOD officials stated that, even
though the project was unsuccessful, they gained a better understanding of
microwave technologies and are continuing to develop these technologies
for other applications.
The long-term impact of the programs will also be difficult to determine
because the technology transition programs have a limited set of metrics
to gauge project success or the impact of program funding over time. While
each funded project had to identify potential impact in terms of dollar
savings, performance improvements, or acceleration to the field as part of
the proposal process, actual impact of specific projects as well as the
transition programs as a whole is not being tracked consistently. The
value of having performance measures as well as DOD's progress in adopting
them for these transition programs is discussed in the next section of
this report.
To ensure that new technologies can be effectively transitioned and
integrated into acquisitions, transition programs need to establish
effective selection, management and oversight, and assessment processes.
For example, programs must assure that proposals being accepted have
established a sound business case, that is, technologies being
transitioned are fairly mature and in demand and schedules and cost for
transition fit within the program's criteria. Once projects are selected,
there needs to be continual and effective communication between labs and
acquisition programs so that commitment can be sustained even when
problems arise. To assure that the return on investment is being
maximized, the impact of programs must be tracked, including cost and time
savings as well as performance enhancements. Our work over the past 7
years has found that high-performing organizations adopt these basic
practices as a means for successfully transitioning technologies into
acquisitions. Moreover, several larger DOD technology transition programs,
such as the ACTD program
Selection, Management and Oversight, and Assessment Processes Could Be
Improved by Adopting Additional Practices
Selection
and some Defense Advanced Research Projects Agency (DARPA) projects,
embrace similar practices and have already developed tools to help sustain
commitment, such as memorandums of agreement between technology developers
and acquirers. Both DARPA and ACTD manage budgets that are considerably
larger than the programs included in this review. As such, the level of
detail and rigor associated with their management processes may not be
appropriate for TTI, DACP, or Quick Reaction Fund. However, the concepts
and basic ingredients of their criteria and guidance could serve as a
useful starting point for the smaller programs to strengthen their own
processes.
The three programs we reviewed adopted these practices to varying degrees.
Overall, the DACP had disciplined and well-defined processes for selecting
and managing, and overseeing projects. The TTI had disciplined and
well-defined processes for selecting projects, but less formal processes
for management and oversight. The Quick Reaction Fund was the least formal
and disciplined of all three, believing that success was being achieved
through flexibility and a high degree of senior management attention. All
three programs had limited performance measures to gauge progress and
return on investment. Generally, we found that the more the programs
adopted structured and disciplined management processes, the fewer
problems they encountered with individual efforts.
Success in transitioning technologies from a lab to the field or an
acquisition program hinges on a transition program's ability to choose the
most promising technology projects. This includes technologies that can
substantially enhance an existing or new system either through better
performance or cost savings and those with technologies at a fairly mature
stage, in other words, suitable for final stages of testing and
evaluation. A program can only do this, however, if it is able to clearly
communicate its purpose and reach the right audience to submit proposals
in the first place. It is also essential that a program have a systematic
process for determining the relative technical maturity of the project as
well as for evaluating other aspects of the project, such as its potential
to benefit specific acquisition programs. Involving individuals in the
selection process from various functions within an organization-e.g.,
technical, business, and acquisition-further helps to assure that the
right projects are being chosen and that they will have interested
customers. An analytical tool that can be particularly useful in selecting
projects is a technology readiness level (TRL) assessment, which assesses
the maturity level of a technology ranging from paper studies (level 1),
to prototypes that can be tested in a realistic environment (level 7), to
an actual system that has proven itself in mission operations (level 9).
Our prior work has
found TRLs to be a valuable decision-making tool because it can presage
the likely consequences of incorporating a technology at a given level of
maturity into a product development.
As further detailed in table 5, the DACP program has a fairly robust
selection process. The program relies on internet-based tools to
communicate its goals and announce its selection process and ensure a
broad audience is targeted. As a result, it receives a wide array of
proposals from which the program office assesses their potential for
generating improvements to existing programs as well as actual interest
from the acquisition community. The DACP also solicits technical experts
from inside and outside DOD to assess potential benefits and risks. Once
the number of projects is whittled down, the program takes extra steps to
secure commitments from acquisition program managers as well as program
executive officers. The program's popularity, however, has had some
drawbacks. For example, the sheer number of proposals have tended to
overwhelm DACP staff and slowed down the selection process, particularly
in the first year. In addition, while technology benefits and risks are
assessed in making selection decisions, DACP does not formally confirm the
technology readiness levels being reported. The TTI program also has a
fairly rigorous selection process, with specific criteria for selection,
including technology readiness, and a team of representatives of
higher-level DOD S&T officials in charge of disseminating information
about the program in their organization, assessing their organization's
proposals based on TTI criteria as well as other criteria they developed,
and ranking their top proposals. The program, which is focused on reaching
DOD's S&T community rather than outside industry, had been communicating
in a relatively informal manner and it was unclear during our review the
extent to which the TTI was reaching its intended audience. The program,
however, has been taking steps to strengthen its ability to reach out to
the S&T community. In addition, TTI does not confirm TRLs. At the time of
our review, the Quick Reaction Program selection process was much less
structured and disciplined than DACP and TTI. This was by design, because
the program wants to select projects quickly and get them out to the field
where they can be of use in military operations in Iraq, Afghanistan, and
elsewhere. However, the program experienced problems related to selection
and as a result-for example, significant gaps in knowledge about
technology readiness led to the cancellation of one project. To program
officials, the risk associated with less formal selection is worth the
benefit of being able to move rapidly evolving technologies into an
environment where they can begin to immediately enhance military
operations and potentially save lives. Nevertheless, the program is now
taking steps to strengthen selection processes.
Table 5: Details of Selection Process
DACP TTI Quick Reaction Fund
How program is communicated Each year, the program office issues an
electronic broad agency announcement to communicate program goals and
funding availability to both internal and external technology developers.
Program officials are satisfied that this approach has helped them reach
their target audience as evidenced by the large number of proposals
received overall and also by the number received by small and
non-traditional businesses. For example, almost 400 proposals were
received during the first and second solicitation periods combined. Of
those, about 52 percent were submitted by small and nontraditional
businesses. TTI established a working group comprised of senior-level S&T
representatives from each of the military services to help disseminate
information about program goals and funding availability to their
respective organizations.
It is unclear whether the TTI is reaching its intended audience. At the
time of our review, six project managers we met with stated they found out
about the TTI by happenstance and not through any formal mechanism.
Furthermore, even though they are now aware of the program, many of these
project managers still have not seen the program formally advertised
throughout their respective organizations. The program office has been
working to improve its ability to reach its target audience and assure
that it is considering the best projects for funding by including
presentations about the program at S&T forums and seminars and developing
a program brochure.
Communication was informal. Each military service and selected defense
agency was asked to submit its top 10 proposals.
A new website was recently developed to automatically distribute the
proposal data call and collect proposals.
DACP TTI
How projects are DACP's selection process is modeled TTI proposals are
judged against a set of evaluated and after DOD's FCT program, which eight
criteria, including time needed to selected already has a well-defined
selection transition, technology readiness, whether
process. The process begins with the project satisfies a critical
program officials performing an initial requirement, and willingness on
the part
review of all proposals to filter out the of an acquisition program to
share costs
ones that are not relevant or feasible. of transition. Working group
members
Next, proposals are reviewed by a play a key role in the selection process
panel of technical experts. If the by reviewing proposals from their
acquisition program office and prime organization and ranking their top
contractor are interested in the proposals for program office
proposed technology, the acquisition consideration. Program officials
believe
program office must develop a final, this approach expedites the selection
more in-depth proposal for the process because it reduces the number
program office to consider for funding. of proposals the program office
needs to
Overall, projects are assessed against review and also gives the officials
an
24 criteria, including acquisition idea of which proposals are the most
funding commitment, technology important to the service or agency.
readiness, and whether the project
addresses a valid requirement, which
are identified in the broad area
announcement used to solicit
proposals.
However, the program office has had
some problems processing the large
volume of proposals received with a
small staff of about 3.5 full-time
equivalent positions. Currently, it
takes about 9 months for a project to
be selected for funding. According to
the program manager, the expert
technical review is the most time
consuming part of the process. To
date, the DACP has had to solicit the
services of over 1,000 experts from
within the Department, private
industry, and academia nationwide to
review proposals. Program officials
are now considering identifying thrust
areas for prioritizing proposals as a
way to reduce cycle time. This was
permitted by the legislation that set up
DACP.
Quick Reaction Fund
Selection was not based on well defined criteria at the time of our
review. Instead, 2-page
proposals were evaluated by the office running the Quick Reaction Fund
with an eye toward whether they could bring immediate benefits to ongoing
military operations. It usually takes 30 to 45 days between the time the
solicitation is issued to the time projects are selected for funding.
DACP TTI Quick Reaction Fund
How TRLs are DACP guidance specifically states
used that technology must have been demonstrated using a representative
model or prototype system in a relevant environment, the equivalent of a
technology readiness level 6 on a scale of 1-9, to be considered for
funding. Technology readiness is also considered by technical experts
reviewing proposals. However, DACP does not have a formal mechanism in
place to ensure that the technology readiness levels have actually been
achieved.
One of the four criteria program officials consider during the selection
process is whether the proposed new technology is mature. However, TTI
does not have a formal mechanism to confirm technology readiness levels
identified in the proposals.
Program officials have limited information about maturity levels.
Proposals are required to disclose technical risks to assist in this
evaluation, but do not identify specific technology readiness levels. We
identified one project where the Quick Reaction Fund invested $1.5 million
but later needed to stop the project because technology was not as mature
as the project manager originally thought.
According to the program manager, TRL maturity will be included as an
evaluation factor in the fiscal year 2006 assessment process.
Management and Oversight
Source: GAO analysis.
Selecting promising projects for funding is not enough to ensure
successful transition. Program managers must also actively oversee
implementation to make sure that project goals are being met and the
program is working as intended and to identify potential barriers to
transition. They must also sustain commitment from acquirers. Moreover,
the transition program as a whole must have good visibility over progress
and be positioned to shift attention and resources to problems as they
arise.
A tool that has proven particularly useful for other established DOD
technology transition programs is designating individuals, preferably with
experience in acquisitions or operations and/or the S&T world, as "deal
brokers" or agents to facilitate communication between the lab and the
acquisition program and to resolve problems as they arise. DARPA employs
such individuals, for example, as well as some Navy-specific transition
programs. Both have found that these agents have been integral to
transition success. Another tool that is useful for sustaining commitment
from the acquirers is a formal agreement. Our previous work found that
best practice companies develop agreements with cost and schedule targets
to achieve and sustain buy-in and that the agreements are modified as a
project progresses to reflect more specific terms for accepting or
rejecting a technology. DARPA develops similar agreements that describe
how projects will be executed and funded as well as how projects will be
terminated if the need arises. The agreements are signed
by high-level officials, including the director of DARPA and senior-level
representatives of the organizations DARPA is working with. The ACTD
program develops "implementation directives" that clarify roles and
responsibilities of parties executing an ACTD, time frames, funding, and
the operational parameters by which military effectiveness is to be
evaluated. The agreements are also signed by high-level officials.
DACP has fairly robust management and oversight mechanisms. Status is
monitored via formal quarterly reporting as well as interim meetings
which, at a minimum, involve the customer, the developer, and the DACP
project manager. The meetings provide an opportunity to ensure the
acquisition program is still committed to the project and to resolve
problems. Though formal memoranda of agreements are not usually employed,
the program establishes test and evaluation plans that detail pass/fail
criteria so that funding does not continue on projects that experience
insurmountable problems. TTI also employs periodic status reports and
meetings; however, communication has not been as open. In two cases,
projects ran into significant problems, such as loss of acquisition
program office support in one case and logistics issues that had not been
addressed to transition a technology smoothly in the other, which had not
come to the attention of the TTI program office. As a result, the TTI
office thought the projects had transitioned when in actuality,
significant problems still needed to be addressed. Per legislation, TTI
had also established a formal council comprised of high-level DOD
officials to help oversee the program; however, the Council has only met
once in 2 years, while the act requires that it meet at least
semiannually. In addition, there is some confusion among Council members
and others we spoke with as to what the purpose of the Council should
be-that is, focused on TTI only or broader transition issues.
Congressional officials expressed that they intended for the Council to
focus on broader transition issues and how best to solve them. Although
the Quick Reaction Fund does not require status reports to assess
progress, project managers are required to submit after-action reports.
However, these were not regularly reviewed by the office. We identified
several problems that arose during transition that were not known to the
Quick Reaction Fund program manager. The program manager is currently
taking steps to improve the management and oversight of projects. For
example, a website has been developed to help monitor and execute the
program. Among other things, the website will allow for the automatic
collection of monthly status reports.
Table 6: Details on Management and Oversight
DACP TTI Quick Reaction Fund
How commitment is secured DACP employs a formal commitment process that
includes pass/fail criteria that will either sustain commitment or result
in the cancellation of a project based upon testing outcomes.
DACP guidance includes practices that help the project manager achieve and
maintain buy-in until a technology has been successfully transitioned. As
part of the selection criteria, both the acquisition program office and
prime contractor must agree to accept a technology if it is successfully
demonstrated. For example, one of the projects we reviewed established
pass/fail criteria for a new technology to repair titanium alloy cracks on
the aft section of the B-2 aircraft. Further, selected projects are
expected to use integrated product teams (which include at least the
vendor, developer, and DACP manager) to exchange information and deal with
potential problems very quickly.
TTI employs a formal commitment process; it has not used formal
agreements.
Initially, all projects must show the acquirer has included funding for
the technology in a future budget if the project is to be considered for
funding. For the most part, once this happens, there is no requirement for
additional interaction through methods like formal agreements. We
identified one project that was not able to sustain support from the
acquisition program-the IROS3 Spartan project, which is intended to
enhance the Navy's shipboard protection. TTI officials believed it had
transitioned to an acquisition program, when, in fact, the customer had
dropped its support. According to the project manager, the Naval Surface
Warfare Center at Crane was supposed to develop the first block of
software for the project and a contractor would be selected to develop
later blocks. However, after successful field-testing, the acquisition
program office decided that it would be less risky to select a contractor
to develop all three software blocks than to hand off the
government-developed software to the contractor to maintain and upgrade.
No formal commitment process used because the program goal is to
demonstrate the military utility of emerging new technologies through
field testing, and not necessarily to transition those technologies at
this time.
DACP TTI Quick Reaction Fund
How status is tracked Project managers are required to submit quarterly
progress reports and final reports once a project has been completed. In
addition to these reports, periodic meetings are held with various
participants to assess progress. Reports contain data on project
accomplishments, planned actions for the next quarter, issues that need to
be addressed, the transition strategy, and the current funding plan. All
quarterly reports are submitted and maintained on an electronic system
that can be accessed by program officials and the project manager.
Periodic status reports are required and interim progress meetings are
held, though not with the same range of representation as DACP.
We identified one project-the Terminal Attack Communications Earplug
System- where problems had arisen during transition that had not come to
the attention of TTI officials. Specifically, problems related to how the
earplug's use could be actualized (for example, training technicians to
pour the new, custom ear molds and what organization would pay for the
training) had not been brought to the attention of the TTI office. In
fact, the office had thought the transition had gone smoothly.
After-action reports are required, though not always reviewed for purposes
of tracking progress.
At the time of our review, the Quick Reaction Fund manager had problems
addressing concerns identified in status reports and obtaining reports
from some project managers. For example, the program manager was unaware
that one project, thought to have been successfully field tested, actually
ran out of funding prior to field testing. Although the project manager
reported that the project had not been completed because it ran out of
funding, the Quick Reaction Fund manager did not have enough time to
thoroughly review the report.
The program manager expects the newly established web-based system to
improve his ability to track project progress.
Use of deal No formal use of "deal No formal use of No formal use of
brokers", though "deal brokers." "deal
brokers involvement of vendors, brokers."
developers, and
customers in status
discussions have
helped to identify and
resolve transition
problems.
Assessment
Source: GAO analysis.
Though the transition programs we reviewed are relatively small in scale
compared to other transition programs in DOD, the government's investment
is still considerable and it will continue to grow if DOD's funding plans
for the programs are approved. As a result, it is important that these
programs demonstrate that they are generating a worthwhile return on
investment-whether through cost savings to acquisition programs, reduced
times for completing testing and evaluation and integrating technologies
into programs, and/or enhanced performance or new capabilities. Developing
such information can enable transition
program managers to identify what is or is not working well within a
program; how well the program is measuring up to its goals, as well as to
make trade-off decisions between individual projects. On a broader level,
it can enable senior managers and oversight officials to compare and
contrast the performance of transition programs across DOD.
Finding the right measures to use for this purpose is challenging,
however, given the wide range of projects being pursued, the different
environments to which they are being applied, and difficulties associated
with measuring certain aspects of return on investment. For example,
measuring long-term cost savings could be problematical because some
projects could have impacts on platforms and systems that were not part of
the immediate transition effort. As a result, the best place to start may
be with high-level or broad metrics or narratives that focus on the
spectrum of benefits and cost savings being achieved through the program,
complemented by more specific quantifiable metrics that do not require
enormous efforts to develop and support, such as time saved in transition
or short-term cost savings. At this time, however, the transition programs
have limited measures to gauge individual project success and program
impact or return on investment in the long term. At best, they are
collecting after action reports that describe the results of transition
projects, and occasionally identify some cost savings, but not in a
consistent manner. In addition, there are inconsistencies in how the
reports are being prepared, reviewed, and used. The Quick Reaction Fund
program manager, in fact, had trouble just getting projects to submit
after action reports.
Officials from all three transition programs we reviewed as well as higher
level officials agreed that they should be doing more to capture
information regarding return on investments for the programs. They also
agreed that there may already be readily available starting points within
DOD. For example, the Foreign Comparative Testing Program has established
metrics to measure the health, success, and cost-effectiveness of the
program and has developed a database to facilitate return on investment
analyses. The program also captures general performance enhancements in
written narratives. The program has refined and improved its metrics over
time and used them to develop annual reports. The specific metrics
established by the FCT program may not be readily transferable to DACP,
TTI, or the Quick Reaction Fund because the nature of FCT projects is
quite different-technologies themselves are more mature and costs savings
are achieved by virtue of the fact that DOD is essentially avoiding the
cost of developing the technologies rather than applying the technologies
to improve larger development efforts. However, the process by which the
program came to identify useful
metrics as well as the automated tools it uses could be valuable to the
other transition programs. In addition, DOD has asked the Naval Post
Graduate School to study metrics that would be useful for assessing the
ACTD program. The results of this study may also serve as a starting point
for the transition programs in developing their own ways to assess return
on investment.
Conclusions
The ability to spur and leverage technological advances is vital to
sustaining DOD's ability to maintain its superiority over others and to
improve and even transform how military operations are conducted. The
three new transition programs are all appropriately targeted on what has
been a critical problem in this regard-quickly moving promising
technologies from the laboratory and commercial environment into actual
use. Moreover, by tailoring processes and criteria to focus on different
objectives, whether that may be saving time or money or broadening the
industrial base, DOD has had an opportunity to experiment with a variety
of management approaches and criteria that can be used to help solve
transition problems affecting the approximately $69 billion spent annually
on advanced stages of technology development. Already, it is evident that
an element missing from all three programs is good performance
measurement. Without having this capability, DOD will not be able to
effectively assess which approaches are working best and whether the
programs individually or as a whole are truly worthwhile. In addition, it
is evident that having well-established tools for selecting and managing
projects as well as communicating with technology developers and
acquisitions helps programs to reduce risk and achieve success, and that
there are opportunities for all three programs for strengthening their
capabilities in this regard. In light of its plans to increase funding for
the three programs, DOD should consider actions to strengthen selection
and management capabilities, while taking into account resources needed
for implementing them as well as their impact on the ability of the
programs to maintain flexibility.
Recommendations for We recommend that the Secretary of Defense take the
following five actions:
Executive Action To optimize DOD's growing investment in the Technology
Transition Initiative, the Defense Acquisition Challenge Program, and the
Quick Reaction Fund, we recommend that the Secretary of Defense direct the
Under Secretary of Defense (Acquisition, Technology, and Logistics) to
develop data and measures that can be used to support assessments of the
performance of the three transition programs as well as broader
assessments of the return on investment that would track the long-term
impact of the programs. DOD could use measures already developed by other
transition programs, such as FCT, as a starting point as well as the
results of its study on performance measurement being conducted by the
Naval Post Graduate School. To complement this effort, we recommend that
DOD develop formal feedback mechanisms, consisting of interim and after
action reporting, as well as project reviews if major deviations occur in
a project. Deviations include, but are not limited to, changes in the
technology developer, acquirer, or user, or an inability for the
technology developer to meet cost, schedule, or performance parameters at
key points in time.
We also recommend that the Secretary of Defense direct the Under Secretary
of Defense (Acquisition, Technology, and Logistics) to implement the
following, as appropriate, for each of the transition programs: (1) formal
agreements to solidify up-front technology development agreements related
to cost, schedule, and performance parameters that must be met at key
points in time and (2) confirmation of technology readiness levels as part
of the proposal acceptance process.
In addition, we recommend that DOD identify and implement mechanisms to
ensure that transition program managers, developers, and acquirers are
able to better communicate to collectively identify and resolve problems
that could hinder technology transition. There may be opportunities to
strengthen communication by improving the structure and content of interim
progress meetings and possibly even designating individuals to act as deal
brokers.
Lastly, as DOD considers solutions to broader technology transition
problems, we recommend that Secretary of Defense direct the Under
Secretary of Defense (Acquisition, Technology, and Logistics) to assess
how the Technology Transition Council can be better used.
DOD provided us with written comments on a draft of this report. DOD
partially concurred with four of the five recommendations and concurred
with one recommendation. The reason DOD only partially concurred with four
of the recommendations is because it does not believe the Quick Reaction
Fund fits the definition of a transition program. However, we continue to
believe it is important for DOD to institute better management controls
and have better visibility of the Quick Reaction Fund as it
Agency Comments
and Our Evaluation
increases its investment in this program over the next several years. DOD
comments appear in appendix I.
DOD partially concurred with our recommendation that the programs develop
data and measures that can be used to support assessments of the
performance of the three transition programs as well as broader
assessments of return on investment that would track the long term impact
of the programs. DOD agreed that performance measures for the DACP and TTI
programs could be improved but does not believe that measuring the impact
of the Quick Reaction Fund is necessary because it does not technically
fit the definition of a transition program. We disagree. DOD should track
the progress of its various programs to determine if the programs are
worthwhile and should be continued, if the program should receive
additional funding, or if changes should be made in the selection or
implementation process that could result in better outcomes. Further,
failure to track even the most basic information, such as the number of
projects completed, could result in a lack of ability to manage the
program properly and poor stewardship of taxpayer money.
DOD partially concurred with our recommendation that the three programs
develop formal feedback mechanisms consisting of interim and after action
reporting, as well as project reviews if major deviations occur in a
project. DOD agrees that the TTI and DACP can be improved and has recently
taken steps in this regard. However, DOD believes that due to the limited
scope and duration of Quick Reaction Fund projects, formal feedback
mechanisms may not be necessary for this program. We believe that regular
feedback on the progress of each program is important to help program
managers mitigate risk. As stated in the report, the Quick Reaction Fund
program manager was unaware that one project ran out of funding prior to
field testing the technology. Had the program manager been aware of the
problem, money that had not yet been allocated could have been used to
finish the project. In addition, based upon our discussions with the
current program manager, DOD is planning to require monthly status reports
for funded projects.
DOD partially concurred with our recommendation that the programs
implement, as appropriate: (1) formal agreements to solidify up-front
technology development agreements related to cost, schedule, and
performance parameters that must be met at key points in time and (2)
confirmation of technology readiness levels as part of the proposal
acceptance process. DOD indicated that it recently implemented Technology
Transition Agreements for the TTI, and the DACP program also uses formal
agreements. However, DOD does not believe formal
agreements are necessary for the Quick Reaction Fund because it is not
intended to be a transition program. Also, it does not believe TRLs should
be a factor in the proposal acceptance process. As stated in the report,
we agree that formal agreements may not be appropriate for Quick Reaction
Fund projects. However, TRLs should be considered during the selection
process. Since the goal of this particular program is to prototype a new
technology in 12 months or less, it is important that DOD has some
assurance that the technology is ready to be field tested. As discussed in
the report, the Quick Reaction Fund had to cancel one project, after $1.5
million had already been spent, because it had only achieved a TRL 3. Had
the selecting official known the TRLs of each proposed project during the
selection phase, he may have decided to fund another, more mature project
instead.
DOD also partially concurred with our recommendation that the programs
identify and implement mechanisms to ensure that transition program
managers, developers, and acquirers better communicate and collectively
identify and resolve problems that could hinder technology transition. DOD
established a Transition Overarching Integrated Product Team earlier this
year to provide the necessary oversight structure to address this issue,
but does not believe this recommendation applies to the Quick Reaction
Fund program. We believe that if DOD receives monthly status reports on
the Quick Reaction Fund, as planned by the program manager, it should be
in a good position to identify and resolve problems that could hinder the
testing of new technology prototypes.
DOD concurred with our recommendation that the Under Secretary of Defense
(Acquisition, Technology and Logistics) assess how the Technology
Transition Council can be better used as DOD considers solutions to
broader technology transition problems. Although DOD did not indicate how
it plans to do this, the Deputy Under Secretary of Defense (Advanced
Systems and Concepts) has a goal that the Council not limit itself to just
the TTI program, but look at broader technology transition issues across
DOD.
We are sending copies of this report to the Secretary of Defense, the
Director of the Office of Management and Budget, and interested
congressional committees. We will also make copies available at no charge
on the GAO Web site at http://www.gao.gov.
If you or your staff have any questions concerning this report, please
contact me at (937) 258-7915. Key contributors to this report were
Cristina
Chaplain, Cheryl Andrew, Art Cobb, Gary Middleton, and Sean D. Merrill.
Michael J. Sullivan
Director
Acquisition and Sourcing Management
GAO's Mission
Obtaining Copies of GAO Reports and Testimony
The Government Accountability Office, the audit, evaluation and
investigative arm of Congress, exists to support Congress in meeting its
constitutional responsibilities and to help improve the performance and
accountability of the federal government for the American people. GAO
examines the use of public funds; evaluates federal programs and policies;
and provides analyses, recommendations, and other assistance to help
Congress make informed oversight, policy, and funding decisions. GAO's
commitment to good government is reflected in its core values of
accountability, integrity, and reliability.
The fastest and easiest way to obtain copies of GAO documents at no cost
is through GAO's Web site (www.gao.gov). Each weekday, GAO posts newly
released reports, testimony, and correspondence on its Web site. To have
GAO e-mail you a list of newly posted products every afternoon, go to
www.gao.gov and select "Subscribe to Updates."
Order by Mail or Phone The first copy of each printed report is free.
Additional copies are $2 each. A check or money order should be made out
to the Superintendent of Documents. GAO also accepts VISA and Mastercard.
Orders for 100 or more copies mailed to a single address are discounted 25
percent. Orders should be sent to:
U.S. Government Accountability Office 441 G Street NW, Room LM Washington,
D.C. 20548
To order by Phone: Voice: (202) 512-6000 TDD: (202) 512-2537 Fax: (202)
512-6061
To Report Fraud, Contact:
Waste, and Abuse in Web site: www.gao.gov/fraudnet/fraudnet.htm
E-mail: [email protected] Programs Automated answering system: (800)
424-5454 or (202) 512-7470
Gloria Jarmon, Managing Director, [email protected] (202)
512-4400Congressional U.S. Government Accountability Office, 441 G Street
NW, Room 7125 Relations Washington, D.C. 20548
Public Affairs Paul Anderson, Managing Director, [email protected] (202)
512-4800 U.S. Government Accountability Office, 441 G Street NW, Room 7149
Washington, D.C. 20548
PRINTED ON RECYCLED PAPER
*** End of document. ***