[Senate Hearing 111-149]
[From the U.S. Government Publishing Office]



                                                        S. Hrg. 111-149
 
 ACQUISITION OF MAJOR WEAPONS SYSTEMS BY THE DEPARTMENT OF DEFENSE AND 
       S. 454, THE WEAPON SYSTEMS ACQUISITION REFORM ACT OF 2009

=======================================================================

                                HEARING

                               before the

                      COMMITTEE ON ARMED SERVICES
                          UNITED STATES SENATE

                     ONE HUNDRED ELEVENTH CONGRESS

                             FIRST SESSION

                               __________

                             MARCH 3, 2009

                               __________

         Printed for the use of the Committee on Armed Services


                  U.S. GOVERNMENT PRINTING OFFICE
53-267                    WASHINGTON : 2009
-----------------------------------------------------------------------
For Sale by the Superintendent of Documents, U.S. Government Printing Office
Internet: bookstore.gpo.gov  Phone: toll free (866) 512-1800; (202) 512ï¿½091800  
Fax: (202) 512ï¿½092104 Mail: Stop IDCC, Washington, DC 20402ï¿½090001


  

                      COMMITTEE ON ARMED SERVICES

                     CARL LEVIN, Michigan, Chairman

EDWARD M. KENNEDY, Massachusetts     JOHN McCAIN, Arizona
ROBERT C. BYRD, West Virginia        JAMES M. INHOFE, Oklahoma
JOSEPH I. LIEBERMAN, Connecticut     JEFF SESSIONS, Alabama
JACK REED, Rhode Island              SAXBY CHAMBLISS, Georgia
DANIEL K. AKAKA, Hawaii              LINDSEY GRAHAM, South Carolina
BILL NELSON, Florida                 JOHN THUNE, South Dakota
E. BENJAMIN NELSON, Nebraska         MEL MARTINEZ, Florida
EVAN BAYH, Indiana                   ROGER F. WICKER, Mississippi
JIM WEBB, Virginia                   RICHARD BURR, North Carolina
CLAIRE McCASKILL, Missouri           DAVID VITTER, Louisiana
MARK UDALL, Colorado                 SUSAN M. COLLINS, Maine
KAY R. HAGAN, North Carolina
MARK BEGICH, Alaska
ROLAND W. BURRIS, Illinois

                   Richard D. DeBobes, Staff Director

               Joseph W. Bowab, Republican Staff Director

                                  (ii)

  
?



                            C O N T E N T S

                               __________

                    CHRONOLOGICAL LIST OF WITNESSES

 Acquisition of Major Weapons Systems by the Department of Defense and 
       S. 454, the Weapon Systems Acquisition Reform Act of 2009

                             march 3, 2009

                                                                   Page

Sullivan, Michael J., Director, Acquisition and Sourcing 
  Management, Government Accountability Office...................     7
Gansler, Hon. Jacques S., Chairman, Defense Science Board Task 
  Force on Industrial Structure for Transformation...............    19
Kaminski, Hon. Paul G., Chair, Committee on Pre-Milestone A 
  Systems Engineering, Air Force Studies Board, National Research 
  Council........................................................    25
Adolph, Charles E. (Pete), Chairman, Defense Science Board Task 
  Force on Developmental Test and Evaluation.....................    40

                                 (iii)


ACQUISITION OF MAJOR WEAPONS SYSTEMS BY  THE  DEPARTMENT  OF  DEFENSE  
     AND  S. 454, THE WEAPON SYSTEMS ACQUISITION REFORM ACT OF 2009

                              ----------                              


                         TUESDAY, MARCH 3, 2009

                                       U.S. Senate,
                               Committee on Armed Services,
                                                    Washington, DC.
    The committee met, pursuant to notice, at 9:36 a.m. in room 
SH-216, Hart Senate Office Building, Senator Carl Levin 
(chairman) presiding.
    Committee members present: Senators Levin, Lieberman, 
Akaka, Bill Nelson, Webb, McCaskill, Udall, Hagan, Begich, 
Burris, McCain, Chambliss, Thune, Martinez, Burr, and Collins.
    Committee staff members present: Richard D. DeBobes, Staff 
Director, and Leah C. Brewer, Nominations and Hearings Clerk.
    Majority staff members present: Creighton Greene, 
professional staff member; Peter K. Levine, general counsel; 
John H. Quirk V, professional staff member; Arun A. Seraphin, 
professional staff member; and William K. Sutey, professional 
staff member.
    Minority staff members present: Joseph W. Bowab, Republican 
staff director; Daniel A. Lerner, professional staff member; 
David M. Morriss, minority counsel; Lucian L. Niemeyer, 
professional staff member; and Christopher J. Paul, 
professional staff member.
    Staff assistants present: Jessica L. Kingston, Brian F. 
Sebold, and Breon N. Wells.
    Committee members' assistants present: Jay Maroney, 
assistant to Senator Kennedy; Bonni Berge, assistant to Senator 
Akaka; Christopher Caple, assistant to Senator Bill Nelson; Jon 
Davey, assistant to Senator Bayh; Gordon I. Peterson, assistant 
to Senator Webb; Stephen C. Hedger, assistant to Senator 
McCaskill; Jennifer Barrett, assistant to Senator Udall; 
Michael Harney, assistant to Senator Hagan; David Ramseur, 
assistant to Senator Begich; Brady King, assistant to Senator 
Burris; Sandra Luff, assistant to Senator Sessions; Clyde A. 
Taylor IV, assistant to Senator Chambliss; Jason Van Beek, 
assistant to Senator Thune; Erskine W. Wells III, assistant to 
Senator Wicker; and Kevin Kane, assistant to Senator Burr.

       OPENING STATEMENT OF SENATOR CARL LEVIN, CHAIRMAN

    Chairman Levin. Good morning, everybody. The committee 
meets today to consider the performance of the Department of 
Defense's (DOD) acquisition programs at a time when cost growth 
on these programs has reached levels that we cannot afford, 
including consideration of our bill, S. 454, the Weapon Systems 
Acquisition Reform Act of 2009, which Senator McCain and I 
recently introduced.
    Since the beginning of 2006, nearly half of the DOD's 
largest acquisition programs have exceeded the so-called Nunn-
McCurdy cost-growth standards established by Congress to 
identify seriously troubled programs.
    As Secretary Gates pointed out in his testimony before our 
committee last month, the list of big-ticket weapon systems 
that have experienced contract or program performance problems 
spans the Services and includes the Air Force tanker, the CSAR-
X, the VH-71, the Osprey, the Future Combat Systems (FCS), the 
Armed Reconnaissance Helicopter, the Littoral Combat Ship 
(LCS), the Joint Strike Fighter (JSF), and so on.
    Overall, DOD's 95 defense acquisition programs, known as 
Major Defense Acquisition Programs (MDAPs), have exceeded their 
research and develop budgets by an average of 40 percent, seen 
their acquisition costs grow by an average of 26 percent, and 
experienced an average schedule delay of almost 2 years. Last 
summer, the Government Accountability Office (GAO) reported 
that cost overruns on DOD's MDAPs now total $295 billion over 
the original program estimates. That's true, even though we've 
cut unit quantities and reduced performance expectations on 
many programs, in an effort to hold costs down.
    These cost overruns happen because of fundamental flaws 
that are endemic to our acquisition system. We have a pretty 
good idea of what those flaws are. Major acquisition programs 
fail because DOD: one, continues to rely on unreasonable cost 
and schedule estimates; two, establishes unrealistic 
performance expectations; three, insists on the use of immature 
technologies; and four, adopts costly changes to program 
requirements, production quantities, and funding levels in the 
middle of ongoing programs.
    Earlier this year, Under Secretary of Defense for 
Acquisition John T. Young, Jr., wrote a memo in which he sought 
to explain the cost growth on some of DOD's largest programs. 
This is what his memorandum said: ``A number of programs had a 
poor foundation and Milestone B, the starting point for major 
development and manufacturing design. . . Fundamentally, these 
programs moved past Milestone B with inadequate foundations 
built on artificially low cost estimates, optimistic schedules 
and assumptions, immature design or technology, fluid 
requirements, and other issues.''
    Mr. Young then went on to list the flaws of each MDAP. The 
JSF: too little understanding of the design; the FCS: fluid 
program strategy; the V-22: immature technology and Congress 
reversed DOD termination; the C-17: development issues and 
underfunding; the Army's Family of Medium Tactical Vehicles: 
design flaws; the CH-47F: low estimates and invalid 
remanufacture assumptions; the advanced EHF Satellite: 
optimistic schedule; the LPD-17: flawed lead ship design 
process and knowledge base; and the F-22A: immature, exquisite 
technology.
    Now, the first two of these programs, JSF and FCS, account 
for almost $80 billion in cost overruns, with average unit 
costs that have already increased by roughly 40 percent each 
over original program estimates, and are likely to rise 
further. According to GAO, both programs were initiated with 
insufficiently mature technologies and overly optimistic 
assumptions about system performance.
    With regard to the JSF, GAO reports that initial estimates 
assumed that commonality between the three variants of the 
aircraft could cut development costs by about 40 percent; 
however, this level of commonality has proven impossible to 
achieve. Twelve years after the program started, three of the 
JSF's eight critical technologies are still not mature, its 
production processes are not mature, and its designs are still 
not fully proven and tested.
    With regard to FCS, GAO reports that the estimated lines of 
code needed to support FCS's software and development are 
almost three times the original assumptions, leading to an 
increase in software development costs that now approaches $8 
billion. Eight years after the program started, only 3 of the 
FCS's 44 critical technologies are fully mature. GAO tells us 
that the Army has not advanced the maturity of 11 critical 
technologies since 2003, and that 2 other technologies, which 
are central to the Army's plans, are now rated less mature than 
when the program began.
    This is the price that we have paid for our failure to 
complete needed systems engineering tasks, perform appropriate 
developmental testing, and build prototypes. Particularly at 
this time, when the Federal budget is under immense strain as a 
result of the economic crisis, we cannot continue this kind of 
waste and inefficiency.
    That is why Senator McCain and I have introduced the Weapon 
Systems Acquisition Reform Act of 2009. This bill is designed 
to help put MDAPs on a sound footing from the outset by 
addressing program shortcomings in the early phases of the 
acquisition process.
    In particular, our bill would address unreasonable 
performance requirements by requiring DOD to rebuild its 
systems engineering capability, reestablish the position of 
director of developmental testing, and use the Joint 
Requirements Oversight Council (JROC) to make early tradeoffs 
between cost schedule and performance requirements.
    Our bill will address unreasonable cost and schedule 
estimates by establishing a new director of independent cost 
assessment to ensure that cost estimates for MDAPs are fair, 
reliable, and unbiased.
    Our bill will reduce the use of immature technologies by 
requiring DOD to make greater use of prototypes, including 
competitive prototypes, and requiring the Director of Defense 
Research and Engineering to periodically review and provide 
independent assessments of the maturity of critical 
technologies on major weapon systems.
    Our bill, finally, addresses costly changes in the middle 
of programs by ensuring, through preliminary design review, 
that requirements are well understood before a program receives 
Milestone B approval, by providing an incentive for contractors 
to improve performance in ongoing programs by developing 
mechanisms to maintain competitive pressure through the program 
cycle, and by tightening the so-called Nunn-McCurdy 
requirements for underperforming programs by providing for the 
termination of any program that cannot be justified after a 
complete reexamination and revalidation.
    Today, we will hear from two distinguished former Under 
Secretaries of Defense for Acquisition, Paul Kaminski and 
Jacques Gansler. We will also hear from Pete Adolph, a former 
DOD Director of Developmental Testing, and Mike Sullivan, the 
GAO Director of Acquisition and Sourcing Management. Each of 
our witnesses has great experience in the area of weapon 
systems acquisition; and, in the course of the last year, each 
has completed a major report recommending significant 
improvements and reforms. We all look forward to their 
testimony on these issues.
    I now call on Senator McCain.

                STATEMENT OF SENATOR JOHN McCAIN

    Senator McCain. Thank you, Mr. Chairman, for today's 
hearing, and more importantly, your leadership on the bill that 
is the subject of today's hearing, the Weapons Acquisition 
Reform Act of 2009. I join you in welcoming our expert 
witnesses today.
    Let me set the overall context of today's discussion, and 
I'll do so very simply. A train wreck is coming. Look at the 
President's 10-year budget and you'll see an overall decrease 
in defense spending. Unless difficult decisions are made and 
serious reform measures are undertaken, our ability to provide 
for our national security will be, over time, fundamentally 
compromised.
    Clearly, the endless cycle of runaway costs, prolonged 
delivery schedules, and poor performance in the acquisition of 
major weapons has, in my view, us mired in a form of unilateral 
disarmament. Since scrutinizing the tanker lease scandal years 
ago, I'm not sure that things are any better. For example, how 
could DOD award a multibillion-dollar contract based on a 
proposal it later found was fundamentally unexecutable? That's 
exactly what happened on the Navy's VH-71 program, the program 
to replace the President's own helicopters.
    Just over the last few years, the VH-71 program has 
doubled, with an additional cost of $6 billion for 28 aircraft 
that will likely cost taxpayers well over $400 million each. 
How could DOD laden a multibillion-dollar shipbuilding program 
with so many requirements that the program more than doubled in 
cost, with DOD basically asleep at the switch? That happened 
with the Navy's LCS program. At times, the program saw change 
orders averaging 75 per week.
    How could a multibillion-dollar program for next-generation 
fighter jets produce planes that are operating below 
satisfactory readiness rates and could end up being too 
expensive to operate? That happened with the Air Force's F-22 
Raptor program.
    How could DOD spend billions for the Army's biggest 
transformational program, valued at almost $200 billion, only 
for it to be, in many respects, closer to the beginning of 
development than it is to the end? That's the FCS program. At 
this point, it's not been clear when, or even if, the 
information network at the heart of the FCS concept can be 
built.
    On our military satellite program, how could a design flaw 
recently emerge that will take at least 1 year, and up to $1 
billion to fix? That's the Air Force's Space-Based Infrared 
System (SBIRS), high satellite program. More cost and schedule 
increases are likely there.
    But, to understand the depth and breadth of our acquisition 
problems, one needs to go no further than to look at the status 
of particular programs. Across all the Services, the top 75 
programs have unfunded cost overruns of at least $295 billion. 
From 2000, the number of MDAPs has increased from 75 to 95, and 
the cost of those programs has doubled from $790 billion to 
$1.6 trillion, leaving unfunded acquisition commitments equal 
to more than 10 years' worth of major weapons procurement 
funding.
    In other words, in the current fiscal environment we find 
ourselves, the DOD acquisition plan is unaffordable. In my 
view, meaningful reform is only going to happen if DOD itself 
decides to change, develops an overarching management 
philosophy, sets up clear lines of authority and 
accountability, brings discipline and control over the 
requirement process, shuts the revolving door, and restores the 
corps of qualified and experienced acquisition and contracting 
professionals. That's what this legislation helps to do.
    In this bill, the chairman and I built on previous reform 
initiatives by focusing on costs and risk. The bill reflects 
that a key to managing defense procurement programs effectively 
is starting them right by requiring key program reviews up 
front to catch costly design flaws and technology risks before 
we actually buy them.
    Probably the most aggressive feature of the bill gives DOD 
a big stick, bigger than anything available under current law, 
to wield against the very worst-performing programs. It does so 
by giving DOD additional tools to enforce fair, reliable, and 
unbiased independent cost estimates with the creation of a new 
director. Unlike merely promulgated DOD instructions, which 
apply only to new programs, that provision will capture 
chronically-poor performers that are in the development 
pipeline now.
    Mr. Chairman, I don't want to go all over the features of 
the bill; I want to hear from our witnesses. But, for truly 
meaningful reform to endure, the commitment to reform must 
begin with the fiduciaries of the taxpayers dollars within the 
DOD itself.
    I thank you, Mr. Chairman. I welcome the witnesses.
    [The prepared statement of Senator McCain follows:]

               Prepared Statement by Senator John McCain

    Thank you, Mr. Chairman, for today's hearing and, more importantly, 
your leadership on the bill that is the subject of today's hearing, 
S.454, ``The Weapons Systems Acquisition Reform Act of 2009.''
    I appreciate the opportunity to serve as an original co-sponsor on 
it with you, and I too welcome our expert witnesses today.
    Before I turn to the bill, let me briefly set the overall context 
of today's discussion, and I'll do so simply.
    A train wreck is coming, and unless hard decisions are made and 
serious reform measures undertaken, our ability to see to our national 
security interests will be over time fundamentally compromised.
    Let me be clear. The endless cycle of runaway costs, prolonged 
delivery schedules and poor performance we have seen in major weapons 
has us, in my view, mired in a form of unilateral disarmament.
    Since we closely scrutinized the tanker lease scandal years ago, 
I'm not sure that things have gotten much better. For example:

         How could the DOD have awarded a multibillion contract 
        based on a proposal it later found was ``fundamentally 
        unexecutable.'' That's exactly what happened on the Navy's VH-
        71 program--the program to replace the President's own 
        helicopters. Just over the last few years, the program has 
        increased by about $6 billion--for aircraft that will likely 
        cost taxpayers well over $400 million each.
    How could the DOD have loaded up a multibillion shipbuilding 
program with so many requirements that the program doubled in cost (by 
about $400 million) without the DOD really noticing until it was too 
late? That happened--on the Navy's Littoral Combat Ship (LCS) program. 
At times, that program saw change orders averaging 75 per week.
         How could a multibillion dollar program for next 
        generation fighter jets have produced planes that may end up 
        being too expensive to operate? That too happened--on the F-22 
        Raptor program. While being the Nation's most expensive fighter 
        aircraft, those jets continue to operate below satisfactory 
        reliability rates.
         How could the DOD have spent billions for the Army's 
        biggest transformational program, valued at about $200 billion, 
        only for it to be (in many respects) closer to the beginning of 
        development than it is to the end? That's the Future Combat 
        Systems (FCS) program. At this point, it's not even clear when 
        (or even if) the information network at the heart of the FCS 
        concept can be built.
         On a military satellite program, how could a design 
        flaw have recently emerged that will take at least 1 year and 
        up to $1 billion to fix? That's the Air Force's Space-Based 
        Infrared System High satellite program. More cost and schedule 
        increases are likely there.

    We're supposed to have laws in the books that are supposed to 
prevent these sorts of things from happening. Why didn't they work? To 
the person--who is responsible?
    The fact that we're asking those questions (with billions of 
dollars of taxpayer money at stake) and that we have no answers, lays 
out what we're dealing with far better than I can possibly describe 
using facts and figures.
    So, I don't have to mention that the top 75 programs across all of 
the Services have unfunded cost overruns of at least $295 billion. Or, 
that (since fiscal year 2000) the number of major defense acquisition 
programs has increased from 75 to 95. Or, that within that period the 
cost of those programs doubled from $790 billion to $1.6 trillion. Or, 
that this left unfunded acquisition commitments equal to more than 10 
years worth of major weapons procurement funding.
    I don't have to talk about how risky developing most of those 
programs are; or the likelihood that they too will balloon in costs; or 
how much other government-wide priorities will constrain defense 
spending going forward. I don't have to go into all that to make the 
point that the DOD's acquisition plan--as it currently stands--is 
itself likely unaffordable.
    However one looks at it, the honeymoon is over.
    In my view, meaningful reform is only going to happen if DOD itself 
decides to change. DOD has to:

         Develop an overarching management philosophy that 
        dictates an overall approach to ensuring the timely delivery of 
        major weapons that satisfy the needs of the joint warfighter at 
        the most reasonable cost to the taxpayer.
         Set up clear lines of authority and accountability for 
        managing procurement programs.
         Bring discipline and control over the requirements 
        process and get out of the business of gold-plating programs.
         Shut the revolving door. While a leavening of 
        experienced DOD procurement officials working for defense 
        contractors (and vice-versa) is healthy, the lack of meaningful 
        controls on this revolving door is creating an unhealthy 
        tolerance of conflict-of-interest.
         Restore the corps of qualified and experienced 
        acquisition and contracting professionals that DOD had in the 
        1980s before it gave its functions over to contractor/lead 
        systems integrators, thereby letting the fox guard the hen 
        house.

    Until and unless administration and DOD leadership do these sorts 
of things--things that set, if you will, a ``command climate'' that's 
conducive to investing the taxpayers' money responsibly--I fear that 
reform efforts may amount to only rearranging the deck chairs on the 
Titanic. That's something that the chairman and I have no interest in 
doing.
    At the end of the day, we in Congress can only give DOD tools that 
it can use to pursue truly lasting solutions that ensure desirable 
cost/scheduling and performance outcomes in our most complicated, most 
expensive weapons systems. That's what this bill helps do.
    In this bill, the chairman and I build on previous reform 
initiatives by focusing on costs and risk. The bill reflects that a key 
to managing defense procurement programs effectively is starting them 
right--by requiring key program review upfront to catch costly design 
flaws and technology risk before we actually buy the system.
    In so doing, we continue our efforts to move DOD closer towards 
fixed price-type procurement contracting by requiring that technology 
and integration risk can be meted out of a program early. So, by the 
time a program heads into procurement, if a contractor isn't ready to 
sign a good fixed price-type contract, the government shouldn't be 
signing a contract with that company to buy that system at all. There's 
still too much risk.
    Probably the most aggressive feature of the bill gives DOD a big 
stick--bigger than anything available under current law--to wield 
against the very worst performing programs. With that provision we 
intend DOD to, if you will, ``enforce'' fair, reliable, and unbiased 
cost estimates verified by a new director of independent cost 
estimates, also created by this bill. Unlike rules that DOD recently 
put in place, that provision doesn't apply to just new programs. It 
will capture chronically poor performers that are in the development 
pipeline now.
    Another important provision requires DOD to consider a broad range 
of cost-effective measures to help maximize competition throughout the 
life of a weapons program. The industry consolidation that occurred 
after the end of the Cold War went too far. Some 50 prime contractors 
merged into only 6. That's far too few to support a competitive base 
for our current and future needs. It's resulted in a serious decline in 
innovation.
    Other provisions in the bill elaborate in the ``starting programs 
right'' theme by:

         Renewing focus on systems engineering early;
         Requiring the completion of preliminary design reviews 
        before a program can move into the development phase; and
         Strengthening DOD's developmental testing and 
        evaluation capability.

    Other helpful provisions include those that:

         Require DOD budget, requirements, and acquisition 
        officials to consult each other and make trade-offs between 
        cost, schedule, and performance early in the process; and
         Crack down on the frequent changes to programs, which 
        tend to cause many cost increases.

    While this bill is not intended as a panacea to cure all that ails 
the defense procurement process, it is an important next step in 
Congress' continuing efforts to help DOD culturally reform the system.
    As I said a moment ago, for meaningful reform to truly endure, the 
commitment to reform must begin with the fiduciaries of the taxpayers' 
dollars within the Department itself.
    Thank you, Mr. Chairman. I look forward to hearing from our 
witnesses today.

    Chairman Levin. Senator McCain, we thank you.
    Now we'll call on our witnesses. First, we'll call on 
Michael Sullivan from the GAO.
    Would you please proceed?

  STATEMENT OF MICHAEL J. SULLIVAN, DIRECTOR, ACQUISITION AND 
     SOURCING MANAGEMENT, GOVERNMENT ACCOUNTABILITY OFFICE

    Mr. Sullivan. Chairman Levin, Senator McCain, members of 
the committee, I'm pleased to be here today to discuss DOD's 
acquisition outcomes and the legislation proposed by this 
committee to improve them. I'll make a brief oral statement and 
ask that my written testimony be placed in the record.
    Chairman Levin. It will be.
    Mr. Sullivan. We've been reporting for years on poor cost 
and schedule outcomes on DOD's major weapon system 
acquisitions. As the chairman noted, most recently we reported 
that 95 programs in DOD's current portfolio have grown in cost 
by $295 billion and are, on average, delivered about 21 months 
late. We believe there are problems at the strategic and at the 
program levels that cause these outcomes.
    At the strategic level, DOD's three systemic processes for 
building its investment strategy are fragmented and broken. The 
requirement-setting process, known as the Joint Capabilities 
and Integration Development System (JCIDS) is stovepiped, it 
does not consider resources, and it approves nearly every 
proposal that it reviews. The funding process accepts programs 
with unrealistic cost estimates, and does not fully fund their 
development costs. These two processes are poorly integrated, 
and this poor communication leads to unhealthy competition, 
where too many programs are chasing too few dollars.
    Finally, at the program level, the acquisition process 
initiates programs with unreliable cost estimates and without 
knowledge from proper systems engineering analysis to 
understand each weapon system's requirements and the resources 
that will be needed to achieve them. These programs move 
forward with too much technology, design, and manufacturing 
risk as a result.
    DOD understands this and has recently revised its policies 
to address some of these problems. Its new acquisition policy, 
for instance, encourages more systems engineering activity 
earlier in programs, competitive prototyping to gain knowledge 
more quickly and to maintain competition, earlier milestone 
reviews, and steering boards to protect programs against the 
desire to add more requirements once they've started.
    Recent decisions by DOD on some programs have been 
encouraging, and some of the newer programs appear to have 
undergone more disciplined reviews.
    For many years, there's been a broad consensus that weapon 
system acquisition problems are serious and their resolution is 
overdue. With the Federal budget under increasing strain from 
the economic crisis facing our Nation, the time to change is 
now.
    In testimony before this committee last month, Secretary of 
Defense Gates identified many of the systemic problems 
associated with acquisitions and indicated that efforts are 
underway to address them.
    We believe that the legislation this committee has proposed 
will help address the toughest problems, and we 
enthusiastically support it. We believe it precisely targets 
key problem areas, provides much-needed oversight, and provides 
increased authority and independence to the critical functions 
of cost estimating and development testing by requiring them to 
report to the Secretary and to Congress.
    Among other things, its provision to require a full 
inventory of DOD's current systems engineering skills is an 
excellent beginning to rebuilding that sorely-needed 
capability. Its addition of a termination criterion for Nunn-
McCurdy breaches sends a strong signal to programs to have 
realistic cost estimates when they start.
    It is important to state that there is also a need for 
changes to the overall acquisition culture and the incentives 
it provides. The culture should begin to change by resisting 
the urge to achieve the revolutionary, but unachievable, 
capability in one step by allowing technologies to mature in 
the technology base rather than forcing them on the acquisition 
programs too early, by ensuring that urgent requirements are 
well-defined and quickly achievable, and by instituting 
shorter, more predictable development cycles.
    These changes will not be easy to make. Tough decisions 
must be made about DOD's overall portfolio of weapon programs 
and about specific programs; and stakeholders from DOD, the 
military services industry, and Congress will have to play a 
constructive role in this decisionmaking. We see the proposed 
legislation discussed here today as a very healthy step in that 
direction.
    Mr. Chairman, this concludes my statement. I'll wait to 
answer questions.
    [The prepared statement of Mr. Sullivan follows:]

               Prepared Statement by Michael J. Sullivan

    Mr. Chairman and members of the committee: I am pleased to be here 
today to discuss the Department of Defense's (DOD) management of its 
major weapon system acquisitions--an area that has been on the 
Government Accountability Office's (GAO) high-risk list since 1990. 
Prior to and since that time, Congress and DOD have continually 
explored ways to improve acquisition outcomes without significant 
results. While the technological sophistication of DOD weapon systems 
is unparalleled, major weapon programs continue to cost more, take 
longer to complete, and deliver fewer quantities and capabilities than 
originally planned. Last year we reported that the cumulative cost 
growth in DOD's portfolio of 95 major defense acquisition programs was 
$295 billion from first estimates and the average delay in delivering 
promised capabilities to the warfighter was 21 months. Clearly, some 
problems are to be expected in developing weapon systems given the 
technical risks and complexities involved. However, all too often we 
have found that cost and schedule problems are rooted in poor planning, 
execution, and oversight.
    Investment in weapon systems is now at its highest level in two 
decades, and DOD plans to spend more than $357 billion over the next 5 
years on major defense acquisition programs. Effective management of 
this substantial investment is critical as competition for funding has 
increased dramatically within the department and across the government. 
DOD faces a number of fiscal pressures: the ongoing military campaigns 
in Afghanistan and Iraq, rising personnel costs, and the rebuilding and 
modernization of the force. In addition, the economic and fiscal crises 
now facing the Nation have required unprecedented spending by the 
Federal Government, and budget deficits are projected to remain high 
for many years to come. At a time when the Federal budget is strained 
by spending needs for a growing number of national priorities, it is 
imperative that DOD get the best value for every dollar it invests in 
weapon system programs. Every dollar wasted during the development and 
acquisition of weapon systems is money not available for other 
priorities within DOD and elsewhere in the government.
    Today, I will discuss: (1) the systemic problems that have 
contributed to cost, schedule, and performance problems in DOD's 
acquisition of major weapon systems; (2) recent actions the department 
has taken to address these problems; (3) our observations on the 
committee's proposed acquisition reform legislation; and (4) steps that 
Congress and the department need to take to improve the future 
performance of acquisition programs. The statement includes findings 
from our July 2008 report on a knowledge-based funding approach and 
February 2009 report on potential changes to DOD's acquisition 
management framework.\1\ It also draws from our extensive body of work 
on DOD's acquisition of weapon systems. A list of our key products is 
provided at the end of this statement. This work was conducted in 
accordance with generally accepted government auditing standards. Those 
standards require that we plan and perform the audit to obtain 
sufficient, appropriate evidence to provide a reasonable basis for our 
findings and conclusions based on our audit objectives. We believe that 
the evidence obtained provides a reasonable basis for our findings and 
conclusions based on our audit objectives.
---------------------------------------------------------------------------
    \1\ GAO, Defense Acquisitions: A Knowledge-Based Funding Approach 
Could Improve Major Weapon System Program Outcomes, GAO-08-619 
(Washington, DC: July 2, 2008), and Defense Acquisitions: Perspectives 
on Potential Changes to DOD's Acquisition Management Framework, GAO-09-
295R (Washington, DC: Feb. 27, 2009).
---------------------------------------------------------------------------
 FRAGMENTED INVESTMENT DECISIONMAKING, UNEXECUTABLE PROGRAMS, AND LACK 
          OF ACCOUNTABILITY UNDERLIE POOR ACQUISITION OUTCOMES

    Over the past several years our work has highlighted a number of 
underlying systemic causes for cost growth and schedule delays at both 
the strategic and program levels. At the strategic level, DOD's 
processes for identifying warfighter needs, allocating resources, and 
developing and procuring weapon systems--which together define DOD's 
overall weapon system investment strategy--are fragmented. As a result, 
DOD fails to effectively address joint warfighting needs and commits to 
more programs than it has resources for, thus creating unhealthy 
competition for funding. At the program level, a military Service 
typically establishes and DOD approves a business case containing 
requirements that are not fully understood and cost and schedule 
estimates that are based on overly optimistic assumptions rather than 
on sufficient knowledge. Once a program begins, it too often moves 
forward with inadequate technology, design, testing, and manufacturing 
knowledge, making it impossible to successfully execute the program 
within established cost, schedule, and performance targets. 
Furthermore, DOD officials are rarely held accountable for poor 
decisions or poor program outcomes.

DOD Lacks an Integrated Approach to Balance Weapon System Investments
    At the strategic level, DOD largely continues to define warfighting 
needs and make investment decisions on a Service-by-Service and 
individual platform basis, using fragmented decisionmaking processes. 
This approach makes it difficult for the department to achieve a 
balanced mix of weapon systems that are affordable and feasible and 
that provide the best military value to the joint warfighter. In 
contrast, we have found that successful commercial enterprises use an 
integrated portfolio management approach to focus early investment 
decisions on products collectively at the enterprise level and ensure 
that there is a sound basis to justify the commitment of resources.\2\ 
By following a disciplined, integrated process--during which the 
relative pros and cons of competing product proposals are assessed 
based on strategic objectives, customer needs, and available resources, 
and where tough decisions about which investments to pursue and not to 
pursue are made--companies minimize duplication between business units, 
move away from organizational stovepipes, and effectively support each 
new development program they commit to. To be effective, integrated 
portfolio management must have strong, committed leadership; empowered 
portfolio managers; and accountability at all levels of the 
organization.
---------------------------------------------------------------------------
    \2\ GAO, Best Practices: An Integrated Portfolio Management 
Approach to Weapon System Investments Could Improve DOD's Acquisition 
Outcomes, GAO-07-388 (Washington, DC: Mar. 30, 2007).
---------------------------------------------------------------------------
    DOD determines its capability needs through the Joint Capabilities 
and Integration Development System (JCIDS). While JCIDS provides a 
framework for reviewing and validating needs, it does not adequately 
prioritize those needs from a joint, departmentwide perspective; lacks 
the agility to meet changing warfighter demands; and validates almost 
all of the capability proposals that are submitted. We recently 
reviewed JCIDS documentation related to new capability proposals and 
found that most--almost 70 percent--were sponsored by the military 
Services with little involvement from the joint community, including 
the combatant commands, which are responsible for planning and carrying 
out military operations.\3\ Because DOD also lacks an analytic approach 
to determining the relative importance of the capabilities needed for 
joint warfighting, all proposals appear to be treated as equal 
priorities within the JCIDS process. By continuing to rely on 
capability needs defined primarily by the Services, DOD may be losing 
opportunities for improving joint warfighting capabilities and reducing 
the duplication of capabilities in some areas. The JCIDS process has 
also proven to be lengthy and cumbersome--taking on average up to 10 
months to validate a need--thus undermining the department's efforts to 
effectively respond to the needs of the warfighter, especially those 
needs that are near term. Furthermore, the vast majority of capability 
proposals that enter the JCIDS process are validated or approved 
without accounting for the resources or technologies that will be 
needed to acquire the desired capabilities. Ultimately, the process 
produces more demand for new weapon system programs than available 
resources can support.
---------------------------------------------------------------------------
    \3\ GAO, Defense Acquisitions: DOD's Requirements Determination 
Process Has Not Been Effective in Prioritizing Joint Capabilities, GAO-
08-1060 (Washington, DC: Sept. 25, 2008).
---------------------------------------------------------------------------
    The funding of proposed programs takes place through a separate 
process, the department's Planning, Programming, Budgeting, and 
Execution (PPBE) system, which is not synchronized with JCIDS. While 
JCIDS is a continuous, need-driven process that unfolds in response to 
capability proposals as they are submitted by sponsors, PPBE is a 
calendar-driven process comprising phases occurring over a 2-year 
cycle, which can lead to resource decisions for proposed programs that 
may occur several years later. In addition, because PPBE is structured 
by military Service and defense programs and not by the joint 
capability areas being used in JCIDS, it is difficult to link resources 
to capabilities. The PPBE process also largely allocates resources 
based on historical trends rather than on a strategic basis. Service 
shares of the overall budget have remained relatively static for 
decades, even though DOD's strategic environment and warfighting needs 
have changed dramatically in recent years. Because DOD's programming 
and budgeting reviews occur at the back end of the PPBE process--after 
the Services have developed their budgets--it is difficult and 
disruptive to make changes, such as terminating programs to pay for 
new, higher-priority programs.
    We recently reviewed the impact of the PPBE process on major 
defense acquisition programs and found that the process does not 
produce an accurate picture of the department's resource needs for 
weapon system programs, in large part because it allows too many 
programs to go forward with unreliable cost estimates and without a 
commitment to fully fund them.\4\ The cost of many of the programs we 
reviewed exceeded the funding levels planned for and reflected in the 
Future Years Defense Program (FYDP)--the department's long-term 
investment strategy (see fig. 1). DOD's failure to balance its needs 
with available resources promotes an unhealthy competition for funding 
that encourages sponsors of weapon system programs to pursue overly 
ambitious capabilities and underestimate costs to appear affordable. 
Rather than limit the number and size of programs or adjust 
requirements, DOD opts to push the real costs of programs to the 
future. With too many programs underway for the available resources and 
high cost growth occurring in many programs, the department must make 
up for funding shortfalls by shifting funds from one program to pay for 
another, reducing system capabilities, cutting procurement quantities, 
or in rare cases terminating programs. Such actions not only create 
instability in DOD's weapon system portfolio, they further obscure the 
true future costs of current commitments, making it difficult to make 
informed investment decisions.
---------------------------------------------------------------------------
    \4\ GAO-08-619.
---------------------------------------------------------------------------
      
    
    
      
Initiating Programs with Inadequate Knowledge of Requirements and 
        Resources Often Results in Poor Outcomes
    At the program level, the key cause of poor outcomes is the 
approval of programs with business cases that contain inadequate 
knowledge about requirements and the resources--funding, time, 
technologies, and people--needed to execute them. Our work in best 
practices has found that an executable business case for a program 
demonstrated evidence that: (1) the identified needs are real and 
necessary and that they can best be met with the chosen concept; and 
(2) the chosen concept can be developed and produced within existing 
resources. Over the past several years, we have found no evidence of 
the widespread adoption of such an approach for major acquisition 
programs in the department. Our annual assessments of major weapon 
systems have consistently found that the vast majority of programs 
began system development without mature technologies and moved into 
system demonstration without design stability.
    The chief reason for these problems is the encouragement within the 
acquisition environment of overly ambitious and lengthy product 
developments--sometimes referred to as revolutionary or big bang 
acquisition programs--that embody too many technical unknowns and not 
enough knowledge about the performance and production risks they 
entail. The knowledge gaps are largely the result of a lack of early 
and disciplined systems engineering analysis of a weapon system's 
requirements prior to beginning system development. Systems engineering 
translates customer needs into specific product requirements for which 
requisite technological, software, engineering, and production 
capabilities can be identified through requirements analysis, design, 
and testing. Early systems engineering provides the knowledge a product 
developer needs to identify and resolve performance and resource gaps 
before product development begins by either reducing requirements, 
deferring them to the future, or increasing the estimated cost for the 
weapon system's development. Because the government often does not 
perform the proper upfront requirements analysis to determine whether 
the program will meet its needs, significant contract cost increases 
can and do occur as the scope of the requirements changes or becomes 
better understood by the government and contractor. Not only does DOD 
not conduct disciplined systems engineering prior to the beginning of 
system development, it has allowed new requirements to be added well 
into the acquisition cycle. We have reported on the negative impact 
that poor systems engineering practices have had  on  several  
programs,  such  as  the  Global  Hawk  Unmanned  Aircraft  System,  F-
22A, Expeditionary Fighting Vehicle, and Joint Air-to-Surface Standoff 
Missile.\5\
---------------------------------------------------------------------------
    \5\ GAO, Best Practices: Increased Focus on Requirements and 
Oversight Needed to Improve DOD's Acquisition Environment and Weapon 
System Quality, GAO-08-294 (Washington, DC: Feb. 1, 2008).
---------------------------------------------------------------------------
    With high levels of uncertainty about requirements, technologies, 
and design, program cost estimates and related funding needs are often 
understated, effectively setting programs up for cost and schedule 
growth. We recently assessed the service and independent cost estimates 
for 20 major weapon system programs and found that while the 
independent estimates were somewhat higher, both estimates were too low 
in most cases.\6\ In some of the programs we reviewed, cost estimates 
have been off by billions of dollars. For example, the initial service 
estimate for the development of the Marines' Expeditionary Fighting 
Vehicle was about $1.1 billion. The department's Cost Analysis and 
Improvement Group (CAIG) estimated the development cost of the program 
to be $1.4 billion, but development costs for the program are now 
expected to be close to $3.6 billion. In the case of the Future Combat 
System (FCS), the Army's initial estimate for the development cost was 
about $20 billion, while CAIG's estimate was $27 billion. The 
department began the program using the program office's estimate of $20 
billion, but development costs for the FCS are now estimated to be $28 
billion and the program is still dealing with significant technical 
risk. Estimates this far off the mark do not provide the necessary 
foundation for sufficient funding commitments and realistic long-term 
planning.
---------------------------------------------------------------------------
    \6\ GAO-08-619.
---------------------------------------------------------------------------
    The programs we reviewed frequently lacked the knowledge needed to 
develop realistic cost estimates. For example, program Cost Analysis 
Requirements Description documents--used to build the program cost 
estimate--often lack sufficient detail about planned program content 
for developing sound cost estimates. Without this knowledge, cost 
estimators must rely heavily on parametric analysis and assumptions 
about system requirements, technologies, design maturity, and the time 
and funding needed. A cost estimate is then usually presented to 
decisionmakers as a single, or point, estimate that is expected to 
represent the most likely cost of the program but provides no 
information about the range of risk and uncertainty or level of 
confidence associated with the estimate.

Lack of Accountability for Making Weapon System Decisions Hinders 
        Achieving Successful Outcomes
    DOD's requirements, resource allocation, and acquisition processes 
are led by different organizations, thus making it difficult to hold 
any one person or organization accountable for saying no to a proposed 
program or for ensuring that the department's portfolio of programs is 
balanced. DOD's 2006 Defense Acquisition Performance Assessment study 
observed that these processes are not connected organizationally at any 
level below the Deputy Secretary of Defense and concluded that this 
weak structure induces instability and inhibits accountability. 
Furthermore, a former Under Secretary of Defense for Acquisitions, 
Technology, and Logistics has stated that weapon system investment 
decisions are a shared responsibility in the department and, therefore, 
no one individual is accountable for these decisions. Frequent turnover 
in leadership positions in the department exacerbates the problem. The 
average tenure, for example, of the Under Secretary of Defense for 
Acquisition, Technology, and Logistics over the past 22 years has been 
only about 20 months.\7\
---------------------------------------------------------------------------
    \7\ The position of Under Secretary of Defense for Acquisition was 
established in 1986 and the title was subsequently changed to the Under 
Secretary of Defense for Acquisition, Technology, and Logistics in 
2000. Since 1986, there have been 11 Under Secretaries.
---------------------------------------------------------------------------
    When DOD's strategic processes fail to balance needs with resources 
and allow unsound, unexecutable programs to move forward, program 
managers cannot be held accountable when the programs they are handed 
already have a low probability of success. Program managers are also 
not empowered to make go or no-go decisions, have little control over 
funding, cannot veto new requirements, and have little authority over 
staffing. At the same time, program managers frequently change during a 
program's development, making it difficult to hold them accountable for 
the business cases that they are entrusted to manage and deliver.
    The government's lack of control over and accountability for 
decisionmaking is further complicated by DOD's growing reliance on 
technical, business, and procurement expertise supplied by contractors. 
This reliance may reach the point where the foundation upon which 
decisions are based may be largely crafted by individuals who are not 
employed by the government, who are not bound by the same rules 
governing their conduct, and who are not required to disclose any 
financial or other personal interests they may have that conflict with 
the responsibilities they have performing contract tasks for DOD. For 
example, while the total planned commitments to major acquisition 
programs have doubled over recent years, the size of the department's 
systems engineering workforce has remained relatively stable, leading 
program offices to rely more on contractors for systems engineering 
support. Further, in systems development, DOD typically uses cost-
reimbursement contracts in which it generally pays the reasonable, 
allocable, and allowable costs incurred for the contractor's best 
efforts, to the extent provided by the contract. The use of these 
contracts may contribute to the perpetuation of an acquisition 
environment that lacks incentives for contractors to follow best 
practices and keep costs and schedules in check.

 RECENT DOD POLICY CHANGES COULD IMPROVE FUTURE PERFORMANCE OF WEAPON 
                            SYSTEM PROGRAMS

    The department understands many of the problems that affect 
acquisition programs and has recently taken steps to remedy them. It 
has revised its acquisition policy and introduced several initiatives 
based in part on direction from Congress and recommendations from GAO 
that could provide a foundation for establishing sound, knowledge-based 
business cases for individual acquisition programs. However, to improve 
outcomes, DOD must ensure that its policy changes are consistently 
implemented and reflected in decisions on individual programs--not only 
new program starts but also ongoing programs as well. In the past, 
inconsistent implementation of existing policy has hindered DOD's 
efforts to execute acquisition programs effectively. Moreover, while 
policy improvements are necessary, they may be insufficient unless the 
broader strategic issues associated with the department's fragmented 
approach to managing its portfolio of weapon system investments are 
also addressed.
    In December 2008, DOD revised its policy governing major defense 
acquisition programs in ways intended to provide key department leaders 
with the knowledge needed to make informed decisions before a program 
starts and to maintain disciplined development once it begins. The 
revised policy recommends the completion of key systems engineering 
activities before the start of development, includes a requirement for 
early prototyping, establishes review boards to identify and mitigate 
technical risks and evaluate the impact of potential requirements 
changes on ongoing programs, and incorporates program manager 
agreements to increase leadership stability and management 
accountability. The policy also establishes early milestone reviews for 
programs going through the pre--systems acquisition phase. In the past, 
DOD's acquisition policy may have encouraged programs to rush into 
systems development without sufficient knowledge, in part, because no 
formal milestone reviews were required before system development. If 
implemented, these policy changes could help programs replace risk with 
knowledge, thereby increasing the chances of developing weapon systems 
within cost and schedule targets while meeting user needs. Some aspects 
of the policy were first pilot-tested on selected programs, such as the 
Joint Light Tactical Vehicle program, and indications are that these 
programs are in the process of acquiring the requisite knowledge before 
the start of systems development. Some key elements of the department's 
new acquisition policy include:

         a new materiel development decision as a starting 
        point for all programs regardless of where they are intended to 
        enter the acquisition process,
         a more robust Analysis of Alternatives (AOA) to assess 
        potential materiel solutions that address a capability need 
        validated through JCIDS,
         a cost estimate for the proposed solution identified 
        by the AOA,
         early program support reviews by systems engineering 
        teams,
         competitive prototyping of the proposed system or key 
        system elements as part of the technology development phase,
         certifications for entry into the technology 
        development and system development phases (as required by 
        congressional legislation),
         preliminary design review (PDR) that may be conducted 
        before the start of systems development, and
         configuration steering boards to review all 
        requirements and technical changes that have potential to 
        affect cost and schedule.

    As part of its strategy for enhancing the roles of program managers 
in major weapon system acquisitions, the department has established a 
policy that requires formal agreements among program managers, their 
acquisition executives, and the user community setting forth common 
program goals. These agreements are intended to be binding and to 
detail the progress the program is expected to make during the year and 
the resources the program will be provided to reach these goals. DOD 
also requires program managers to sign tenure agreements so that their 
tenure will correspond to the next major milestone review closest to 4 
years. The department acknowledges that any actions taken to improve 
accountability must be based on a foundation whereby program managers 
can launch and manage programs toward successful performance, rather 
than focusing on maintaining support and funding for individual 
programs. DOD acquisition leaders have also stated that any 
improvements to program managers' performance depend on the 
department's ability to promote requirements and resource stability 
over weapon system investments.
    Over the past few years, DOD has also been testing portfolio 
management approaches in selected capability areas--command and 
control, net-centric operations, battlespace awareness, and logistics--
to facilitate more strategic choices for resource allocation across 
programs. The department recently formalized the concept of capability 
portfolio management, issuing a directive in 2008 that established 
policy and assigned responsibilities for portfolio management. The 
directive established nine joint capability area portfolios, each to be 
managed by civilian and military co-leads. While the portfolios have no 
independent decisionmaking authority over requirements determination 
and resource allocation, according to some DOD officials, they provided 
key input and recommendations in this year's budget process. However, 
without portfolios in which managers have authority and control over 
resources, the department is at risk of continuing to develop and 
acquire systems in a stovepiped manner and of not knowing if its 
systems are being developed within available resources.

        OBSERVATIONS ON PROPOSED ACQUISITION REFORM LEGISLATION

    Overall, we believe that the legislative initiatives being proposed 
by the committee have the potential, if implemented, to lead to 
significant improvements in DOD's management of weapon system programs. 
Several of the initiatives--including the increased emphasis on systems 
engineering and developmental testing, the requirement for earlier 
PDRs, and the strengthening of independent cost estimates and 
technology readiness assessments--could instill more discipline into 
the front end of the acquisition process when it is critical for 
programs to gain knowledge. Establishing a termination criterion for 
Nunn-McCurdy cost breaches could help prevent the acceptance of 
unrealistic cost estimates as a foundation for starting programs.\8\ 
Having greater involvement by the combatant commands in determining 
requirements and requiring greater consultation between the 
requirements, budget, and acquisition processes could help improve the 
department's efforts to balance its portfolio of weapon system 
programs. In addition, several of the proposals as currently drafted 
will codify what DOD policy already calls for, but are not being 
implemented consistently in weapon programs.
---------------------------------------------------------------------------
    \8\ 10 U.S.C. Sec. 2433(a)(5) requires the Secretary of Defense to 
report to Congress when a program's acquisition unit cost increases by 
at least 25 percent over the current baseline estimate or increases by 
at least 50 percent over the original baseline estimate.
---------------------------------------------------------------------------
Section 101: Systems Engineering Capabilities
    Requires DOD to: (1) assess the extent to which the department has 
in place the systems engineering capabilities needed to ensure that key 
acquisition decisions are supported by a rigorous systems analysis and 
systems engineering process; and (2) establish organizations and 
develop skilled employees to fill any gaps in such capabilities.
    The lack of disciplined systems engineering analysis conducted 
prior to starting system development has been a key factor contributing 
to poor acquisition outcomes. Systems engineering activities--
requirements analysis, design, and testing--are needed to ensure that a 
weapon system program's requirements are achievable and designable 
given available resources, such as technologies. In recent years, DOD 
has taken steps to improve its systems engineering capabilities by 
establishing a Systems and Software Engineering Center of Excellence 
and publishing guidance to assist the acquisition workforce in the 
development of systems engineering plans, education, and training. 
However, as the National Research Council recently reported, DOD's 
systems engineering capabilities have declined over time and shifted 
increasingly to outside contractors.\9\ A comprehensive assessment to 
determine what systems engineering capabilities are in place and what 
capabilities are needed, as recommended in the proposed legislation, is 
a critical first step in enhancing the function of systems engineering 
in DOD acquisitions. At the same time, it will be important for DOD to 
implement steps to ensure systems engineering is applied in the right 
way and at the right time.
---------------------------------------------------------------------------
    \9\ National Research Council, Pre-Milestone A and Early-Phase 
Systems Engineering: A Retrospective Review and Benefits for Future Air 
Force Systems Acquisition (Washington, DC: February 2008).
---------------------------------------------------------------------------
Section 102: Developmental Testing
    Requires DOD to reestablish the position of Director of 
Developmental Test and Evaluation and requires the Services to assess 
and address any shortcomings in their developmental testing 
organizations and personnel.
    Robust developmental testing efforts are an integral part of the 
systems development process. They help to identify, evaluate, and 
reduce technical risks, and indicate whether the design solution is on 
track to satisfy the desired capabilities. As the Defense Science Board 
reported in 2008, developmental testing in weapon system programs needs 
to be improved.\10\ We believe that developmental testing would be 
strengthened by a formal elevation of its role in the acquisition 
process and the reestablishment of a Director of Developmental Test and 
Evaluation position. Furthermore, requiring the Director to prepare an 
annual report for Congress summarizing DOD's developmental test and 
evaluation activities would provide more accountability. We also agree 
that the military Services should be required to assess their 
respective developmental testing entities and address any shortcomings. 
This action would help ensure that the Services have the knowledge and 
capacity for effective developmental test efforts.
---------------------------------------------------------------------------
    \10\ Defense Science Board, Report on Developmental Test & 
Evaluation (Washington, DC: May 2008).
---------------------------------------------------------------------------
Section 103: Technical Maturity Assessments
    Makes it the responsibility of the Director of Defense Research and 
Engineering (DDR&E) to periodically review and assess the technological 
maturity of critical technologies used in major defense acquisition 
programs.
    Ensuring that programs have mature technology before starting 
systems development is critical to avoiding cost and schedule problems, 
yet for many years we have reported that a majority of programs go 
forward with immature technologies and experience significant cost 
growth. Legislation enacted by Congress in 2006, requiring DOD to 
certify that the technology in a program has been demonstrated in a 
relevant environment before it receives approval to start system 
development, has begun to help address this problem. Since the 
legislation was enacted, DOD has asked the DDR&E to conduct independent 
reviews of technology readiness assessments for system development 
milestone decisions. Although DDR&E reviews are advisory in nature, we 
have seen reviews that have pushed programs to do more to demonstrate 
technology maturity. The improvements that this proposed legislation, 
as currently written, is intended to bring about may already be 
occurring in DOD. Congress, however, may wish to consider requiring the 
DDR&E to conduct technology readiness reviews not just periodically, 
but for all major defense acquisition programs, and whether or not 
DDR&E has the capacity and resources to effectively conduct technology 
assessments.

Section 104: Independent Cost Assessment
    Establish a Director of Independent Cost Assessment to ensure that 
cost estimates for major defense acquisition programs are fair, 
reliable, and unbiased.
    Within DOD, the CAIG is the organization responsible for conducting 
independent costs estimates for major defense acquisition programs. The 
CAIG reports to the department's Director of Program Analysis and 
Evaluation, but its principal customer is the Under Secretary of 
Defense for Acquisition, Technology, and Logistics. We believe that 
establishing an independent assessment office that reports directly to 
the Secretary or Deputy Secretary of Defense and to Congress--similar 
to the Office of the Director of Operation Test and Evaluation--would 
more fully integrate cost estimating with the acquisition management 
framework and provide an increased level of accountability. We see no 
reason why CAIG should not form the basis of the proposed organization. 
Congress may also wish to consider appointing the Director for a time-
certain term and making the Director responsible for prescribing cost-
estimating policy and guidance and for preparing an annual report 
summarizing cost estimates for major acquisition programs. Ultimately, 
however, improved cost estimating will only occur if there is a better 
foundation for planning and acquiring weapon system programs--one that 
promotes well-defined requirements, is knowledge-based and informed by 
disciplined systems engineering, requires mature technology, and 
adheres to shorter development cycle times.

Section 105: Role of Combatant Commanders
    Requires the Joint Requirements Oversight Council (JROC) to seek 
and consider input from the commanders of the combatant commands in 
identifying joint military requirements.
    Requirements determination in DOD, particularly for major weapon 
system programs, continues to be driven largely by the military 
Services. Studies by the Defense Science Board, Center for Strategic 
and International Studies, and others have revealed that although the 
combatant commands--which are responsible for planning and executing 
military missions--are the principal joint warfighting customer in DOD, 
they have played a limited role in determining requirements. Currently, 
the JROC is doing more to seek out and consider input from the 
combatant commands through regular trips and meetings to discuss 
capability needs and resourcing issues. However, many of the combatant 
commands do not believe that their needs, which are reflected through 
the Integrated Priority List process, are sufficiently addressed 
through the department's JCIDS process. For the combatant commands to 
meet this proposed legislative mandate and have more influence in 
establishing requirements, DOD should consider providing the combatant 
commands with additional resources to establish robust analytical 
capabilities for identifying and assessing their capability needs. 
Ultimately, the department must better prioritize and balance the needs 
of the military Services, combatant commands, and other defense 
components, and be more agile in responding to near-term capability 
needs.

Section 201: Trade-offs of Cost, Schedule, and Performance
    Requires consultation between the budget, requirements, and 
acquisition processes to ensure the consideration of trade-offs between 
cost, schedule, and performance early in the process of developing 
major weapon systems.
    As currently structured, DOD's budget, requirements, and 
acquisition processes do not operate in an integrated manner. The 
function and timing of the processes are not sufficiently synchronized, 
and the decisionmakers for each process are motivated by different 
incentives. These weaknesses have contributed to the development of a 
portfolio with more programs than available resources can support and 
programs that launch into system development without executable 
business cases. We have recommended that the department establish an 
enterprisewide portfolio management approach to weapon system 
investment decisions that integrates the determination of joint 
warfighting needs with the allocation of resources, and cuts across the 
Services by functional or capability area.\11\ To ensure the success of 
such an approach, we believe that the department should establish a 
single point of accountability with the authority, responsibility, and 
tools to implement portfolio management effectively.
---------------------------------------------------------------------------
    \11\ GAO-07-388.
---------------------------------------------------------------------------
Section 202: Preliminary Design Review
    Require the completion of a PDR and a formal post-PDR assessment 
before a major defense acquisition program receives approval to start 
system development.
    We have found that a key deliverable in a knowledge-based 
acquisition process is the preliminary design of the proposed solution 
based on a robust systems engineering assessment prior to making a 
large financial commitment to system development. Early systems 
engineering provides the knowledge needed by a developer to identify 
and resolve gaps, such as overly optimistic requirements that cannot be 
met with current resources, before product development begins. 
Consequently, DOD would have more confidence that a particular system 
could successfully proceed into a detailed system development phase and 
meet stated performance requirements within cost, schedule, risk, and 
other relevant constraints. The recently revised DOD acquisition policy 
places an increased emphasis on programs planning for PDR prior to the 
start of system development but does not go as far as making it a 
requirement to do so. We support any effort to add controls to the 
acquisition process to ensure that timely and robust systems 
engineering is conducted before major investment decisions, such as the 
approval to start system development, are made.

Section 203: Life-Cycle Competition
    Require DOD to adopt measures recommended by the 2008 Defense 
Science Board Task Force on Defense Industrial Structure for 
Transformation--such as competitive prototyping, dual sourcing, open 
architectures, periodic competitions for subsystem upgrades, and 
licensing of additional suppliers--to maximize competition throughout 
the life of a program.
    We have reported in the past on the problem of diminishing 
competition and the potential benefits of more competition.\12\ In 
discussing the environment that leads to poor acquisition outcomes, we 
have noted that changes within the defense supplier base have added 
pressure to this environment. We noted that in 2006, a DOD-commissioned 
study found that the number of fully competent prime contractors 
competing for programs had fallen from more than 20 in 1985 to only 6, 
and that this has limited DOD's ability to maximize competition in 
order to reduce costs and encourage innovation. However, avenues exist 
for reducing costs through competition. For example, we reported that 
although continuing an alternate engine program for the Joint Strike 
Fighter would cost significantly more in development costs than a sole-
source program, it could, in the long run, reduce overall life-cycle 
costs and bring other benefits.
---------------------------------------------------------------------------
    \12\ GAO, Defense Acquisitions: Better Weapon Program Outcomes 
Require Discipline, Accountability, and Fundamental Changes in the 
Acquisition Environment, GAO-08-782T (Washington, DC: June 3, 2008).
---------------------------------------------------------------------------
Section 204: Nunn-McCurdy Breaches
    Requires that a major defense acquisition program that experiences 
a critical Nunn-McCurdy cost breach be terminated unless the Secretary 
of Defense certifies that: (1) continuing the program is essential to 
national security and the program can be modified to proceed in a cost-
effective manner; and (2) the program receives a new milestone approval 
prior to the award of any new or modified contract extending the scope 
of the program.
    In order for DOD to improve its program outcomes, realistic cost 
estimates must be required when programs are approved for development 
initiation. DOD often underestimates costs in large part because of a 
lack of knowledge and overly optimistic assumptions about requirements 
and critical technologies. This underestimation is also influenced by 
DOD's continuing failure to balance its needs with available resources, 
which promotes unhealthy competition among programs and encourages 
programs to overpromise on performance capabilities and underestimate 
cost. This false optimism is reinforced by an acquisition environment 
in which there are few ramifications for cost growth and delays. Only 
in very rare instances have programs been terminated for poor 
performance. When DOD consistently allows unsound, unexecutable 
programs to begin with few negative ramifications for poor outcomes, 
accountability suffers. As section 204 proposes, the strengthening of 
the Nunn-McCurdy provision--by including the potential termination of 
programs that experience critical cost growth--could facilitate a 
change in DOD's behavior by preventing the acceptance of unrealistic 
cost estimates as a foundation for program initiation and placing more 
accountability on senior DOD leadership for justifying program 
continuation. Programs may thus be forced to be more candid and upfront 
about potential costs, risks, and funding needs, and the likelihood of 
delivering a successful capability to the warfighter at the cost and in 
the time promised may grow.

Section 205: Organizational Conflicts of Interest
    Prohibits systems engineering contractors from participating in the 
development or construction of major weapon systems on which they are 
advising DOD, and requires tightened oversight of organizational 
conflicts of interest by contractors in the acquisition of major weapon 
systems.
    The defense industry has undergone significant consolidation in 
recent years which has resulted in a few large, vertically integrated 
prime contractors. This consolidation creates the potential for 
organizational conflicts of interest where, for example, one business 
unit of a large company may be asked to provide systems engineering 
work on a system being produced by another unit of the same company. As 
the Defense Science Board has recognized, these conflicts of interest 
may lead to impaired objectivity, which may not be mitigated 
effectively through techniques such as erecting a firewall between the 
employees of the two units. While the Federal Acquisition Regulation 
currently covers some cases of potential organizational conflicts of 
interest involving the systems engineering function, there may be a 
need for additional coverage in this area. In general, we would support 
efforts to enhance the oversight of potential organizational conflicts 
of interest, particularly in the current environment of a heavily 
consolidated defense industry.

Section 206: Acquisition Excellence
    Establishes an annual awards program to recognize individuals and 
teams that make significant contributions to the improved cost, 
schedule, and performance of defense acquisition programs.
    We support the creation of an annual awards program to recognize 
individuals and teams for improving the cost, schedule, and performance 
of defense acquisition programs. We have reported that meaningful and 
lasting reform will not be achieved until the right incentives are 
established and accountability is bolstered at all levels of the 
acquisition process. The need for incentives emerged as a significant 
issue in our recent discussions with acquisition experts examining 
potential changes to the acquisition processes enumerated in last 
year's defense authorization act. The discussions revealed that those 
changes may not achieve the desired improvement in acquisition outcomes 
unless they are accompanied by changes in the overall acquisition 
environment and culture, and the incentives they provide for success.
           concluding observations on what remains to be done
    A broad consensus exists that weapon system problems are serious 
and that their resolution is overdue. With the Federal budget under 
increasing strain from the Nation's economic crisis, the time for 
change is now. DOD is off to a good start with the recent revisions to 
its acquisition policy, which, if implemented properly, should provide 
a foundation for establishing sound, knowledge-based business cases 
before launching into development and for maintaining discipline after 
initiation. The new policy will not work effectively, however, without 
changes to the overall acquisition environment. Resisting the urge to 
achieve the revolutionary but unachievable capability, allowing 
technologies to mature in the science and technology base before 
bringing them onto programs, ensuring that requirements are well-
defined and doable, and instituting shorter development cycles would 
all make it easier to estimate costs accurately, and then predict 
funding needs and allocate resources effectively. But these measures 
will succeed only if the department uses an incremental approach. 
Constraining development cycle times to 5 or 6 years will force more 
manageable commitments, make costs and schedules more predictable, and 
facilitate the delivery of capabilities in a timely manner.
    Acquisition problems are also likely to continue until DOD's 
approach to managing its weapon system portfolio: (1) prioritizes needs 
with available resources, thus eliminating unhealthy competition for 
funding and the incentives for making programs look affordable when 
they are not; (2) facilitates better decisions about which programs to 
pursue and which not to pursue given existing and expected funding; and 
(3) balances the near-term needs of the joint warfighter with the long-
term need to modernize the force. Achieving this affordable portfolio 
will require strong leadership and accountability. Establishing a 
single point of accountability could help the department align 
competing needs with available resources.
    The department has tough decisions to make about its weapon systems 
and portfolio, and stakeholders, including military Services, industry, 
and Congress, have to play a constructive role in the process toward 
change. Reform will not be achieved until DOD changes its acquisition 
environment and the incentives that drive the behavior of its 
decisionmakers, the military Services, program managers, and the 
defense industry.
    Mr. Chairman, this concludes my prepared statement. I would be 
happy to answer any questions you may have at this time.
                     contacts and acknowledgements
    For further information about this statement, please contact 
Michael J. Sullivan (202) 512-4841 or [email protected]. Contact points 
for our Offices of Congressional Relations and Public Affairs may be 
found on the last page of this statement. Individuals who made key 
contributions to this statement include John Oppenheim, Charlie 
Shivers, Dayna Foster, Matt Lea, Susan Neill, Ron Schwenn, and Bruce 
Thomas.

    Chairman Levin. Thank you so much, Mr. Sullivan.
    Dr. Gansler?

STATEMENT OF HON. JACQUES S. GANSLER, CHAIRMAN, DEFENSE SCIENCE 
  BOARD TASK FORCE ON INDUSTRIAL STRUCTURE FOR TRANSFORMATION

    Dr. Gansler. Mr. Chairman, members of the committee, thank 
you very much for this honor of appearing before you at what I 
think is a critical period and on such an important topic.
    I don't have to tell this committee of the incredible 
national security challenges that the United States is facing 
in the 21st century, brought on by the rather dramatic world 
changes that I believe require a new, holistic view of 
security--DOD, Department of State, Department of Homeland 
Security, Director of National Intelligence, and so forth--and 
utilizing both hard and soft power, and addressing a very broad 
spectrum of the security missions with great unpredictability 
and covering the full spectrum, from terrorism all the way 
through nuclear deterrence.
    I would also emphasize that we need to take full advantage 
of globalization of the technology of industry, not restricting 
or gaming the benefits from globalization through restrictive 
legislation.
    In recognizing the long-term national security implications 
of the global financial crisis, the need for energy security, 
worldwide pandemics, the impact of climate change, growing 
anti-globalization backlash, and the challenging U.S. 
demographics. We have to do all of this, as Senator McCain 
emphasized, in a likely fiscally-constrained budget 
environment.
    Now, to address these challenges, I believe four highly 
interrelated acquisition issues must be addressed, and they 
have to be addressed by both DOD and Congress. First, what 
goods and services to buy; that's the requirements process. 
Second, how to buy them; that's the acquisition reform. Third, 
who does the acquiring; we have major issues in the acquisition 
workforce. Fourth, from whom it is acquired; namely, the 
industrial base.
    Now, I wish I could tell you that there is some silver 
bullet to address all of these needed changes, but it truly 
requires a very broad set of initiatives in each of the four 
areas if the Nation is to achieve the required 21st century 
national security posture.
    In my prepared testimony, which, Mr. Chairman, I would 
appreciate being put into the record----
    Chairman Levin. It will be.
    Dr. Gansler.----I listed the required actions in each of 
these four areas, and I'd be pleased to discuss any of these 
with you at any time. However, for now let me summarize.
    I believe this is a very critical period, perhaps somewhat 
similar to the period following the launch of Sputnik or the 
fall of the Berlin Wall. Today, the security world is changing 
dramatically, especially since September 11--geopolitically, 
technologically, threats, missions, warfighting, commercially, 
et cetera--and this holistic perspective that I mentioned is 
required. Moreover, a decade of solid budget growth, which I 
believe will almost certainly change, has deferred the 
difficult choices--for example, between more 20th century 
equipment versus 21st century equipment--and the controlling 
acquisition policies, practices, laws, and so forth, as well as 
the Services' budgets and requirements priorities have not been 
transformed sufficiently to match the needs of this new world. 
In fact, there's still an emphasis on resetting versus 
modernization.
    Now, leadership is required to achieve the needed changes. 
You look at the literature on culture change, which I think 
this clearly is, two things are required to successfully bring 
about the needed changes. First is the recognition of the need, 
a crisis. In this case, I believe it is a combination of the 
economic--the budget, if you will, crisis--and the changing 
security needs, along with the shortage of the senior 
acquisition experienced personnel to address these needs. 
Second, leadership, with a vision, a strategy, and an action 
plan. I honestly believe that President Obama, Congress, and 
Secretary Gates support the needed changes; however, it's 
pretty clear that the changes can be expected to be severely 
resisted. Significant change always is.
    I would start, as my highest priority, with the important 
role of the service chiefs and secretaries in recognizing and 
promoting senior acquisition personnel, military and civilian. 
Over the last decade, the DOD acquisition workforce has been 
greatly undervalued. DOD leadership now must demonstrate their 
personal recognition of the critical nature of senior 
experienced acquisition personnel and of the smart acquisition 
practices that they would bring to America's military posture 
in the 21st century.
    As my second priority, I would emphasize the importance of 
weapons costs as a true military requirement, to achieve 
adequate numbers of weapons in a resource-constrained 
environment. This will require enhanced systems engineering, 
including cost-performance tradeoffs, throughout both the 
government and industry, and incentives to industry for 
achieving lower cost.
    By the way, this has been done before; for example, with 
the Joint Direct Attack Munition missile, where the Air Force 
Chief of Staff said it should hit the target and cost under 
$40,000 each. It now sells for under $20,000 and precisely hits 
the targets.
    Finally, as my third priority, I would emphasize the value 
of rapid acquisition, from both its military and its economic 
benefits, which will require the full use of spiral 
development, with each block based on proven, tested technology 
and continuous user and logistician feedback for the subsequent 
block improvements, and with the option of continuous effective 
competition, at the prime or at the sublevel. If they're not 
continuously achieving improved performance at lower costs, 
then they should be competed.
    Achieving these required changes will take political 
courage and sustained, strong leadership by both the executive 
and legislative branches, working together. I hope, and firmly 
believe, that it can be achieved. The American public, and 
particularly our fighting men and women, deserve it, and the 
Nation's future security depends upon it.
    Thank you.
    [The prepared statement of Dr. Gansler follows:]
           Prepared Statement by Hon. Jacques S. Gansler \1\
---------------------------------------------------------------------------
    \1\ Dr. Gansler is Professor and Roger C. Lipitz Chair, as well as 
Director of the Center for Public Policy and Private Enterprise at the 
School of Public Policy, University of Maryland. He served as Under 
Secretary of Defense (Acquisition, Technology, and Logistics) from 
1997-2001.
---------------------------------------------------------------------------
    Mr. Chairman, members of the committee, thank you for the honor of 
appearing before you at this critical period, and on such an important 
topic.
    I need not tell you that the U.S., in the 21st century, faces 
incredible national security challenges--brought on by dramatic world 
changes that require:

         A new, Holistic View of Security (e.g., DOD, State, 
        DHS, DNI, etc.)--utilizing both ``hard'' and ``soft'' power
         Addressing a Broad Spectrum of Security Missions--with 
        great unpredictability (from Terrorism to Nuclear Deterrence)
         Taking full advantage of Globalization (of Technology, 
        Industry, etc.)
         Recognizing the long-term national security 
        implications of:

                 The global financial crisis
                 The need for energy security
                 Worldwide pandemics
                 The impact of climate change
                 The growing anti-globalization backlash
                 The challenging U.S. demographics

         To do all of this in a likely fiscally-constrained 
        budget environment

    To address these challenges, four, highly-interrelated acquisition 
issues must be addressed (by the DOD and Congress):

         What goods and services to buy (the ``requirements'' 
        process)
         How to buy them (``acquisition reform'')
         Who does the acquiring (the acquisition workforce)
         From whom is it acquired (the industrial base)

    I wish I could tell you that there was a ``silver bullet'' to 
address the needed changes; but this truly requires a broad set of 
initiatives in each of the four areas--if the Nation is to achieve the 
required 21st century national security posture.
    This need, for the four sets of broad changes, was emphasized in a 
recent Defense Science Board report; where they found:

         ``DOD policies, processes, and management of the 
        Defense Acquisition Enterprise (broadly defined) impede the 
        transition to an effective, agile, and affordable overall, 
        joint military force for the 21st century.''
         ``U.S. Government policies, practices, and processes 
        do not facilitate the development, deployment, and support of 
        the innovative, affordable, and rapidly acquired weapons, 
        systems, and services needed for the 21st century forces.''
         ``The absence of many of the needed skills, (e.g., 
        experienced program management, systems engineering, biotech, 
        advanced IT) in DOD's acquisition workforce, (particularly at 
        the senior military and civilian levels), combined with the 
        coming retirement and prior, large acquisition workforce 
        reductions, significantly impedes the development, production, 
        support, and oversight of the military capabilities needed for 
        the 21st century.''
         ``Government acquisition policies and industry trends 
        (e.g., further horizontal and vertical consolidations) will not 
        produce the required competitive, responsive, efficient, and 
        innovative National Security Industrial Base.''

    So let me (very briefly) summarize the changes required in each of 
the four, critical (and interrelated) areas: [in priority order within 
each category]

What is acquired:

    To meet the wide range of challenges, within a resource-constrained 
environment, the Nation must focus on:

          1. Lower cost systems and services
          2. Optimized, net-centric systems-of-systems (vs. individual 
        ``platforms'')
          3. A ``reserve'' of resources to rapidly respond to combat 
        commanders' urgent needs
          4. More ``balanced'' allocation of resources (to address 
        ``irregular'' operations): C\3\ISR, unmanned systems, Special 
        Forces, ``Land Warriors,'' cyberdefense, etc; [and these 
        resources must be moved from the Supplementals into the base 
        budget]
          5. Interoperability of ``joint'' systems; and coalition 
        systems
          6. Planning, equipping, and exercising ``as we'll fight'': 
        with allies, multi-agencies, and ``contractors on the 
        battlefield''

How goods and services are acquired:

    To achieve higher performance at lower costs and faster:

          1. Require ``cost'' as a design/military ``requirement'' 
        (because cost, in a resource-constrained environment, is 
        numbers)
          2. Provide viable, continuous ``competition options'' (as the 
        incentive for higher performance at lower costs) e.g. 
        competitive prototypes, competitive split-buys, etc.
          3. Fully utilize ``spiral development,'' with demonstrated 
        technologies (because it is lower cost, lower risk, faster to 
        field; maintains the option of competition; avoids 
        obsolescence; can respond rapidly to combat needs)
          4. Make maximum use of commercial products and services (at 
        all levels--utilizing Other Transactions Authority; especially 
        at lower tiers)
          5. Institutionalize a ``Rapid acquisition,'' parallel process 
        (to respond to COCOM urgent needs)
          6. Create incentives for contractors to achieve desired 
        results (in cost, schedule, and performance)
          7. Implement modern, integrated, enterprise-wide IT systems 
        (logistics, business, personnel, etc.)--including linking 
        Government and Industry
          8. Address Conflict of Interest concerns (from 
        LSIMake/BuySETA); but don't reduce the value 
        of relevant experience

Who does the acquiring:

    A flexible, responsive, efficient, and effective acquisition 
program (for sophisticated, hi-tech goods and services) requires 
``smart buyers.''
    This depends on both quantity and quality of senior and experienced 
military and civilian personnel (especially for expeditionary 
operations). In the last decade-plus, this ``requirement'' has not been 
met! In fact, the acquisition workforce declined on seniority and 
quantity even as procurement appropriations increased.
      
    
    
      
    Therefore, one of the Nation's highest priorities (not just in the 
DOD) must be to address the acquisition workforce.

         The DOD, especially, has an acquisition workforce 
        problem:

                 Greatly reduced senior officers and SESs

                         In 1990 the Army had five general 
                        officers with contract background; in 2007 they 
                        had zero.
                         In 1995, the Air Force had 40 General 
                        Officers in Acquisition, today 24; and in 1995, 
                        87 SESs and today 49
                         The Defense Contract Management Agency 
                        (25,000 people in 1990 down to 10,000 today; 
                        and 4 General Officers to 0)

    These reductions (due to the under valuing of the importance of the 
acquisition workforce) introduce ``opportunities'' for ``waste, fraud 
and abuse'' (e.g., 90 fraud cases under review from war zone; examples 
of poor acquisition process results, such as the Air Force Tanker, the 
Presidential Helicopter, etc.). These Government acquisition workforce 
issues must be addressed. I believe that President Obama, Congress, and 
Secretary Gates all agree on this (but it will take the priority 
attention of the Service Chiefs and Secretaries to make it a priority).

From whom goods and services are acquired:

    To quote, again, from the recent Defense Science Board study (on 
the desired 21st century defense industry):

        ``The last two decades have seen a consolidation of the Defense 
        Industry around 20th Century Needs--The next step is DOD 
        leadership in transforming to a 21st Century National Security 
        Industrial Structure.''

    The ``vision'' for this 21st Century National Security Industrial 
Base (which appropriate government actions, i.e. acquisitions, 
policies, practices, and laws, must incentivize and facilitate) are:

          1. Efficient, responsive, technologically advanced, highly-
        competitive (at all levels, including public and private 
        sectors)
          2. Globalized (utilizing ``best in class'')--requires 
        significant changes to U.S. export controls (i.e., changes to 
        ITAR, EAR, etc.)
          3. Healthy (profitable); and investing in IR&D and capital 
        equipment (rules should separate IR&D and B&P)
          4. Includes commercial firms and equipment, and maximizes 
        dual-use facilities and workforce (barriers must be removed)
          5. ``Independent'' systems-of-systems architecture and 
        systems engineering firms (to support the Government--as the 
        integrator)
          6. Merger and Acquisition policy guidelines to be based on 
        this vision
          7. Strong Government-Industry Communications encouraged
          8. All non-inherently-governmental work to be done 
        competitively (public vs. private, for current government work)
          9. Structural changes to eliminate appearance, or reality, of 
        conflict of interest (regarding ``vertical integration'')--but 
        great care to assure relevant--experienced firms and people 
        involved

    In summary, I believe this is a critical period, perhaps similar to 
the period following the launch of Sputnik or the fall of the Berlin 
Wall. Today the security world is changing dramatically--especially 
since September 11, 2001 (geopolitically, technologically, threats, 
missions, warfighting, commercially, etc.)--and a holistic perspective 
is required (including State, DHS, and DNI, as well as coalition 
operations). Moreover, a decade of solid budget growth--which will 
almost certainly change--has deferred difficult choices (between more 
20th century equipment vs. 21st century equipment). The controlling 
acquisition policies, practices, laws, etc. and the Services' budgets 
and ``requirements'' priorities have not been transformed sufficiently 
to match the needs of this new world (in fact, there is still an 
emphasis on ``resetting'' vs. ``modernization'').
    Leadership is required to achieve the needed changes! All of the 
literature on ``cultural change'' (which this clearly must be) state 
that two things are required to successfully bring about the needed 
changes:

         Recognition of the need (a ``crisis'')

    In this case, I believe it is the combination of economic/budget 
``crisis,'' the changing security needs, and the shortage of the senior 
acquisition-experienced personnel to address the needs; and

         Leadership--with a ``vision,'' a ``strategy,'' and an 
        ``action plan.'' I believe that President Obama, Congress, and 
        Secretary Gates support the needed changes. However, the 
        changes can be expected to be severely resisted--significant 
        change always is!

    I would start with the important role of the Service Chiefs and 
Secretaries in recognizing, and promoting senior acquisition personnel 
(military and civilian) in order to demonstrate their personal 
recognition of the critical nature of smart acquisition practices to 
American's military posture in the 21st century. As my second priority, 
I would emphasize the importance of weapons costs as a military 
requirement (to achieve adequate members of weapons in a resource-
constrained environment)--which will require enhanced systems 
engineering (throughout both government and industry) and incentives to 
industry for achieving lower cost systems. Finally, as my third 
priority, I would emphasize the value of ``rapid acquisition'', for 
both its military and economic benefits--which will require the full 
use of ``spiral development'' (with each ``block'' based on proven/
tested technology, and continuous user and logistician feedback, for 
subsequent ``block'' improvement--and with the option of effective 
competition (at the prime and/or sub-level, if they are not 
continuously achieving improved performances at lower and lower costs).
    Achieving these required changes will take political courage and 
sustained, strong leadership--by both the executive and legislative 
branches (working together). I hope, and firmly believe, it can be 
achieved. The American public, and particularly, our fighting men and 
women, deserve it--and the Nation's future security depends upon it.
    Thank you.

    Chairman Levin. Dr. Gansler, thank you so much.
    Dr. Kaminski?

  STATEMENT OF HON. PAUL G. KAMINSKI, CHAIR, COMMITTEE ON PRE-
   MILESTONE A SYSTEMS ENGINEERING, AIR FORCE STUDIES BOARD, 
                   NATIONAL RESEARCH COUNCIL

    Dr. Kaminski. Mr. Chairman, Senator McCain, and members of 
the committee, first of all, I want to thank you for your 
leadership on these critical acquisition issues and for the 
invitation to testify.
    Since you've asked me to testify, first, in my role as 
Chairman of the National Research Council's Study on Pre-
Milestone A Systems Engineering, with your permission I would 
ask that my statement, which includes a full summary, be put in 
the record, and then I will proceed to provide a short verbal 
summary of the summary.
    Chairman Levin. All the statements will be made part of the 
record. Thank you.
    Dr. Kaminski. Thank you, sir.
    Recent years have seen a serious erosion in our ability to 
field new weapon systems quickly in response to changing 
threats, as well as a large increase in the cost of these 
weapon systems. Our programs today for developing weapon 
systems take two to three times longer than they did 30 years 
ago. I note that time is money in this process, and time also 
leaves room for disruptions, uncertainty, and changes in 
commercial technology. In a 15- or 20-year period, we're seeing 
commercial technology turnover three, four, or five times. When 
a weapon system takes 15 to 20 years to develop, the technology 
that you start with isn't going to be supported when it's 
fielded. So, we have to vigorously attack this time issue.
    Our committee also noted the importance of systems 
engineering in reducing this acquisition time, when combined 
with development planning. We further underscored the 
importance of an early systems engineering effort, in that, the 
decisions made prior to and the key Milestone A and B decisions 
impact somewhere between 75 percent and 85 percent of the total 
life-cycle cost. So, the time to address those issues is up 
front, before those decisions are made.
    Our committee also noted that many of the conclusions that 
we reached have been reached in several previous studies. So, 
the issue isn't disagreement on what the recommendations are, 
the issue is implementing those recommendations. So, once 
again, we thank you for your leadership in creating a forum for 
that kind of implementation.
    Let me address now the issues that you asked me to address.
    First of all, just one overall comment on systems 
engineering. I agree with Secretary Gates, who, when asked 
about acquisition, said, ``There is no one silver bullet that 
is going to correct all the problems.'' But, I do believe that 
good systems engineering, coupled with effective development 
planning early on, are two of the most important contributors 
to successful acquisition.
    Our report provided some formal definitions of ``systems 
engineering,'' but they tend to be arcane, so I thought I might 
start with a couple of examples. I'll briefly describe some 
examples of good systems engineering in the work we've done, 
and also where have we seen poor systems engineering.
    One of the really good examples is the Apollo program. That 
program, from a dead start, put man on the moon in about 8 
years. When that program was started, we didn't have mature 
technology. What we did was good upfront systems engineering 
and development planning, so we could proceed in a sequential 
way, step by step, with each new step building on the previous 
step. In building hardware, we were also building the 
experience of our acquisition workforce and our industry, so we 
could, step by step, increase our capabilities, eventually 
going to the moon.
    Another really good example is the Air Force 
Intercontinental Ballistic Missile (ICBM) programs that were 
done in the 1970s and 1980s. What we saw there is that we would 
never start a full-scale development contract for a new ICBM 
until we had done the upfront systems engineering and 
development planning. The development planning produced an 
inertial guidance system for the ICBM, as well as critical 
propulsion components, and a reentry vehicle. That not only 
reduced the risk of the hardware development and integration in 
the future, it also gave domain experience to our key people in 
government and industry, so that when we threw the switch and 
started full-scale development, we could typically expect a 
first flight in 3 to 4 years, as a result of that experience 
base. That's what we need to restore.
    You asked: ``What were systemic contributors to acquisition 
problems?'' I listed five.
    The first of these is the lack of this early and continuing 
systems engineering, coupled to a development planning program 
early on, right upfront.
    The second key impediment is the lack of alignment of 
responsibility, authority, and accountability of the program 
manager. A program manager needs to be able to exercise his or 
her judgment. Much of the program manager's authority has been 
taken away by one-size-fits-all approaches to acquisition and 
by the oversight process, which has some onerous elements that 
are nonvalue-added.
    A third major impediment is the lack of stability in 
program funding. Many contribute to that.
    A fourth is the lack of early attention to test and 
evaluation, with insufficient planning and investment in the 
tools, such as modeling and simulation, test equipment, 
facilities, and personnel, to provide us with the timely and 
meaningful results needed by program management and for 
continuous systems engineering to refine our performance 
objectives and development plans.
    Finally, the root fundamental issue here is this excessive 
time to acquire that I had spoken about. Time is money. As this 
time increases from a few years in the past to 15 years today, 
it undermines our entire process, causing the key participants 
to lose what I call the ``recipe'' for how we move forward and 
also to lose a sense of accountability. When we see new 
capabilities that are developed and fielded in 5 years, the 
engineers, the managers, the testers, the cost analysts are all 
able to benefit and apply their experience from previous 
programs, and they can also be held accountable, since they can 
be in place managing the programs deliverables during one 
assignment. That all changes when we move to 15-year 
acquisitions and we have five rollovers of management, 
engineers, and cost analysts, and five rollovers of the 
technology in the process.
    So, attacking acquisition time is fundamental. I would say, 
a testament to our failure today is the fact that we have to 
discard our current acquisition approaches to deal with our 
urgent needs and field systems, such as the Mine Resistant 
Ambush Protected vehicle and counter-improvised explosive 
devices (IEDs) by forming and using rapid-reaction 
organizations, because our existing ones don't work. They can't 
respond to the cycle time that we need.
    So, what do we do about this? Again, I've listed in my 
statement five steps.
    The first is to ensure that we not only restore, but 
enhance, this early and continuing systems engineering work, 
coupled with development planning. This means restoring funding 
upfront in the programs, and using independent estimates to 
ensure we have enough funding upfront. It also means attracting 
best and brightest to the critical systems engineering work, 
and providing a path to career advancement, career tracking, 
and leadership for the key people that we need to rebuild in 
cadre.
    Second issue is the alignment of the responsibility, 
authority, and accountability of the program manager. I've 
listed several steps in my statement about what's needed to be 
done to do that.
    Third issue is improving funding stability. We pay a great 
deal for the instability we cause by making funding adjustments 
to program. My experience shows that every time we make a cut 
in a program, for financial or other reasons unrelated to 
performance, we end up eventually putting in three times what 
we cut to restore the program later and get it back to a base.
    Fourth item is giving early and serious attention to the 
test and evaluation issues that I noted earlier, so they can be 
part of a rapid process. When we wait for test and evaluation 
results because we haven't done a good job planning and 
preparing, what we have is hundreds of people sitting on their 
hands, waiting for results, and we're paying all those people 
while they wait for results.
    Finally, the last item is fundamentally attacking this 
problem of long development times by the combination of the 
previous four items.
    I believe action on these five issues will have a 
significant and demonstrable impact on our serious acquisition 
problems. I believe that we need to move now with the same 
urgency and priority that we expect in combat operations to 
permit the timely and effective development and fielding of new 
capabilities and services with what I expect will be more 
limited future defense dollars.
    Thank you, Mr. Chairman.
    [The prepared statement of Dr. Kaminski follows:]

               Prepared Statement by Dr. Paul G. Kaminski

    Chairman Levin, and Ranking Member McCain:
    Thank you for your leadership on the Department of Defense (DOD) 
acquisition, and for the invitation to testify on these important 
acquisition issues. Since you have asked me to testify in my role as 
Chair, Committee on Pre-Milestone A Systems Engineering, Air Force 
Studies Board, National Research Council, I will begin by providing a 
summary of our report, which was approved by the Governing Board of the 
National Research Council and published in 2008. The report is 
available to the public at http://www.nap.edu. After the report 
summary, I will provide my personal views on systems engineering and 
respond to the key issues you asked that I address.

                                SUMMARY

    Recent years have seen serious erosion in the ability of U.S. 
forces to field new weapons systems quickly in response to changing 
threats, as well as a large increase in the cost of these weapons 
systems. Today the military's programs for developing weapons systems 
take two to three times longer to move from program initiation to 
system deployment than they did 30 years ago. This slowdown has 
occurred during a period in which threats have been changing more 
rapidly than ever and when technology advances and accumulated 
experience should have been accelerating rather than slowing the 
development process.
    Many causes for this trend have been suggested, including the 
increased complexity of the tasks and the systems involved from both 
technological and human/organizational perspectives; funding 
instability; loss of ``mission urgency'' after the end of the Cold War; 
bureaucracy, which increases cost and schedule but not value; and the 
need to satisfy the demands of an increasingly diverse user community. 
The difficulty of focusing on a specific, homogeneous, post-Cold War 
threat made problems even worse. Yet although the suggested causal 
factors have merit, a common view is that better systems engineering 
(SE) and development planning could help shorten the time required for 
development, making it more like what it was 30 years ago.
    Simply stated, SE is the translation of a user's needs into a 
definition of a system and its architecture through an iterative 
process that results in an effective system design. SE applies over the 
entire program life cycle, from concept development to final disposal.
    The Committee on Pre-Milestone A Systems Engineering was tasked by 
the U.S. Air Force to examine the role that SE can play during the 
defense acquisition life cycle in addressing the root causes of program 
failure, especially during the pre-Milestone A and early phases of a 
program. Currently, few formal SE processes are applied to Air Force 
development programs before the Milestone A review.\1\
---------------------------------------------------------------------------
    \1\ This is a result of the elimination in the 1990s of the 
development planning function that had existed in the Air Force Systems 
Command.
---------------------------------------------------------------------------
    The committee devoted considerable time and space in its report to 
trying to define a minimum set of systems engineering processes. The 
most important of these processes are summarized in the checklist in 
Box S-1 below. A few of the things that need to be taken care of before 
Milestone A and just after it are the following: the consideration of 
alternative concepts (solutions) up front; the setting of clear, 
comprehensive key performance parameters (KPPs) and system 
requirements; and early attention to interfaces and interface 
complexity, to the concept of operations, and to the system 
verification approach. It is these early-stage processes that are 
covered in this report. The importance of stable requirements and 
funding between Milestone B and the achievement of initial operational 
capability (IOC) is stressed, as are processes including good 
configuration management and change control. The committee further 
stresses in the report what it regards as six of the most important 
process areas in its discussion of six ``seeds of failure''.
      
    
    
      
    Figure S-1 DOD life-cycle acquisition process. Points A, B, and C 
at the top of the figure represent Milestones A, B, and C. LCC, life-
cycle cost. Source: Richard Andrews, 2003, An Overview of Acquisition 
Logistics. Fort Belvoir, VA; Defense Acquisition University. Available 
at http://www.afcea.org/events/pastevents/documents/
Track4Session4AMCEmphasisonCustomerFocusedITInitiatives.ppt#364,12,Slide
 12. Last accessed on November 20, 2007.

         SYSTEMS ENGINEERING AND THE DOD ACQUISITION LIFE CYCLE

    The use of formal systems engineering practices throughout the life 
cycle of an acquisition program is critical to fielding the required 
system on time and within budget. Across the top of Figure S-1 are the 
points at which important management decisions are made: Milestones A, 
B, and C. Concept development and refinement occur before Milestone A, 
and further technology development, to reduce system design and 
development (SDD) risk, occurs before Milestone B. Only after Milestone 
B does a program become an enterprise with dedicated funding. 
Importantly, Figure S-1 shows that about three-quarters of total system 
life-cycle costs are influenced by decisions made before the end of the 
concept refinement phase at Milestone A, while about three-quarters of 
life-cycle funds are not actually spent until after Milestone C. This 
means that although high-quality SE is necessary during the entire 
acquisition cycle, the application of SE to decisions made in the pre-
Milestone A period is critical to avoiding (or at least minimizing) 
cost and schedule overruns later in a program. Much of the value of 
early, high-quality SE will be manifested as success in fulfilling 
Milestone B requirements.

                   MAIN FINDINGS AND RECOMMENDATIONS

    The committee's main findings and recommendations are given below.
Finding
    Attention to a few critical systems engineering processes and 
functions particularly during preparation for Milestones A and B is 
essential to ensuring that Air Force acquisition programs deliver 
products on time and on budget.
    Today's weapons systems provide unprecedented capabilities but also 
involve complex interfaces with external command, control, and 
communications systems and rely on a greater volume of software than 
ever before. Early decisions on the weapons system requirements and 
capabilities have a disproportionately large impact on program cost and 
schedule. The committee also recognizes that a lack of flexibility (a 
result of overly rigid processes or a lack of trust among program 
participants or stakeholders) can limit the ability of a program 
manager to change early decisions that warrant changing.
    The committee found many gaps and inconsistencies in the way that 
the Air Force manages pre-Milestone A activities. The committee heard 
from presenters of some cases for which required documents were 
completed pro forma and filed away, never to be seen again, or for 
which required steps were skipped completely. The current practice of 
initiating programs at Milestone B denies the acquisition review 
authority the earlier opportunity (at Milestone A) to make judgments 
about the maturity of the technologies on which the program is based 
and to decide whether technologies need to be further developed prior 
to making a Milestone B commitment to system development and 
demonstration.
Recommendation
    The Air Force leadership should require that Milestones A and B be 
treated as critical milestones in every acquisition program and that a 
checklist such as the ``Pre-Milestone A/B Checklist'' suggested by the 
committee (see Box S-1 in this Summary) be used to judge successful 
completion.
    A rigorous, standard checklist of systems engineering issues should 
be addressed by each program through both the pre-Milestone A and pre-
Milestone B phases. The committee's recommended 20-item checklist is 
shown in Box S-1.
      
    
    
      
    
    
      
    
    
      
    
    
      
    While the committee considers that each item on the checklist is 
important, it calls attention to several items that warrant further 
discussion. Item 2 recognizes that the world changes too fast to be 
friendly to long development cycles. The committee believes that the 
Air Force should strive to structure major development programs so that 
initial deployment is achieved within, say, 3 to 7 years. Thirty years 
ago, this was a typical accomplishment--for example, nearly 40 years 
ago, the Apollo program put the first man on the Moon in fewer than 8 
years.
    The development time issue is addressable by applying systems 
engineering to Items 3, 4, and 13 through 15 before Milestones A and B. 
The definition of clear KPPs by Milestone A and clear requirements by 
Milestone B that can remain stable through IOC can be essential to an 
efficient development phase. It is also important that critical 
technologies be sufficiently mature prior to starting SDD. The 
committee observed that although today's systems are not necessarily 
more complex internally than those of 30 years ago, their ``external 
complexity'' often is greater, because today's systems are more likely 
to try to meet many diverse and sometimes contradictory requirements 
from multiple users. This kind of complexity can often lead to 
requirements being changed between Milestone B and IOC, and it can lead 
to relying on immature technology.
    Item 19 of the checklist stresses the importance of placing 
experienced, domain-knowledgeable managers in key program positions. 
The committee has observed that many of the truly extraordinary 
development programs of the past, such as Apollo, the Manhattan 
Project, the early imaging satellite programs, the U-2, the fleet 
ballistic missile system, and nuclear submarines, were managed by 
relatively small (and often immature) agencies with few established 
processes and controls. In that environment, dedicated managers driven 
by urgent missions accomplished feats that often seem incredible today.
    The committee believes that the accumulation of processes and 
controls over the years--well meant, of course--has stifled domain-
based judgment that is necessary for timely success. Formal SE 
processes should be tailored to the application. But they cannot 
replace domain expertise. In connection with item 19, the committee 
recommends that the Air Force place great emphasis on putting seasoned, 
domain-knowledgeable personnel in key positions--particularly the 
program manager, the chief system engineer, and the person in charge of 
``requirements''--and then empower them to tailor standardized 
processes and procedures as they feel is necessary.
    One key pre-Milestone A task is the analysis of alternatives (AoA), 
which entails evaluating alternative concepts and comparing them in 
terms of capabilities, costs, risks, and so on. Checklist items 1 
through 4, 12, and 13 should be completed before the AoA, while items 5 
through 11 and 14 through 20 may be addressed after the AoA.

Finding
    The creation of a robust systems engineering process is critically 
dependent on having experienced systems engineers with adequate 
knowledge of the domain relevant to a contemplated program.
    While the systems engineering process has broad use, effective 
application depends on having domain experts who are aware of what has 
gone wrong (and right) in the past, recognize the potential to repeat 
the successes under new circumstances, and avoid repeating the errors.
    Ideally, a person or persons with domain knowledge would have had 
experience working on exactly the same problem, or at least a problem 
related to the one at hand. If that is not so (and it might not be if 
the problem has never been addressed before, as was the case for Apollo 
and nuclear submarines), the term could be taken to refer to academic 
training in the relevant field of engineering or science. It could also 
refer to the practice of critical thinking and problem solving that 
comes with learning to be a systems engineer and then building on that 
foundation to gain the experiential knowledge and understanding of 
engineering in the context of an entire system. Systems engineering is 
enabled by tools that have been developed to assist in the management 
of systems engineering (not to be confused with the practice of systems 
engineering).
    Both industry and Air Force presenters told the committee that 
there are not enough domain-knowledgeable and experienced systems 
engineers to support all of the programs that need them.

Recommendation
    The Air Force should assess its needs for officers and civilians in 
the systems engineering field and evaluate whether either its internal 
training programs, which include assignments on Air Force programs that 
provide mentoring by experienced people and hands-on experience in the 
application of systems engineering principles, or external 
organizations are able to produce the required quality and quantity of 
systems engineers and systems engineering skills. Based on this 
assessment, the Air Force first should determine how and where students 
should be trained, in what numbers, and at what cost, and then 
implement a program that meets its needs.
    The Air Force needs to attract, develop, reward, and retain systems 
engineers across the full spectrum of relevant domains, engage them in 
the early (pre-Milestone A) phase of new programs (or modification 
programs), and sustain their participation throughout the life of the 
programs. One important step in this process would be to create an Air 
Force occupational code for systems engineering so that engineers' 
experience and education can be tracked and managed more effectively. 
The Air Force should support an internal systems engineering career 
track that rewards the mentoring of junior systems engineering 
personnel, provides engineers with broad systems engineering 
experience, provides appropriate financial compensation to senior 
systems engineers, and enables an engineering career path into program 
management and operations.

Finding
    The Government, federally Funded Research and Development Centers 
(FFRDCs), and industry all have important roles to play throughout the 
acquisition life cycle of modern weapons systems.
    Since the need for a new or upgraded weapons system is most often 
first recognized by the military user, it is appropriate for the 
military to codify its requirements and, with support from FFRDC and 
independent systems engineering and technical assistance contractors, 
to explore materiel and nonmateriel solutions (such as doctrinal, 
organizational, or procedural changes) as well as to assess the 
potential for new technology to provide enhanced capabilities. While it 
is appropriate and usually desirable to engage development contractors 
in the pre-Milestone B process using competitive study contracts, the 
source selection for system development and demonstration should not be 
made until after the work associated with Milestones A and B is 
complete.

Recommendation
    Decisions made prior to Milestone A should be supported by a 
rigorous systems analysis and systems engineering process involving 
teams of users, acquirers, and industry representatives.
    Working together, government and industry can develop and explore 
solutions using systems engineering methodology to arrive at an optimal 
systems solution.

Finding
    The Air Force used to have a development planning organization that 
applied pre-Milestone A systems engineering processes to a number of 
successful programs, but that organization was allowed to lapse.
    The role of the Air Force development planning organization, which 
was within the Air Force Systems Command, was to provide standard 
evaluation tools and perform pre-Milestone A systems engineering 
functions across acquisition programs. The early 1990s saw an erosion 
of this front-end planning organization along with its funding as the 
Air Force Systems Command (now the Air Force Materiel Command) began to 
play a decreasing role in program execution. In the opinion of several 
speakers who met with the committee, one main reason for the erosion of 
funding was a lack of congressional support for the planning function.

Recommendation
    A development planning function should be established in the 
military departments to coordinate the concept development and 
refinement phase of all acquisition programs to ensure that the 
capabilities required by the country as a whole are considered and that 
unifying strategies such as network-centric operations and 
interoperability are addressed.
    The Air Force and the other military services should establish a 
development planning organization like that which existed in the early 
1990s.
    The roles and functions of the various organizations involved in 
acquiring major weapons systems need to be clearly defined. The 
responsibility for executing systems engineering and program management 
in the pre-Milestone A and B phases should be vested in the military 
departments that do the actual development planning functions. This 
should not be the responsibility of the Office of the Secretary of 
Defense (OSD) or of the Joint Staff. Instead, those offices need to 
enable the creation and functioning of military department development 
planning organizations with policy measures and, where appropriate, 
resources. The Joint Staff, under the auspices of the Joint 
Requirements Oversight Council, may help to define the requirements for 
major programs in the course of the development planning process, but 
it should not run the process itself.
    The existence of ``joint'' programs or a program such as Missile 
Defense, which has several related systems being developed by different 
military services, requires clear guidance from both OSD and the Joint 
Staff about who is in charge. These programs need to be harmonized and 
integrated by the responsible integrating agency. However, development 
planning activities should still take place in the military departments 
where the expertise resides. Consequently, the development planning 
should be managed by that agency.
    While this committee cannot predict how Congress will view the 
revival of a good planning process to support pre-Milestone A program 
efforts, it is still important for the Air Force and DOD to make the 
case for the critical importance of this process before Congress and 
others. A development planning process is important not to start new 
programs, but rather to ensure that any new program (or a new start of 
any kind) is initiated with the foundation needed for success. Funding 
for this planning function needs to be determined by the military 
services, including both the acquisition communities and those (the 
warfighters) who generate the operational requirements.

                          CONCLUDING THOUGHTS

    Many of the conclusions reached and recommendations made by the 
committee are similar to those of previous reviews. Most of the past 
recommendations were never implemented, so one of this committee's most 
critical thoughts relates to the importance of implementation. A 
sampling of key findings and recommendations from previous studies 
follows:

         Government Accountability Office (GAO) 2,3
---------------------------------------------------------------------------
    \2\ GAO, 2003, Defense Acquisitions: Improvements Needed in Space 
Systems Acquisition Management Policy, September. Available at http://
www.gao.gov/new.items/d031073.pdf. Last accessed April 2, 2007.
    \3\ GAO, 2005, Space Acquisitions: Stronger Development Practices 
and Investment Planning Needed to Address Continuing Problems, July. 
Available at http://www.gao.gov/new.items/d05891t.pdf. Last accessed 
April 2, 2007.

                 Separate technology development from systems 
                acquisition. Commit to a program only if the technology 
                is sufficiently mature. Set the minimum Technology 
                Readiness Level (TRL).
                 Stabilize the requirements early.
                 Employ systems engineering techniques before 
                committing to product development.
                 Employ evolutionary approaches that pursue 
                incremental increases in capability.
                 Address shortfalls in science, engineering, 
                and program management staff.

         National Defense Industrial Association (NDIA) \4\
---------------------------------------------------------------------------
    \4\ NDIA Systems Engineering Division, 2003, Task Report: Top Five 
Systems Engineering Issues in Defense Industry, January, Arlington, VA: 
NDIA.

                 Increase SE awareness and recognize SE 
                authority in the program formulation and decision 
                process.
                 Incentivize career SE positions within the 
                government.

         Defense Science Board (DSB) \5\
---------------------------------------------------------------------------
    \5\ DSB/Air Force Scientific Advisory Board (AFSAB) Joint Task 
Force, 2003, Acquisition of National Security Space Programs, May, 
Washington, DC.
---------------------------------------------------------------------------
                 Overhaul the requirements process.
                 Stabilize acquisition tours.
                 Establish a robust SE capability.

         Defense Acquisition Performance Assessment (DAPA) \6\
---------------------------------------------------------------------------
    \6\ Ronald Kadish, Gerald Abbott, Frank Cappuccio, Richard Hawley, 
Paul Kern, and Donald Kozlowski. 2006. Defense Acquisition Performance 
Assessment. Available at http://www.acq.osd.mil/dapaproject/documents/
DAPA-Report-web/DAPA-Report-web-feb21.pdf. Last accessed on April 2, 
2007.

                 Strategic technology exploitation is a key 
                U.S. advantage. Opportunities need to be identified 
                early.
                 The U.S. economic and security environments 
                have changed--for example, there are fewer prime 
                contractors, smaller production runs, reduced plant 
                capacity, fewer programs, and unpredictable threats.
                 The acquisition system must deal with 
                instability of external funding.
                 The DOD management model is based on a lack of 
                trust. Quantity of review has replaced quality. There 
                is no clear line of responsibility, authority, or 
                accountability.
                 Oversight is preferred to accountability.
                 Oversight is complex, not process- or program-
                focused (as it should be).
                 The complexity of the acquisition process 
                increases costs and draws out the schedule.
                 Incremental improvement applied solely to the 
                ``little a'' acquisition process \7\ requires all 
                processes to be stable--but they are not.
---------------------------------------------------------------------------
    \7\ The Acquisition--``Big A''--system is often believed to be a 
simple construct that efficiently integrates three independent 
processes: requirements, budgeting, and acquisition. ``Little a'' on 
the other hand, refers to the acquisition process that focuses on ``how 
to buy'' in an effort to balance cost, schedule, and performance; it 
does not include requirements and budgeting.

    The committee notes that successful implementation of these 
recommendations requires the ``zipper concept''--making connections at 
all levels, from the senior leadership of the Air Force and DOD down to 
the working levels within key program management offices and 
supervisory staffs.
   committee on pre-milestone a systems engineering: a retrospective 
      review and benefits for future air force systems acquisition
    Paul Kaminski, Chair, Technovation, Inc.
    Lester Lyles, Vice Chair, U.S. Air Force, (Ret.)
    Dev Banerjee, The Boeing Company
    Thomas Blakely, Lockheed Martin Aeronautics Company
    Natalie Crawford, RAND Corporation
    Stephen Cross, Georgia Tech Research Institute
    Gilbert Decker, Independent Consultant
    Llewelln Dougherty, Raytheon
    John Farr, Stevens Institute of Technology
    James Frey, Frey Associates
    Robert Fuhrman, Lockheed Martin (Ret.)
    David Gorney, The Aerospace Corporation
    John Griffin, Griffin Consulting
    Wesley Harris, Massachusetts Institute of Technology
    Ronald Kadish, Booz Allen Hamilton
    Robert Latiff, Science Applications International Corporation
    Alden Munson, Independent Consultant
    Mark Wilson, Mark Wilson Consulting
Staff
    Michael A. Clarke, Board Director
    James C. Garcia, Study Director
    Greg Eyring, Senior Program Officer
    Carter W. Ford, Associate Program Officer
    William E. Campbell, Senior Program Associate
    Lanita Jones, Program Associate

    Having summarized the findings and recommendations of the Committee 
on Pre-Milestone A Systems Engineering, let me now add my personal 
views on systems engineering, and the two additional questions that you 
asked me to address: 1) the systemic issues that have contributed to 
cost, schedule, and performance problems in the acquisition of major 
weapon systems; and 2) the steps that Congress and the Department need 
to take to improve performance of the Department's acquisition 
programs.

                          SYSTEMS ENGINEERING

    I agree with Secretary Gates who said that there is no one silver 
bullet that will correct all of the DOD acquisition problems. But I 
believe that good systems engineering coupled with effective 
development planning are the two most important contributors to 
successful acquisition. Our report provided formal definitions of 
systems engineering and development planning that are somewhat arcane. 
So rather than provide further definition, I find it easier to 
illustrate by choosing examples of good and bad systems engineering and 
development planning. Examples of good work include the Apollo Program 
and the U.S. Air Force intercontinental ballistic missile (ICBM) 
programs (e.g., Minuteman, MX) in the 1970s-1980s. Apollo succeeded in 
putting men on the moon in about 8 years. At the start of the program, 
almost all of the key technologies were immature. But good systems 
engineering and development planning were applied to develop a 
systematic approach, reducing risk by taking a series of limited steps, 
and applying the learning and domain experience gained from each step 
to the subsequent step. The U.S. Air Force ICBM programs used a similar 
approach, beginning with conceptual studies and technology development, 
and holding initiation of full scale development (FSD) contracts until 
key guidance system, re-entry system and propulsion technologies had 
been demonstrated. As a result, the time from initiation of FSD until 
first flight was typically 3-4 years. An example of poor systems 
engineering is the SBIRS program, in which a lack of domain experience 
and analysis led to a failure to anticipate the possibility of severe 
radio frequency interference between two key payloads--discovering this 
problem years after program initiation. Inadequate systems engineering 
and development are therefore the first of five items listed below as 
systemic contributors to acquisition problems.

             SYSTEMIC CONTRIBUTORS TO ACQUISITION PROBLEMS

          1. The lack of early and continuing systems engineering and 
        the absence of a closely-coupled development planning program 
        are a fundamental contributor, as identified in our report. The 
        root causes include: (a) lack of sufficient personnel (in both 
        government and industry) with adequate education, training and 
        domain experience (this includes personnel in requirements 
        development as well as in acquisition); and (b) lack of 
        sufficient front end investment necessary to understand the key 
        tradeoffs in cost/schedule/performance, and to identify and 
        address the key risks in a systematic manner.
          2. Lack of alignment of responsibility, authority, and 
        accountability of the program manager. In many cases the 
        program manager's authority is diffused by many levels of 
        oversight in both the Department and in Congress, and the 
        financial and performance constraints imposed do not allow 
        sufficient freedom of action to apply informed judgment in a 
        timely manner. Flexibility is further limited by application of 
        a ``one-size-fits-all'' approach imposed by the DOD 5000 
        system, and the oversight practiced by the DOD and Congress. A 
        program manager needs the freedom to tailor the acquisition 
        approach to the problem, to ensure that the program response 
        time will fall within the response time of the threat, and to 
        apply a variety of tools and techniques (such as the use of 
        prototypes, competitive prototypes, modeling and simulation, 
        critical subsystem and component demostration). For this to 
        work, we need program managers with the education, training, 
        and domain experience needed to enable timely responses and 
        excellent judgment relevant to the domain.
          3. Lack of stability in program funding.
          4. Lack of early attention to test and evaluation, with 
        insufficient planning and investment in the tools (e.g., 
        modeling and simulation, test equipment, facilities, and 
        personnel) to provide the timely and meaningful results needed 
        by program management and systems engineering to continually 
        refine performance objectives and development plans.
          5. Excessive (and growing) time from program initiation to 
        fielding. As this time increases from a few years to 15 years 
        or more, it undermines the entire acquisition process by 
        causing key participants to ``lose the recipe'', and lose a 
        sense of accountability as well as a sense of being able to 
        make a difference. When new capabilities are developed and 
        fielded in 5 years, engineers, managers, testers, cost 
        analysts, etc. are able to benefit from and apply the 
        experience gained from a previous program or program phase. 
        They can also see the results of their decisions and be held 
        accountable. We can also meaningfully employ past performance 
        of the contractor as a factor in the award of future programs--
        an important factor in incentivizing contractor performance. 
        This all changes dramatically when the time extends to 15 
        years, and we have five roll-overs of management, engineers, 
        cost analysts, and commercial technology during this time 
        period. This long and growing time period is a result of the 
        inflexibility inherent in our entire system of requirements 
        development, budgeting and acquisition, and it creates a 
        vicious cycle in which it further exacerbates the contributors 
        above, and they in turn further increase the time and cost 
        growth. We see the result when we must discard our current 
        acquisition system in order to deal with urgent needs and field 
        systems such as MRAP and jammers to counter IED's by forming 
        and using rapid reaction organizations. This cycle must be 
        broken by attacking the root causes.
          steps that congress and the department need to take
          1. The first step is to insure that we not only restore, but 
        enhance early and continuing systems engineering coupled with 
        effective development planning. This will require commitment of 
        more significant investment dollars earlier in our acquisition 
        programs, and a commitment to build a cadre of systems 
        engineers and development planners with the education, training 
        and domain experience needed to be effective. Attracting ``best 
        and brightest'' to this work--and keeping them--will require a 
        personnel system that will identify and track these important 
        human resources and establish a career path to allow those who 
        are successful to advance to senior program management and 
        leadership positions. Their domain experience will be enhanced 
        by managing the building of critical subsystems during the 
        development planning program, reducing risk and building skills 
        and experience at the same time. Congress and the Department 
        can assist by providing incentives for attracting and keeping 
        key personnel (not only financial incentives, but educational, 
        training, recognition, and most important--the opportunity to 
        take on challenging developments and see that they can make a 
        difference). We will need metrics to assess how well we are 
        doing in building and applying this cadre, and we must 
        recognize that this will not be accomplished in 4, or even 8 
        years. But we must begin in earnest and begin now. Finally, we 
        need a means to insure that we have adequate funding upfront 
        for new programs; one approach would require a report at 
        program initiation from an independent cost analyst working 
        with system engineers and development planners who have 
        developed their skills on previous programs.
          2. Alignment of the responsibility, authority, and 
        accountability of the program manager requires that a degree of 
        trust be established between the program manager and those 
        responsible for our oversight mechanisms. We must be prepared 
        to delegate authority to the program manager, and provide him 
        or her with some flexibility to manage--to adjust levels and 
        allocation of funding, to adjust the allocation of performance 
        parameters, to adjust schedule, and to tailor the acquisition 
        approach to be responsive to the need. Clearly, there must be 
        bounds established beyond which the program manager must seek 
        approval from oversight authorities. But I believe these bounds 
        are too narrow and inflexible today. One size does not fit all 
        programs. Congress and the Department should be willing to 
        consider and tailor many of the restrictions which 
        unnecessarily limit and delay program managers today. I have 
        seen many of our successful classified special programs benefit 
        from greater management flexibility than that afforded to their 
        in conventional program counterparts. The good managers of 
        these special programs have used that flexibility to the 
        benefit of the program and the Department by operating with 
        transparency and maintaining trust. I realize that it seems 
        counterintuitive to recommend greater flexibility and trust in 
        an environment rife with acquisition problems, but I believe we 
        need to break the current cycle. One way to begin is with a 
        limited number of pilot programs, with first priority to those 
        programs addressing urgent needs, and assignment of our most 
        experienced program managers to meet those urgent needs. Since 
        these programs will be moving with dispatch, they offer the 
        best opportunity to produce early indications of whether this 
        is a sound approach which should be extended to other programs.
          3. Improving funding stability will require that the 
        Department and Congress be willing to give up some of their 
        flexibility in making annual (or more frequent) adjustments in 
        funding. Doing so will require tradeoffs of the costs and 
        benefits, and I believe it is time to make explicit 
        consideration of these tradeoffs. I have seen the projected 
        benefits of stable funding by looking at theoretical Monte-
        Carlo simulations (which show efficiency improvements of 
        perhaps 8-10 percent as a result of holding a small capital 
        Reserve of less than 10 percent). We can also see the benefits 
        of multiyear procurements saving similar or greater amounts. I 
        have also seen many examples in which funding cuts of x dollars 
        today result in later additions of 3x dollars to catch up.
          4. Giving early and serious attention to test and evaluation 
        will require strengthening our test and evaluation 
        organizations and personnel. Test and evaluation is often an 
        afterthought, and contracts are often written without any 
        mention of how we will test the product. We spend large amounts 
        of money when a large development team waits for test results. 
        The alternative is to spend less money and time by considering 
        testing and investment in test resources as part of our systems 
        engineering and development planning efforts. The actions 
        recommended in paragraph 1 above are the same actions required 
        to address these critical test and evaluation needs.
          5. Reducing the time from program initiation to fielding will 
        require the combination of all actions suggested above. Further 
        benefits will be derived by placing more emphasis on time-
        certain acquisition. This will be helped by better development 
        planning and alignment of incentives. With good development 
        planning, we can assign managers to develop prototypes, 
        critical systems or components needed to better understand 
        cost/performance trades and reduce risk. It is reasonable to 
        expect that many of these developments can be completed in 2-4 
        years, so one manager will be in place from start to delivery. 
        This will help align authority and accountability in both 
        government and industry. As these critical subsystems are 
        delivered and tested, the risk reduction and domain experience 
        gained in both government and industry will allow us to reduce 
        the time required to develop, integrate and test the full 
        system. We can also apply meaningful incentive programs to link 
        profits to demonstrated performance, and use that performance 
        as a factor in making future competitive awards. We can rely on 
        the experience gained during development planning to apply 
        informed judgment to adjust requirements to improve value, 
        reduce time, and better estimate and manage costs. The 
        Department and Congress can assist by placing more emphasis on 
        time-certain acquisition, with the opportunity for milestone 
        reviews at the completion of major development planning 
        activities.

    I believe action on these five issues will have a significant and 
demonstrable impact on our serious acquisition problems. We need to 
move now with the same urgency and priority that we expect in combat 
operations to permit the timely and effective development and fielding 
of new capabilities and services with what I expect will be more 
limited future defense dollars.

    Chairman Levin. Thank you so much, Dr. Kaminski.
    Mr. Adolph?

   STATEMENT OF CHARLES E. (PETE) ADOLPH, CHAIRMAN, DEFENSE 
 SCIENCE BOARD TASK FORCE ON DEVELOPMENTAL TEST AND EVALUATION

    Mr. Adolph. Chairman Levin, Senator McCain, members of the 
committee, I'd like to thank you for inviting me today.
    I chaired a recent Defense Science Board (DSB) study of 
developmental test and evaluation, and during my opening 
remarks, I'll summarize the key points from the study. I ask 
that my written testimony, which addresses the major findings 
and recommendations in more detail, be put into the record.
    Chairman Levin. It will be.
    Mr. Adolph. The task force was originally convened in 2007 
to investigate the causal factors for the high percentage of 
programs completing initial operational test and evaluation in 
recent years, which have been evaluated as not operationally 
effective and/or suitable.
    The task force was asked to assess roles and 
responsibilities for test and evaluation oversight in the 
Office of the Secretary of Defense (OSD). We were also tasked 
to recommend changes to facilitate the discovery of suitability 
problems earlier, and thus, improve the likelihood of 
operational suitability during initial operational test and 
evaluation.
    Very early in the study, it became obvious that the high 
suitability failure rates were the result of systemic changes 
that had been made to the acquisition process, and that changes 
in test and evaluation alone could not remedy poor program 
formulation and execution.
    A number of major changes in the last 15 years have had a 
significant impact on the acquisition process. First, 
congressional direction from 1996 through 1999 reduced the 
acquisition workforce, which, of course, includes developmental 
test and evaluation. In many instances, services acquisition 
organizations went well beyond the mandated cuts, some making 
up to 60 percent reductions in organizations providing 
acquisition support.
    Concurrent with acquisition reform, the general practice of 
reliability growth during development was deemphasized and, in 
most cases, eliminated. This departure from the widely 
recognized best practice may not have been a direct result of 
acquisition reform, but may instead be related to the loss of 
key personnel and experience, as well as shortsighted attempts 
to save acquisition funds at the expense of increased 
sustainment and life-cycle costs. Numerous studies have 
conclusively demonstrated that investing in system reliability 
during development will yield a substantial reduction in 
support costs.
    Our study reached the conclusion that the single most 
important step necessary to correct high suitability failure 
rates is to ensure that programs are formulated to execute a 
viable systems engineering strategy, including a robust 
reliability, availability, and maintainability program, as an 
integral part of design and development.
    Moving on to government test organizations, in the last 15 
years, with some exceptions, there's been a significant 
decrease in government involvement in test planning, conduct, 
and execution. One of our task force members observed that, in 
many instances, the government has gone from oversight to 
insight to out-of-sight. Our task force recommends that 
government test organizations reconstitute and retain a cadre 
of experienced test and evaluation personnel to perform the 
test oversight function.
    Regarding OSD roles and responsibilities for test 
oversight, the study team found that the developmental test 
office, which had existed for decades, was disestablished in 
the late 1990s. Currently there is no OSD organization with 
comprehensive developmental test oversight, responsibility, 
authority, or staff. We recommend that the office be 
reestablished as a direct report to the Deputy Under Secretary 
for Acquisition and Technology, as outlined in the proposed 
legislation.
    I'd like to make a few additional observations about the 
systemic issues that have contributed to the current problems.
    First, during a time of increased programmatic and 
technical complexity, there has been a loss of a large number 
of the most experienced management and technical personnel 
without an adequate replacement pipeline. Solutions to 
acquisition problems must begin with reconstituting a trained 
and experienced government acquisition workforce, which 
includes program managers, subject-matter experts, as well as 
systems engineers, contracts personnel, testers, and 
evaluators.
    Second, more attention must be paid to technology 
readiness, to include prototyping and testing crucial 
technologies.
    Finally, I believe that the major recommendations in the 
recent study chaired by Dr. Kaminski on pre-Milestone A systems 
engineer would, if implemented and combined with a revitalized 
acquisition workforce, go a long way towards correcting many of 
the current acquisition problems.
    Thank you.
    [The prepared statement of Mr. Adolph follows:]

             Prepared Statement by Charles ``Pete'' Adolph

    Mr. Chairman, members of the committee, I am Pete Adolph, the 
chairman of a recent Defense Science Board (DSB) Task Force study of 
Developmental Test and Evaluation (DT&E). I am pleased to present a 
summary of the study results. The findings and recommendations I will 
discuss represent a consensus of the Task Force members and do not 
reflect an official position of the Department of Defense (DOD).
    A DSB Task Force on DT&E was convened in the summer of 2007 to 
investigate the causal factors for the high percentage of programs 
entering Initial Operational Test and Evaluation (IOT&E) in recent 
years which have been evaluated as both not operationally effective and 
not operationally suitable. The following are the specific issues which 
the Task Force was asked to assess:

         Office of the Secretary of Defense (OSD) organization, 
        roles, and responsibilities for Test and Evaluation (T&E) 
        oversight. Compare organization, roles, and responsibilities in 
        both DT&E and OT&E. Recommend changes that may contribute to 
        improved DT&E oversight, and facilitate integrated T&E.
         Changes required to establish statutory authority for 
        OSD DT&E oversight. Title 10, U.S.C. has an OT&E focus, and 
        does not address OSD authority in oversight of DT&E. Recommend 
        changes to title 10 or other U.S. statutes that may improve OSD 
        authority in DT&E oversight.
         Many IOT&E failures have been due to lack of 
        operational suitability. Specific problems have been in the 
        materiel readiness sustainment areas of reliability, 
        maintainability, and availability. Recommend improvements in 
        DT&E process to discover suitability problems earlier, and thus 
        improve likelihood of operational suitability in IOT&E.

                           PROBLEM DEFINITION
 
   In recent years, there has been a dramatic increase in the number 
of systems not meeting suitability requirements during IOT&E. 
Reliability, Availability and Maintainability (RAM) deficiencies 
comprise the primary shortfall areas. DOD IOT&E results from 2001 to 
2006 are summarized in Figures 1 through 3. These charts graphically 
depict the high suitability failure rates during IOT&E resulting from 
RAM deficiencies.
      
    
    
      
      
    
    
      
    Early in the DSB study, it became obvious that the high suitability 
failure rates were the result of systemic changes that had been made to 
the acquisition process; and that changes in DT&E could not remedy poor 
program formulation and execution. Accordingly, the Task Force study 
was expanded to address the broader programmatic issues, as well as the 
issues previously identified.
    A number of major changes in the last 15 years have had a 
significant impact on the acquisition process. First, congressional 
direction in National Defense Authorization Acts for Fiscal Year 1996, 
1997, 1998, and 1999 reduced the acquisition workforce (which includes 
DT&E). Several changes resulted from the implementation of Acquisition 
Reform in the late 1990s. The use of commercial specifications and 
standards was encouraged, unless there was justification for the use of 
military specifications. Industry was encouraged to use commercial 
practices. Numerous military specifications and standards were 
eliminated in some Service acquisition organizations. The requirement 
for a reliability growth program during development was also 
deemphasized, and in most cases, eliminated. At the same time, systems 
became more complex, and systems-of-systems integration became more 
common. Finally, there was a loss of a large number of the most 
experienced management and technical personnel in government and 
industry without an adequate replacement pipeline because of the 
personnel cuts. The loss of personnel was compounded in many cases by 
the lack of up-to-date standards and handbooks, which had been allowed 
to atrophy, or in some cases, eliminated. It should be noted that 
Acquisition Reform included numerous beneficial initiatives. There have 
been many programs involving application of poor judgment in the last 
15 years that can be attributed to acquisition/test workforce 
inexperience and funding reductions. It is probable that these problems 
would have occurred independently of most Acquisition Reform 
initiatives.
    All Service acquisition and test organizations experienced 
significant personnel cuts, the magnitude varying from organization to 
organization. Over time, in-house DOD offices of subject matter experts 
(who specialized in multiple areas, such as promoting the use of proven 
reliability development methods) were drastically reduced, and in some 
cases, disestablished. A summary of reductions in developmental test 
personnel follows. The Army essentially eliminated their military 
Developmental Testing (DT) component and declared the conduct of DT by 
the government to be discretionary in each program. The Navy reduced 
their DT workforce by 10 percent but no shift of ``hands-on'' 
government DT to industry DT occurred. The trend within the Air Force 
gave DT conduct and control to the contractor. Air Force test personnel 
have been reduced by approximately 15 percent and engineering personnel 
supporting program offices have been reduced by as much as 60 percent 
in some organizations. The reduction of Acquisition Program Office and 
Test personnel in the Services occurred during a time when programs 
have become increasingly complex (e.g., significant increases in 
software lines of code, off-board sensor data integration, and systems 
of systems testing).

                 PRINCIPAL FINDINGS AND RECOMMENDATIONS

Reliability, Availability, and Maintainability
    As a result of industry recommendations in the early 1970s, the 
Services began a concerted effort to implement reliability growth 
testing as an integral part of the development process. This 
implementation consisted of a reliability growth process wherein a 
system is continually tested from the beginning of development, 
reliability problems are uncovered, and corrective actions are taken as 
soon as possible. The Services captured this practice in their 
reliability regulations, and DOD issued a new military standard on 
reliability, which included reliability growth and development testing 
as a best practice task. The goal of this process from 1980 until the 
mid-1990s was to achieve good reliability by focusing on reliability 
fundamentals during design and manufacturing rather than merely setting 
numerical requirements and testing for compliance towards the end of 
development.
    The general practice of reliability growth was discontinued in the 
mid- to late 1990s. This discontinuance may not be a direct result of 
Acquisition Reform, but may be related instead to the loss of key 
personnel and experience, as well as short-sighted attempts to save 
acquisition funds at the expense of increased life-cycle costs. With 
the current DOD policy, most development contracts do not include a 
robust reliability growth program. The lack of failure prevention 
during design, and the resulting low initial Mean Time Between Failure 
and low growth potential are the most significant reasons that systems 
are failing to meet their operational reliability requirements.
    According to Army studies, almost 90 percent of the sustainment 
costs are directly correlated with the reliability of the system. Given 
the amount of resources consumed during sustainment, investments in 
reliability enhancements can provide a very large return on that 
investment. A case study conducted by the Logistics Management 
Institute, provided data that indicated an investment in total program 
reliability would yield a substantial reduction in support costs.

    Findings
         Acquisition personnel reductions combined with 
        acquisition system changes in the last 15 years had a 
        detrimental impact on RAM practices

                 With some exceptions, the practice of 
                reliability growth methodologies was discontinued 
                during System Design and Development
                 Relevant military specifications, standards, 
                and other guidance were not used
                 Suitability criteria, including RAM, were de-
                emphasized

         Improved RAM decreases life-cycle costs and reduces 
        demand on the logistics system
         The Deficiency Report can be a valuable tool for early 
        identification of RAM-related suitability problems, when used 
        in conjunction with an adequately resourced deficiency 
        correction system

    Recommendations
    The single most important step necessary to correct high 
suitability failure rates is to ensure programs are formulated to 
execute a viable systems engineering strategy from the beginning, 
including a robust RAM program, as an integral part of design and 
development. No amount of testing will compensate for deficiencies in 
RAM program formulation. To this end, the following RAM-related actions 
are required as a minimum:

         Identify and define RAM requirements during the Joint 
        Capabilities Integration Development System, and incorporate 
        them in the Request for Proposal (RFP) as a mandatory 
        contractual requirement
         During source selection, evaluate the bidders' 
        approaches to satisfying RAM requirements
         Ensure flow-down of RAM requirements to subcontractors
         Require development of leading indicators to ensure 
        RAM requirements are met
         Make RAM, to include a robust reliability growth 
        program, a mandatory contractual requirement and document 
        progress as part of every major program review
         Ensure that a credible reliability assessment is 
        conducted during the various stages of the technical review 
        process and that reliability criteria are achievable in an 
        operational environment
         Strengthen program manager accountability for RAM-
        related achievements
         Develop a military standard for RAM development and 
        testing that can be readily referenced in future DOD contracts
         Ensure a adequate cadre of experienced RAM personnel 
        are part of the Service acquisition and engineering office 
        staffs

Roles and Responsibilities of Government Test and Evaluation 
        Organizations
    The role of the government in the DT process has evolved over the 
past 50 years. With some exceptions, there has been a significant 
decrease in government involvement in test planning, conduct and 
execution, in the last 15 years.
    The traditional role of the government during the DT planning phase 
included the identification of the test resource requirements and 
government test facilities, the development of the test strategy and 
detailed T&E plans, as well as the actual conduct of T&E. When a 
program moved from the planning phase to the test execution phase, the 
government traditionally participated in test conduct and analysis; 
performing an evaluation of the test results for the program office. 
With some exceptions, this is no longer the case. Until recently, it 
was recognized that there should be some level of government 
involvement and oversight even when the contractor has the primary 
responsibility regarding planning and execution of the DT program.
    In addition to the reduction in the number of government 
acquisition and test personnel, the experience level of both government 
and industry personnel has steadily diminished in recent years. A 
significant percentage of the workforce became eligible to retire since 
2000, and due to prior downsizing, there has not been a steady pipeline 
of younger technical personnel to replace them.

    Findings
    The changes in the last 15 years, when aggregated, have had a 
significant negative impact on DOD's ability to successfully execute 
increasingly complex acquisition programs. Major contributors include 
massive workforce reductions in acquisition and test personnel, a lack 
of up-to-date process guidance in some acquisition organizations, 
acquisition process changes, as well as the high retirement rate of the 
most experienced technical and managerial personnel in government and 
industry without an adequate replacement pipeline.

         Major personnel reductions have strained the pool of 
        experienced government test personnel
         A significant amount of DT is currently performed 
        without a needed degree of government involvement or oversight 
        and in some cases, with limited government access to contractor 
        data
    Recommendations
         As a minimum, government test organizations should 
        develop and retain a cadre of experienced T&E personnel to 
        perform the following functions:

                 Participate in the translation of operational 
                requirements into contract specifications, and in the 
                source selection process, including RFP preparation
                 Participate in DT&E planning including Test 
                and Evaluation Master Plan (TEMP) preparation and 
                approval
                 Participate in technical review processes
                 Participate in test conduct, data analysis, 
                and evaluation and reporting; with emphasis on analysis 
                and reporting

         Utilize red teams, where appropriate, to compensate 
        for shortages in skilled, experienced T&E domain and process 
        experts
         Develop programs to attract and retain government 
        personnel in T&E career fields so that the government can 
        properly perform its role as a contract administrator and as a 
        ``smart buyer''

Integrated Test and Evaluation
    Integrated testing is not a new concept within the DOD, but its 
importance in recent years has been highlighted, due in part to the 
growth of asymmetric threats and the adoption of net-centric warfare. A 
December 2007 OSD Test and Evaluation Policy Revisions memorandum 
reinforces the need for integrated testing. Implementation of 
integrated test concepts has been allowed to evolve on an ad-hoc basis. 
The time has come to pursue more consistency in integrated test 
planning and execution.
    Collaboration between developmental and operational testers to 
build a robust integrated test program will increase the amount of 
operationally relevant data that can be used by both communities. DT 
and Operational Test (OT) planning is separate and this inhibits 
efforts by the Services to streamline test schedules, thereby 
increasing the acquisition timeline and program test costs.
    Additionally, there is a widely held assumption by many in the OT 
community that only data from independent OT is acceptable for 
operational evaluation purposes. While not all information from DT may 
be useable by the Operational Test Agency to support IOT&E, a 
significant amount of developmental test data can be used to partially 
satisfy OT requirements. More importantly, an operational perspective 
earlier in the developmental process has often proven to be a catalyst 
to early identification and correction of problems.
    DOD policy should mandate integrated test planning and execution on 
all programs to the extent possible. To accomplish this, programs must 
establish a team made up of all relevant organizations (including 
contractors, developmental and OT&E communities) to create and manage 
the approach to incorporate integrated testing into the T&E Strategy 
and the TEMP.
    Findings
         Service acquisition programs are incorporating 
        integrated testing to a limited degree through varying 
        approaches
         Additional emphasis on integrated testing will result 
        in greater T&E process efficiency and program cost reductions
    Recommendations
         Implement OSD and Service policy mandating integrated 
        DT&E/OT&E planning and execution throughout the program

                 Require sharing and access to all appropriate 
                system-level and selected component-level test and 
                model data by government DT and OT organizations, as 
                well as the prime contractor, where appropriate
                 Integrate test events, where practical, to 
                satisfy OT and DT requirements

Operational Test Readiness Review
    Each Service has an Operational Test Readiness Review (OTRR) 
process. Although it varies from Service to Service, the process 
generally results in in-depth reviews of readiness to undergo an IOT&E 
event.
    Findings
         A DOD Instruction requires that ``the Service 
        Acquisition Executive shall evaluate and determine materiel 
        system readiness for IOT&E''

                 Decision authority is frequently delegated to 
                the appropriate Program Executive Officer
                 Materiel developer is also required to furnish 
                DT&E report to the Under Secretary of Defense for 
                Acquisition, Technology and Logistics (USD[AT&L]) and 
                Director, OT&E

         Shortcomings in system performance, suitability, and 
        RAM are usually identified during the OTRR
         In most cases, the operational test readiness 
        certifying authority is well aware of the risk of not meeting 
        OT criteria when major shortcomings exist
         Because of funding constraints, the low priority given 
        to sustainment, as well as the urgency in recent years to get 
        new capabilities to the Warfighter, major suitability 
        shortcomings have rarely delayed the commencement of dedicated 
        IOT&E
    Recommendations
         Conduct periodic operational assessments to evaluate 
        progress and the potential for achieving pre-determined 
        entrance criteria for operational test events
         Conduct an independent Assessment of Operational Test 
        Readiness prior to the OTRR
         Include a detailed RAM template in preparation for the 
        OTRR
         Require the Command Acquisition Executive to submit a 
        report to OSD that provides the rationale for the readiness 
        decision

OSD Test and Evaluation Organization
    The Task Force was asked to assess OSD roles and responsibilities 
for T&E oversight. T&E has been a visible part of OSD since the early 
1970s, reporting to the Research and Engineering command section when 
it was in charge of acquisition oversight and subsequently to the Under 
Secretary of Defense for Acquisition (now AT&L). The early T&E office 
was responsible for all T&E, ranges, resources oversight, and policy. 
In 1983, Congress established an independent Director, OT&E 
organization, reporting directly to the Secretary of Defense (SECDEF), 
responsible for OT&E policy, budget review, and assessments of 
operational effectiveness and suitability. The Live Fire Test (LFT) 
oversight function was created and added to the DT&E office 
responsibilities in the mid-1980s. Later, the LFT oversight function 
was moved to the DOT&E organization.
    In 1999, the DT&E organization was disestablished. Many functions 
were moved to DOT&E, including test ranges and resources, and joint T&E 
oversight. Some of the remaining T&E personnel billets were eliminated 
to comply with a congressionally mandated (AT&L) acquisition staff 
reduction. The residual DT&E policy and oversight functions were 
separated and moved lower in the AT&L organization.
    A 2000 DSB Task Force Study on Test and Evaluation Capabilities 
recommended that DOD create a T&E resource enterprise within the office 
of the DOT&E to provide more centralized management of T&E facilities. 
This recommendation ultimately led to removing the test ranges and 
resources oversight from DOT&E, abandoning the notion of centralized 
management, and the establishment of the Test Resource Management 
Center (TRMC) in AT&L (as directed by the National Defense 
Authorization Act for Fiscal Year 2003).
    Findings
    Current policy as of December 2007 mandates that developmental and 
operational test activities be integrated and seamless throughout the 
system life cycle. There must be enough experts in OSD with the ability 
to understand and articulate lessons learned in early testing and the 
ability to execute the new T&E policy. That policy is to ``take into 
account all available and relevant data and information from 
contractors and government sources'' in order to ``maximize the 
efficiency of the T&E process and effectively integrate developmental 
and operational T&E.''

         Currently there is not an OSD organization with 
        comprehensive DT oversight responsibility, authority or staff 
        to coordinate with the operational test office

                 The historic DT organization has been broken 
                up and residual DT functions were moved lower in 
                organization in 1999, and lower yet in 2002
                 Programmatic DT oversight is limited by staff 
                size and often performed by generalists vice T&E 
                experts
                 Recruitment of senior field test personnel is 
                hampered by DT's organizational status
                 Existing residual organizations are fragmented 
                and lack clout to provide DT guidance
                 System performance information and DT lessons 
                learned across DOD has been lost
                 DT is not viewed as a key element in AT&L 
                system acquisition oversight
                 Documentation of DT results by OSD is minimal

         Access to models, data, and analysis results is 
        restricted by current practice in acquisition contracting, and 
        the lack of expertise in the DT organization
         TRMC has minimal input to program-specific questions 
        or interaction with oversight organizations on specific 
        programs

                 Organizational separation is an impediment
    Recommendations
         Implementation of integrated and seamless DT and OT 
        will require, at a minimum, greater coordination and 
        cooperation between all testing organizations
         Consolidate DT-related functions in AT&L to help 
        reestablish a focused, integrated, and robust organization

                 Program oversight and policy, and Foreign 
                Comparative Test (FCT)
                 Have Director, DT&E directly report to Deputy 
                Under Secretary of Defense, Acquisition and Technology 
                (DUSD[A&T])
                 Restore TEMP approval authority to Director, 
                DT&E

         Integrate TRMC activities early into DT program 
        planning

                 Make TRMC responsible for reviewing the 
                resources portion of the TEMP

         If such an organization is established and proves 
        itself effective, consider as part of a future consolidation 
        moving LFT back to its original DT location (this would require 
        congressional action and DOT&E concurrence)

    Most of the organizational changes recommended above are within the 
purview of AT&L. The LFT change requires the concurrence of DOT&E and a 
legislative change to Title 10 because of the change in reporting 
official. All the other recommendations can be implemented within 
current DOD authority.

    Chairman Levin. Thank you so much.
    I think we'll try a 7-minute round. I'm not sure there will 
be time for a second round; I think we have a vote scheduled 
around 11:30 a.m., if I'm not mistaken.
    Let me start with this question. Mr. Sullivan, you 
commented on the reform bill, the Weapon Systems Acquisition 
Reform Act of 2009, which I've introduced with Senator McCain. 
Each of you, in a way, has commented on it. Let me start with 
you, Dr. Gansler. Would you comment on the bill, any parts you 
like, any parts you don't like, any additions you might be able 
to recommend at this time?
    Dr. Gansler. Overall, Mr. Chairman, I thought it was in the 
right direction and important. I think that we have to 
recognize that simply writing a memo, passing a law, doesn't 
change the system. Each of the areas that you highlighted, I 
think are in the right direction. What we have to be careful of 
is not going too far in one direction. For example, in the 
conflict of interest, clearly we have to avoid conflict of 
interest, but we don't have to go so far that we have only 
people who have no experience in those positions. That's the 
danger of going too far, in terms of the legislation of it.
    In each of the areas, I think that there's some clarity 
that could be added, but, in general, I think you've gone in 
the right direction. For example, you emphasize systems 
engineering. I think you need to define that as including costs 
so that there's no ambiguity as to whether that's a pure 
engineering problem or a cost issue, from a design perspective.
    I think the importance of test and evaluation that you 
highlight is clearly something that's very important. It could 
be emphasized without really changing the title of the office. 
But, nonetheless, when I was Under Secretary, I felt it was a 
very critical piece. But, it's important, when you talk about 
test and evaluation, that it be viewed as a part of a 
development process. We learn from that testing. We do it 
early, as Mr. Adolph indicated, but we also don't view it as a 
pass-fail final exam because we're doing spiral development, 
and we're continuing to learn from the testing as it goes 
along, and some people have tended to think of the test process 
as a final exam.
    In the area of independent cost analysis, it is absolutely 
essential. On the other hand, we have the Cost Analysis 
Improvement Group (CAIG) office, which I used all the time, and 
I felt it was critically important. The problem is, people 
don't want to use their numbers. If they say it's realistically 
going to cost more, and government and industry want to put in 
the low bid in order to get a program into the overall budget, 
that's a management question; it's not a matter of what 
organization you set up. But, it's a very important function, 
as you highlighted.
    In terms of getting the combatant commanders involved in 
the requirements process, that was exactly what we intended 
with the Packard Commission and Goldwater-Nichols intended with 
the establishment of the Vice Chairman of the Joint Chiefs. 
That was the purpose of that statement. Somehow that hasn't 
gotten the intended strength, it has been much more the 
suppliers than the users, if you will, the warfighters. I think 
it is important to get the combatant commanders much more into 
the loop on the requirements process, and I think you've gotten 
that properly emphasized.
    I think, in the acquisition area I would emphasize that 
projected unit costs are a military requirement because that's 
one of the things we've lost in some of the programs. Global 
Hawk started out that way, got off track. You mentioned the 
JSF. That started off as a unit cost being one of the 
requirements for globalization of that program. It got out off 
track because it lost sight of that unit cost as being one of 
its principal considerations in design.
    As I mentioned, in terms of competition, I think it's 
really important that we view competition as an option 
throughout the programs, at the prime and at the lower tiers, 
but not as a law. You don't compete for its own sake; you 
compete when the current contractors, prime or sub, aren't 
getting better performance at lower cost. That's their 
incentive for doing it. If you tell them we're going to compete 
it anyhow, they have no incentive. If you tell them that if 
they can get higher performance and lower cost, then, in fact, 
they will try to achieve that so that they don't have to 
compete in the next round, and it's doing exactly what the 
government wants them to do.
    In the same way, if they achieve the objectives, we ought 
to use, in effect, the same way that the commercial world does: 
through price elasticity. If you get a lower cost, we'll buy 
more of them. We don't take the money away and put it in the 
general treasury, so there will be a need to create incentives 
for industry, as well as government, for doing a better job of 
achieving higher performance at lower cost, continuously.
    In terms of conflict of interest, I think we need to focus 
on some structural ways to address conflict of interest. We do 
this, in terms of foreign ownership, through limited liability 
corporations, in effect, the special boards are set up. Maybe 
there's some ways we can do that in order to address conflict 
of interest without the sort of blanket requirement that 
someone in an engineering job shouldn't know anything about 
that job. That's wrong.
    Those are the suggestions that I would have. But, overall, 
I think you are definitely going in the right direction with 
the bill.
    Chairman Levin. Thank you. We would welcome any specific 
language changes you would recommend.
    Now, Mr. Sullivan, you already commented. Do you have any 
additional comments?
    Mr. Sullivan. Yes, just briefly. I would say that, given 
the package that we've looked at, we support everything in it. 
We thought it was very well-targeted to the key problems on 
acquisition programs, and that the idea to give more authority 
to the combatant commanders, in terms of getting urgent needs 
met, was a good provision.
    I think the most important thing on an acquisition program 
is, at the outset--we've all discussed it here--to have more 
knowledge about the requirements that you're going to build to, 
before you begin, than they have now.
    So, the two provisions that we think are most important--
first is the cost-estimating provision. We wrote that we don't 
see any reason why the CAIG actually couldn't fulfill that 
position. The important thing there is to probably provide the 
CAIG with more resources so they can do their jobs on a more 
regular basis and rather than just periodically, at every 
milestone. The other critical thing there is that if the 
director was moved up, out of the bureaucracy a little bit 
more, the estimates that the CAIG make might be looked at as 
less personality-driven, if you will, depending upon who's in 
office.
    As Under Secretary of Defense, Dr. Gansler used the CAIG, 
others may not use it as much. If they're reporting to a higher 
level, and they owe Congress a report, I think that'll really 
improve their authority and their visibility and their 
independence. But, the key is always going to go back to the 
requirement-setting process, and jelling that with cost 
estimates. If you begin with not enough information about what 
it is you're going to, what you want to build, you are not 
going to get a good cost estimate. So, it really has to be 
based on knowledge.
    So, the cost-estimating provision, I think, is very good. 
The systems-engineering provisions that you have in there go a 
long way to providing the knowledge that the cost estimators 
would need up front.
    Chairman Levin. Thank you.
    Dr. Kaminski, do you have any comment?
    Dr. Kaminski. Yes, sir, I do. I made some specific comments 
in my statement, but let me make a couple of big-picture 
comments, if I may.
    If there's a direction in which I would try to move 
language in the proposed legislation, it would be to focus, not 
only on process, on oversight mechanisms, but a focus on 
people; people that make this system work. I don't care how 
good a process you put in place, if you don't have people who 
are experienced and know what they're doing, I think you're 
going to end up with problems.
    When I say ``people,'' I am referring very broadly to our 
acquisition workforce. Think about this for a minute. These 
people and requirements development skills are also key for 
those who are doing development of requirements. They have to 
participate in this tradeoff process to consider the cost of 
what they're asking for and how those tradeoff with the 
performance capabilities that are desired.
    Testing is a critical piece of this process. If you think 
about it--I know my experience is, there isn't any program I've 
ever worked on in which I didn't know a heck of a lot more 
about the program 6 months or a year into it than I did when I 
started. So this is a continuing learning process as we find 
out things from testing--what's hard to do, what's easier to 
do. We need to have a continuing dialogue in this requirements 
tradeoff loop.
    So, requirements developers and acquisition personnel have 
to have training and experience in these systems engineering 
tools and techniques. We would never let a fighter pilot get 
into an aircraft without a very extensive training program to 
prepare him or her for that operation. It's one of the reasons 
we do so well with our forces. When I compare our requirements 
for operational requirements with the training and education 
requirements of people going into acquisition requirements 
generation, they pale in comparison. We have to be able to 
develop the training, education, and the domain experience that 
go with this to make it work.
    I also believe it is worth it for us to look very hard at: 
How can we systematically reduce this long development time? 
That's just killing us. There are programs in which we can do 
time-certain acquisition. Time is money, here. By doing good 
development planning to be able to reduce the risk, following 
the example of Apollo, following the example of our ICBM 
programs, I believe we can compress that time. But, we need 
some targets, and some incentives to do that.
    We also need to make better and more extensive use of 
prototypes, in a sensible way. Competitive prototypes for some 
programs, perhaps single-thread programs for others. We also 
need to realize that one size does not fit all, here. For 
example, in dealing with IEDs, we find that we need to have an 
acquisition system whose cycle time is measured in weeks. 
That's a different acquisition system than you need for a 
strategic bomber. The acquisition system has to fit the cycle 
of the threat that we're dealing with, so we have to tailor it 
in that way.
    The last comment I would make is that, with respect to 
implementation, what better place could we find to start to 
implement some of these processes, procedures, and people 
development than in these urgent programs that are doing rapid 
acquisition. Why start there? One, it's urgent. Two, we'll be 
able to see the impact of changes that we make more quickly in 
programs that are operating in cycle times of weeks or months, 
and see what's benefiting us and what is not. I'd like to see 
that commitment to implementation because it'll happen much 
more quickly than if we simply write new processes or new 5000 
series in the DOD.
    Chairman Levin. Thank you.
    My time's way up, but, Mr. Adolph, do you have a brief 
comment on the bill? Any changes?
    Mr. Adolph. Very brief comment. In general, I believe that 
the proposed legislation will go a long way toward alleviating 
the major acquisition problems which have occurred in recent 
years.
    I also have a quick comment regarding the legislation 
pertaining to the director of the developmental test function. 
I would add one responsibility, that the director participate 
in the acquisition program reviews conducted by the Under 
Secretary for Acquisition and submit a status of the 
developmental testing for the programs under review. That was 
the norm in the past, for many years, and I think that's 
important.
    Chairman Levin. Thank you so much.
    Senator McCain.
    Senator McCain. Thank you, Mr. Chairman. Again, thanks, to 
the witnesses.
    I think there's general agreement on the part of the 
witnesses that there's been a dramatic erosion in acquisition 
workforce and test and evaluation personnel. Maybe I could ask 
you, Dr. Kaminski, what happened?
    Dr. Kaminski. I think a number of things happened. Just as 
there's no one silver bullet, there's no single entity or 
action to blame. But, let me list some of the things that 
happened. I think, as we looked at major programs of 
acquisition reform, we tried to do more with less. We also had 
some pretty strong direction from some portions of Congress on 
this. I can recall the chairman of a key committee who publicly 
made statements that said we had too many ``shoppers'' in the 
Department, and we really needed to reduce this acquisition 
workforce. So, over a period of 3 or 4 years, nearly 50 percent 
of the acquisition workforce was taken out. Now, the Department 
was a partner in that, agreeing to those reductions.
    Senator McCain. We removed incentives for people to remain 
in the acquisition workforce, in the form of lack of promotion 
or career enhancement?
    Dr. Kaminski. Exactly. If we are able to attract and train 
the right system engineers--if that system engineer doesn't see 
some path for advancement in the DOD, he or she is going to go 
find a place where they can make a difference and have an 
opportunity for advancement.
    Senator McCain. Which means a revolving door evolves?
    Dr. Kaminski. Yes, sir. But, that revolving door actually 
may be the commercial industry. Again, I return to the comment 
I was making about the importance of cycle time in the 
acquisition system. If you have someone who's really worth 
their salt and able to make contributions, and they get into a 
DOD acquisition system which is going to produce something in 
15 or 20 years, it won't take them long to realize that their 
knowledge base is going to erode so that they will no longer be 
valuable to commercial industry, which is producing things in 
2- or 3-year timeframes.
    So, to be able to have some revolving door from commercial 
industry back to DOD will benefit from getting these cycle 
times down.
    Senator McCain. Dr. Gansler?
    Dr. Gansler. Yes, Senator McCain. In my prepared testimony, 
I actually showed a graph that came out of the commission that 
I ran. I was shocked at how much the acquisition workforce has 
been allowed to deteriorate. In fact, as we came out of the 
Cold War, the procurement budgets dropped, the number of 
acquisition people came out with it. Then, as Dr. Kaminski 
said, there was a legislative mandate in 1996 to take another 
25 percent out. So we've ended up, now, dramatically, where we 
had about 500,000 people in 1990, we now have about 200,000 
people. But, the acquisition dollars have gone up dramatically, 
so you have this huge gap between the dollars and the people.
    But, much more important is the point that you just made 
about the officers and the senior people. In 1990, the Army had 
five general officers; in 2007, when we did the study, there 
were zero general officers with a contracting background. In 
the contract management organization, the Defense Contract 
Management Agency (DCMA), they had 25,000 people in 1990, they 
have 10,000 today. There's basically an undervaluing. They used 
to have four general officers; they have zero. So, as you 
suggest, if you're a young major, you're not getting into that 
career field, and, as a result, it's been just totally 
undervalued. Without those experienced senior people, both 
civilian and military, they don't know what questions to ask, 
and they are not going to be able to make the right judgments.
    Senator McCain. Obviously, we need to have some personnel 
policy changes, as well. Mr. Sullivan, the issue of Nunn-
McCurdy--when it was first passed, we thought it was really 
important and effective. For a while, it was. I think a breach 
of Nunn-McCurdy was a big deal. Now it seems to be a routine 
kind of event that the notification comes over, we see it, and 
ho-hum. Are we in danger of experiencing the same thing with 
this measure?
    Mr. Sullivan. We have noticed that the Nunn-McCurdy 
breaches we see take a lot longer to resolve and come out with 
a new program, and the funding is continuing on that program as 
they do that.
    As I read the proposal that the committee has now, I think 
it's really good to have a termination criteria like that. I 
think GAO thinks that's a good thing. In other words, a program 
termination would not just have someone do a review, but also 
probably look at what happens, what the triggers are for that. 
I think, right now, it states that the program cannot change 
the scope of work, cannot start new contracts, and there might 
be one other thing that the program can do. I don't know if 
legislation can basically say that that program can no longer 
obligate money, but that would, to me, be a much more direct 
way to get the point across. If a program is automatically 
terminated when it passes a threshold, and cannot obligate 
money, that might get people's attention.
    Senator McCain. I can imagine the blowback when some vital 
program is shut down because of our failure to act, but I think 
we're in agreement that there has to be a more robust oversight 
and ability to exercise that oversight as we see these costs 
spiral completely out of control.
    Mr. Sullivan. Yes, sir.
    Senator McCain. Mr. Adolph, on the issue of Nunn-McCurdy, 
do you think it's sufficient measure to impose more discipline 
on the cost overruns?
    Mr. Adolph. I believe it's sufficient. But, the issue that 
concerns me, and the remedy has already been discussed, is to 
get the technology readiness right, at the outset; and second, 
to get a realistic cost estimate. There's too much concurrency, 
there are actions that are taken in the program to kick the can 
down the road on Nunn-McCurdy, which, in the long run, in my 
opinion, adversely impact the program. The example from the F-
35 is getting rid of two of the test articles.
    Again, I believe Nunn-McCurdy is fundamentally sound. The 
problems in recent years are a combination of issues which I 
believe the proposed legislation, if it's really implemented in 
the Services--and, again, back to Dr. Kaminski and Dr. 
Gansler's point--in order to implement it, we have to 
reconstitute the acquisition workforce. That's the first step.
    Senator McCain. Also I think it would be important for us 
to have the combatant commanders more involved in the 
requirements process, as well. I think sometimes we have 
neglected that aspect of the equation.
    I thank you, Mr. Chairman. I thank the witnesses.
    Chairman Levin. Thank you, Senator McCain.
    Senator Akaka.
    Senator Akaka. Thank you very much, Mr. Chairman.
    I want to add my welcome to the witnesses before us today, 
and to point out that I'm glad that, Mr. Chairman, you are 
holding this hearing on acquisition. For me over the years, I'm 
beginning to feel that we need to change, or try to change, the 
heart and soul of DOD, and, really, the culture of DOD, and to 
get to acquisition.
    Gentlemen, as you well know, the reform of DOD acquisition 
process is an extraordinarily complex undertaking involving 
many, many moving parts. However, I believe the first step to 
tackling any problem is to prioritize. So, let me ask this 
question to each of you. In order to most effectively reform 
the DOD acquisition process, what do we need to focus on first?
    First, Dr. Gansler.
    Dr. Gansler. I believe it's the people. I think if we've 
undervalued the importance of this area, in terms of promotion, 
in terms of experience, in terms of numbers, all across the 
board, both civilians and military, that we're not going to get 
there, even if we pass all the laws in the world. We need the 
people who are going to be driving this process. That is my 
number-one priority. We have neglected it and, of course, in 
the last 8 years we've been living in a rich man's world, so 
money doesn't matter, and if people overrun or they don't 
perform, ``Let's spend more money.'' Now, that's not going to 
be the case, and we need people who are smart, experienced, and 
competent to run their programs--with flexibility, though. They 
have to make tradeoffs of cost and performance, systems 
engineering kinds of work, test and evaluation, so forth. That 
requires management judgment, and you can't just legislate 
that, and therefore, you need people with experience to be able 
to make those management judgments. That's my number one 
priority.
    Senator Akaka. Dr. Gansler, would you also say that another 
part of that would be inadequate staffing?
    Dr. Gansler. Yes.
    Senator Akaka. From what I gathered, there were positions 
that were not filled.
    Dr. Gansler. Absolutely. I was shocked to find that, in 
Iraq and Afghanistan, only 35 percent of the people that were 
in their jobs were qualified for those jobs, even with the 
minimal qualifications that Dr. Kaminski mentioned. Besides 
that, most of the positions weren't even filled, and they were 
almost all volunteer civilians in the warzone. We need to be 
able to get some senior military there, as well. So, there's a 
great lack of people, numbers, but you don't want just numbers, 
you also want qualified people. Numbers won't do it. It has to 
be qualified, experienced people. Some of those by the way, can 
come from industry. You can rotate people from industry, 
without conflict of interest, very easily. That's what we've 
had with many of the people with past experience. We do that in 
DARPA, we bring them in and out, and we've done it in other 
parts of the government. I think that people are out there with 
experience; we just have to make sure that we make it 
attractive to get them in these government acquisition 
positions.
    Senator Akaka. Mr. Sullivan.
    Mr. Sullivan. I think, at a strategic level, one of the 
biggest problems the DOD faces is that it is unable to 
prioritize what weapon system capabilities should be put into 
programs. They have 95 weapon system acquisition programs, 
major acquisitions, that are underway right now. That's up from 
about 75 or so in 2000.
    There is a tendency to have too many programs vying for the 
acquisition dollars that are available to DOD on an annual 
basis. When that happens, you get a very unhealthy competition, 
where many programs and few dollars drive the requirements-
setting process, which is stovepiped in many ways by the 
Services.
    So, the Services are all vying for a solution, they want 
requirements that are very, very tough to make, so that their 
weapon system can do the most. They, as a result of that, tend 
to put in optimistic cost estimates. The funders, of course, 
are looking at those very optimistic cost estimates that are 
very heavy on assumption and very light on facts, data, and 
actual costs. The acquisition process begins with the lack of 
the systems engineering that we've talked about here today.
    So, you have too many programs chasing too few dollars, 
with business cases that are unexecutable. The whole system is 
segregated in such a way with process owners and stakeholders 
that, in a way, it works for everyone. That's the culture. It's 
a performance-driven culture, and we all understand that and 
accept that, but there's also a lot of players involved in the 
culture that create this kind of unhealthy competition at the 
outset. I think that's the culture change that has to take 
place in this DOD, and that is almost intractable, when you 
think about it: the difficulty of changing that. I think 
legislation can go a long way to requiring people to do certain 
things, but, in the end, I agree with Dr. Gansler, it falls on 
the people to have the right principles, if you will, to change 
what this system is really supposed to create.
    Senator Akaka. Dr. Kaminski, with your research and study 
background, do you care to comment?
    Dr. Kaminski. Yes. Let me not repeat because I agree with 
everything that's been said. Let me put another dimension on 
the people. I, too, would answer: most important is people. 
But, what you have to do is recognize the dynamics that involve 
people. If we're going to attract our very best and brightest 
people to an activity, I've found that the principal incentive 
usually isn't money. With the salaries we pay military officers 
and civilians in DOD, we are still able to attract very capable 
people to key jobs.
    What attracts them? What usually attracts them is the 
ability to make a difference, to see that they can have a 
personal impact on a major program, on the security of this 
Nation.
    So, I want to come back again to my issue about time. When 
programs are taking 15 or 20 years, many of those best and 
brightest people say, ``What's the difference? If I'm not going 
to see something happen for 15 or 20 years, why don't I go 
someplace where I can make something happen sooner?'' If they 
don't have any freedom to make decisions and influence things 
because of excessive oversight processes and complexity of the 
process, they'll go find another place to work.
    I want to share with you a perspective that I got from a 
different position. Rather than a perspective of a previous 
Under Secretary of Defense, I want to share the perspective 
that I gained when I served on Active Duty in the Air Force. 
I'd say I spent two-thirds of my career there working on 
special access programs, part of that career, in the early days 
of the National Reconnaissance Office, where I was a program 
manager for one of our National Reconnaissance spacecraft 
that's up and flying today, and for several years in the  
stealth  program.  Let  me  just  pick  one  example  there,  
the  F-117 program. I was heavily involved in that program. 
When we initiated that program, our plan was, from the 
beginning of full-scale development to field the aircraft in 3 
years. We missed that, sir. We missed it by a year. We fielded 
the aircraft in 4 years. Everybody who worked on that program 
could see that they were making a difference. They could see 
that capability coming along to be fielded. There was excellent 
interation and tradeoffs in that process, between the testers 
and the program managers and the users who were going to use 
that aircraft. We made continual adjustments. So, that 
motivation of people and reduced time go together. To attract 
the right kind of people, we have to work on this time issue.
    Senator Akaka. Mr. Adolph?
    Mr. Adolph. As I mentioned in my earlier remarks, people 
are certainly the highest priority, and that's been reiterated 
here by the previous comments.
    Another area for improvement is certainly overly ambitious 
requirements. We need to continue to push technology, without a 
doubt. But, at the same time, we need to ensure that the 
technology is sufficiently matured to incorporate in a weapon 
system. That means prototyping and testing, and testing the 
prototype item in an environment in which it's going to be 
placed in a combat environment. So, again, first, people. 
Second, getting the requirements right and making sure that 
they're not overly ambitious. Again, Dr. Kaminski's study 
really addresses the latter issue quite well.
    Senator Akaka. Thank you very much, Mr. Chairman.
    Chairman Levin. Thank you, Senator Akaka.
    Senator Collins.
    Senator Collins. Thank you. Thank you, Mr. Chairman.
    Mr. Chairman, I first want to salute you and Senator McCain 
for your initiative. I think you've brought forth a bill that 
will really make a difference. I also want to point out that 
I'm very pleased to hear the witnesses today all stress the 
importance of the acquisition workforce. This is an area that 
Senator Lieberman, Senator Akaka, and many of us, have said is 
the number-one problem, over and over again. When I brought 
this up at the White House Conference last week, however, some 
of my colleagues felt that it was a lesser problem. So, I was 
very pleased to see the panel of experts before us list this as 
perhaps the greatest problem that we're facing.
    Dr. Gansler, you and I also worked together, many, many 
years ago, on how to increase competition in Federal 
contracting, and it's very good to see you here again, as well.
    I want to ask the panel about some of the specific 
provisions in the Levin-McCain bill. In particular, this bill 
would require that costs be considered right up front, when the 
requirements are set. That is a pretty dramatic change from how 
military requirements are set now, when they're done in an 
environment that does not consider costs, but, rather, an 
idealized world, where costs would not be a factor.
    Dr. Gansler, you endorsed including cost as a design 
military requirement right up front, so I'm going to skip over 
you for this question and go to the rest of the panel and ask 
all of you: Should costs be considered up front when military 
requirements are first established?
    Mr. Sullivan?
    Mr. Sullivan. I think that is a good idea. Dr. Gansler said 
that should be one of the key performance parameters on any 
major weapon systems, in their business case, and I think we 
would agree with that.
    When you're trying to set requirements, and you're doing 
the requirements analysis that's needed, I think you begin, at 
first, in an unconstrained manner, and try to get from the user 
what the user would like, in an unconstrained environment. But 
then it's critical, at some point, to start bonding that with 
the realities of the time it's going to take to get that to the 
user, the amount of money, and the technologies you have 
available to do it. So, I don't think you could do that without 
precluding the ability to think, unconstrained. But the exit 
criteria would be something that is constrained, at least in a 
cost range. Then once it exits the requirements process, there 
is a stage before it would become a set business case and begin 
as an acquisition program, where that cost range could be 
further reduced to more of a point estimate by continuing to 
make trades.
    Senator Collins. Dr. Kaminski?
    Dr. Kaminski. Senator Collins, I also agree that cost 
should be an upfront factor. I'd add another factor to go with 
it, though.
    Senator Collins. Yes?
    Dr. Kaminski. Back to my comment about time. Time and money 
go together. When we're doing requirements tradeoffs, if we're 
going to have an acquisition cycle time that operates within 
the threat cycle time, those developing requirements have to 
look at the time they want something fielded, as well. So, that 
needs to be an important consideration in this process, and we 
need to manage the time.
    The one other comment I wanted to make with respect to cost 
estimates: we talked earlier about breaches in Nunn-McCurdy. 
The surveillance system that finds the breaches, I think, is 
fine. But, one of the things we want to do is look at root 
causes. What's causing us to get into Nunn-McCurdy? One of the 
constructive uses for the independent cost estimate that was in 
the bill might be to add another consideration. We've talked 
about the importance of the upfront work in systems engineering 
and development planning. One of the things that would be 
useful for us all to ensure is that there is adequate funding 
at the beginning of a program, between Milestone A and B, for 
us to apply our systems engineering and development planning 
capabilities to get a good handle on what those cost estimates 
are, and do a thorough job involving stakeholders, the 
requirements part of the equation, and the program manager in 
that process, along with the CAIG.
    I, like Dr. Gansler, used the CAIG very heavily. In fact, I 
advocated that we fund programs to the CAIG estimate, but we 
hold the program manager and the contractor to the estimates 
they developed, so we had some finance reserve between the two 
estimates.
    Senator Collins. Thank you.
    Mr. Adolph?
    Mr. Adolph. Senator Collins, certainly we need to consider 
the cost issue upfront. Another very important driver early on 
is technology readiness. In program after program after 
program, we're into full-scale development and discover that 
some of the critical technologies simply aren't mature enough. 
So the program is delayed, and that drives the schedule, and 
these slippages occur.
    A key to getting the costs right is to ensure that the 
technology is really sufficiently mature.
    Senator Collins. Thank you.
    Dr. Gansler, an issue that hasn't been discussed today is 
the impact of a lack of stability and predictability on a 
program's cost growth. That obviously can drive up the unit 
cost. When the military starts out with one plan for acquiring 
a weapon system, and then switches direction or reduces the 
number of units, doesn't that also drive up the cost?
    The reason I bring this up is, we talk a lot about the 
errors made by contractors, we talk about weapon systems that 
get gold-plated because additional requirements are added, but 
there's also an important issue, as far as the stability of 
funding and the lack of predictability driving up the unit 
cost. Could you address that issue?
    Dr. Gansler. Yes. But, let me just briefly comment on your 
first question, though, because frequently the military doesn't 
think that cost is a military requirement. What they neglect is 
the fact that numbers are a military requirement. If you're 
resource-constrained, the total dollars that you have divided 
by the unit cost gives you the numbers. Numbers really matter 
in military operations, whether it's by Lancaster's law of N-
squared or by numbers. Either way, numbers really matter, and 
therefore, cost really matters. That's why it's so important to 
have the unit cost as part of the requirements.
    Now you get to your changes, and unless we estimate the 
cost of those changes and their impact on the ultimate cost of 
the equipment, we let things get out of hand. So, if cost is a 
requirement, then every time a change comes in, and as Senator 
McCain said earlier, it was 75 a week on the LCS, you have to 
price those each out to make sure it is not having a big cost 
impact on the program.
    Then, when programs, in general, become relatively stable, 
you don't get this ripple effect through the budget, which is 
the point that you're really making, Senator. If I want to pay 
for program A, I take it out of program B, not recognizing that 
program B now is in really bad shape because they don't have 
the stability of the funding. That stability of funding is a 
critical issue.
    Senator Collins. Thank you.
    Chairman Levin. Thank you, Senator Collins.
    Senator Burris.
    Senator Burris. Thank you, Mr. Chairman.
    This is an interesting experience for me, to hear the 
distinguished testimony, Mr. Chairman. I would rather listen 
than to talk. I yield my time.
    Chairman Levin. Thank you.
    Senator Burr is next on this side; he's not there.
    Senator Chambliss.
    Senator Chambliss. Thank you, Mr. Chairman.
    Gentlemen, it's good to see all of you, and thanks for your 
service over the years, and thanks for your being here on what 
I think is, if not the most important issue at the Pentagon 
today, certainly it ranks up pretty high. With these budget 
times that we're in, trying to figure out a way to buy the 
weapon systems that we need within the timeframe we need is 
virtually impossible. That's why your testimony is so critical.
    I want to go back to what Senator Collins was talking about 
there, and what all of you have alluded to in some part, and 
that's this issue of instability, whether it's requirements, 
whether it's personnel, whether it's funding, or whether, as 
you say, Mr. Adolph, it's the technology aspect of it, where 
we, too many times, tend to come up with a great idea, and by 
the time we get into the production phase, we've wasted, not 
only time, we've wasted money, but oftentimes, there's a new 
idea that has been developed in the meantime.
    I'm amazed, Dr. Kaminski, that you say, on the F-117, that 
we were in production in 4 years. Gosh, if we had done that 
with the F-22, we wouldn't be having the arguments we're having 
today, and we'd have a great airplane, and we'd be worrying 
about other issues.
    But, it doesn't make any difference whether you're talking 
about an aircraft carrier, whether you're talking about FCS or 
a tactical fighter, we come up with this idea, and we get into 
the R&D phase, and there is, all of a sudden, a great idea, but 
instability in all four of those areas runs that cost up 
tremendously. Then, you throw in what Senator Collins alluded 
to, about the number of these units that we're going to buy, 
and all of a sudden it explodes again and it becomes such a 
negative at the Pentagon, rather than the positive that it 
started out to be.
    My question for a comment from each of you is: How do we 
get back to this? How do we get back to the point to where we 
come up with this idea? If it's a tactical fighter, it's 
supposed to be air-to-air, or supposed to be air-to-ground, 
whatever it may be, how do we develop that and get it into 
production right away, without technology intervening and all 
of a sudden having to add this and add that? How do we get our 
arms around that issue?
    Jack, let's start with you.
    Dr. Gansler. I think one of the main opportunities we have 
is to accept the concept of spiral development, that for the 
block 1 system, we have a fixed set of requirements, we have a 
fixed price that we're trying to get a fixed schedule, as Dr. 
Kaminski said, and we go ahead with block 1 under the 
assumption that if we can then demonstrate new technology, if 
we find that the user needs something different, if we find 
that even the logistician has a problem with maintenance of 
that equipment or the reliability as we deploy it, that becomes 
block 2, block 3, block 4. But, block 1 has to use proven 
technology and get out there quickly, all with a set of 
constraints. So, it's a stable program, as you point out.
    The most successful acquisition that, in fact, Congressman 
Aspin used to always highlight, was the Navy's Polaris, 
Poseidon, Trident. When I was in industry, I always knew how 
much money I was going to get next year for that program. I 
could hire, I could plan my workforce, and so forth. That 
stability is very important for efficiency. I think, if we go 
to a spiral development model, whereby block 1 is stable, and 
block 2 is being developed while block 1 is being deployed, you 
have the concept of stability built in, and evolutionary 
systems are still stable. That's the way the real world, the 
commercial world, works. You constantly are upgrading the 
software, the hardware of computers, but you're constantly 
getting higher performance at lower cost. That has to the be 
objective of each of the blocks as we're going along in spiral 
development.
    Senator Chambliss. Mike?
    Mr. Sullivan. Goldwater-Nichols legislation, from years 
ago, tried to bring jointness into the DOD. I think it 
succeeded on the operations side. We now have the combatant 
command's matrix, the military forces that fight wars jointly 
very well. The same thing did not occur on the acquisition 
side. I think if you look at what's going on now, there's a 
kind of a stovepipe system for how you get programs started, 
and that creates this kind of competition for big requirements 
and cost estimates that are heavy on assumption.
    I would agree with everything that Dr. Gansler just said. 
If you can work on that and get a more joint requirement 
setting and funding system, and try to get the proper balance 
of weapon systems started, try to get rid of the stovepipes, 
you'd have an environment that could do what Dr. Gansler, I 
think, is describing, a little more easily.
    Another program I would throw out is the F-16 program. Back 
in the 1970s, it was a block program. It was the capabilities 
that the Air Force wanted for the F-16. They knew that they 
needed an aircraft faster than they could develop the 
technologies to get those capabilities, so they had blocks. If 
you look at the F-16s, and, for that matter, the F-15s, 
performance over the last 30 years, it's pretty impressive. 
They basically upgraded those aircraft pretty efficiently as 
they went because they started without that big revolutionary 
leap, that one-step, big-bang kind of a thing.
    So, we've done this before, and I think it's possible to 
get back to it, but this is where the culture comes in. I think 
there's some culture change that needs to take place.
    Dr. Gansler has other things that I read in his report that 
would help this significantly. Open systems, for example, on 
these weapon systems, when, if you can make interfaces on the 
weapon systems uniform, you can keep proprietary data that 
subcontractors have that supply subsystems to them, and all 
they have to do is have the proper interface. Then you can open 
up competition.
    One more point I would make is the difference between 
technology development and product development probably needs 
to be better understood. Technology is the kind of thing you 
should think of when you think of scientists and lab coats and 
trial and error, and it's done in a smaller-dollar environment, 
where you can test, use trial and error, and make mistakes. You 
have to keep that off of these acquisition programs. I think 
someone up here, I don't know if it was Dr. Kaminski or Mr. 
Adolph, said that when you have a technology that's not mature, 
and it's on an acquisition program that's driving towards 
production, you have an entire workforce--an entire supply 
chain, for that matter--that's waiting for that technology to 
mature. The burn rate is pretty big on that workforce that you 
have.
    Dr. Kaminski. I'd emphasize that point. While everybody's 
waiting, we're paying, and so what you want to do is decouple 
those two.
    In terms of approaches to do this, I agree completely with 
the spiral development approach. One problem I see is in the 
application of spiral. In the few spirals I've seen that we 
actually implemented, we had everything but the kitchen sink in 
the first spiral, rather than stretching this out over a period 
of time, like we did in Apollo. So, we need development 
planning. You have to have a plan with stability to do this. 
The program manager has to have the discipline and the 
experience to reject things that aren't in the plan or that 
aren't mature enough to be harvested.
    I have found that one of the characteristics of a good 
program manager is a big lower right-hand drawer in their desk, 
and what went into that drawer were all the ideas for 
improvement. The drawer was kept closed until such time as we 
fielded the first system; that is the time open the drawer and 
look to see what development plans we need to deal with 
shortfalls or upgrades for that system.
    This time-certain development is important because, as time 
goes by, the technology gets old, and new ideas are introduced 
that end up being disruptive to the process. So, time is a key 
factor here.
    This stability issue is really key. If I look back through 
my whole career, there is only one program I ever worked on 
where we actually produced the system at the rate we planned. 
That was the F-117. We built one a month. Every other program I 
can think of, by the time we were done with development, we 
couldn't afford the build rate that we planned.
    Senator Chambliss. Anything to add, Mr. Adolph?
    Mr. Adolph. Just one thing, since I certainly agree with 
the other panel members, but with my background in tests, I'll 
add that issue. In the case of the F-15 and F-16, which were 
mentioned earlier, I worked on both programs, and I was out in 
the field, test business, working for the Air Force.
    Chairman Levin. Please talk a little bit louder, if you 
would.
    Mr. Adolph. Yes. In the case of the F-15 and F-16, I was 
working for the Air Force in the field at the time, and the F-
15 test program was structured well; we had 19 test articles 
and sufficient articles in the case of the F-16 as well. In the 
latter F-16 case, the propulsion system had been matured. In 
order to move a program along, you need an adequate number of 
test articles, and you need to be able to do as much testing as 
you can. Avionics is a good example, where considerable testing 
can be accomplished on a test bed platform, rather than the 
developmental platform.
    So, that's, I think, one key component of keeping a program 
moving. When a program stagnates, when you only have one or two 
test articles, and you have a `standing army' of test personnel 
waiting, and the fixed cost of those people is almost as great 
as the incremental cost of doing additional testing, 
particularly when you recover the article. That's not the case 
in a missile program, where the test article is destroyed on 
each test.
    Chairman Levin. Thank you, Senator Chambliss.
    Senator McCaskill.
    Senator McCaskill. Thank you, Mr. Chairman.
    Let me start with Mr. Sullivan, and there have been several 
references that sometimes we have excessive oversight with no 
value added. That's a frustrating comment to me, in that I have 
watched the Defense Contracting Audit Agency basically get 
taken to task by their peers for failure to even follow basic 
auditing standards. I've read, I can't tell how many of your 
reports that have, in fact, identified weapons acquisition as 
high risk, since 1990. It is not as though the oversight's not 
occurring, it's just that it's being ignored. It's not adding 
value because nobody's paying any attention to it.
    Let me ask you, in that regard, about JCIDS, Planning, 
Programming, Budgeting, and Execution (PPBE), and Future Years 
Defense Program (FYDP). Now, first of all, I think we ought to 
pass a law that they quit talking in initials because you all 
know what those things are I just said, but I guarantee you, 
nobody that I work for in Missouri has any idea what JCIDS is, 
PPBE is, or FYDP. What they are is: one, is all the Services 
getting together and basically giving each other what they 
want; two, is a 2-year calendar-driven process, where they're 
supposed to be figuring how they're going to spend the money; 
and three, it is the Secretary of Defense trying to low-ball 
what it's going to cost, long-term, in order to make sure that 
the other two go along with it. Is that an accurate summary of 
what those three are?
    Mr. Sullivan. The--that---- [Laughter.]
    I would say, number one, that the JCIDS is a requirement-
setting process; it's where all of the Services tend to get 
together and figure out what it is that they require.
    Senator McCaskill. But, in your testimony, Mr. Sullivan, 
you pointed out, they never say no.
    Mr. Sullivan. That's true.
    Senator McCaskill. Have they ever said no that you're aware 
of since you've been looking at this?
    Mr. Sullivan. We issued a report for this committee, I 
think, about a year ago, where we looked at that, and it was--I 
think the JCIDS process then, if I'm not mistaken--this may not 
be exact, but I can get it for you--about 90 percent of the 
proposals that went in were granted.
    [Excerpt of GAO-08-1060 follows:]
      
    
    
      
    
    
      
    Senator McCaskill. Explain to me, so I understand, what is 
the makeup of the three groups of people. These three are 
really the stovepipes that cause a lot of the problem because 
you have, ``Here's what we want, here's how we pay for it, and 
here's how we figure out how much it's going to cost in the 
long run.''
    Mr. Sullivan. Yes, ma'am.
    Senator McCaskill. What I don't understand is: what is 
keeping them from making that one thing, so they all have to do 
all of that at once?
    Mr. Sullivan. The requirement-setting process, the JCIDS, 
is run by the JROC, and the JROC is made up of the military 
service chiefs, more or less. It's run by the Vice Chief of 
Staff. So, it, in a sense, is a matrix organization, but it 
receives most of the proposals for needs to be validated as 
weapon systems from the three Services, as stovepipes.
    Now, the JCIDS was established to have something called 
``functional capability boards,'' which were supposed to be a 
matrixed organization based on looking at things like 
battlespace awareness or force protection or force projection, 
looking at it functionally instead of across the Services. What 
we found when we did that study was that the DOD has not 
staffed those functional capability boards properly, it hasn't 
resourced them properly, so they don't really do a lot of joint 
decisionmaking to send proposals forward to the JROC. So, 
mostly what they are receiving is proposals for capabilities 
that are coming from the Air Force, the Navy, and the Army, and 
they compete with each other.
    Senator McCaskill. The sense is, at JCIDS, that if you say 
no to what they want, then they're going to say no to what you 
want. Isn't that part of the problem, in terms of the way this 
is actually supposed to be oversight with no value added.
    Mr. Sullivan. I would say that the oversight is far from 
perfect. There's not a lot of value to it.
    Senator McCaskill. The idea of the Joint Staff is that 
they're supposed to be picking winners and losers. What is your 
sense of how effective the organization of the Joint Chiefs 
office has been, in terms of weapon acquisition and picking 
winners and losers?
    Mr. Sullivan. In terms of that organization that does that 
picking of the winners and losers, the JROC, there are a lot of 
redundancies right now in the weapon system portfolio because 
they can't make proper decisions, it seems to us.
    I'll give you an example. Right now, there are some 
unmanned aerial systems that are in development that we 
believe, and I think the Department actually believes, should 
have been joint programs, but the Services had unique-enough 
requirements and missions to be separate. Right now, there is 
the Predator, which has been very valuable in Iraq and 
Afghanistan and is an Air Force program. Air Force is currently 
making a bigger, more powerful Predator, called the Reaper. 
Another one of the Services, I believe it's the Army, has 
started what they call the Sky Warrior. Both drones are done by 
the same contractor and both with very similar requirements, 
but the Services have determined that they're different enough 
that they each have to have their own acquisition program. 
That's the sort of thing that the JROC is contending with; it's 
a very parochial kind of an attitude.
    Senator McCaskill. So, is this not fixable?
    Mr. Sullivan. This is the cultural aspects of this, I 
think, that we've all been addressing, where you can write 
legislation, you can have policies.
    Senator McCaskill. It doesn't do any good.
    Mr. Sullivan. But, unless you change the culture, and I 
guess that is the number one question: How do you change that 
culture, that has been in existence for so long, to try to turn 
it a little bit, to do things a little more efficiently? It's 
very much a culture issue.
    Senator McCaskill. I also think, Mr. Chairman, one of the 
things we have to do, in some instances, is take out a mirror 
because I think there are times that, when the military has 
tried, either by the way they've done the budget or by actually 
being so bold as to say we need to wind down a program, that 
Congress decides, because of our parochial interests, that it's 
important that we go to bat to augment the budget to take care 
of the weapon system that we think is important in our part of 
the world. So we contribute to this problem, and I think we 
shouldn't complete this hearing without at least acknowledging 
that sometimes Congress has their hand in this stew.
    Mr. Sullivan. Yes, ma'am. I would say, sometimes we should 
think of this as a system that is in equilibrium because there 
are very many stakeholders in this system that are getting 
specific things, even the GAO. It's a pretty good employment 
program for us. We report on cost schedule and performance 
problems, and we have been doing it 30 years. So, culturally 
speaking, if you examine it as a system that, maybe, is in 
equilibrium, in a sense--it's not necessarily broken for the 
people that are involved in it.
    Senator McCaskill. Thank you.
    Thank you, Mr. Chairman.
    Chairman Levin. Thank you very much.
    Just one quick comment on that. We do, in this bill, 
attempt to make the JROC system cover some of the issues that 
Senator McCaskill talked about by requiring it to make these 
early tradeoffs, by looking at cost and at schedule, by the 
way, as well as the requirements and the performance 
requirements.
    Mr. Sullivan. Yes, sir.
    Chairman Levin. There is an effort in this bill, Senator 
McCaskill. I just want to give you assurance that at least this 
bill attempts to do what we can, legislatively, to put those 
elements into the JROC process, which would, hopefully, cut 
down the parochialism by forcing consideration of cost and 
schedule, not just requirements. So, I'd just get that on the 
record.
    Thank you, Senator McCaskill.
    Senator Martinez is next.
    Senator Martinez. Mr. Chairman, thank you very much.
    Dr. Gansler, I wonder if I could ask of you to define for 
me a little more broadly what you mention as a ``holistic 
approach'' to defense needs.
    Dr. Gansler. Yes. In fact, when I even looked at the 
situation in Iraq and Afghanistan, I was surprised to find that 
the State Department, United Sates Agency for International 
Development, and DOD were all there, but not integrated, in 
terms of the buying, the contracting, and planning purposes. 
But, much more at a higher level than that, it seems to me that 
the world of the 21st century is going to require us to combine 
hard and soft power in the kinds of operations involving 
expeditionary operations or insurgent operations that we get 
involved with around the world. So, we're going to need a much 
closer tie between, in that case, State and Defense. But, I 
would go further and say, Homeland Security and Defense. It 
shocked me, in fact, that Saddam Hussein didn't try and pull 
some terrorist actions at the same time as we attacked him. I 
would expect that's going to happen in the future. So, it is an 
integrated, holistic perspective that is the meaning of 
``national security'' in the 21st century, that will involve 
Homeland Security, that will involve State, will involve the 
intelligence community, will involve the DOD.
    I think that's basically what National Security Advisor 
General Jim Jones is now trying to do, some of the 
restructuring that's taking place at the national security 
level within the President's office. I think this combination 
of soft and hard power is going to be required, very clearly. 
That's what I was thinking of, in terms of the ``holistic 
perspective.'' But, also the types of threats. Think about it, 
the energy case, the pathogen spreading worldwide, the economic 
crisis that we're in, these are all national security issues 
for the 21st century that we have to start to incorporate into 
our thinking of national security. That's what I had in mind.
    Senator Martinez. Thank you. I think it is an intriguing 
future that we all are stepping into. I think, by the way, 
Saddam would have if he could have.
    But, the LCS program is one that I'm very much fond of, and 
one that I think is essential to the national security 
interests of our country. I wonder how you believe the Navy got 
so far off track on that particular program? Was it too many 
requirements being put on the platform by the Navy? Was it the 
length it's taken to develop it? We now have two hulls being 
developed. So, to the extent that any of you could speak to the 
LCS program and what you see, going forward, it would be 
helpful to me.
    Dr. Gansler. I just actually published, I think last week, 
a DSB report which looks specifically at the LCS and the 
presidential helicopter, a couple of programs of that sort, 
where the initial concept was: get something relatively fast, 
take something ``off-the-shelf'' that could be used, and 
addressing Dr. Kaminski's point about rapid acquisition--then 
maybe block 2, 3, and 4 would add some of the additional 
things.
    What happened on the LCS: the first thing they said was, 
``Has to go through Sea State 8.'' Well, that's like going 
through a hurricane and it wasn't initially designed for that. 
Then the next thing they said was, ``Well, it has to have a new 
Navy sprinkler system.'' The sprinkler that was in the system 
in those two ships that you talked about, it wasn't going to be 
adequate, for some reason or other. So, each of these special 
requirements ended up basically changing the original block 1 
system and introduced the instability, cost growth, and 
schedule impact that we've talked about in all these other 
programs.
    Yes, we badly needed the LCS, but is it going to have to be 
a battleship? Does it have to do everything that a battleship 
does? How it's going to be used by the Navy was resisted, in 
terms of the nontraditional solutions. So, this is the culture 
change that we've been talking about, as well. If you kept the 
cost and the schedule, and got a block 1 system out there much 
faster--I like the idea that we did it competitively. I think 
that was a very important step. So, I would encourage that to 
be done in these earlier demonstration systems.
    Senator Martinez. Dr. Kaminski?
    Dr. Kaminski. Yes, sir. I think the concept initially was a 
good concept. What was missing was the upfront systems 
engineering and development planning that I spoke about. I can 
remember back to the time I was serving in the acquisition 
executive's job in DOD; I was a big proponent of commercial 
practice, and also in buying commercial systems where that made 
sense. But, if somebody came to me and said we were going to 
buy a commercial ship for this mission, my first question to 
him was, ``What are we going to change in the mission to be 
able to do it with a commercial ship?'' If the answer is 
``nothing,'' then I have to ask a second question, ``Wait a 
minute, this commercial ship doesn't have the kind of military 
requirements you would have for fire-safe cables, for a fire 
sprinkler system, or a whole variety of things. What are you 
going to do about those?'' I don't think we started asking 
those questions about the LCS program until we were well into 
the program, so we missed this upfront set of tradeoffs. Those 
are tradeoffs that you have to make. They can be made sensibly 
if you approach them, understand them, look at the costs, the 
performance, and the schedule to make those tradeoffs. I don't 
believe we made those tradeoffs upfront. That, for me, does not 
necessarily damn the LCS program. There may still be value 
derived from looking at these tradeoffs and now making sensible 
decisions to go forward. I agree with Dr. Gansler about the 
advantage of having a competitive environment to be able to do 
that.
    Senator Martinez. Mr. Adolph?
    Mr. Adolph. I'm not really familiar with the LCS. I've had 
no involvement with the program.
    Senator Martinez. Okay, and you, Mr. Sullivan? I don't know 
whether you've had any experience.
    Mr. Sullivan. I don't have any specific experience with 
that, but we do have a team that looks at all of our Navy 
ships, and I'm sure we've had a report on that recently, which 
I could submit for the record.
    Senator Martinez. That would be great. I'd appreciate that.
    Mr. Sullivan. Okay.
    [Excerpt of GAO-05-255 follows:]
      
    
    
      
    
    
      
    Senator Martinez. I'm sure going to try to look up your 
article, as well, Dr. Gansler.
    You haven't written on DGD-1000 and the DGD-51 to date, 
have you?
    Dr. Gansler. Not lately.
    Senator Martinez. Okay. Do any of you share an opinion on 
the needs of the Navy as it relates to these two programs?
    Mr. Sullivan. I would just say that we're going to come out 
with our annual assessment of programs, and I think both the 
DGD-1000 and LCS are going to be programs that are covered in 
that, so you will get our take on them, probably in the next 
couple of weeks. I'll follow up, as well.
    Senator Martinez. Very good. Thank you.
    [Excerpts of GAO-09-326SP follows:]
      
    
    
      
    
    
      
    
    
      
    
    
      
    
    
      
    
    
      
    
    
      
    
    
      
    Senator Martinez. My time's up, thank you very much.
    Chairman Levin. Thank you, Senator Martinez.
    Senator Begich.
    Senator Begich. Thank you. Thank you very much, Mr. 
Chairman.
    It has been very interesting, listening to all the 
testimony. I will try to ask my questions very quickly, and I 
may not ask every single one of you to answer because I want to 
get some specifics here.
    Before I talk about the personnel issue, I'm a former 
mayor; I've just become a Senator. So, I believe I'm a mayor 
who just happens to be a Senator, so I like some of the 
comments you've made on personnel issues that I'll talk about 
in a second.
    I'm not familiar with the CAIG. Is that the CAIG? I looked 
through the bill as quickly as I could, and I see an annual 
report to Congress. Do you think it would be helpful for us to 
require more? I believe it's a combination of oversight; it's 
not just internal oversight, but this body needs to do more. I 
think this bill attempts to do that. But, do you think there 
should be a report--and maybe there is and I'm just not 
familiar with it--that comes to this committee on a regular 
basis, maybe quarterly or twice a year, that shows what the 
CAIG said it would cost before a system is started?
    Mr. Gansler, you made the comment that--I may get these 
words wrong--but it's almost like they ignore it. It comes out, 
it says, maybe, what it might really cost, but then they kind 
of push it down. I'm familiar with this, as a mayor. We call 
them HMS, Inc., studies, where the contractor wants to build 
it, as well the person inside the system who wants to build it, 
always seems to have a different price. When it's all done, 
it's pretty close to the one that HMS, Inc., did. Is that 
something we should have in this legislation, so we can see, 
before these systems start kicking off, here is what the real, 
or what another group said it could cost, so we at least have 
some understanding? I don't know who can answer that.
    Dr. Gansler. The idea of the independent cost analysis, 
right upfront, when we're doing the early systems engineering, 
is to be able to see the impact of the various requirements, 
and to be able to trade requirements, schedule, and cost as 
part of that early design requirements setting. Cost is 
actually an engineering challenge, just as schedule and 
technical performance are. So, if you, upfront, try to say, 
``What's the cost impact of this?'' and the independent cost 
analysis group will say, ``Historically here's what it's 
been,'' and then people will come back and say, ``But, this 
time it's going to be different.''
    Senator Begich. Everything's different the next time.
    Dr. Gansler. Right, and so it's important to keep that in 
perspective, that the new technology comes along, and so forth. 
But, it's absolutely essential to get that independent 
estimate. Now, do you want to have Congress legislate what the 
price should be? I don't think so.
    Senator Begich. No, I'm not asking that. But here we are, 
complaining about all these cost overruns. I agree with all 
your comments. You can write all the legislation you want, but 
you do not change the culture and remove people who are not 
doing the job they should be doing, and putting people in there 
who should be doing the job, you don't change anything, we'll 
be back here in a couple of years. So, I'm not saying, 
legislate the price, but we become more aware, so we are 
putting the pressure where it should be. Let's be honest about 
the pricing, so when we do the budgeting authorization and 
appropriation, we don't go from 75 to 95, with two-thirds 
probably in the planning and design stage, and we always get 
the answer, ``We're this far, we have to do a little bit more, 
a little bit more, a little bit more.'' So what I'm asking is, 
should we have a more regular reporting period? Because once 
you're a year into a project, even though some are longer, 
they're already obligating more money. By the time this system 
here moves, you could be 2 years into having any commentary on 
it.
    That's my question. Should it be more regular reporting?
    Dr. Gansler. I think the thing you're missing is the fact 
that, in the development of a program, the program changes 
weekly. There's always technical issues that come up, there's 
always schedule problems, there's personnel problems, and so 
forth. You don't want Congress to be micromanaging the 
programs. You do want to make sure that the process is a good 
process. That's where I think the independent cost analysis is 
a very important thing and that your emphasis on it is the 
right thing to do. But, I don't think you want to get down to 
the point where you have a weekly report from the DOD.
    Senator Begich. I didn't ask that. I'm trying to get to the 
point. Is annual enough?
    Dr. Gansler. I think that--if you've convinced the DOD that 
you care about this, that you are going to be monitoring it, 
and that's what you're trying to do with the threshold numbers 
and the controls on that--I think that you're giving the 
message that the DOD needs to care about cost.
    Senator Begich. Okay. Let me make one comment. I forget who 
said it, about, ``stop the obligation of money at a certain 
point.'' I know, as a mayor, that's what you do. You turn the 
dollars off, and suddenly you get a response, and you get 
people paying more attention. So, I might be a little different 
than earlier comments that were made. I'm a little more direct 
on that. But, the personnel issue, to me, this is the 
challenge. If you don't change the culture, nothing changes. 
We'll be back here, and the numbers will grow, as they have 
over the years. It's not about adding more people, and I think 
Congress made a huge error by reducing down the amount of 
people. That was a huge mistake. We basically took the people 
who manage our programs out of the equation.
    So, besides putting more people into the system to make 
sure we have more folks out there, do you think--and I'm not 
sure I want to ask this question because I'm not sure you'll 
want to answer it--but, do you think, within the system that 
currently exists today, we have to change the deck? When I say 
``change the deck,'' change personnel, people, and not just add 
more people and move people around so we satisfy their issues. 
This is a very hard, direct question. I've had to do this, as 
mayor. You might have half a dozen, a dozen, or 100 people; it 
doesn't really matter--they're in the wrong place. Anyone dare 
to want to answer that question?
    Mr. Sullivan. I would just say, just real quickly, that a 
lot of this is organizational. I think the people that work in 
the DOD now are great people, and really, really capable 
people, and good public servants and everything else.
    Senator Begich. Right.
    Mr. Sullivan. But, when we talk about culture, I guess, 
it's more the way things are organized. For example, I think, 
in our written statement, we have that the Under Secretary of 
Defense for Acquisition, Technology, and Logistics probably 
should have more ability to make the final decisions on things, 
acquisition, than he or she has now. One of the things that 
probably gets in the way of that is the fact that that position 
had an average turnover of 20 months since it started in 1986. 
I think that's part of the problem; there's an accountability 
issue, people change over too much, there's not a lot of direct 
communication. The three processes, I think, that this 
legislation does address, that you're trying to get the three 
big processes that we've talked about to communicate with each 
other more and to share in decisionmaking; right now, that's 
not there. So, I don't know if the people are good people. The 
structures and the way they're organized and the way they come 
and go is the big issue.
    Dr. Gansler. If I might comment that if we implemented what 
Goldwater-Nichols says to do, relative to promotion potential 
for the acquisition community, that you'd get a big step 
forward there. Instead of putting someone into a four-star 
position who has no acquisition background, but happens to be 
called an acquisition job, that's where we lose out. Each of 
the promotion-potential reviews and so forth need to really 
show that we value the acquisition workforce, civilians and 
military, and that's a critical point, I think, in order to 
keep people coming in. Dr. Kaminski said they're not doing it 
for the money, they're doing it so they can have an impact, and 
they need to have promotion potential.
    Senator Begich. I know my time's out, but you're about to 
jump out of your seat, Mr. Adolph.
    Chairman Levin. Can you do it real quickly?
    Mr. Adolph. Very quickly. There are three issues: numbers, 
training, and people. Particularly in the Air Force, they need 
to plus up because they drew down their acquisition workforce 
to a greater extent.
    I think the training that Defense Acquisition University 
provides is on the mark, for the most part. They were a part of 
our study. They ground in what we found into their training 
courses. I believe people need to be moved around a bit more, 
particularly in the civilian workforce, so that they get a 
variety of experiences rather than 1 or 2 years' experience 10 
times.
    Senator Begich. Thank you very much, Mr. Chairman.
    Chairman Levin. Thank you, Senator Begich.
    Senator Lieberman.
    Senator Lieberman. Thanks, Mr. Chairman. Thanks to the 
panel. It's probably not been an exciting hearing in content, 
but it's actually very important, an educational hearing, and 
there's a lot on the line in it, so I thank the four of you for 
the accumulated experience and wisdom you've brought to the 
table. I thank Senator Levin and Senator McCain for their 
legislation, S. 454, which certainly would take us forward in 
significant ways. I hope it passes.
    Earlier, Senator Collins referred to the fiscal 
responsibility summit that President Obama convened last week 
at the White House, and then a group of us on this committee 
happened to be in a breakout session on procurement reform. As 
Senator Collins indicated, most, though not all, but most, of 
us focused on the acquisition workforce, and the size of it. I 
want to talk to you about that in the time I have, hopefully, 
and at least one other subject we talked about.
    Dr. Gansler, you had a chart here in your testimony which 
has two lines; one is procurement by DOD, in dollars, from 1990 
to 2006, about as close as we can get to today, and then the 
other is the acquisition workforce. Obviously, the procurement 
dollars go up dramatically as the acquisition workforce goes 
down. But, beneath that, there are some numbers that are quite 
stunning, I think, and I know when I was here, Mr. Sullivan 
referred to them. The acquisition workforce, in 1990, was 
actually 500,000 people, and today it's dropped, but it's still 
200,000 people, which is an enormous number of people in 
acquisition. I note in your testimony that you focus in on the 
DCMA and say that it had 25,000 people in 1990, down to 10,000 
today. Then the other four general officers, and down to zero 
today. Give me some sense of the 200,000. Because my first 
reaction to it is, ``Wow, 200,000 people, isn't it enough to 
handle acquisitions by the DOD, even though acquisitions are so 
large?''
    Dr. Gansler. A large share of those are in the military 
depots that you have insisted do 50 percent of the maintenance 
work. A depot that has 20,000 people, that adds up pretty fast. 
To get to 200,000, you only need 10 of those.
    Senator Lieberman. So, acquisition wouldn't be what most of 
those do?
    Dr. Gansler. That's part of the acquisition workforce 
because logistics is part of the acquisition.
    Senator Lieberman. Got it. But in the conventional meaning, 
I think that's important, so I wanted to bring out.
    Dr. Gansler. Very few people are actually doing contract 
work, program management work, or things like that. As Mr. 
Adolph pointed out, the test and evaluation community is down 
significantly, but they're part of that community. So it's the 
total encompassing the research labs that the government runs 
all the way through the maintenance and logistics support.
    Senator Lieberman. Is there any way, now or maybe 
afterward, to submit for the record, you could give us a sense 
of what we would normally, in conversation, consider to be the 
acquisition workforce--how many people in the DOD are actually 
involved in acquisition, contracting, et cetera?
    Dr. Gansler. It's a small percentage of the people that are 
actually involved in that.
    In other words, you have the comptroller people, you have 
the personnel people, you have the policy people, and you have 
the acquisition workforce.
    Senator Lieberman. Right. I'm curious as to really what the 
size of the real acquisition workforce is, leaving out the 
depots and the rest.
    Mr. Sullivan. I don't have that answer right now. I know 
we're doing work on that.
    Senator Lieberman. Good. Get it for the record, please.
    Mr. Sullivan. Yes, sir.
    [The information referred to follows:]

    In our recent review of the Department of Defense's (DOD) 
acquisition workforce management (GAO-09-342), we found that, according 
to DOD data, the number of civilian and military personnel in the 
Department's acquisition workforce totaled about 126,000 at the end of 
fiscal year 2007 compared to about 129,000 personnel in 2001, a decline 
of about 2.5 percent. During this same time period, the number of 
contracting actions valued at over $100,000 increased by 62 percent and 
dollars obligated on contracts increased by 116 percent, according to 
DOD. Moreover, DOD has reported that the number of major defense 
acquisition programs has increased from 70 to 95. To augment its 
declining in-house acquisition workforce, DOD has relied more heavily 
on contractor personnel. However, DOD does not collect or track 
information on contractor personnel, despite their being a key segment 
of the total acquisition workforce.

    Senator Lieberman. There were two other things that there 
seemed to be an interesting consensus on, and they're quite 
different, about, you might say, principles for acquisition; a 
little different than anything we've talked about today. 
They're not unfamiliar. One was that our original position on 
acquisition should be to favor fixed-price contracts over cost-
plus, and to favor competitive bidding, as opposed to 
negotiated contracts. I want to ask, to the extent that we have 
time--let's just focus on the fixed-price. Our sense, as we 
discussed at our breakout session with probably about 25 
people, was, generally speaking, private sector favors 
competition. So, why are we favoring cost-plus? Does the 
taxpayer really benefit from that?
    Mr. Adolph, you always get asked last because we're going 
left to right, so let's start from the right and ask you about 
that.
    Mr. Adolph. I think the other three panel members have more 
expertise in this issue. But, when you're in the basic research 
arena, it's very difficult to do on a fixed-price basis. It 
should be accomplished using cost-plus contracts.
    Once you get beyond research, then the next challenge in 
the development is system integration. That's not an 
insignificant task with the very complex systems we're 
developing today. But, once you get beyond that point then I 
think you reach a point where you can really consider going to 
fixed-price for downstream procurement.
    Senator Lieberman. Right. That's interesting.
    Dr. Kaminski, I think, just to clarify--if I could really 
make it too simple--what if we had a law that said, ``Defense 
contracts ought to be done on a fixed-price basis unless the 
Secretary certifies that there's a good reason not to''?
    Dr. Kaminski. I think that would end up requiring a lot of 
certification, Senator Lieberman, for the following reason.
    Senator Lieberman. Right.
    Dr. Kaminski. I believe fixed-price contracts are 
completely appropriate when we know precisely what it is we're 
going to buy. If there's uncertainty in what we're going to 
buy, and we know we're going to change, and we don't know yet 
quite how we're going to change, I think we end up on the wrong 
end of the bargain negotiating a fixed-price contract and then 
having to go back and renegotiate that effort for every change 
that occurs, especially when the contractor has already priced 
in some contingency in the fixed price.
    So, there's a time, for example, in the program, where 
perhaps we are working through this in development, and then we 
settle in on what we want to buy, and we're ready to enter a 
well-defined production program. That would be a fine time to 
do a fixed-price contract.
    So, I think you have to pass that criteria, knowing what it 
is you want to buy, and have it be predictable.
    Senator Lieberman. Okay. My time is running out, but, Mr. 
Sullivan, Dr. Gansler, if you'd give me a quick answer to a big 
question and follow up with writing.
    Mr. Sullivan. For major developments, I think it would be 
very difficult to go to a fixed-price contract for that because 
of the unknowns that are involved. But, I would say that if you 
work on requirements and try to do some of the things that 
we've been talking about here today, in terms of staying in 
what is doable, and having shorter cycle times to get these 
things done, you could have cost-plus development contracts 
that don't get so out of control. It really goes back to how 
much knowledge you have when you set out.
    Senator Lieberman. Okay.
    Dr. Gansler?
    Dr. Gansler. I think that it's a question of what's the 
meaning of a ``fixed-price contract''? As Senator McCain said, 
you get 75 changes every week. The contract continues to change 
hourly, in effect. I think it's very clear, when you have a 
stabilized and lower-risk program, that a fixed-price makes a 
lot of sense, it does give an incentive for the contractors. On 
the other hand, the cost-plus, I would say, we haven't been 
using the incentives that are available with the cost-plus-type 
contracts as well as we should, and I think, clearly, for 
research-and-development-type activities, cost-plus is an 
appropriate way to do it, but the ``plus'' part is an incentive 
rather than a fixed fee, I think. I would use the incentive 
more.
    Senator Lieberman. Those are very helpful answers. You 
encourage me to think that we ought to take a look, not at 
fixed prices on across-the-board answer, but to apply it by 
some selective means, and that, in doing so, we might benefit 
the taxpayers.
    Thanks very much.
    Chairman Levin. Thank you, Senator Lieberman.
    Senator Webb.
    Senator Webb. Thank you, Mr. Chairman. I'd like to add my 
appreciation for your taking on this issue. As has been so 
clear today, we're burning up a lot of money, and we're not 
getting a lot of product right now, particularly in the 
shipbuilding programs and aircraft programs.
    Dr. Gansler, when you're talking about people, which 
everyone seems to agree is the major issue, I was thinking 
about all the different years and different positions I've had 
on different sides of the table, here, working on this issue. 
It's so clear that what we need is disciplined management, not 
only on the people side, but in the system itself. ``People'' 
include people in government, on this side of government, it 
includes people in business. We have challenges because there 
are not a lot of people in the military who use the business 
concepts, quite frankly, and they're asked to manage these 
programs. There are not a lot of businesspeople who are used to 
how product comes through a governmental system.
    I believe it was Mr. Sullivan who mentioned the creation of 
the Under Secretary of Defense for Acquisition in 1986. I was 
actually on Caspar Weinberger's staff when that position was 
created. We had a very talented individual who came into the 
position. He was bewildered with all the different steps that 
were required to get a system through the process. It's 
something you just don't see in the normal business world. 
There are lots of checks and balances. Some of them are 
appropriate, and some of them are less than appropriate.
    Dr. Gansler, I found your analogy with the LCS brought back 
a lot of different memories. They say that a camel is a horse 
created by a committee. I can go all the way back to the M-16, 
when they first developed the M-16, and they developed it 
directly toward a jungle environment, and then the different 
requirements were put on it to be able to be used in the desert 
and other different places. They put a different round in it 
that carboned up the chamber so people were dying in Vietnam 
because the weapon system requirements had changed as it 
evolved.
    I was in DOD when we were trying to do the Bradley fighting 
vehicle, and there were different requirements put on it here 
in Congress, so that it was very similar to what you were 
talking about with the LCS. They were saying it should perform 
different functions from the original design, and then there 
were all these press reports about the Bradley fighting vehicle 
falling over when it was going through a water obstacle because 
it got top heavy.
    Or the FG-7s, the USS Stark-class ships, which were 
designed to build to cost. So, we have fixed cost that we were 
going to build a ship toward, and then you go inside one of 
these FG-7s, you could plink the bulkheads on a FG-7, they were 
so thin. So when an Exocet missile hit the USS Stark in the 
Persian Gulf when I was Secretary of the Navy, it went all the 
way through the ship because they had had to make adjustments 
based on the cost rather than on some other areas.
    So it's a very complicated question. I think the key, when 
I look back, is if you can find the right leadership at the top 
on a program, negotiate and agree on general requirements--
there are always going to be fixes--we'll get a program 
through. Probably the best example of that is when they put Al 
Gray, who later became Commandant of the Marine Corps, and he 
had the Development Center, they put him on the Light Armored 
Vehicle, and he got that program through in about a year. He 
just pushed it through, made all the negotiations, was very 
firm with people over here in Congress, as to what the 
requirements were for the Marine Corps, and it was a very 
successful program.
    I have one question I would like to put in front of you 
because I'm very concerned about it, and that is the state of 
all these programs in the United States Navy right now. You 
talk about the Polaris as having been probably the best 
analytical prototype of how to build a weapon system. One of 
the things about our submarine programs is that we built the 
frame, and then we added the technology onto the frame, similar 
to, say, the C-130, rather than continually building a new 
frame with all the costs and the time that goes into that. I've 
just been really struck over the last couple of years at how 
difficult the Navy procurements appear to be, and I'm trying to 
get my arms around why. I would be happy to hear from any of 
you who would like to begin answering that question.
    Dr. Kaminski. I'll make one comment. I think a key thing 
that impacts the programs are stability. So, if you see a 
program that we've had in production for some period of time, 
they have very good learning curves on those ships when we have 
a stable production program. Our issues sometimes are with 
first-ship cost. But, if you look at the cost of subsequent 
ships, what's happening there is very competitive with 
commercial-kind of production experience. Where we're producing 
ships regularly--one of the ship families in which we're doing 
that is the DDG-51. We have two yards. We have some competitive 
arrangements. Not quite head-to-head competitions, but there 
are some incentives in those programs. But, it is a well-
planned, stable program that we've been producing. I think that 
approach would benefit us. It's the areas where we've had 
instability where we've more problems.
    Dr. Gansler. Yes, I guess I would approach it by thinking 
about, ``What is it the Navy really needs for the 21st 
century?'' and what types of ships they're going to want. There 
is a resistance to change. Take the Arsenal ship, for example, 
which was primarily to support the Marines and the Army 
onshore, it was resisted significantly in terms of it being a 
low-cost ship for few people. The highest costs in the Navy are 
for people and fuel if you look at the life-cycle costs of a 
ship. So trying to drive down the number of people on the ship 
and improve the fuel utilization are things we need to stress. 
Those are not the traditional things that are emphasized in the 
Navy construction of ships. I think it's a different look that 
we need to think about. The LCS has the same thing. Is that 
something the Navy really wanted or really resisted? So it's 
more the institutional inertia that has to be changed, I think, 
in terms of what the future Navy for the 21st century is going 
to need.
    Senator Webb. Do any of you see this as leadership failures 
in the Navy? [No response.]
    In terms of defining these objectives?
    Dr. Gansler. There have been some real successes. The F-
18E/F on the Navy program was extremely well-managed, but that 
was because they had some really top people doing it, they had 
a clear objective, it was a incremental version of a prior 
demonstrated program, and it was well done.
    Another big success is the patrol frigate. It kept up 
competition throughout its life, and it had the steepest 
learning curves of any ship in the Navy. So, there are some 
success stories, but I think lessons learned haven't been 
widely applied.
    Senator Webb. Thank you, Mr. Chairman. My time's up.
    Chairman Levin. Thank you, Senator Webb.
    Any other questions of this panel before we excuse them? 
[No response.]
    Thank you all for your time. Some of you came some distance 
to get here, at some inconvenience. At least one of you had to 
give up a family commitment, and we won't identify who that was 
because the family is better off not knowing, maybe. 
[Laughter.]
    But we're very grateful to all of you for your testimony. 
It's very, very helpful.
    I will also submit for the record the text of the Weapon 
Systems Acquisition Reform Act of 2009; a statement from Ken 
Krieg, former Under Secretary of Defense for Acquisition, 
Technology, and Logistics; a statement from Moshe Schwartz from 
the Congressional Research Service; and the DOD Inspector 
General Acquisition Workforce Count Report.
    [Additional material included for the record.]
      
    
    
      
    
    
      
    
    
      
    
    
      
    
    
      
    
    
      
    
    
      
    
    
      
      
    
    
      
    
    
      
    
    
      
    
    
      
    
    
      
    
    
      
    
    
      
    
    
      
    
    
      
    
    
      
    
    
      
    
    
      
    
    
      
    
    
      
    
    
      
    
    
      
    
    
      
    
    
      
    
    
      
    
    
      
    
    
      
    
    
      
    
    
      
    
    
      
    
    
      
    
    
      
    
    
      
    
    
      
    
    
      
    
    
      
    
    
      
    
    
      
    
    
      
    
    
      
    
    
      
    
    
      
    
    
      
    
    
      
    
    
      
    
    
      
    
    
      
    
    
      
    
    
      
    
    
      
    
    
      
    
    
      
    
    
      
    
    
      
    
    
      
    
    
      
    
    
      
    
    
      
    
    
      
    
    
      
    
    
      
    
    
      
    
    
      
    
    
      
    
    
      
    
    
      
    
    
      
    
    
      
    
    
      
    
    
      
    
    
      
    
    
      
    
    
      
    
    
      
    
    
      
    
    
      
    
    
      
    
    
      
    
    
      
    
    
      
    
    
      
    
    
      
    
    
      
    
    
      
    
    
      
    
    
      
    Chairman Levin. We'll stand adjourned.
    [Questions for the record with answers supplied follow:]
           Questions Submitted by Senator Joseph I. Lieberman

                  STATUS OF CURRENT ACQUISITION SYSTEM

    1. Senator Lieberman. Mr. Sullivan and Dr. Gansler, I have several 
concerns with the acquisition system as it currently operates. Most 
importantly, it seems that the requirements validation process under 
the Joint Capabilities Integration and Development System (JCIDS) is 
not synchronized with the Planning, Programming, Budgeting, and 
Execution (PPBE) funding process. Can any effort at reform succeed 
without addressing this fundamental flaw in acquisition management?
    Mr. Sullivan. We do not believe that there can be success until 
there is better synchronization between the Department of Defense's 
(DOD) requirements, funding, and acquisition processes. The Government 
Accountability Office (GAO) has previously reported that the lack of 
integration between the JCIDS and PPBE processes is a key factor that 
contributes to the Department's inability to achieve a balanced 
portfolio of weapon system programs that meets the needs of the joint 
warfighter and matches needs with available resources. The JCIDS 
process has not been effective in prioritizing needs from a joint, 
Department-wide perspective, and it largely approves capability needs 
without accounting for the resources or technologies that will be 
needed to acquire the desired capabilities. Resource allocation 
decisions in DOD take place through the PPBE process, which is separate 
from JCIDS. PPBE largely allocates resources on a Service-by-Service 
basis and does not effectively link resources to capabilities. In 
addition, the process allows too many programs to start development 
with unreliable cost estimates and without a commitment to fully fund 
them. Until DOD establishes a more integrated approach to weapon system 
acquisition, it will continue to struggle to effectively prioritize 
warfighting needs, make informed trade-offs, and achieve a balanced mix 
of weapon systems that are affordable, feasible, and provide the best 
value to the warfighter.
    Dr. Gansler. I believe ``cost'' should be an essential part of the 
``requirements process''--since, in a resource-constrained environment, 
it directly determines ``numbers'' (of systems procured) and that 
determines force effectiveness (even more than individual weapon's 
effectiveness).

    2. Senator Lieberman. Mr. Sullivan and Dr. Gansler, what are your 
recommendations for effectively balancing the requirement validation 
process with the calendar-driven funding process?
    Mr. Sullivan. To improve DOD's ability to deliver weapon systems at 
the right time and right cost, GAO has recommended that the Department 
implement an enterprise-wide portfolio management approach to making 
weapon system investments that integrates the determination of 
warfighting needs with available resources and cuts across the Services 
by functional or capability areas. Such an approach would focus on 
assessing weapon system investments collectively from an enterprise 
level, rather than as independent and unrelated initiatives. By 
following a disciplined, integrated process--during which the relative 
pros and cons of competing weapon system proposals are assessed based 
on strategic objectives, warfighter needs, and available resources, and 
where tough decisions about which investments to pursue and not to 
pursue are made--DOD could minimize duplication between Service 
components and more effectively support each new development program it 
commits to. Furthermore, to ensure effective weapon investment 
decisions are made, GAO has recommended that a single point of 
accountability be established at the Department level with the 
authority, responsibility, and tools to implement portfolio management.
    Dr. Gansler. My answer to question #1 (above) will greatly aid in 
this; but, in general, the ``requirements process'' must look well 
beyond the near-term budget (funding) process. Nonetheless, the 
frequent budget changes (particularly, by Congress) introduce great 
instability and inefficiency into the implementation of the weapons 
acquisition process.

                           JOINT REQUIREMENTS

    3. Senator Lieberman. Mr. Sullivan and Dr. Kaminski, I am concerned 
that DOD has historically done a poor job of procuring capabilities 
that address joint requirements, and more specifically, that the 
Services have little incentive to spend what they see as their limited 
budgets on capabilities that will principally benefit sister Services. 
One particular instance of this is the Multi-Platform Radar Technology 
Insertion Program, and the Air Force's apparent reluctance to procure a 
system that will principally serve the Army's battlefield management 
requirements. How can the acquisition process better prioritize and 
procure such crucial joint capabilities?
    Mr. Sullivan. GAO has recommended that DOD should determine and 
allocate appropriate resources for more effective joint capabilities 
development planning. The functional capabilities boards, which were 
established to manage the JCIDS process and facilitate the 
prioritization of needs, have not been staffed or resourced to 
effectively carry out these duties. Similarly, the combatant commands 
(COCOMs) also lack the analytic capacity and resources to become more 
fully engaged in JCIDS. GAO recently reviewed JCIDS documentation 
related to new warfighting capabilities and found that most--almost 70 
percent--were sponsored by the military Services with little 
involvement from the joint community, including the COCOMs, which are 
responsible for carrying out military missions. The Services drive the 
determination of capability needs, in part because they retain most of 
DOD's analytical capacity and resources for requirements development.
    Within the Department's PPBE process, the individual military 
Services are responsible for budgeting and allocating resources under 
authority that is commonly understood to be based on title 10 of the 
United States Code--to organize, train, and equip military forces. In 
this structure, the budget is based more on individual, service-focused 
needs than on joint warfighting needs. In the past, GAO has reported 
that this situation has contributed to interoperability problems among 
weapon systems and unnecessary duplication of capabilities in some 
areas. The Office of the Secretary of Defense (OSD) reviews and makes 
adjustments to the Services' budgets, but this takes place toward the 
end of the PPBE cycle when it can be difficult and disruptive to make 
changes, such as moving funds to higher-priority programs or to support 
joint needs. DOD has recently taken steps to establish capability 
portfolio management in selected areas in an attempt to overcome the 
service-centric nature of the resource allocation process. However, 
because the portfolios are largely advisory and lack authority and 
control over the allocation of resources, there may be limits to 
achieving joint capabilities.
    Dr. Kaminski. The priority must be set in the budget process, with 
key oversight from OSD and the Joint Chief of Staff. The procurement 
can be improved by first developing (a several year process) then 
assigning key program management personnel with the requisite training 
and domain experience. Alignment of the responsibility, authority, and 
accountability of the program manager will require that a degree of 
trust be established between the program manager and those responsible 
for our oversight mechanisms. We must be prepared to delegate authority 
to the program manager, and provide him or her with some flexibility to 
manage--to adjust levels and allocation of funding, to adjust the 
allocation of performance parameters, to adjust schedule, and to tailor 
the acquisition approach to be responsive to the need. Clearly, there 
must be bounds established beyond which the program manager must seek 
approval from oversight authorities. But I believe these bounds are too 
narrow and inflexible today. We also need to provide sufficient upfront 
funding, and maintain funding stability throughout program execution.

    4. Senator Lieberman. Mr. Sullivan and Dr. Kaminski, are either the 
JCIDS or PPBE as currently constructed able to overcome parochial 
service interests in this regard?
    Mr. Sullivan. No, I do not believe that the JCIDS and PPBE 
processes are constructed effectively to address the service-centric 
manner in which weapon systems are acquired in DOD. Past studies 
chartered by DOD and other organizations have reported similar 
conclusions and made a number of recommendations to improve the 
acquisition of joint warfighting capabilities.
    Dr. Kaminski. Both help, but I don't believe they are sufficient.

          DEPARTMENT OF DEFENSE SCIENCE AND TECHNOLOGY PROGRAM

    5. Senator Lieberman. Dr. Gansler and Dr. Kaminski, as we try to 
reduce risk in acquisition programs, what role does (or should) DOD 
science and technology (S&T) program play?
    Dr. Gansler. S&T are critical to both ``staying ahead'' (which is 
the basis of our national security strategy of ``technological 
superiority'') as well as to risk reduction--since we need to prove out 
the S&T first, before introducing it into weapons acquisition.
    Dr. Kaminski. With good S&T programs supporting development 
planning, we can assign managers to develop prototypes, critical 
systems or components needed to better understand cost/performance 
trades and reduce risk. With the appropriate technology base in place, 
it is reasonable to expect that many of these developments can be 
completed in 2-4 years, so one manager will be in place from start to 
delivery of those critical subsystems during the development planning 
phase. This should allow the full system development to proceed on a 
shorter schedule as a result of the risk reduction.

    6. Senator Lieberman. Dr. Gansler and Dr. Kaminski, do you think 
that the Department's S&T programs, and in particular the DOD 
laboratories, are sufficiently resourced and staffed to provide support 
to the acquisition programs and address the problems you are discussing 
today?
    Dr. Gansler. The DOD's S&T program must be adequately funded to 
``stay ahead''; and, traditionally, whenever there is a shortage of DOD 
money it is one of the areas the Services tend to cut (since it is 
long-term). Thus, in the likelihood of a coming budget crunch, it must 
be protected--not just in the DOD labs, but particularly in the 
universities and in industry.
    Dr. Kaminski. No. We are short in both quantity and quality of 
staffing, and are not using the latest tools to support early modeling, 
simulation, and testing.

    7. Senator Lieberman. Dr. Gansler and Dr. Kaminski, what 
recommendations do you have for this committee to improve the quality 
of the DOD S&T program and the quality of the DOD laboratories?
    Dr. Gansler. Adequate funding, and allowance for foreign graduate 
students, scholars, and researchers (per Presidential Decision 
Directive-189) to work on DOD S&T, along with ``peer review'' of all 
proposals, are critical to the United States staying ahead. The SBIR 
and STTR programs are also a great benefit, and must be maintained.
    Dr. Kaminski. The first step is to ensure that we not only restore, 
but also enhance early and continuing systems engineering coupled with 
the effective development planning needed to drive our S&T program. 
This will require commitment of more significant investment dollars 
earlier in our acquisition programs, and a commitment to build a cadre 
of systems engineers and development planners with the education, 
training, and domain experience needed to be effective. Attracting 
``best and brightest'' to this work--and keeping them--will require a 
personnel system that will identify and track these important human 
resources and establish a career path to allow those who are successful 
to advance to senior program management and leadership positions. Their 
domain experience will be enhanced by managing the building of critical 
technology demonstrators and subsystems during the development planning 
program, reducing risk, and building skills and experience at the same 
time.
                                 ______
                                 
                Question Submitted by Senator Jack Reed

                           S. 454 LEGISLATION

    8. Senator Reed. Dr. Gansler, in the proposed S. 454 legislation, 
section 203 focuses on maximizing competition throughout the life cycle 
of major acquisition programs. One proposed provision is that modular, 
open architectures are used to enable competition for upgrades. Please 
comment on how the implementation of this recommendation would 
positively impact major weapons programs that have been sole sourced 
for decades, and how the Services can rapidly implement these changes 
to achieve maximum benefits. In addition, please comment on the 
feasibility of applying this provision to current programs, not just 
new programs.
    Dr. Gansler. Let me begin by noting that the objective should not 
be ``maximizing competition''; rather, it should be ``maximizing the 
effectiveness of competition.'' Competition for its own sake, can be 
detrimental. For example, there must always be the ``option'' of 
competition, but if a firm is constantly improving performance and 
lowering costs, they should be rewarded by not recompeting the program; 
but there must always be the option of introducing competition if costs 
are rising (so the ``threat'' of competition is what creates the 
incentive for lowering costs; and the follow-on is the reward).
    Open architectures are clearly one way to allow competition to be 
introduced at the subsystem level (which often is 70-80 percent of the 
total weapons cost); so it should be greatly encouraged--both on 
upgrades of current systems and on new ones.
                                 ______
                                 
             Questions Submitted by Senator Daniel K. Akaka

                 WEAPON SYSTEMS ACQUISITION REFORM ACT

    9. Senator Akaka. Mr. Sullivan, section 105 of the Weapon Systems 
Acquisition Reform Act recommends the Joint Requirement Oversight 
Council (JROC) seeks input from the combatant commanders in identifying 
joint military requirements. Currently, the combatant commanders 
produce an annual Integrated Priority List (IPL) that attempts to 
satisfy this same goal. In your prepared testimony, you stated some of 
the combatant commanders do not feel their needs are sufficiently 
addressed. What must be done within DOD to fully implement the intent 
of this legislation to ensure combatant commander's influence 
throughout the acquisition system to decrease the Service-centric 
environment we've seen for years?
    Mr. Sullivan. GAO and several other studies have recommended that 
DOD increase joint analytic resources for a less stovepiped 
understanding of warfighting needs. While each COCOM submits an IPL to 
the Chairman of the Joint Chiefs of Staff each year, which defines the 
COCOM's highest-priority needs, it does not appear that the COCOM needs 
are well integrated with JCIDS. According to officials from several 
COCOMs that we have met with, needs identified through IPLs are not 
typically developed into JCIDS capability proposals. Some COCOM 
officials pointed out to us that because of their limited resources, 
they must pick and choose capability needs and persuade one or more of 
the military Services to sponsor a proposal that addresses the COCOM's 
need. The Department has been working to give more time and priority to 
the commanders' IPLs, but more attention is needed.

    10. Senator Akaka. Mr. Sullivan, in your prepared testimony, you 
mentioned the establishment of the DOD capability portfolio management 
process. DOD identified nine joint capability area portfolios to 
mitigate the tendency to develop Service-centric, stovepiped solutions. 
However, the capability portfolio managers lack the necessary authority 
and control required to be successful. How can DOD reform their 
capability portfolio management process to mitigate the tendency to 
develop Service-centric, stovepiped solutions?
    Mr. Sullivan. DOD's capability portfolios are intended to advise 
the Deputy Secretary of Defense on how to optimize investments within 
their respective capability areas. They have no independent 
decisionmaking authority and must operate within the Department's 
existing framework for acquiring weapon systems. That is, capability 
needs are still determined through JCIDS and resources are allocated 
through PPBE. Although it is too soon to assess the impact of 
capability portfolio management, according to some DOD officials, the 
portfolios provided key input and recommendations in the 2010 budget 
process. However, without portfolios in which managers have authority 
and control over resources, DOD is at risk of continuing to acquire 
weapon systems in a stovepiped manner. Ultimately, DOD needs to develop 
an integrated portfolio approach that brings the determination of 
requirements together with the allocation of resources. To be 
effective, such an approach must have committed leadership, empowered 
portfolio managers, and accountability at all levels of the Department.

    11. Senator Akaka. Dr. Gansler, Dr. Kaminski, and Mr. Adolph, 
changes made to a system after acceptance of review cost many times 
more than the same changes made during the requirements gathering 
phase. How can we improve the requirements gathering phases of the 
acquisition of major weapons systems to front-load these changes to 
achieve significant cost savings?
    Dr. Gansler. Please refer to my answer to question #1. If cost is a 
requirement, then changes that come in must not only have showed how 
they will improve performance but how they will live within the cost 
requirement.
    Clearly, the earlier this is done, the better; but technology 
changes, threat changes, et cetera, do come along, and as long as they 
are explicitly within the overall cost requirement they can be allowed 
in.
    Dr. Kaminski. Requirements adjustments during execution of 
acquisition programs are not necessarily bad if they are conducted 
within the framework of value propositions. Good program management 
requires continuing adjustment of the requirements flow-down and 
allocation to major subsystems based upon expected performance, 
schedule, and cost. Even top level requirements require continuing 
assessment and adjustment based upon similar value propositions. As we 
discover more about costs and performance during execution, good 
program managers should be interacting with requirements developers to 
adjust as needed to provide the best value for the user. This requires 
some flexibility in cost allocation, and it also places demands upon 
requirements developers to become involved in tradeoff decisions. To be 
effective, requirements developers as well as acquisition managers must 
have experience with systems engineering tools and techniques, and both 
must have sufficient domain experience to engage productively in cost-
performance tradeoffs.
    Mr. Adolph. A disciplined analysis of alternatives process should 
be employed to support the JCIDS and acquisition processes, from 
capability needs identification to include system design and 
development as well as life-cycle improvement. Emphasis must be placed 
on improving the processes for relating mission effectiveness and cost 
to system design, system performance, and suitability. Effective 
feedback processes are of special importance for spiral developments to 
identify enhancements which will improve performance and suitability. 
Improving the quality and speed of this feedback is increasingly 
important in responding to rapid changes in threat environments.

    12. Senator Akaka. Dr. Gansler, Dr. Kaminski, and Mr. Adolph, 
significant cost overruns are incurred from the instability of mission 
and design requirements during testing and even, in some cases, 
production. To what extent do you think it would be advantageous to 
more rigorously enforce a freeze on fundamental requirements at the 
system requirements review gate?
    Dr. Gansler. The greatest causes of cost growth, after program 
initiation, are budget changes (often caused by Congress) and 
``requirements'' changes--both of which can be controlled by 
discipline. (See question #11.)
    Dr. Kaminski. It is always beneficial to have a firm understanding 
of very top level requirements at the start of a program. But 
adjustments of subordinate requirements during execution of acquisition 
programs are not necessarily bad if they are conducted within the 
framework of value propositions. Even top level requirements need 
continuing assessment based upon evolving value propositions. As we 
discover more about costs and performance during execution, good 
program managers should be interacting with requirements developers to 
adjust as needed to provide best value for the user. This requires some 
flexibility in cost allocation, and it also places demands upon 
requirements developers to become involved in tradeoff decisions. To be 
effective, requirements developers as well as acquisition managers must 
have experience with systems engineering tools and techniques, and both 
must have sufficient domain experience to engage productively in cost-
performance tradeoffs.
    Mr. Adolph. Rigorous enforcement of key requirements thresholds 
should be the norm when entering system development and demonstration. 
There needs to be more rigor and discipline in the entire requirements 
definition and acquisition process. Issues that need to be addressed in 
relation to requirements setting include technology readiness (see 
response to question 13 below), the translation of requirements into 
design criteria, with attention to testability at the subsystem and 
system levels, as well as defining thresholds for key performance 
parameters.

    13. Senator Akaka. Dr. Gansler, Dr. Kaminski, and Mr. Adolph, cost 
overruns are also caused by the inherent unpredictability of the 
development of new technologies in the course of fulfilling system 
contracts. Would you recommend restricting the proposed designs for 
weapons systems to proven technologies while adequately funding 
contractual vehicles for pure research?
    Dr. Gansler. I would restrict the first ``block'' of the system to 
proven technology; then--using ``spiral development''--evolve the 
system's future blocks as future technology is proven out; and as long 
as the future technology either improves performance (at the same cost) 
or lowers the cost. This future technology should be funded in parallel 
with the system, until it is proven.
    Dr. Kaminski. This will be helped by better development planning 
and alignment of incentives. With good development planning, we can 
assign managers to develop prototypes, critical systems or components 
needed to better understand cost/performance trades and reduce risk. It 
is reasonable to expect that many of these developments can be 
completed in 2-4 years, so we do not need to restrict ourselves to 
proven technology at the beginning of development planning. As these 
critical subsystems are delivered and tested, the risk reduction and 
domain experience gained in both government and industry will allow us 
to prove the critical technologies involved and reduce the time 
required to develop, integrate, and test the full system. We can rely 
on the experience gained during development planning to apply informed 
judgment to adjust requirements to improve value, reduce time, and 
better estimate and manage costs.
    Mr. Adolph. Weapon system design should be based on proven 
technologies. History has shown repeatedly that doing otherwise 
introduces unacceptable risk in to weapons system development which 
necessarily translates into cost growth and schedule delays. Assessing 
whether a technology is sufficiently mature is currently accomplished 
through the Technology Readiness Levels (TRL) construct which should 
employ government test and evaluation (T&E) as the key component in 
moving a technology from TRL 4 to TRL 6. Recent consideration has been 
given to strengthening early involvement of the test community by 
reinvigorating the government developmental test process and by 
involvement of OT&E in readiness assessments. These are sound 
initiatives and should be focused on providing rigor to the TRL 
assessments in terms of the test methodology as well as the test 
environment. Implementation of this more disciplined approach will 
require a reversal to the trend of cutting back on government test 
personnel as well as strengthening their role in supporting the 
acquisition process. In that regard, the Director of Developmental Test 
and Evaluation should make an assessment of technological maturity and 
integration risk, based on test results and report the findings to the 
Undersecretary of Defense for Acquisition, Technology and Logistics.
    There may be rare exceptions when a requirement is so pressing that 
risks must be taken to design a system which incorporates an unproven 
technology. In those instances, a high-priority, adequately funded, and 
closely monitored effort must be undertaken to mature the technology. 
This must include an early, rigorous assessment of the difficulties 
associated with the development effort as well as testing to fully 
assess the requisite maturity.
    Additionally, the TRL process has been focused on the technology of 
system components. Problems with manufacturing these components in 
production quantities have also led to significant cost growth for 
weapons systems. Previous legislation established the Manufacturing 
Technology Program to identify and develop initiatives to improve 
manufacturing quality, productivity and technology. Consideration 
should be given to expanding the TRL process to evaluate the maturity 
of the production methods by which parts are manufactured.
                                 ______
                                 
               Questions Submitted by Senator Bill Nelson

                        MODELING AND SIMULATION

    14. Senator Bill Nelson. Dr. Gansler, Dr. Kaminski, and Mr. Adolph, 
computer modeling and simulation (M&S) tools can be very useful in 
discovering problems in major systems before we spend a lot of money or 
time on hardware development. DOD has some programs in M&S--which in my 
opinion are underfunded and disjointed. What is your assessment of M&S 
activities within DOD and their ability to address some of the 
acquisition problems we are discussing here today?
    Dr. Gansler. M&S can be a great aid in system design and 
performance evaluation, and should be an important element in the 
development of all systems.
    Dr. Kaminski. M&S can have a very significant positive impact on 
acquisition performance if there is sufficient emphasis on continuing 
validation of M&S tools and integration with T&E. Giving early and 
serious attention T&E will require strengthening our T&E organizations 
and personnel. T&E is often an afterthought, and contracts are often 
written without any mention of how we will test the product. We spend 
large amounts of money when a large development team waits for test 
results. The alternative is to spend less money and time by early 
development of M&S tools, continuously validating the models with 
testing at critical points, and considering the investment in M&S and 
test resources as a critical part of our systems engineering and 
development planning efforts.
    Mr. Adolph. With some exceptions, the capabilities and limitations 
of simulations, simulators and physics-based models of individual 
subsystems are well understood. These tools are generally used where 
appropriate. However the use of modeling to support requirements 
definition, analysis of alternatives, and test design needs to be 
pursued more vigorously. The expectations regarding the use of M&S to 
support elements of the acquisition process, including system 
integration, T&E, and manufacturing have increased along with advances 
in computer technology. Distributed T&E (and training) using multiple 
ranges, laboratories, and facilities have benefited from the use of 
live, virtual, and constructive simulations. However, the availability 
of adequate models and modeling tools involving human dynamics for use 
in T&E, mission planning, systems engineering, and risk assessment has 
not kept pace.
    M&S to support acquisition/test programs has been the norm for 
decades, and received additional impetus in the 1990s with advances in 
information processing technology. In many cases, the incremental 
improvements as related to acquisition were oversold in the late 1980s 
and 1990s. Additionally, the requirements for verification, validation, 
and accreditation have proven to be more costly and timeconsuming than 
anticipated. Many of the M&S activities initiated in the last 15 years 
were disjointed, with few successful attempts to coordinate 
requirements, build and share common models, correlate performance, and 
ensure that the models were adequate for the intended applications. 
Numerous recent M&S developments have been terminated after spending 
millions because of the failure to produce a useful product.
    Building models to meet every need is not feasible. An overall M&S 
vision is required to identify where efforts should be focused, and to 
ensure that new efforts are within that vision. If high priority M&S 
efforts within that vision could be identified, it would be possible to 
establish requirements, understand what is currently available, and 
then determine shortfalls in M&S capability. One approach is 
exemplified by the Testing in a Joint Environment Roadmap, which was 
recently developed by DOT&E and approved by the Deputy Secretary of 
Defense. While focused on T&E usage, this capability would meet a 
significant percentage of contractor development and testing M&S 
requirements. The T&E roadmap identifies a requirement for a 
distributed live, virtual, constructive T&E capability; largely 
supported by existing models of friendly and threat systems. Much of 
this capability could come from existing models of acquisition systems 
developed by program contractors, in conjunction with threat models 
available from the intelligence agencies. However, there has been 
virtually no effort to identify requirements for high priority joint 
missions, to determine what friendly and threat force representations 
are required, assess the availability of existing models that meet 
those requirements, and then use that information to define M&S 
shortfalls. Additionally, there are insufficient mechanisms and 
processes to feed back data from operational tests and field exercises 
to further validate and refine models. Archiving mechanisms that can 
store and locate data for future applications are needed.
    There are also supporting M&S requirements (engineering and 
physics-based models) which are required in the validation of higher-
level engagement models. Higher level models, such as envisioned in the 
T&E roadmap, could be used to identify areas with the greatest risk and 
uncertainty to better define areas where engineering and physics-based 
models are required. The initial effort should be to review the past 
studies on M&S in the DOD and determine those requirements that could 
contribute the most to improving M&S capability. While most studies 
have identical or similar recommendations, the majority of those 
recommendations have not been implemented.

    15. Senator Akaka. Dr. Gansler, Dr. Kaminski, and Mr. Adolph, how 
would you recommend addressing shortfalls in DOD activities in M&S?
    Dr. Gansler. To address shortfalls, I would ask an independent 
group (such as the Defense Science Board) for an assessment and a set 
of recommendations (including appropriate funding).
    Dr. Kaminski. We need upfront investment in M&S tools to understand 
the value propositions and the tradeoffs among cost, schedule, and 
performance of the mission. We spend large amounts of money when a 
large development team waits for test results. The alternative is to 
spend less money and time by early development of M&S tools, 
continuously validating the models with testing at critical points, and 
considering the investment in M&S and test resources as a critical part 
of our systems engineering and development planning efforts.
    Mr. Adolph. Focused M&S business plans must be developed. Separate 
plans are needed for requirements definition, acquisition/test, 
training and force structure evaluations. As discussed above, the 
latter plans should be used to identify critical joint missions and 
provide a basis for prioritizing M&S requirements; to understand 
current capabilities and identify major shortfalls.
    An issue that needs to be addressed is the availability of existing 
models to meet current requirements. Many models were developed by 
system contractors to meet specific needs. In some cases they are 
proprietary and there are rarely provisions for models to be maintained 
current so they represent a system in the field with normal 
improvements and modifications, or changes to the treats. Often 
contractors are not funded to make their models generally available. 
The government must address issues relating to long-term configuration 
control, upgrades, ownership, and funding throughout the system 
lifecycle, to include training. In addition, addressing the 
requirements for a single joint mission doesn't capture requirements 
across several potential joint missions and scenarios.
    However, it would provide a starting point for future requirements 
and identify the most sensitive issues in implementation of the overall 
concept. Addressing one major joint mission scenario would identify a 
large percentage of requirements for other joint missions.
    Finally, an important consideration is the physical location of 
major simulation laboratories and related facilities. There are often 
valid reasons to locate a simulation capability at a contractor 
development facility to support initial development; however it places 
limitations on the future utility and accessibility of the simulation, 
as well as the need to replicate parts of the simulation capability at 
a government test facility or logistics center. Prior to making a 
decision regarding the location of simulation facilities for each major 
program, an assessment should be made as to the most cost-effective 
locations(s), taking into account follow-on and future programs. This 
decisionmaking process should include the appropriate Service MRTFB 
representatives, as well as the TRMC. (See also my response to question 
#17.)

                     TEST AND EVALUATION ENTERPRISE

    16. Senator Akaka. Mr. Adolph, a major step in developing weapons 
systems is ensuring that they are appropriately tested during their 
development and before their operational use. I believe that DOD is 
letting its testing enterprise decline precipitously in terms of 
testing equipment, infrastructure, and workforce. What role do you feel 
that the DOD testing enterprise plays in ensuring acquisition programs 
deliver their systems on schedule and on budget?
    Mr. Adolph. The role of the DOD testing enterprise is addressed in 
detail in the recent Defense Science Board Task Force Report on 
Developmental T&E. My written testimony covers the major findings and 
recommendations from that study, and while it emphasizes the impacts of 
reliability, availability, and maintainability on life-cycle costs, the 
implementation of some acquisition reform initiatives diminished the 
role of government T&E in the acquisition process.
    A brief summary response follows. The traditional role of the 
government during the test planning phase included the identification 
of test resource requirements and test facilities, the development of a 
test strategy and detailed T&E plans. When programs moved from the 
planning phase to the execution phase, the government traditionally 
performed operational assessments, participated in test conduct and 
analysis, and performed an evaluation of the test results for the 
government Program Office. With a few exceptions, this is no longer the 
case. In many instances the government testing enterprise has gone from 
vigorous test participation to oversight to out-of-sight.
    As a minimum, government test organizations should reconstitute and 
retain a cadre of experienced T&E personnel to perform the following 
functions:

         Participate in the translation of operational 
        requirements into contract specifications and key performance 
        parameters
         Participate in RFP preparation and source selection, 
        including assurance that contractor developed models and 
        simulations are available to the government
         Participate in technology maturity determination and 
        prototype evaluations
         Perform operational assessments in conjunction with 
        the systems engineering process
         Participate in developmental T&E planning and 
        technical reviews
         Participate in test conduct, analysis, reporting 
        capabilities, and limitations, with emphasis on government 
        analysis and reporting

    17. Senator Akaka. Mr. Adolph, what is your assessment of the state 
of the DOD's testing facilities and their ability to perform their 
designated missions?
    Mr. Adolph. The focus of DOD's testing facilities is the Major 
Range and Test Facility Base (MRTFB) which is comprised of those ranges 
and facilities designated most critical to supporting the T&E needs of 
DOD development and acquisition programs. Since the 1970s, DOD's intent 
has been to ensure the capabilities and capacities of these MRTFB 
facilities/ranges meet the T&E requirements of emerging technologies 
and weapon system programs, taking into consideration the long-lead 
times necessary to acquire these capabilities, which often dictates 
that new test technologies be developed and matured. Likewise, a 
skilled T&E workforce must be in place to address the challenges of 
testing and evaluating new weapon systems with increasing complexities 
of technologies, integration, and missions. Given that this is the 
intention of the MRTFB, where do we stand today?
    As I noted in my written statement, several changes resulted from 
the implementation of acquisition reform initiatives in the late 1990s. 
Among them was a de-emphasis on the use of government test facilities, 
testers, and evaluators. When programs need to use government test 
facilities, the first inclination has sometimes been to use non-MRTFB 
capabilities to avoid reimbursement costs required by the MRTFB charge 
policies, often using training ranges having little or no T&E 
infrastructure. If the MRTFB facilities/ranges are used, programs 
typically prefer to request only the raw data to perform their own 
evaluations. Consequently, the Services have been reducing their 
budgets and T&E workforce for sustaining the MRTFB. Not only has the 
MRTFB experienced loss of a large number of the experienced subject 
matter engineering experts, scientific, and mathematical personnel, 
several MRTFB test facilities are being mothballed or closed. The 
government skills to restore or replace them will be lost over time.
    Investments in new test capabilities in the MRTFB for anticipated 
weapon system technologies and test programs have been increasingly 
awaiting the first program to need them for funding rather than using 
T&E institutional funds. This practice is counter to the intent of 
having an MRTFB; i.e., anticipating and having T&E capabilities in 
place when new development or acquisition programs require them. Some 
new T&E capabilities require long-lead times to plan, budget, and to 
develop to be ready when needed by a program, thus can cause the test 
program schedules to slip, inadequate capabilities to be accepted as 
work-arounds, or not to be tested at all. Furthermore, these facilities 
are normally specialized for the programs paying for them, and may have 
little or no residual use for other programs, particularly when they 
are co-located with the prime contractor's manufacturing facility. 
While the Test Resources Management Center (TRMC) ``Strategic Plan for 
DOD Test and Evaluation Resources'' serves as a good guide to the 
actions required to sustain a capable MRTFB, the TRMC has virtually no 
power to require the military departments to sustain adequate operating 
or investment funding beyond the funds managed through the Central Test 
and Evaluation Investment Program and T&E/S&T Program. As a result, my 
assessment is that in some instances, the MRTFB no longer meet its 
intended purpose of having adequate capabilities and capacity available 
for future acquisition programs.
                                 ______
                                 
           Questions Submitted by Senator E. Benjamin Nelson

                         FISCAL RESPONSIBILITY

    18. Senator Ben Nelson. Mr. Sullivan, Dr. Gansler, Dr. Kaminski, 
and Mr. Adolph, cost overruns and fiscal irresponsibility are great 
concerns for me. During his hearing in January, the Secretary of 
Defense stated that many programs that cost more than anticipated are 
built on an inadequate initial foundation, which in my opinion begins 
with defining requirements. It has been reported that many of the 
COCOMs do not believe their requirements are sufficiently represented 
by the current process, which seems to be centered on each Service, not 
the Armed Forces as a whole. This results in duplication of 
capabilities, non-alignment of requirements to capability, and 
interoperability problems, all of which lead to expensive requirements 
creep. Should COCOMs play a lead role in defining needed capabilities?
    Mr. Sullivan. Yes, we believe that the COCOMs should play a 
stronger role in defining capability needs. Requirements determination 
in DOD, particularly for major weapon system programs, continues to be 
driven largely by the military Services. Our work has revealed that 
although the COCOMs--which are responsible for planning and executing 
military missions--are the principal joint warfighting customer in DOD, 
they have played a limited role in determining requirements. In 
analyzing requirements documentation related to new capability 
proposals, we recently reported that most--almost 70 percent--were 
sponsored by the military Services with little involvement from the 
COCOMs. Other studies have also raised concerns that the Services and 
the COCOMS do not routinely collaborate to identify possible joint 
capability solutions.

         The Defense Acquisition Performance Assessment Panel 
        concluded that JCIDS resulted in capabilities that did not meet 
        warfighter needs in a timely manner and recommended that JCIDS 
        be replaced with a COCOM-led requirements process in which the 
        Services and defense agencies compete to provide solutions.
         The Defense Science Board similarly reported that 
        JCIDS has not provided for increased warfighter influence, but 
        instead actually suppresses joint needs in favor of military 
        Service interests, and recommended an increase in the formal 
        participation role of the COCOMs in the JCIDS process.
         The Center for Strategic and International Studies has 
        also recommended that the Joint Forces Command take the lead in 
        conducting capabilities development planning for the COCOMs and 
        become a formal member of the JROC.

    By continuing to rely on capability needs defined primarily by the 
Services, DOD may be losing opportunities for improving joint 
warfighting capabilities and reducing the duplication of capabilities.
    Dr. Gansler. The Packard Commission and (subsequently) Goldwater-
Nichols stated that all requirements were ``joint'' (since that is the 
way we fight). The JROC was set up to represent the COCOMs in the 
``requirements process.'' Interoperability should always be a critical 
design and test requirement. The COCOMs should be the source of 
requirements for both ``urgent needs'' and for JCTDs.
    Dr. Kaminski. I believe that the COCOMs should play a strong role. 
But to do so, they need to develop the expertise needed to engage in 
cost, schedule, and performance tradeoffs. Even top level requirements 
need continuing assessment based upon evolving value propositions. As 
we discover more about costs and performance during execution, good 
program managers should be interacting with requirements developers to 
adjust as need to provide best value for the user. This requires some 
flexibility in cost allocation, and it also places demands upon 
requirements developers to become involved in tradeoff decisions. To be 
effective, requirements developers as well as acquisition managers must 
have experience with systems engineering tools and techniques, and both 
must have sufficient domain experience to engage productively in cost-
performance tradeoffs.
    Mr. Adolph. I don't have the background or experience to respond to 
this question.

    19. Senator Ben Nelson. Mr. Sullivan, Dr. Gansler, Dr. Kaminski, 
and Mr. Adolph, how successful are DOD's processes for matching 
warfighter needs with resources?
    Mr. Sullivan. GAO has previously reported that DOD's inability to 
achieve a balanced portfolio of weapon system programs that matches 
joint warfighting needs with available resources is due in part to the 
fragmented structure of the Department's requirements, funding, and 
acquisition processes. The JCIDS process has not been effective in 
prioritizing needs from a joint, Department-wide perspective, and it 
largely approves capability needs without accounting for the resources 
or technologies that will be needed to acquire the desired 
capabilities. Resource allocation decisions in DOD take place through 
the PPBE process, which is separate from JCIDS. PPBE largely allocates 
resources on a Service-by-Service basis and does not effectively link 
resources to capabilities. In addition, the process allows too many 
programs to start development with unreliable cost estimates and 
without a commitment to fully fund them. Until DOD establishes a more 
integrated approach to weapon system acquisition, it will continue to 
struggle to effectively prioritize warfighting needs, make informed 
trade-offs, and achieve a balanced mix of weapon systems that are 
affordable, feasible, and provide the best value to the warfighter.
    While DOD recently reported that the JROC is doing more to seek out 
and consider input from the COCOMs through regular trips and meetings 
to discuss capability needs and resourcing issues, we found that many 
of the COCOMs still do not believe their needs--reflected through the 
IPL process--are sufficiently addressed through DOD's requirements 
process. In order to grant the COCOMs more influence in identifying 
requirements, DOD should consider providing the COCOMs with additional 
resources to establish robust analytical capabilities for identifying 
and assessing their capability needs. In addition, DOD and the Joint 
Staff should ensure the JROC gives equal consideration to COCOM needs.
    Dr. Gansler. As Secretary Gates has stressed, we need a better 
``balance'' between likely warfighter needs and resource allocations. 
Today, the budgets are unbalanced in the direction of potential, future 
peer competitors.
    Dr. Kaminski. The success is mixed. Warfighters are often very able 
to address their immediate needs, but addressing future needs can be 
challenging. Warfighters also need to be open to new conops--new 
approaches to performing their mission that can be derived from new, 
enabling technologies. Similarly, developers and cost estimators are 
more able to address near-term estimates involving technologies that 
are mature and familiar, and less able to do so with future 
technologies that are less mature and less familiar. Matching needs and 
resources must be a continuing, iterative process. This process must 
have a solid foundation in systems engineering, and the systems 
engineering must in turn have some foundation in M&S which is validated 
by building and testing. Effective value proposition assessments will 
occur only with skilled, trained, and experienced personnel from 
requirements and acquisition communities working together within a 
systems engineering framework to refine needs and solutions.
    Mr. Adolph. I don't have the background or experience to respond to 
this question.

                TECHNOLOGICAL CHANGE AND WEAPONS SYSTEMS

    20. Senator Ben Nelson. Mr. Sullivan, Dr. Gansler, Dr. Kaminski, 
and Mr. Adolph, one of the challenges we face with new technology and 
the rapid advancement of technology is the desire to outfit our 
military men and women with the latest and most advanced equipment. 
While we all want to provide our military with the latest advances in 
technology, I worry that as technology changes we are in an ever state 
of trying to upgrade a system while it is in development. How can we 
best handle constant technology changes for weapons systems that take 
years to develop and deliver?
    Mr. Sullivan. First, technology development must be placed on the 
critical path of any new acquisition program. Technology development 
requires trial and error, discovery, and invention, which should all be 
done in a S&T environment before technologies are brought onto 
programs. Second, GAO has found that successful commercial companies 
use an evolutionary or incremental approach that reduces development 
risks and enables quicker delivery of products to the customer. These 
commercial companies have implemented an evolutionary approach by 
establishing time-phased plans to develop a new product in increments 
(5 years or less) based on technologies and resources achievable now 
and later. They set requirements for products that require only 
technologies that they know will work, and wait to put more risky 
technologies on future increments once their risks have been 
diminished. An open architecture approach to development is also 
important so that product components, such as technologies, can be 
upgraded or added in subsequent product development increments, without 
having to do a major redesign of the product.
    Historically, DOD's approach to developing new weapon systems often 
attempts to satisfy the full capability in a single step, ``big-bang'' 
approach regardless of the design complexity or the maturity of the 
technologies. Under this approach, the warfighter can wait over a 
decade to receive any new capabilities and by the time the capabilities 
are delivered, they may be out of date. Implementing knowledge-based 
evolutionary acquisition practices--where individual program increments 
are defined on the basis of mature technologies and a feasible design 
that incorporates an open architecture approach--would allow DOD to 
more effectively manage technology changes in acquisition programs. 
Shorter, more manageable development cycles would also provide the 
warfighter with useful technologies quicker and provide more confidence 
that technologies can be developed within program cost estimates.
    Dr. Gansler. See my answer to question #13.
    Dr. Kaminski. I have always found that the mark of a good program 
manager is a large lower right hand desk drawer. In this drawer are 
placed all the ideas for incorporation of new technology advancements 
until the first version of the system is fielded. At that time the 
drawer is opened, and representatives from the requirements and 
acquisition communities meet and apply systems engineering tools and 
techniques to assess shortfalls in the fielded system, evolving needs, 
and evolving technologies to examine value propositions and decide what 
to incorporate in the next block of that fielded system.
    Mr. Adolph. With platforms and major systems being procured less 
frequently and taking many years in development, there is often a 
tendency to over-reach since another opportunity to provide advanced 
technology to the warfighter may not occur for many years. The first, 
and most, beneficial action would be to work toward shortening the 
acquisition cycle. For various reasons, the acquisition cycle has 
become overly extended, due in part, as discussed in my response to 
question #13, to initiating acquisition programs based on immature 
technologies. This trend must be reversed. Recommendations have been 
made repeatedly in numerous studies that would result in a shortened 
cycle.
    One approach to a shortened cycle for large programs would be to 
incorporate new technologies in ``block upgrades.'' This approach has 
been successful for many defense systems, sometimes as a result of 
necessity and sometimes as preplanned improvements. The latter approach 
has been implemented on many aircraft programs where planned multi-
staged improvement programs are executed at regular intervals. 
Strategies such as spiral acquisition and pre-planned product 
improvement have been pursued with various degrees of success. If too 
many new technologies are incorporated into a system, the risk is 
compounded by the complex task of integrating advanced technologies 
with sometimes insufficient technical maturity. These risks may be 
mitigated through disciplined processes to properly evaluate technology 
maturity, larger investments in technical domain expertise as well as 
in systems engineering. More emphasis should also be placed on analysis 
of alternatives, to include an OSD-level review. Designs that support 
``plug and play,'' e.g. standard interfaces are also a partial 
solution. Producing the technology and integrating it into the 
logistics system as well as training maintenance personnel, et cetera, 
can still be challenges if the technology has never been deployed to 
the warfighter. (See also my response to question #13.)

                         ACQUISITION PERSONNEL

    21. Senator Ben Nelson. Mr. Sullivan, Dr. Gansler, Dr. Kaminski, 
and Mr. Adolph, a consistent theme throughout the hearing was the 
insufficiency of the current acquisition workforce. What would your 
recommendation be for increasing the number of Active Duty billets 
specifically for acquisition?
    Mr. Sullivan. Although we did not examine DOD's decisionmaking 
process for using military versus civilian personnel, we recently 
reported that DOD lacks critical Department-wide information to 
determine the sufficiency of its acquisition workforce to meet its 
mission needs. We recommended that DOD identify the number and skill 
sets of its total acquisition workforce--including civilian, military, 
and contractor personnel--and conduct analyses using this information 
to inform acquisition workforce decisions regarding the appropriate mix 
of civilian, military, and contractor personnel. We are encouraged by 
the Department's recent announcement that it plans to significantly 
increase the size of DOD's acquisition workforce by converting 11,000 
contractors and hiring an additional 9,000 government acquisition 
professionals by 2015--beginning with 4,100 in fiscal year 2010.
    Dr. Gansler. In the Commission I chaired (the ``Commission on Army 
Acquisition and Program Management in Expeditionary Operations'') we 
stressed that it is not just numbers that matter, but rank as well as 
experience. The senior military acquisition positions should be staffed 
with General Officers that have extensive acquisition experience. As to 
numbers, we should at least get back the 25 percent that Congress 
mandated be cut in fiscal year 1996 (which, across the DOD acquisition 
workforce, military and civilian, is approximately 50,000 additions--of 
which perhaps 20 percent are military).
    Dr. Kaminski. Numbers alone aren't the answer. We need an 
appropriate mix of quality and quantity. We need personnel able to 
provide early and continuing systems engineering and closely-coupled 
development planning. We need: a) sufficient personnel (in both 
government and industry) with adequate education, training and domain 
experience (this includes personnel in requirements development as well 
as in acquisition); and b) sufficient front end investment in the tools 
necessary to understand the key tradeoffs in cost/schedule/performance, 
and to identify and address the key risks in a systematic manner.
    Mr. Adolph. As a minimum, the government acquisition workforce, 
which includes military and civilian personnel, should be increased to 
the levels that existed in the mid-1990s prior to the congressional and 
Service reductions. The military/civilian acquisition position mix 
should be left to the individual Services as each is procuring 
different commodities, and has a different philosophy regarding a 
career military acquisition workforce. As I noted in my opening 
statement, during a time of increased programmatic and technical 
complexity, there has been a loss of a large number of the most 
experienced management and technical personnel, without an adequate 
replacement pipeline. Solutions to the acquisition problems must begin 
with reconstituting a trained and experienced government workforce, 
which includes programs managers, domain subject matter experts as well 
as systems engineers, contracts personnel, testers and evaluators.

    22. Senator Ben Nelson. Mr. Sullivan, Dr. Gansler, Dr. Kaminski, 
and Mr. Adolph, to ensure we get the quality and expertise we require, 
how do you recommend we recruit and retain this force?
    Mr. Sullivan. DOD has just announced plans to substantially 
increase its in-house acquisition workforce over the next few years. 
Our recent report on DOD's acquisition workforce included practices of 
leading companies that could provide insights for DOD as it moves 
forward to hire and retain these employees. We found, for example, that 
the companies employed a variety of recruitment and retention 
initiatives. More importantly, some companies assessed their efforts at 
filling workforce gaps by tracking data on specific recruiting and 
retention metrics. One company assessed the quality of their hiring 
sources by assessing the performance of new hires over their first 2 
years.
    Dr. Gansler. Promotion potential and acquisition-experienced 
leaders are the keys to recruiting, developing, and retaining top 
acquisition personnel.
    Dr. Kaminski. You are quite right in noting that we must address 
retention as well as recruitment to build the requisite work force. We 
want people who want to make a difference. When we recruit these 
people, we must expect that we won't keep them if they can't see that 
they in fact are making a difference. Excessive (and growing) time from 
program initiation to fielding is a big problem for young people who 
want to see the impact of their work. As this time increases from a few 
years to 15 years or more, it undermines the entire acquisition process 
by causing key participants to ``lose the recipe'', and lose a sense of 
accountability as well as a sense of being able to make a difference. 
When new capabilities are developed and fielded in 5 years, engineers, 
managers, testers, cost analysts, et cetera, are able to benefit from 
and apply the experience gained from a previous program or program 
phase. They can also see the results of their decisions and be held 
accountable. We can also meaningfully employ past performance of the 
contractor as a factor in the award of future programs--an important 
factor in incentivizing contractor performance. This all changes 
dramatically when the time extends to 15 years, and we have five roll-
overs of management, engineers, cost analysts, and commercial 
technology during this time period. This long and growing time period 
is a result of the inflexibility inherent in our entire system of 
requirements development, budgeting, and acquisition, and it creates a 
vicious cycle in which it further exacerbates the contributors above, 
and they in turn further increase the time and cost growth. We see the 
result when we must discard our current acquisition system in order to 
deal with urgent needs and field systems such as MRAP and jammers to 
counter IEDs by forming and using rapid reaction organizations. This 
cycle must be broken by attacking the root causes.
    Mr. Adolph. When the acquisition workforce reductions were made in 
the late 1990s, they were implemented by encouraging early retirements 
as well as hiring freezes, which often lasted for years. As a 
consequence of the latter, the pipeline of new civilian employees was 
shut off, and today there are large gaps in experience as well as 
inadequate numbers of personnel. Some Service test organizations are 
aggressively addressing these issues and have more insight into what 
works in today's environment than I do. Based on my past experience, 
initial hiring, particularly for positions at remote test locations, is 
the biggest challenge. It requires aggressive recruiting at 
universities, along with co-op programs. The challenge of the work 
environment and the learning experiences, dealing with the latest 
technologies, combined with well-funded full-time advanced degree 
educational opportunities (with associated follow-on career 
commitments), are powerful positive retention forces after people are 
brought on board. On-the-job learning experiences have been diminished 
in some test organizations, where government personnel have been 
relegated to less challenging supporting functions.

    23. Senator Ben Nelson. Mr. Sullivan, Dr. Gansler, Dr. Kaminski, 
and Mr. Adolph, what can we do in the meantime to handle the personnel 
shortage since it will take many years to develop the level of 
expertise required for proper acquisition reform?
    Mr. Sullivan. As we recently reported in our review of DOD's 
acquisition workforce management, DOD is currently implementing 
initiatives aimed at improving the capacity of its acquisition 
workforce using the newly established Acquisition Workforce Development 
Fund. These efforts are focused in three key areas: (1) recruiting and 
retention; (2) training and development; and (3) retention and 
recognition. DOD has also recently announced plans for significantly 
increasing its in-house acquisition workforce over the next few years. 
While DOD integrates these additional employees into its acquisition 
workforce, the Department can also leverage the experience and 
knowledge of its current workforce. DOD can take steps to retain 
experienced personnel to aide in the development of newer staff and to 
ensure it maintains and transfers key institutional knowledge.
    Dr. Gansler. To fill the gap, while developing a new group, I would 
bring into the government some middle and senior people from industry; 
and I would use contractors (with experience, but without conflicts) to 
fill the rest of the slots.
    Dr. Kaminski. We can begin to hire people with education in 
engineering and systems engineering. We can send our current personnel 
back to universities to enhance their education. We can send our 
current personnel to industry to gain domain experience. We can bring 
back retirees on a part time basis to provide mentoring to our new 
hires and help them gain domain experience. We need to do all of these.
    Mr. Adolph. During the period when the normal pipeline is being 
replenished, Centers of Excellence must be created in selected test and 
acquisition engineering support organizations, with a focus on 
technical domain subject matter expertise. These cadres of expertise 
will provide the basic oversight functions such as those I outlined in 
my answer to question #16. The first step is to provide oversight in 
key technical disciplines at key programmatic times, e.g., RFP 
preparation, source selection, TEMP preparation, test reporting to 
support program reviews; transitioning to more active, continuous 
involvement as the workforce increases. The Services may be reluctant 
to take this approach. These Service cadres can also be augmented by 
subject matter experts from FFRDCs. The use of red teaming by outside 
experts (e.g., cross-Service, FFRDCs, and SETAs) should also be 
employed to provide the requisite expertise, as well as a measure of 
independence which is too often lacking in government program office-
centric reviews.

      MANDATORY PROCEDURES FOR MAJOR DEFENSE ACQUISITION PROGRAMS

    24. Senator Ben Nelson. Mr. Sullivan, Dr. Gansler, Dr. Kaminski, 
and Mr. Adolph, what oversight and reporting mechanisms do you 
recommend we establish to ensure the Mandatory Procedures for Major 
Defense Acquisition Programs (MDAPs) meet warfighter requirements while 
eliminating redundancy and wasteful spending?
    Mr. Sullivan. DOD has recently revised its acquisition policy in 
ways intended to provide key Department leaders with the knowledge 
needed to make informed decisions before a program starts and to 
maintain disciplined development once it begins. The revised policy 
recommends the completion of key systems engineering activities before 
the start of development, requires early prototyping, establishes 
review boards to identify and mitigate technical risks, and implements 
early milestone reviews. If implemented, these policy changes could 
help programs replace risk with knowledge, thereby increasing the 
chances of developing weapon systems within cost and schedule targets 
while meeting user needs. To achieve improved outcomes in acquisition 
programs, DOD must ensure its policy changes are consistently 
implemented and reflected in decisions on individual programs. However, 
DOD could do more by requiring new programs to have manageable 
development cycles and establish knowledge-based cost and schedule 
estimates. Limiting the length of development cycles would make it 
easier to more accurately estimate costs, predict future funding needs, 
effectively allocate resources, and hold decision makers accountable.
    The acquisition reform legislation recently proposed by the Senate 
Armed Services Committee should also help achieve further improvements. 
Provisions increasing the emphasis on systems engineering, requiring 
early preliminary design reviews, and strengthening independent cost 
estimates and technology readiness assessments should make the critical 
front end of the acquisition process more disciplined. Establishing a 
termination criterion for critical cost breaches could help prevent the 
acceptance of unrealistic cost estimates at program initiation. In 
addition, having greater COCOM involvement in determining requirements 
and greater consultation between the requirements, budget, and 
acquisition processes could help improve DOD's efforts to balance its 
portfolio of weapon system programs.
    While policy revisions and legislation may lead to improvements, 
they will not be effective without changes to DOD's overall acquisition 
environment and the incentives that drive the behavior of its 
decisionmakers, the military Services, program managers, and the 
defense industry. Changing the environment will require strong 
leadership and accountability within the Department.
    Dr. Gansler. I believe we currently have enough oversight and 
reporting mechanisms for MDAPs (and more would only slow the system 
down even more). The critical need is to use these mechanisms more 
effectively--with experienced personnel (who know what to look for, and 
what questions to ask).
    Dr. Kaminski. We first need to realize that ``one size does not fit 
all.'' For example, programs dealing with countering IEDs need cycle 
times measured in weeks, while a next generation bomber program must 
plan for fielding and support cycles measured in years.
    In most cases today the program manager's authority is diffused by 
many levels of oversight in both the Department and in Congress, and 
the financial and performance constraints imposed do not allow 
sufficient freedom of action to apply informed judgment in a timely 
manner. Flexibility is further limited by application of a ``one-size-
fits-all'' approach imposed by the DOD 5000 system, and the oversight 
practiced by the DOD and Congress. A program manager needs the freedom 
to tailor the acquisition approach to the problem, to ensure that the 
program response time will fall within the response time of the threat, 
and to apply a variety of tools and techniques (such as the use of 
prototypes, competitive prototypes, M&S, critical subsystem and 
component demonstration). For this to work, we need program managers 
with the education, training, and domain experience needed to enable 
timely responses and excellent judgment relevant to the domain. They 
need the flexibility to apply their expertise.
    Mr. Adolph. I believe the current oversight and reporting 
mechanisms, when combined with the changes in the proposed legislation, 
will provide adequate direction and guidance for the acquisition 
process. The key is putting more discipline in every step of the 
process by the Services and OSD. This was achieved in numerous past 
successful programs in the 1960s, 1970s, and 1980s without being overly 
bureaucratic. One impediment is the recent trend towards contractual 
vehicles which relieve development contractors of reporting 
requirements, including regular status reports on costs and technical 
performance. The government must have continued access to cost and 
performance data to provide effective oversight.
    Most of the basic policies and directives for the acquisition 
process represent best practices based on decades of experience. The 
fundamental problem is a lack of adequate discipline throughout the 
process. This begins with proper definition of operational requirements 
at the outset of a program and stating those requirements in terms that 
are reasonable and quantifiable for design and test purposes. Another 
major issue is the lack of adequate technology maturity (See my 
response to question #13.)
    There have been some problems with military specifications and 
standards in the past, in that they were not properly tailored to the 
system to be procured. However, these standards have evolved and were 
updated over decades and were, for the most part, excellent compendiums 
of best practices and lessons learned. If properly used, they provide 
guidance for system development to ensure a systematic and disciplined 
approach. Proper application of these guidance documents also ensures 
that problems, which are a normal part of any complex high technology 
development, are identified early. Many of these documents have been 
allowed to atrophy over the last several years or have been eliminated. 
Previously eliminated specifications and standards should be 
selectively updated and reinstated; while retaining the option to use 
commercial specifications and standards when available, provided the 
latter adequately address the requisite military performance and 
suitability requirements in the intended operating environment. 
Finally, the most important ingredient remains a capable and 
experienced government acquisition workforce with equally capable 
leaders, who remain on the job long enough to realize the consequences 
of their programmatic decisions.
                                 ______
                                 
              Questions Submitted by Senator David Vitter

                    PERSONNEL IN ACQUISITION REFORM

    25. Senator Vitter. Dr. Gansler, a recurring message from the panel 
is the importance of improving the quality and experience of personnel 
in acquisition reform, and I'm glad that Chairman Levin and Ranking 
Member McCain included section 206 in their acquisition reform bill to 
reward excellence. This section requires the establishment of a program 
recognizing excellent performance and the award of cash bonuses. 
However, it was stated that cash isn't necessarily the greatest 
incentive for encouraging excellence, and making a difference is the 
best incentive. In your opinion, what can DOD implement in an 
excellence program to demonstrate improvements so that personnel can 
track and actually see their work reflected in tangible results to 
provide and maintain motivation?
    Dr. Gansler. Combining appropriate metrics with rewards for success 
(including recognition, and even token financial rewards) are 
desirable--but the managers must be allowed to have sufficient 
flexibility to exercise their judgment, if they are to succeed.

    26. Senator Vitter. Dr. Gansler, how much of the problem in 
attracting the right quantity of qualified acquisition personnel is a 
product of the overall Federal employee hiring rules established by 
Office of Personnel Management (OPM) or the specific rules established 
by DOD?
    Dr. Gansler. The DOD must be able to compete with industry for 
``the best and brightest.'' The process must be flexible and fast--yet 
fair. Salary will not be the driver (government jobs offer the rewards) 
but the process can be neither bureaucratic nor political.

    27. Senator Vitter. Dr. Gansler, last year, Congress authorized 
$300 million to rebuild the acquisition workforce. Do you believe this 
is sufficient?
    Dr. Gansler. It is an important start, and will likely need to be 
continued for a few years.

    28. Senator Vitter. Dr. Gansler, cost overruns were endemic in the 
1980s and 1990s when the acquisition workforce was robust. Is it likely 
then that adding more people will solve the problem of cost overruns?
    Dr. Gansler. As I noted in question #21, numbers alone are not the 
answer, senior and experienced people with an improved acquisition 
process (as covered in my answers above--especially numbers 1, 8, and 
13).

    29. Senator Vitter. Dr. Gansler, it seems that there is a problem 
with keeping quality program managers in their roles for an extended 
period of time. How can we ensure continuity at these positions?
    Dr. Gansler. Goldwater-Nichols intended the program managers to 
remain at least through the next milestone. In the Packard Commission 
we emphasized that they be allowed to be promoted in place (so that the 
extended period is a reward, not a career-limiting assignment).

                   FUNDING FOR DEVELOPMENTAL TESTING

    30. Senator Vitter. Mr. Adolph, to what extent does a lack of 
funding for developmental testing (DT) early in a program's schedule 
result in cost increases and delays later in its schedule?
    Mr. Adolph. The lack of adequate funding for DT has a major adverse 
impact on developmental cost and schedule. There are numerous issues 
relating to DT funding. The weapons timelines for RDT&E continue to 
increase, driven by inadequate resourcing of the entire developmental 
process as well as the inclusion of immature technologies in systems. 
Inadequate funding of DT, which includes the number of test assets, 
results in delayed identification and correction of problems, many of 
which are a normal part of a high-technology developmental program. 
Late identification results in more difficulty and expense involved in 
fixes. Most recent programs have had an inadequate number of assets to 
execute a robust DT program in a timely manner. When there are an 
inadequate number of test assets, the DT program is drastically 
impacted, often for months, when a test vehicle must be laid up for a 
retrofit. By the time a program reaches the full-scale platform test 
phase, there is a huge cadre of test and test support personnel who 
cannot be efficiently used whenever a test program comes to a halt. The 
fixed cost of maintaining this cadre often exceeds the incremental 
variable costs of conducting test missions. Another factor in efficient 
testing is the availability of adequate test personnel and test 
facilities. The latter includes physical test support facilities and 
assets, instrumentation, and data processing (see also my response to 
question #17). Numerous developmental programs have been delayed 
because of the lack of adequate facilities, capacity, and people to 
accomplish the basic data processing tasks quickly, as well as 
inadequate number of domain subject matter experts to analyze test 
results. This is exacerbated when earlier involvement by DT&E and OT&E 
personnel is limited by a lack of sufficient experienced personnel. 
Commercial programs routinely resource facilities and personnel to 
conduct testing on a multiple shift basis during critical phases of the 
developmental process.

                        COMPETITIVE PROTOTYPING

    31. Senator Vitter. Dr. Kaminski, how much can competitive 
prototyping help in identifying the best product for warfighters?
    Dr. Kaminski. Competitive prototyping is one arrow in our quiver of 
tools and techniques in development planning. Not all prototyping needs 
to be competitive. Prototyping should not be limited to the full system 
level. There are cases where prototyping is best applied only to 
address major platform integration risk areas, or to critical 
subsystems. Again, I don't believe there is a ``one-size-fits-all'' 
solution to prototyping. But I do believe that at a big picture level 
we are not doing enough prototyping in our development planning. 
Reducing risk and developing domain expertise require that we build 
things. M&S are of great value, and so are our computer based analysis 
and design tools, but all of my experience tells me that we need to 
build and test on a continuing basis to make mistakes, learn from those 
mistakes, and apply that learning in the development of new 
capabilities.

    32. Senator Vitter. Dr. Kaminski, the F-16 program, lauded by many 
as an example of how to acquire a capability, was competed to the point 
where the Air Force actually flew competing planes before deciding on 
which defense contractor won the award. Why have we gotten so far away 
from that?
    Dr. Kaminski. We haven't completely departed from that approach. We 
did something very similar in the JSF program. But I agree that we are 
not doing enough. We can do quite a bit more. I believe we can do so 
without increasing costs by spending money upfront to reduce or avoid 
the cost of significant overruns later.

                  JOINT REQUIREMENTS OVERSIGHT COUNCIL

    33. Senator Vitter. Mr. Sullivan, the Vice Chiefs have important 
jobs internally managing their respective Services. Do you believe that 
JROC should instead be staffed with independent personnel who are not 
advocates for a particular Service?
    Mr. Sullivan. GAO has found DOD's requirements process to be too 
Service-centric. The needs of the COCOMs should be given greater 
consideration. There are a number of ways the requirements setting 
process might be changed to address this issue. For example, the 
combatant commanders might supplant the Service Vice Chiefs role in the 
process or might be given an equal say in the process. Several outside 
reviews have suggested such solutions:

         The Beyond Goldwater-Nichols study team recommended a 
        more ``joint'' JROC in which the Service Vices are replaced by 
        COCOM Deputies and civilians responsible for requirements 
        policy.
         The Defense Acquisition Performance Panel (DAPA) 
        recommended replacing the current requirements setting process 
        with a combatant commander-led requirements process in which 
        the Services and defense agencies compete to provide solutions. 
        Each of the combatant commanders would be tasked to prepare 
        extended planning annexes to each of their operational and 
        contingency plans (to be updated on a 2-year cycle) that would 
        provide a 15-year forecast of both capability gaps and excesses 
        relative to mission requirements.
         Some members of the Defense Science Board Summer Study 
        on Transformation felt that the requirements process continues 
        to be dominated by the force providers and the Joint Staff and 
        that COCOM needs are under-represented. They proposed making 
        the COCOMs more equal partners with the force providers from 
        the beginning of the process, particularly when identifying 
        capabilities needed to carry out the Department's operational 
        missions.

    We believe that these proposals deserve further consideration as a 
means to help improve the Department's ability to balance joint 
warfighting needs.

    34. Senator Vitter. Mr. Sullivan, should we have a new, independent 
JROC that could hear all advocates for all programs and then recommend 
continuation or elimination of the program based on current and likely 
future threats to the United States?
    Mr. Sullivan. Because many major defense acquisition programs have 
ambitious and lengthy product developments--in some cases, delivery of 
a weapon system to the warfighter takes as long as 15 years--there can 
be a significant amount of time that elapses between JROC reviews and 
changes in the threat environment can occur. Before a major weapon 
program is approved to begin system development, the JROC must validate 
the capabilities to be developed, including the system's key 
performance parameters. The JROC also validates the production elements 
of a program before it is approved to start production. For programs 
that have lengthy development cycles, more frequent reviews may be 
necessary to assess whether programs are worth continuing in light of 
current and likely future threats. In prior work we have conducted on 
how successful companies manage their portfolios of product development 
efforts, we found that the companies revisit their investment decisions 
at multiple stages throughout product development to ensure products 
are still of high value.

    35. Senator Vitter. Mr. Sullivan, should the Services be required 
to budget their programs to the cost estimates provided by a new 
Director of Independent Cost Assessment?
    Mr. Sullivan. Budgeting programs to an independent cost estimate 
would be an improvement, however, for cost estimates to be effective 
they must be based on a high degree of knowledge about requirements, 
technology, design, and manufacturing.
    A reliable cost estimate helps ensure a program's projected funding 
needs are adequate to execute the program. Less than a quarter of the 
48 programs in GAO's annual assessment of weapon system programs that 
provided data used the estimate made by DOD's Cost Analysis Improvement 
Group (CAIG) as a basis for the program's baseline, while almost 70 
percent of the programs used the program office or Service cost 
estimate. While cost estimates from the CAIG can underestimate a 
program's costs by billions of dollars, GAO has previously found that 
these independent estimates generally underestimate costs by a smaller 
amount than program office and Service estimates.
    Cost estimates are inaccurate in part because they are based on 
limited knowledge about the requirements, technologies, design 
maturity, and the time and funding needed to execute a program. GAO has 
found, for example, that program Cost Analysis Requirements Documents--
used to build the program cost estimate--often lack sufficient detail 
about planned program content for developing sound cost estimates. 
Without this knowledge, cost estimators must rely heavily on parametric 
analysis and assumptions about system requirements, technology, and 
design maturity, and the time and funding needed. The assumptions used 
in developing estimates also tend to be overly optimistic. Furthermore, 
cost estimates that lack knowledge and rely on assumptions have 
inherently high levels of risk and uncertainty that are not typically 
communicated to decision makers.

    36. Senator Vitter. Mr. Sullivan, with the annual Selected 
Acquisition Reports and current Nunn-McCurdy reporting requirements, is 
Congress suffering from a lack of oversight and reports from DOD?
    Mr. Sullivan. In terms of the reports it receives from DOD, 
Congress is suffering from a lack of insight into the risks on programs 
that could lead to cost, schedule, and performance shortfalls. The 
Selected Acquisition Reports and Nunn-McCurdy reporting that Congress 
gets from DOD on its major defense acquisition programs primarily 
report on program outcomes to date--including quantitative measures of 
cost, schedule, and performance over time. While these are useful 
indicators of the health of acquisition programs and whether they are 
meeting their intended goals, they have limited predictive value. DOD 
could improve the information it provides to Congress by reporting on 
knowledge metrics that evaluate whether programs have attained certain 
levels of technology, design, and manufacturing knowledge by key points 
in the acquisition process. This reporting would facilitate the 
identification of potential problems that could lead to cost, schedule, 
and performance shortfalls before they occur.

    37. Senator Vitter. Mr. Sullivan, is there any credence to the 
concern that program managers have ``too many'' reports to file and are 
not focusing on managing their programs?
    Mr. Sullivan. Oversight and reporting requirements need to be 
evaluated both from the perspective of the time they take and the value 
they provide. In the past, we have reported on concerns in both of 
those areas. In a November 2005 report, we reported that program 
managers and program executive officers commented that they spend too 
much time producing data that is not strategic or very useful to them. 
In fact, more than 90 percent of 126 respondents to our survey of 
program managers said that they spent either a moderate, great, or very 
great extent of their time representing their program to outsiders and 
developing and generating information about program progress.
    We have also found that the reporting that is done does not include 
key information that decisionmakers need to decide if a program is 
ready to proceed into the next phase of the acquisition process. For 
example, in transitioning from system integration to system 
demonstration, we have recommended that DOD ensure the capture of 
knowledge about the completion of engineering drawings; completion of 
subsystem and system design reviews; agreement from all stakeholders 
that the drawings are complete; and identification of critical 
manufacturing processes, among other indicators. In the transition to 
production, we recommended that DOD capture knowledge about production 
and availability of representative prototypes along with statistical 
process control data. Our 2005 report indicated that a relatively small 
percentage of programs used these knowledge indicators. For example, 
only 32 percent of 126 program managers who responded to our survey 
said they used design drawing completion extensively to measure design 
maturity; only 26 percent said they used production process controls to 
a great extent. Even fewer program managers reported that their 
immediate supervisor used these measures extensively to evaluate 
progress. Broader use and reporting of these types of metrics would 
help decisionmakers gauge progress and ensure that programs managers 
are: (1) conducting activities to capture relevant product development 
knowledge; (2) providing evidence that this knowledge has been 
captured; and (3) holding decision reviews to determine that the 
requisite knowledge has been captured before proceeding to the next 
phase of the acquisition process.

    [Whereupon, at 11:52 a.m., the committee adjourned.]

                                 
