[House Hearing, 115 Congress]
[From the U.S. Government Publishing Office]





 
                         [H.A.S.C. No. 115-37]

                   CREATING A FLEXIBLE AND EFFECTIVE
                         INFORMATION TECHNOLOGY
                       MANAGEMENT AND ACQUISITION
                    SYSTEM: ELEMENTS FOR SUCCESS IN
                      A RAPIDLY CHANGING LANDSCAPE

                               __________

                                HEARING

                               BEFORE THE

           SUBCOMMITTEE ON EMERGING THREATS AND CAPABILITIES

                                 OF THE

                      COMMITTEE ON ARMED SERVICES

                        HOUSE OF REPRESENTATIVES

                     ONE HUNDRED FIFTEENTH CONGRESS

                             FIRST SESSION
                               __________

                              HEARING HELD

                             APRIL 26, 2017

[GRAPHIC(S) NOT AVAILABLE IN TIFF FORMAT]
 
 
 
 
                                   ______
 
                          U.S. GOVERNMENT PUBLISHING OFFICE 
 
 25-823                         WASHINGTON : 2017 
 -----------------------------------------------------------------------
   For sale by the Superintendent of Documents, U.S. Government Publishing 
   Office Internet: bookstore.gpo.gov Phone: toll free (866) 512-1800; 
          DC area (202) 512-1800 Fax: (202) 512-2104 Mail: Stop IDCC, 
                           Washington, DC 20402-0001
   
 
 
 
 
 
 
 
 
 
 
 
 
 


           SUBCOMMITTEE ON EMERGING THREATS AND CAPABILITIES

                ELISE M. STEFANIK, New York, Chairwoman

BILL SHUSTER, Pennsylvania           JAMES R. LANGEVIN, Rhode Island
BRAD R. WENSTRUP, Ohio               RICK LARSEN, Washington
RALPH LEE ABRAHAM, Louisiana         JIM COOPER, Tennessee
LIZ CHENEY, Wyoming, Vice Chair      JACKIE SPEIER, California
JOE WILSON, South Carolina           MARC A. VEASEY, Texas
FRANK A. LoBIONDO, New Jersey        TULSI GABBARD, Hawaii
TRENT FRANKS, Arizona                BETO O'ROURKE, Texas
DOUG LAMBORN, Colorado               STEPHANIE N. MURPHY, Florida
AUSTIN SCOTT, Georgia
                 Kevin Gates, Professional Staff Member
              Lindsay Kavanaugh, Professional Staff Member
                          Neve Schadler, Clerk
                          
                          
                          
                          
                          
                          
                          
                          
                          
                          
                          
                          
                          
                          
                          
                          
                          
                          
                          
                          
                          
                          
                          
                          
                          
                          
                          
                            C O N T E N T S

                              ----------                              
                                                                   Page

              STATEMENTS PRESENTED BY MEMBERS OF CONGRESS

Langevin, Hon. James R., a Representative from Rhode Island, 
  Ranking Member, Subcommittee on Emerging Threats and 
  Capabilities...................................................     2
Stefanik, Hon. Elise M., a Representative from New York, 
  Chairwoman, Subcommittee on Emerging Threats and Capabilities..     1

                               WITNESSES

Greer, Edward, Former Deputy Assistant Secretary of Defense for 
  Developmental Test and Evaluation..............................     7
Halvorsen, Hon. Terry, Former Department of Defense Chief 
  Information Officer and Department of the Navy Chief 
  Information Officer............................................     5
Levine, Hon. Peter, Former Department of Defense Deputy Chief 
  Management Officer and Under Secretary of Defense for Personnel 
  and Readiness..................................................     3

                                APPENDIX

Prepared Statements:

    Greer, Edward................................................    46
    Halvorsen, Hon. Terry........................................    36
    Levine, Hon. Peter...........................................    27
    Stefanik, Hon. Elise M.......................................    25

Documents Submitted for the Record:

    [There were no Documents submitted.]

Witness Responses to Questions Asked During the Hearing:

    Mr. Scott....................................................    65

Questions Submitted by Members Post Hearing:

    Mr. Franks...................................................    73
    Mr. Shuster..................................................    73
    Ms. Stefanik.................................................    71








CREATING A FLEXIBLE AND EFFECTIVE INFORMATION TECHNOLOGY MANAGEMENT AND 
    ACQUISITION SYSTEM: ELEMENTS FOR SUCCESS IN A RAPIDLY CHANGING 
                               LANDSCAPE

                              ----------                              

                  House of Representatives,
                       Committee on Armed Services,
         Subcommittee on Emerging Threats and Capabilities,
                         Washington, DC, Wednesday, April 26, 2017.
    The subcommittee met, pursuant to call, at 2:03 p.m., in 
room 2118, Rayburn House Office Building, Hon. Elise M. 
Stefanik (chairwoman of the subcommittee) presiding.

 OPENING STATEMENT OF HON. ELISE M. STEFANIK, A REPRESENTATIVE 
FROM NEW YORK, CHAIRWOMAN, SUBCOMMITTEE ON EMERGING THREATS AND 
                          CAPABILITIES

    Ms. Stefanik. The subcommittee will come to order. I would 
like to welcome everyone to this hearing of the Emerging 
Threats and Capabilities Subcommittee of the House Armed 
Services Committee on the important topic of information 
technology [IT].
    Several of us from the committee recently had a discussion 
with Eric Schmidt, who is the executive chairman of Alphabet, 
as well as the chairman of the Department of Defense [DOD] new 
Defense Innovation Board. That discussion was very helpful in 
provoking us to think about how the Department could do a 
better job incorporating best practices from industry. And 
while it is unrealistic to think that DOD will ever operate 
exactly like industry, it is nonetheless imperative that we 
achieve every ounce of efficiency out of our back office and IT 
operations in order to fully invest in combat capability. The 
challenge is finding that correct balance, and today's hearing 
brings us one step closer.
    Information technology represents over $30 billion of the 
Department's total budget; however, too often it is treated as 
a support tool secondary to platforms, weapons, training, and 
the operations of the Department. But anyone who has seen U.S. 
forces operate over the past 25 years understands that our 
military advantage comes from those network systems providing 
the intelligence, precision strike, information fusion and 
warning capabilities our warfighters have come to rely on.
    This committee has been focused on reforming the operations 
of the Department for the past 2 years, from streamlining 
acquisition regulations to streamlining an overly cumbersome 
bureaucracy. The end goal of these efforts has been to enable 
us to buy and develop systems with greater agility and 
flexibility so that state-of-the-art tools get into the hands 
of our warfighters faster.
    I know our witnesses today will help provide us with a 
better framework for understanding how to think about defense 
management and acquisition practices for information technology 
as we explore the critical questions before us, such as: What 
are the characteristics of well-performing programs that we 
should focus on? What leading indicators should we be 
monitoring to determine if programs are going off the rails and 
before it is too late? And how can we better identify, 
encourage, and reward those program managers that are executing 
information technology programs well?
    Let me now turn to the ranking member, Jim Langevin of 
Rhode Island, for any opening comments he would like to make.
    Jim.
    [The prepared statement of Ms. Stefanik can be found in the 
Appendix on page 25.]

  STATEMENT OF HON. JAMES R. LANGEVIN, A REPRESENTATIVE FROM 
RHODE ISLAND, RANKING MEMBER, SUBCOMMITTEE ON EMERGING THREATS 
                        AND CAPABILITIES

    Mr. Langevin. Thank you, Chairwoman Stefanik, and thank you 
to our witnesses for being here today. I definitely look 
forward to hearing what you have to say, given your wealth of 
knowledge in this area.
    So management and acquisition of information technology at 
the Department of Defense is one of the most challenging and 
pressing DOD organizational and administrative issues facing us 
today. For years, Congress, the executive branch, and industry 
have attempted to bring DOD's IT programs and processes into 
the 21st century. Despite attempts like the Joint Information 
Environment and streamlining acquisition processes, DOD's pace 
to improve its IT posture is not progressing with the desired 
speed to achieve serious efficiencies, increase security, and 
take advantage of enhanced capabilities that are readily 
available. For example, the intelligence community has 
developed and is using commercial cloud services, yet 
widespread use of cloud computing has not yet taken place at 
the Department.
    All that said, I will be the first to acknowledge that the 
Department of Defense IT management, modernization, and cyber 
are extraordinarily complex. I will also acknowledge that 
Congress, in an attempt to be helpful, has taken action that 
has seemingly had the opposite effect.
    That is why today's hearing is so important. IT management 
and modernization experts who served at the DOD sit before us. 
I look forward to their concrete recommendations for improving 
DOD's IT posture in an expeditious manner, and I hope they 
provide granularity on legislative, regulatory, and cultural 
barriers, like risk aversity, that are inhibiting the rapid 
change required. And I am certainly glad that they help us 
understand what is working as well as highlight the success 
that the Department has already had in this arena.
    So with that, Madam Chair, I thank you for organizing this 
hearing. I thank our witnesses for being here today. And I 
yield back.
    Ms. Stefanik. Thank you, Jim.
    Today, we welcome three distinguished witnesses, all of 
whom have served as senior officials within the Department of 
Defense, and so they not only understand the government 
challenges with information technology management and 
acquisition, but also how private sector practices can be 
applied or need to be avoided.
    First, the Honorable Peter Levine, former Department of 
Defense Deputy Chief Management Officer [DCMO] and Under 
Secretary of Defense for Personnel and Readiness [P&R]. Next, 
the Honorable Terry Halvorsen, former Department of Defense 
Chief Information Officer [CIO] and Department of the Navy 
Chief Information Officer. And Mr. Ed Greer, former Deputy 
Assistant Secretary of Defense for Developmental Test and 
Evaluation [DT&E].
    Welcome to all of our witnesses. I would like to remind you 
that your testimony will be included in the record, and we ask 
that you summarize key points from that testimony in 5 minutes 
or less for your opening statements.
    And, Mr. Levine, I will start with you. The floor is yours.

 STATEMENT OF HON. PETER LEVINE, FORMER DEPARTMENT OF DEFENSE 
DEPUTY CHIEF MANAGEMENT OFFICER AND UNDER SECRETARY OF DEFENSE 
                  FOR PERSONNEL AND READINESS

    Mr. Levine. Thank you. Chairwoman Stefanik, Ranking Member 
Langevin, and members of the subcommittee, I thank you for the 
opportunity to appear here this afternoon to discuss DOD's 
information technology management and acquisition processes. 
What I would like to focus on, with your permission, is DOD 
business systems acquisition.
    For years, DOD has been trying to buy new business systems 
the same way that it buys weapon systems and other big ticket 
items, and the result has been a series of sometimes 
spectacular failures. The fact is that there is no such thing 
as a stand-alone business system and no such thing as a stand-
alone business system acquisition. The DOD's business systems 
are interconnected not only with each other, but also with the 
business processes that they run.
    When DOD wants to acquire a new financial management system 
or logistic system or HR [human resources] system, it 
inevitably finds links to dozens if not hundreds of other IT 
systems which either feed it data or rely on it for data. Each 
of these systems, no matter how outdated, redundant, or 
ineffective they may seem, has a core of devoted users who are 
likely to resist change.
    DOD tries to ensure that change management and business 
process reengineering take place concurrently with new business 
systems acquisitions, but it has not been easy. Acquisition 
officials don't have the authority or expertise to require 
their customers to do business process reengineering when they 
buy new systems, and management officials responsible for those 
business processes have proven to be incapable of running large 
acquisitions.
    In the last couple of years, Congress has helped this issue 
considerably by streamlining section 2222 of title 10, which 
provides the statutory requirements for DOD business systems 
acquisitions, and by repealing requirements for major automated 
information systems, or MAIS programs, that mimic the 
acquisition process for major weapon systems.
    When I testified with Terry here last year, I promised that 
the Department would take advantage of the new flexibility 
granted by Congress to better coordinate the roles of the DCMO, 
the CIO, and AT&L [Acquisition, Technology, and Logistics] in 
the acquisition of business systems.
    Two weeks after that hearing, I left the position of DCMO 
to become Acting Under Secretary for P&R, but I am pleased to 
say that streamlining effort went on in my absence and that 
just a couple of months ago the Department issued a new DOD 
instruction addressing business systems acquisitions. This new 
instruction appropriately sequences for the first time decision 
points regarding business solutions, IT solutions, and 
acquisition solutions so that we don't have these redundant 
processes going on side by side without referencing each other, 
but we have a coordinated process.
    This is also the first step toward building an IT-specific 
acquisition process as the Defense Science Board [DSB] 
recommended almost a decade ago. Consistent with the DSB's 
recommendations, the new instruction establishes a unique set 
of decision points for business systems acquisitions and 
requires prototyping and incremental delivery of capability.
    I would have to say, though, that the DSB report, which is 
a 2009 report, recommended a unique acquisition system not just 
for the acquisition of business systems, but for the 
acquisition of all IT. It has been a disappointment for those 
who were here with the Congress--I was on the SASC [Senate 
Armed Services Committee] staff at the time that that report 
came out--that that still hasn't been fully implemented for IT.
    Congress felt strongly enough about this that you enacted 
provisions on this twice, first in 2010 and then in 2015, 
telling the Department: Please go look at this DSB report and 
figure out a way to implement the kind of IT-unique acquisition 
system that the DSB is recommending, having looked at 
commercial practices, and figure out how it would best apply to 
the Department. I commend you to look at that report. It is 
still the right thing to do. It is still what the Department 
needs to do.
    So while I think the Department has made good steps in the 
direction of change, there is still a long way to go. And one 
thing I would add before I wrap up and hand over to Terry is 
that the issuance of a new DOD instruction like this one I just 
mentioned is the beginning of a process of change, not the end 
of a process of change.
    One thing that I have seen in the Department is it takes 
every bit as much effort to make sure that the instruction, the 
legislation, the regulation is really implemented, implemented 
in the spirit it was intended, as it does to enact it in the 
first place. So it will be up to the new team at DOD when it 
arrives to ensure that the new processes are appropriately 
implemented in practice, and in particular with regard to 
business systems, that the CMO [Chief Management Officer] will 
have to remain engaged throughout the acquisition life cycle to 
ensure that the Department adopts new and efficient business 
solutions rather than tailoring new business systems to old and 
inefficient processes.
    With that, I thank you again for inviting me here today, 
and I look forward to your questions.
    [The prepared statement of Mr. Levine can be found in the 
Appendix on page 27.]
    Ms. Stefanik. Thank you, Mr. Levine.
    Mr. Halvorsen, you are recognized for 5 minutes.

STATEMENT OF HON. TERRY HALVORSEN, FORMER DEPARTMENT OF DEFENSE 
  CHIEF INFORMATION OFFICER AND DEPARTMENT OF THE NAVY CHIEF 
                      INFORMATION OFFICER

    Mr. Halvorsen. Madam Chairman, Ranking Member, 
distinguished members, thank you for the opportunity to testify 
today before this committee. This is an important topic, and I 
would like to echo much of what Peter just said. Obviously, we 
did testify together on this topic last year, and I did testify 
the last 2 years on this topic.
    DOD has made progress. They continue to face critical 
global challenges. I do think they will meet those challenges. 
But it is faced with an added and unprecedented dimension: This 
is arguably the period in history with the fastest developing 
and most complex technology.
    Unlike previous times, the vast majority of this technology 
growth is occurring in the private sector, not originating with 
the government. This means in addition to identifying the right 
capabilities to meet DOD requirements, DOD must be able to 
acquire and integrate this technology with greater agility. 
Today's environment demands more broadly defining capability 
and not providing detailed requirements that dictate solutions.
    At times, the government process today, because of the 
current thinking, delivers legacy solutions. We really do need 
to think about how we talk about capabilities and not dictating 
a series of technical requirements that will be out [of] date 
the day we publish them.
    DOD needs a better understanding of the commercial 
environment to become more effective and efficient working with 
industry and determining how solutions should be implemented.
    With respect to business systems, DOD really needs to ask 
itself: Should it implement whole commercial solutions or some 
degree of hybrid solutions retaining some government 
capability?
    I will echo what Peter said, that this also needs to focus 
on the process, and I strongly recommend that the going-in 
position for business solutions until proven wrong through 
business case analysis is that the government and DOD should be 
adopting commercial solutions off the shelf. The real question 
really is what businesses should DOD be directly in and where 
should it offload those businesses to the commercial sector.
    Regarding systems that are more aligned with the primary 
mission of DOD, such as national security systems, DOD must 
more carefully weigh the mission risk, the mission security 
requirements, and since these systems are more likely to be 
operated by military and civilian members, DOD must really look 
at the workforce implications of training and sustainment.
    The new, changing environment also means DOD will be 
requiring more services from industry as opposed to just buying 
products. To successfully buy services in this exploding 
technical environment will require DOD to form better 
partnerships with industry and for industry to be more open to 
sharing technical data with DOD.
    To facilitate the building of these critical partnerships, 
I believe this committee and others will need to look at the 
laws governing relationships and contact between DOD officials 
and industry members and expand those programs. The laws we 
have today have been written with great intention, but they are 
outdated. Many of them from a financial perspective still use 
numbers from the 1970s. They don't make any sense, and they 
limit the ability for DOD and other government agencies to 
function correctly.
    We also need to look at the way we test commercial products 
for acceptance and security perspective today, and I elaborate 
this in my written testimony. The security accreditation 
process is costing both the government and industry lots of 
money and doing a disservice to, in the case of DOD, our 
service members for how long it takes to get those products 
certified.
    We insist on testing stuff that has commercially been 
accepted and tested in industry. We need some legislation that 
says: Why can't I accept that testing as the official testing? 
Test can the government operate it correctly, 100 percent, that 
testing should be included, but we really need to look at how 
we do the testing. That has become one of the long poles in the 
tent.
    You have to improve efficiency. I will talk a little bit 
about the McKinsey report, about the 125 million. DOD 
definitely took that report seriously, but some of those 
numbers in there were not well thought out in the end. But they 
should be continuing to refer to that report. It does give the 
right topic areas that DOD should be looking at to gain more 
efficiencies in.
    As Peter, I am happy with the DOD CIO, and the DCMO 
relationship. I think it is strong today, and I think it will 
continue to get better. That is important, because it is a 
combination of the process and the technology.
    In the end, this is a little bit about a cultural solution 
more than it is laws and anything else. We need to think 
differently. I think we have unintentionally been building for 
a long time a culture of distrust and one that was based on 
overregulation and a foundational belief that all the players, 
industry, government, and academia, all needed to be protected 
from itself, and we have done a good job at doing that.
    Somewhere we have lost what was good in the systems we had 
prior. I was quoted at the DOD as saying that our secret weapon 
was our commercial capability and our relationships with 
industry. I would amend that to read our secret weapon is our 
commercial capability, our relationship with industry, and the 
combined efforts of the military, civilian, contractor, and 
commercial workforce to make it all work and deliver the 
results.
    Thank you, and I look forward to your questions.
    [The prepared statement of Mr. Halvorsen can be found in 
the Appendix on page 36.]
    Ms. Stefanik. Thank you.
    Mr. Greer, you are recognized for 5 minutes.

STATEMENT OF EDWARD GREER, FORMER DEPUTY ASSISTANT SECRETARY OF 
         DEFENSE FOR DEVELOPMENTAL TEST AND EVALUATION

    Mr. Greer. Chairwoman Stefanik, Ranking Member Langevin, 
and members of the subcommittee, thank you for the opportunity 
to express my views this afternoon.
    In 2013, this same subcommittee hosted a roundtable 
discussion on IT acquisition reform, and it seems like the same 
challenges and the same issues were discussed 3 or 4 years ago. 
So we do need to be mindful of what kind of progress we have 
made.
    Today, I am going to talk about MAIS [major automated 
information systems] challenges, best practices, the role of 
developmental test and evaluation, and business systems versus 
tactical weapon systems.
    MAIS challenges. The challenging nature of MAIS acquisition 
can be attributed to many factors. I will briefly discuss 
three.
    Program complexity. Typical MAIS programs have to be 
integrated into multiple existing and emerging enterprises that 
contain a large number of government and commercial entities.
    Another challenge: unstable requirements. In many cases, 
the changes are driven by advancements in technology, vendors 
updating hardware, operating system upgrades, and so on, or by 
changing world events.
    And the third challenge: build versus buy. While many IT 
companies regret building enterprise software solutions because 
it was much more expensive than expected, there are times when 
custom software solutions is the right answer. A rational 
build-versus-buy decision starts with well-defined requirements 
and a quantifiable outcome.
    Second, the best practices. There are many best practices 
within the commercial sector and within DOD. I would like to 
highlight just a few.
    Executive leadership participation. Robust and continued 
senior-level attention and participation contributed 
significantly to the success of agile MAIS acquisition programs 
as documented in the DOT&E [Director for Operational Test and 
Evaluation] 2016 annual report. When senior leaders are on-
site, they can add resources as necessary and they can also 
shorten the decision cycle time.
    Another best practice, continuous developmental test and 
evaluation. The program office should have a coherent DT&E 
strategy to find and fix problems as each software component is 
developed and delivered. Deficiencies are much easier to fix 
before integrating into modules and components.
    And the best practice of evolutionary acquisition, which is 
a method intended to reduce cycle time and speed the delivery 
of capabilities to users. Often, in an evolutionary model, the 
development of increments must occur in parallel to deliver 
capability on time.
    The third topic, the role of developmental test and 
evaluation. Conducting DT&E in an agile environment must be 
done early and often. During a major weapon system development 
cycle, 80 percent of the test and evaluation is developmental 
test and evaluation.
    There are DT&E organizations within the services and at OSD 
[Office of the Secretary of Defense]. I have worked in both. 
They serve a different purpose. The DT&E professionals within 
the services are typically funded by the program managers 
[PMs], whether it is acquisition programs, MAIS, or whatnot, 
and they are potentially subject to biased reporting due to the 
pressure from the PMs. I have seen it firsthand. The OSD DT&E 
organization is not funded by the program managers. They are 
mission funded and therefore independent.
    And the last area to talk about is business systems versus 
tactical weapons systems. It is important to make a distinction 
between cyber and IT/IA [information assurance] policies for 
warfighting systems and those pertaining to business systems or 
corporate enablers like email, common business systems, and 
cloud applications. Technical laboratories and communities 
should be held accountable for making recommendations for IT 
consolidation and savings, but should control their own destiny 
in determining the best solutions. There is not a cookie-cutter 
approach.
    To ensure there is an appropriate focus on the cyber 
systems engineering, it is now time to pair a traditional CIO 
function with an RDT&E [research, development, test and 
evaluation] warfighting system cyber assessment function, which 
is focused on the tactical weapon systems and RDT&E 
infrastructure.
    That concludes my remarks, and I look forward to answering 
your questions.
    [The prepared statement of Mr. Greer can be found in the 
Appendix on page 46.]
    Ms. Stefanik. Thank you, Mr. Greer.
    We now turn to members' questions. My question is for all 
three participants in the panel today.
    Industry best practice calls for the use of leading 
indicators as a way to measure the health and performance of a 
sector; however, DOD, as you know, tends to focus more on 
lagging indicators because they are easier to measure, but they 
don't do a very good job of helping decision makers decide if a 
program is performing well or not, at least not until it is too 
late.
    What kinds of leading indicators should we be using to 
measure IT programs as well as performance of our acquisition 
systems? And what are the obstacles to being able to collect 
that sort of information currently?
    Mr. Halvorsen. One of the first things we ought to do is 
concentrate on money. That is what industry does. We do look at 
how is the money being spent, and if you find that you are 
starting to have undocumented rising cost, that is generally a 
pretty good indicator that something is wrong.
    Performance is the next best thing if you look at industry, 
they will look at performance. And it is not the performance of 
how the acquisition process worked. You ask what are one of the 
problems. We grade the acquisition process.
    Now, I am going to say everybody in this room has fault in 
that. I did, and so does the legislation, because it requires 
that I had to report on how I was doing in the process. Most 
acquisitions, the day they were finally declared unfit, still 
had green in checking the acquisition process. That doesn't 
make a lot of sense to me.
    So why aren't we looking at performance in the field? I 
mean, industry, if I am going to grade it, I grade it: Did it 
meet the customer requirements? That ought to be the number 
one, and track the money. If we did those two things, we would 
absolutely have a much better system today. And then get out of 
the rules and laws that require that we have to track and 
produce: Is the acquisition process itself being followed? The 
acquisition process is part of the problem.
    So I really think if we followed--you could take these 
templates right off industry guidelines, we are not different 
in that respect, and track the actual outcome performance 
results, not the process.
    Mr. Levine. I think that when we look at the DOD 
acquisition system not just for IT but for anything, we need to 
recognize that we rely on our contractors for almost all the 
data on their performance. So we need to be careful about what 
kind of data we prescribe, because if we say we want more data, 
then we are adding a contract requirement and a new regulation 
to a system where we don't need a whole lot of new regulation.
    I would add to what Terry said about tracking dollars, 
though. It is important not just to be able to track dollars, 
but to be able to link dollars to performance. And certainly 
with the acquisition system in general, I have always felt that 
earned value management systems are one of the most important 
things that we have, because they don't just track how much 
money you are spending, but track it against measurable 
milestones so you can see whether you are progressing in the 
way you thought that you should be progressing.
    Now, some of our contractors have earned value management 
systems, some of them don't. It depends on whether--you know, 
if they are government contractors, if you are a Lockheed or a 
Boeing or somebody like that, you will have an earned value 
management system. If you are a commercial contractor, you may 
not have one that matches government standards. So we may need 
to figure out how we can work within data that is otherwise 
already provided and collected by commercial industry to 
determine these things without imposing new requirements.
    Mr. Greer. Industry does a very good job at quantifying 
their requirements. They stay rigid with their requirements. 
They developed a product based on quantifiable outcomes. They 
also make it a priority to make sure their senior leaders are 
proactively engaged. They are on-site where the decision time 
is much shorter.
    Proprietary data is an area that, as Peter mentioned, is a 
concern. In some instances, it is difficult to get our hands on 
the data, and when we do, it costs us dearly. So that is an 
issue.
    The last one is a partnership, perhaps, an integrated 
product team with industry so we don't have to worry about 
lagging indicators. We will be right there working with the 
industry developing the leading indicators and monitoring the 
leading indicators.
    Ms. Stefanik. Thank you.
    I now yield to Mr. Langevin.
    Mr. Langevin. Thank you, Madam Chair.
    And I want to thank all of the witnesses for your 
testimony.
    So to carry on on this point, in your opinion, what do you 
think is the single most important action both Congress and the 
new administration can take to enable a rapid modernization of 
IT? I know it is a broad question, but kind of following on 
with our discussion. Please give us your insights.
    And then the other one, I will say, in your opinion, what 
cultural barriers, such as risk aversity or service 
preferences, exist within the Department and what 
recommendations do you have to overcome them?
    Mr. Halvorsen. So the first thing I would say is we need to 
push the authority to acquire down, and we can do that in small 
numbers. So, I mean, here is an interesting thing. In my 
previous job, I was the CIO. I had a $37 billion budget. 
Approved that. It is the only place in the DOD where as a 
single entity, I as CIO, I had to approve that budget, write 
that budget, yet I couldn't authorize directly a million 
dollars if I saw great technology to put right on the table. 
That doesn't seem to make a lot of sense to me. I think that is 
the first thing I would do.
    Smaller numbers, control. Give it a limit of maybe $10 
million total that the CIO could say, ``Yes, that is great 
technology, I want it,'' and waive some of the requirements 
that an institution as big as DOD has to review every piece of 
that against some competition criteria. That does not make 
sense.
    Part of the time, if I can see best practices in industry, 
and that is where industry is going, I ought to be able to say 
as the CIO, ``Industry CIOs do that, they say this is where 
industry is going, I am making the decision,'' and we go. We 
could do that, try that, small amounts of money. I think that 
would really drive some rapid acquisition quickly.
    The second thing, your question about what else do we need 
to do, I would really recommend that this committee establish a 
group, and I think in 90 days they could get some very good 
results. But stack it with both industry and military business 
systems owners, not just the technical guys, because in the 
end, back to our original question, I never had my customers, 
DOD, soldiers, sailors, marines, and civilians are not real 
shallow, and they are not real--they will tell you what the 
problems are really quickly in language you will get to 
understand quickly if you go ask them. We don't ask them 
enough.
    And so I think that is the other problem. Let's go out and 
ask the customers. Bring some customers on this panel, bring 
some industry experts, bring some leadership, 90 days I think 
you could get a really good result on here are some of the 
specific things you could do to really get at the cultural 
issue, because this is cultural.
    Even when it is fixed and even when we have the authority, 
the whole system, which includes the DOD, the way we inspect 
and review, and, frankly, includes the way GAO [Government 
Accountability Office] inspects and reviews, we are using the 
mindset that was 20 years old.
    We have got to change that. That will take time. But I 
think this committee, because it has done it in the past, could 
establish that a committee take a review and put some things in 
legislation and then follow up on it with your personal 
attention, and that would get some good results.
    Mr. Langevin. Thank you.
    Mr. Levine.
    Mr. Levine. I agree with Terry, that I think that the real 
focus ought to be on planning and prioritization so that you 
are spending the money in the right place. We tend to focus in 
a hearing like this on what is wrong with DOD's policies and 
practices and how can we make things better through that and we 
can make things better.
    But if you are really looking for how are you going to 
improve the IT environment at the Department of Defense, 
whether it is in business systems or weapon systems, the issue 
is, we have a lot of low-hanging fruit out there, things that 
need to be improved. We don't necessarily spend our money in 
the right places for the right priorities, and that 
prioritization, I think, is the number one task that we have.
    Mr. Greer. In response to your first question, contracting 
is, in my mind, a major impediment, whether it is acquisition 
weapon systems or IT systems or whatnot. If you look at the 
entire food chain from A to Z in terms of delivering an IT 
agile system, it starts with the statement of work, then it 
goes into a do-loop in the contracting arena because of 
policies. It is not the executors, it is the policymakers, 
whether it is an OSD or the echelon command within the services 
on more pointed here at Congress, legislation that has been 
burdened on the contracting community. Conducting a zero-based 
review of all of the contracting policies certainly might be 
one approach.
    I mentioned earlier about the CIO. My personal view is--and 
I am an engineer, I have the technical background more so than 
the IT side of the house--but certainly the CIO has such a 
broad responsibility. That is another long lead time approval 
cycle. It has been from my experience.
    In terms of cultural barriers, a lot of the government 
organizations do have in-house software development teams. And 
sometimes they make less than informed decisions regarding 
whether or not they should develop in-house software solutions 
or go out to the commercial sector. And I believe that is a 
cultural issue that ought to be addressed.
    Thank you.
    Mr. Langevin. Thank you.
    I yield back. I will have additional questions if we go a 
second round.
    Ms. Stefanik. Mr. Scott.
    Mr. Scott. Thank you, Madam Chair.
    Gentlemen, thank you for your service.
    And, Mr. Greer, you mentioned the proprietary data and how 
much--you said it costs us dearly, I believe is the term that 
you used. And regardless of what weapon system it is or IT 
system it is, I mean, we have seen examples in the past where 
we as the government, we paid for the research, we paid for the 
development, and then in an error in the contracting, even 
though we paid for all of it, some private company ends up 
owning that data.
    I assume that we are not making that mistake anymore? Is 
that something that we have corrected in the contracting?
    Mr. Levine. Congressman, I would be happy to respond to 
that.
    Congress did change the rules on that several years ago, 
about 3 or 4 years ago. I was with the Senate Armed Services 
Committee then. But you changed the rules. In particular, there 
was a rule that said the way that rights and technical data 
work, if it is 100 percent developed through private 
expenditure, the private company owns it; if 100 percent 
developed at government expenditure, the government owns it; 
and if it is mixed expenditure, we get just government purpose 
rights.
    But those rules also said you have to identify in the 
contract up front which data you are going to have ownership of 
as the government. So even if it was developed exclusively at 
government expense, if we failed to identify it up front, we 
didn't own it, and the contractor would come back and tell us, 
``Well, we own it now because you didn't say in the contract, 
so you are going to have to pay through the nose to use what 
you paid for in the future.''
    We changed the law so that it said, given that we were 
supposed to own it in the first place, if we failed to identify 
it in the contract, then the amount that we will owe you to 
provide us that data is the amount that it takes to produce the 
data. We are not going to have to pay you monopoly rights to 
buy it from you again. So we did make that change--or you did 
make that change--in the Congress a couple of years ago
    Mr. Scott. Okay. I just know that as somebody who 
represents a depot, this is a problem for us in supply chain 
management in some of the older systems out there where we paid 
for the development of a product and we are having to then pay 
for the schematic to manufacture the part because they don't 
have it anymore.
    Mr. Levine. It has been a huge problem. It should be better 
going forward, but of course that doesn't get us all the 
technical data from contracts in the past that we wrote under 
this old system.
    Mr. Scott. Absolutely.
    Mr. Greer. And I might add for just a couple of seconds 
here, even if it is just one piece of small source code within 
a weapon system, that will hold us hostage, and that is what is 
happening today. So you can have all of the lawyers get 
together and come up with compromises, generate legislation, 
but at the end of the day, the major prime contractors are 
still holding the government hostage.
    Mr. Levine. The change that we made a few years ago, the 
change that Congress made a few years ago also attempted to 
address that by saying that you have to segregate the smallest 
piece of technical data that you actually spent your money on 
and isolate that and tell us how we can work around it so that 
you can't tell us that we have to pay for all of the technical 
data or rely on you for all the technical data. So there is an 
attempt to address that as well.
    There is also currently, I believe, Congress----
    Mr. Scott. Is that attempt effective, or do we need 
different language to make it more effective?
    Mr. Levine. I believe it is effective, but it needs to be 
implemented and practiced over a period of time to see where 
the problems are. Congress has established a technical data 
panel to look at these issues. And I suspect that, with all due 
regard to my friends in the contractor community, they are 
going to try to reverse some of these changes. So I would urge 
you to be very alert to the recommendations of that panel and 
make sure that there isn't backsliding on those issues that you 
are concerned about.
    Mr. Scott. If you would provide additional information to 
us on that, I would be very appreciative of it.
    [The information referred to can be found in the Appendix 
on page 65.]
    Mr. Scott. Gentlemen, it took the Army 10 years to decide 
on which new pistol they were going to use. I don't know if it 
was because of the law, the bureaucracy. But the iPhone came 
about about 10 years ago, the original iPhone. Flip phones were 
still popular back then. With technology there is certainly a 
timeline that we cannot operate in. Is it the law or is it the 
bureaucracy that needs to be changed?
    Mr. Halvorsen. Yes. Some of the laws still need to be 
restructured on that. There are some things that still treat 
off-the-shelf technology like a weapon system. We have got to 
stop doing it. We really do need to say off-the-shelf 
technology, commoditized technology, does not need to be 
treated like a weapon system, but then we have got to get 
through the bureaucracy.
    Mr. Scott. Do you have any specific language that you could 
suggest, provide for our committee?
    Mr. Halvorsen. I will shoot you--yes, sir, I will do that.
    [The information referred to can be found in the Appendix 
on page 65.]
    Mr. Scott. I would appreciate that.
    Madam Chair, I am down to about 20 seconds. I will yield 
the remainder of the time.
    Ms. Stefanik. Thank you.
    Ms. Gabbard.
    Ms. Gabbard. Thank you. Thank you, gentlemen.
    I wonder if you can comment on what impact, if any, you 
think that Buy America provisions have on obtaining the best 
possible technology.
    Mr. Levine. It has always been a concern in the acquisition 
community that if you have Buy American provisions that become 
too restrictive, it will limit our ability to get the best 
weapon systems for our warfighters.
    The acquisition community always has a bias toward getting 
the best possible for the best possible price. As long as the 
Buy American provisions are in the area where they are now, the 
acquisition community has been able to work around them; but 
there is concern if you ramp it up too much, it might impede 
their ability to do what they need to do.
    Mr. Halvorsen. I will echo what Peter said. There are some 
cases where, unfortunately, you can't Buy American in much of 
the technical side.
    I think the other thing we want to talk about here is that 
part of the Buy American also addresses some security concerns. 
What we really ought to be looking at, and this is maybe a bad 
way to say it is, where don't we want to buy? Very candidly, I 
don't want to buy stuff, I didn't want to in my last job, that 
was manufactured in certain nation-states. I think that needs 
to be more of the focus here.
    And what I would say, the words here, we might in the 
technology area want to change that to Buy American and allied. 
The other problem we have in the technology communications 
business is we don't go to war alone, the DOD does not, and we 
need to be able to communicate effectively with allies. So that 
is something we have really got to make sure that agency by 
agency we don't put out Buy American rules that would preclude 
us from then communicating very well.
    I also think there is great benefit if we focus this around 
Buy American and allied and how we could do some better 
arrangements for everybody in terms of cost, delivery times, 
and production.
    Mr. Greer. Outsourcing integrated circuitry is a concern 
that I have had for quite a while from a cybersecurity and 
trusted systems point of view. We are second to none when it 
comes to manufacturing integrated circuitry and mass producing 
or producing small quantities. But the outsourcing of the large 
quantities is something that we all have to be concerned about 
only from malware perspective and other reasons.
    Mr. Levine. I would echo what Terry said, though, that 
there is a difference between worrying about your supply chain 
vulnerability, where you are worried about countries of 
concern, and what you do with Buy American, which is to say 
that all countries are off the table.
    Ms. Gabbard. Right. With regard to, you know, you 
mentioning Buy American and allied, what are some of the 
approaches that our allies are taking with regard to some of 
these challenges with workforce and improving their own speed 
of acquisition? Are there lessons we can learn?
    Mr. Halvorsen. There are. So an example, like the U.K. 
[United Kingdom] and Australia, their CIOs are permitted up to 
I think it is 10 million pounds to look at existing technology 
and say: Hey, this would fit. Let's buy a small amount of it. 
Let's test it. Let's put a performance objective around it and 
track the cost performance to Peter's.
    They have to have those, but they can make the commitments 
right on spot to do that. That makes them a lot more attractive 
to new technology, because if you are a startup new technology, 
you are on a 6-month funding cycle. If you don't get that 6 
months, you are walking away. You have to. So that is something 
we could do.
    They also have changed their workforce construction. My 
counterpart in the CIO at the U.K. until just recently--he 
retired March 31--was what was they called a crown contractor. 
To get around the pay gap issues, they were able to negotiate 
not quite a civilian salary, but better than the government 
salary, and they were able to hire him, give him full civilian 
authority to make the decisions. But he came out of both 
industry and the military, he had been a retired two-star, he 
did 5 years with British Telecom. I think that is kind of what 
we want to be able to bring into the government, people who 
have both government and industry experience. You need both to 
succeed at the top level.
    The Canadians are doing the same thing. And the way they 
are doing their supply chain also, those nations which are in 
Five Eye, and there is only one nation that doesn't 
have this authority, and it is probably the leading nation in 
the Five Eye community, which would be us, they are allowed to 
look at a computer, say, check a supply chain.
---------------------------------------------------------------------------
     ``Five Eyes'' (or FVEY), refers to an intelligence 
alliance involving Australia, Canada, New Zealand, the United Kingdom, 
and the United States.
---------------------------------------------------------------------------
    So there are some U.S. companies that I wouldn't want to 
buy from because when you look at their components, they are 
all from countries I don't want. That is the most important 
thing. So they are allowed to say: Hey, if that is a U.K. 
company, but all of their parts are sourced out of a place we 
don't like, well, then we are going to go buy an allied 
company. We ought to have that kind of flexibility restriction 
to make those decisions when it is in the best interest of 
defense.
    Ms. Gabbard. Thank you.
    Mr. Levine. One thing that the last administration did, if 
we want to look to a domestic example rather than overseas, is 
to have teams of IT experts who came in from Silicon Valley to 
help DOD and other agencies look at specific problems and 
identify solutions. So they would identify commercial 
solutions. Sometimes they would write software themselves to 
address it.
    I believe that the new administration is considering 
similar kinds of steps. But that gets you ahead of a cycle. You 
know, you don't have to take 5 years----
    Ms. Stefanik. We are running out of time, Mr. Levine.
    Mr. Levine. Thank you.
    Ms. Stefanik. Dr. Abraham.
    Dr. Abraham. Thank you, Madam Chair.
    Gentlemen, thank you for your direct testimony. It is 
appreciated. There was some great solutions or possible 
solutions mentioned in you-all's testimony. And as we know, 
industry has to make a profit. They have a board of directors 
that they report to. And, unfortunately, in the DOD or any 
bureaucracy, that word is not even in their dictionary. And, 
you know, that is a problem.
    Again, it all comes down to accountability. And when you 
have a budget, Mr. Halvorsen, of $37 billion or whatever and 
you can't write a check for something that you see, it is just 
idiotcracy at its finest.
    So it is a national security issue. And I understand we 
have to maintain qualitative and quantitative advantage over 
our enemies. But for these programs that we could just 
literally pick off the shelf and save literally hundreds of 
millions of dollars, it just--it bodes poorly for us in the 
government for us to allow this to happen.
    Are there any examples, can you give me some examples of 
some successful industry partnerships?
    Mr. Halvorsen. Well, you know, I hesitate, because some of 
those I was directly involved in. But I will tell you, the 
Microsoft partnership with DOD, the ability that Microsoft let 
us, looking at the license agreement, let me at that point make 
the decision that the entire Department could go to Windows 10, 
get to a single operating system, without having to renegotiate 
the current contract, that was a partnership with Microsoft.
    That what they saw down the road, and I think this is going 
to happen if the government goes to Windows 10, to one of the 
Representative's comments earlier, it will actually facilitate 
then the follow-on move, which I know is being planned, into 
the 365 and public cloud domain.
    So Microsoft saw that as investment. It was a good, open 
dialogue. That was very good.
    Again, I hate to say it, I will go on the record, I am now 
working for Samsung, but one of the things that happened early, 
we put out a capability statement that said we needed a secure 
smartphone. That is all we put out. Samsung came in. We worked 
with them. We actually co-developed what today they feel is 
their KNOX security system. It kept everybody's prices low. We 
were able to field that at the secret and now top secret level 
in less than 24 months. The products, which are rare in my 
history, were delivered on time and at cost. And in the end, 
they actually cost less than we thought they would.
    Dr. Abraham. That leads somewhat to the second question I 
have. Mr. Scott referenced the iPhone 10 years or less. And, 
again, these products are being developed by a different 
generation of entrepreneurs, engineers, IT people. What are we 
doing to attract some millennials into the fold?
    Mr. Halvorsen. Not enough. We are going to have to address 
a couple things on that. One of them is the pay gap. I will 
tell you, I lost my director of security. You know, he went and 
he quadrupled his salary after spending, you know, 32 years 
serving the government. Good.
    That is one thing we are going to have to address. But we 
are also going to have to address some of the other things that 
the millennials want, which is they want their technology.
    One of the things that when I went in my old job and now in 
my new job, I talk to millennials, ``Why don't you look at 
DOD?'' ``Well, because they will take my phones away. I won't 
be able to do the things I want.''
    Now, that is not true as much anymore. We are using 
multifactor phones, those type of things. But there is the 
perception that the government lags greatly in the ability to 
be flexible in the workplace.
    We have got to get that message out. The rules are there. I 
mean, the laws are there. I mean, I could let my people 
telework. I could do those things. It just wasn't the way we 
thought. That is a cultural issue. The laws are there to make 
that happen.
    That is something that maybe this committee and other 
committees ought to be asking some questions about what are we 
doing on the millennials, what are we doing to be more flexible 
in the workplace? There are some good answers to that, but that 
needs to be the holistic plan, not just individual efforts.
    Dr. Abraham. A minute left. Mr. Levine.
    Mr. Levine. If I could just add one thing to that. We don't 
take advantage enough of the things that we do have to offer. I 
think that the most attractive thing for the Department of 
Defense for people who work there is the mission and the 
feeling of commitment to the mission and that they can make a 
difference. And I think that we don't take enough advantage of 
that. We don't sell that well enough. When we treat civil 
servants as bureaucrats who are just too much, there are too 
many of them, we need to get them out of the way, we need to 
fire them and cut their pay, it doesn't create the image that 
people want to see of themselves. And we need to do a better 
job of selling the mission to get the kind of people we want.
    Dr. Abraham. Very good.
    Mr. Greer.
    Mr. Greer. And just one or two points here. In the IT 
community, I believe that we have to raise the level of our 
professional series. We have a lot of nonprofessional IT 
specialists, and they serve a very good purpose, but where we 
are today, what the cyber vulnerabilities and resiliency 
challenges, I think making sure that we hire enough 
professional series in the IT community is important. And it is 
not necessarily the pay that is attracting, like Peter said, it 
is the quality of work for these millennials.
    Dr. Abraham. Thank you, gentlemen.
    I yield back.
    Ms. Stefanik. Mr. Larsen.
    Mr. Larsen. Thank you, Madam Chair.
    Mr. Greer, can I talk a little bit about your build-versus-
buy conundrum in your testimony, in your written testimony? I 
am sorry for being late, you may have addressed this. But I 
wanted to ask you in your experience on this build-versus-buy 
tension generally are there some criteria that we should be 
thinking about where there is a leverage point within the DOD 
where they ought to build versus they ought to buy in general?
    Mr. Greer. That is a difficult question to answer in a 
short period of time, mainly because there is a hybrid 
solution. There are a lot of commercial software packages like 
SAP that we do purchase, but we use government people to 
program it and code the SAP system. And so that is always an 
issue.
    Mr. Larsen. Sorry. Mr. Halvorsen looks like he is excited 
to answer too. But go ahead.
    Mr. Greer. Well, I am just expressing my views.
    Mr. Larsen. That is great.
    Mr. Greer. Okay. And this is why----
    Mr. Larsen. So then continue.
    Mr. Greer. But in terms of having firm requirements and 
quantifiable outcomes and making that make-buy decision, it 
gets difficult within the government, mainly because of 
cultural issues. There are a lot of software programmers in the 
government, and they inherently lean in the in-house solution 
side.
    Mr. Larsen. So let me ask you this then. Are there specific 
build-versus-buy decisions on cloud computing? Is that treated 
differently than, say, operational software like on the 
Microsoft decision?
    Mr. Halvorsen. It should be, but it is generally not the 
culture. But I want to be very clear on this: If you could buy 
it, buy it. I did not find that to be as difficult for anything 
that wasn't a weapon systems as [inaudible]. If you can buy it, 
we should buy that, and we should put--and we really did try to 
do this.
    I mean, one of the plaques over at my desk was 
``customization bad.'' You should fight every attempt we take 
to customize that software off the shelf, because every time we 
do it, it is a lifelong contract for contractors without much 
gain. Our business processing systems, back to Peter's point, 
don't differ that much from the best in industry. They 
shouldn't, if they do.
    So one of the first criteria is, if it is a business 
system, you should have to come in with a different business 
case that says why I am not buying that solution, why am I 
building any specific software.
    It is getting that way for many communication systems, 
because we are using off-the-shelf communication. If it is off 
the shelf, really consider why you need to put something extra. 
And in the communications it might be for cybersecurity, but I 
have found if I go to industry, when I did that, if I told them 
I wanted that extra, most of the time they were happy to put it 
in, and most of the time they carried that in their commercial 
product because it was going to sell more as security becomes 
more important.
    Mr. Levine. The track record of the Department is, if we 
are buying a commercial-type business system, if we start 
tailoring it, we end up spending more money on the tailoring 
than we spent on the system in the first place. It just doesn't 
make sense to do anything unique unless you absolutely have to.
    Mr. Larsen. I am sure if we went back to the first hearing 
that I attended on this issue when I was first elected in 2001 
a lot of the answers would be the same.
    On the build versus buy then does that principle, can that 
principle apply to the people? For instance, in Washington 
State, we have the example of the National Guard working with 
private sector and public sector folks on cybersecurity and 
doing red teaming and doing some mapping out of how to approach 
defense. And some of these folks are reservists and they are 
trying to attract folks in for 2 weeks a year, two weekends a 
month, in a sense buying people as opposed to building them 
internally. Does that model work in terms of the IT personnel 
acquisition side, if you are acquiring people?
    Mr. Halvorsen. It certainly needs to be part of your plan. 
It can't fill all the balance. And there are some security 
issues that you might not want to put everybody that way. But 
absolutely we need to be taking advantage of that.
    If you look at our last sets of operations, you look at the 
Reserves we put on board, many of them were with those 
technical skill sets. As the CIO when I was there, I would 
reach out and bring Reserves on who were working for Google, 
working for whatever big company. And it is actually part of 
the deal, I will give DOD great credit, they actually have that 
in their plan.
    Continuing that into other areas I think makes sense with 
caution. It is not as easy to do that with people as it is with 
software, but it is something that we certainly ought to be 
looking at.
    Mr. Greer. Contractor support is an area that we need to 
leverage even more. There is always a need for inherently 
governmental personnel to be able to represent the government. 
But I think we can leverage more on the contractor support 
side, especially as your workload goes up and down. You are 
able to make adjustments much more quicker.
    Mr. Larsen. Yeah. Thank you.
    Ms. Stefanik. Thank you very much to each of our three 
panelists, Mr. Levine, Mr. Halvorsen, and Mr. Greer. And thank 
you to the members for their thoughtful questions. This is a 
critical issue that falls under this subcommittee's 
jurisdiction that we will continue to focus on as we prepare 
and draft this year's NDAA [National Defense Authorization 
Act], and we look forward to continuing to engage with you to 
get your suggestions.
    Thank you very much. This hearing is adjourned.
    [Whereupon, at 2:57 p.m., the subcommittee was adjourned.]

      
=======================================================================




                            A P P E N D I X

                             April 26, 2017
      
=======================================================================


              PREPARED STATEMENTS SUBMITTED FOR THE RECORD

                             April 26, 2017

=======================================================================

      
[GRAPHIC(S) NOT AVAILABLE IN TIFF FORMAT]


      
=======================================================================


              WITNESS RESPONSES TO QUESTIONS ASKED DURING

                              THE HEARING

                             April 26, 2017

=======================================================================

      

             RESPONSES TO QUESTIONS SUBMITTED BY MR. SCOTT

    Mr. Levine. The statutory provision to which I referred in my 
testimony is section 815 of the National Defense Authorization Act for 
Fiscal Year 2012. As I indicated, this provision:
      Ensures that DOD may require the delivery of data 
generated in the performance of a contract that is necessary for the 
purpose of reprocurement, sustainment, modification or upgrade, even if 
such delivery was not required by the original contract. In exchange 
for the delivery of the data (which was generated at government 
expense) the government is required to compensate the contractor only 
for reasonable costs incurred to covert and deliver the data in the 
appropriate form.
      Requires the contractor to provide technical data ``is 
necessary for the segregation of an item or process from, or the 
reintegration of that item'' with the rest of the system. In other 
words, if a contractor wants to withhold data on a widget that was 
developed at contractor expense, it has to tell DOD how to work around 
the widget, so that the rest of the technical data, which DOD owns, 
won't be useless. Since I am no longer with the Department, I am unable 
to give a report on the extent to which this provision has been 
successfully implemented. However, section 813(b) of the National 
Defense Authorization Act for Fiscal Year 2016 establishes a 
Government-Industry Advisory Panel to review statutory requirements 
regarding technical data rights. Some may seek to use this panel to 
unwind the 2012 reforms.   [See page 13.]
    Mr. Halvorsen. Language that stresses DOD should use commercial 
business case analysis to include review of DOD business process when 
procuring what is clearly a commercial solution to include hardware, 
software and IT service contracts. Something like the following:
    ``DOD will apply commercial best practices when conducting business 
process and detailed market analysis when procuring readily available 
IT hardware software and services in support of or in replacing 
business systems. If DOD can demonstrate through business intelligence 
and analysis a clear product/service leader(s) in a specific area DOD 
can pursue sole source or limited partnership procurements. These facts 
can include the ability to support a large enterprise like DOD, similar 
to how other large commercial companies develop procurement 
partnerships.
    An example of this would be using the MICROSOFT based Services 
Cloud supported by MICROSOFT 365 and Azure. This is the leading cloud 
solution for the fortune 500 and clearly dominates the market and would 
be considered a best of breed solution. If DOD can make this case it 
should be allowed to by-pass current acquisition rules and go direct to 
procuring a best of breed solution, negotiating and spending time on 
obtaining the best value (pricing and performance) instead of defending 
and deliberating on multiple of solutions. This doesn't mean the 
MICRSOFT would be the guaranteed provider, it does mean that MICROSOFT 
products/services would be the used to form the service cloud 
architecture and operating system.
    I want to make it clear I am not a MICROSOFT employee and receive 
no compensation from MICROSOFT for these recommendations. My current 
employer Samsung receives no direct benefit as a result of these 
recommendations. These recommendations are aligned with statements and 
testimony I made as the DOD CIO. I use MICROSOFT as it is the most 
compelling example and in my opinion moving to the cloud is a necessary 
and critical step to improving DOD security, greatly improving 
productivity and saving dollars.
    With respect to all of the responses I have provided to the 
Questions for the record, they represent my personal opinions and 
beliefs and are in my own words. I am currently an employee of Samsung 
Electronics of America. The responses I have provided have not been 
endorsed, constrained of approved by Samsung Electronics of America or 
any other entity.   [See page 13.]
    Mr. Greer. There are a few acquisition programs that have had 
difficulty obtaining data/software/interface rights by prime 
contractors. Despite language that stipulates the Government has 
purpose rights if Government funds are spent to develop a deliverable, 
industry still presses back when the Government attempts to exert its 
rights. Industry is interpreting that if a deliverable contains their 
proprietary information or reveals the workings of what they claim as 
their intellectual property (IP), they may limit access to it. 
Additionally, for software intensive deliverables, industry has started 
`bundling' IP with the deliverable to the government, and claiming the 
government has less than purpose rights to the overall product. This is 
leading to complex licensing issues for deliverables the Government 
should have purpose rights to. This has happened on F-35 as well as 
many other programs. Language should be clarified to state the 
Government has no less than purpose rights to the code segments (source 
and executable) and interfaces used in an overarching deliverable 
funded in any part by the Government.
    From an acquisition oversight perspective, with the KC-46A program, 
Boeing made it clear during contract negotiations that they would not 
use Boeing 767 reliability data to meet the government requirement. 
That sounded like the right approach at the time given the KC-46A 
differences in configuration, but in practice it meant that Boeing will 
not share any reliability test data from their Boeing 767-2C (the 
provisioned freighter) either from Boeing DT&E or FAA Amended Type 
Certificate (ATC) certification testing. Although we do get some 
anecdotal failure data from these flights in which the government 
participates, it is purely word of mouth from those attending the 
briefings or flying the tests. We do have reliability data from 
flights, which are used to meet contract specifications. The net result 
is that we have very incomplete reliability data. The DASD (DT&E)'s 
official MS C assessment concluded, ``As a result, the Lead 
Developmental Test Organization (LDTO) used data only from flights in 
which the Air Force participated to estimate the system's progress 
against the reliability growth curve, which presents an incomplete 
picture of the system's reliability.''
    Also, the tension over Boeing 767 proprietary data makes it 
difficult and time-consuming to obtain test data that the government 
feels is necessary to evaluate the system under test when that data 
comes from Boeing DT&E or ATC testing. When it was in Boeing's interest 
to share more quickly and ``freely'' during the boom issues prior to MS 
C, the data flowed more freely. Most of the time, it can take weeks or 
longer to get the data the government believes that it needs.
    To the best of my knowledge, Boeing has provided (or will provide) 
the data required for gov't spec verification, though it has been a 
painful process that runs through Boeing Commercial Aircraft (BCA) 
because they technically own the aircraft during EMD. As best as I can 
tell, we will never get the B-757-2C reliability data from Boeing 
internal and ATC testing. I sat in on a meeting between BCA, Boeing 
Defense, and the program office and can relate that BCA was surprised 
that we wanted the requested data. In the commercial world, BCA simply 
delivers aircraft that meet the customers' specifications. The 
commercial customers don't care how the aircraft was built and tested. 
Maybe part of this equation involves the gov't rethinking what data we 
really need, which drives back to the acquisition process and how the 
gov't buys stuff.
    The Department of Defense has tremendous capabilities to modify 
legacy weapon systems if full data rights were available. Government 
personnel could perform the Non Recurring Engineering development for 
small to medium upgrades in-house saving millions of taxpayer's 
dollars. For example, The NAVAIRSYSCOM's Naval Air Warfare Center, 
Aircraft Center has an Aircraft Prototyping Facility for modifying DOD 
weapon systems. The DOD should be taking advantage of facilities like 
this as opposed to sole sourcing modifications to the prime 
contractors. The Army has similar modification facilities.
    The lack of government access to design data related to items 
(platforms, systems, components, hardware, software, etc.) already 
purchased by the government has been and continues to be a major cost 
driver for any subsequent upgrade done to these items during their 
service lifetime. Although IT systems are certainly a critical element 
of this problem, the problem is pervasive. Asserted data rights 
involving organic government prototype development programs costs time 
and money at a minimum and, in the extreme, can simply stop an effort.
    This problem is further compounded on a broad scale by government 
laboratories and field activities that are reticent to reverse-engineer 
the data out of the government-owned items due to ultra-conservative 
legal advice predicting litigation as well as fear of Congressional 
inquiries. This is in stark contrast to the private sector where 
backing data out of owned items through reverse engineering is 
commonplace, protected by legal precedent and an essential element of 
one company modifying or upgrading another's product. Reverse 
engineering is not always practical but it is often an effective work-
around particularly in situations where the prime contractor no longer 
produces the item.
    The complexity in system acquisitions and understanding 
intellectual property rights in Modeling & Simulation (M&S) 
acquisitions, requires a clear understanding from Program Managers, 
System Engineers, and Procuring Contracting Officers to M&S Integrated 
Product Teams of the Government's license rights in intellectual 
property comprised of technical data (e.g. technical packages and other 
information relating to the developed product's design, configuration, 
or to operation, maintenance, installation, or training data) and 
computer software (e.g. executable code, source code, code listings, 
algorithms, formulae, design details). The lack of clear understanding 
among various Program Offices highlights the need for specific 
guidelines for contracting for open M&S systems and the data rights 
required. Open System Architecture is a key pillar of the Department of 
Defense's Better Buying Power (BBP) initiative to implement best 
practices to strengthen the Defense Department's buying power, improve 
industry productivity, and provide an affordable, value-added military 
capability to the Warfighter.   [See page 13.]



      
=======================================================================


              QUESTIONS SUBMITTED BY MEMBERS POST HEARING

                             April 26, 2017

=======================================================================

      

                  QUESTIONS SUBMITTED BY MS. STEFANIK

    Ms. Stefanik. In addition to investments in IT to improve its use 
of data to inform decision-making, DOD also needs to address the 
cultural barriers that hinder effective information sharing throughout 
the Department. For example, RAND found in 2015 that ``institutional 
structure and bureaucratic incentives to restrict data access are 
exacerbated by policy and guidance to protect information. The result 
is a strong conservative bias in labeling and a reluctance to share.'' 
In your opinion, what are the initial steps that the Department needs 
to take to address these cultural barriers, and who should be 
responsible for leading this effort?
    Mr. Levine. There are both good and bad reasons for restricting 
data. Some restrictions on access to data are appropriate to safeguard 
classified and sensitive unclassified data from unauthorized 
disclosure. Widespread dissemination of such data to those without a 
need to know, even within the Department, could lead to compromise.
    Other restrictions appear to be imposed for more parochial reasons: 
program managers, PEOs, and component acquisition executives may seek 
to retain control over their programs by controlling the flow of data 
to Comptrollers, to CAPE, and even to others in their chain of command.
    Before I left the Department, DOD was seeking to address this issue 
through a coordinated ``big data'' project. The objective of the 
project was to identify data elements are needed by users at all levels 
of the acquisition system, as well as by outside stakeholders in the 
Department, and standardize that data to increase transparency. If 
successful, this project should help address the problem identified by 
the RAND review.
    Ms. Stefanik. In addition to investments in IT to improve its use 
of data to inform decision-making, DOD also needs to address the 
cultural barriers that hinder effective information sharing throughout 
the Department. For example, RAND found in 2015 that ``institutional 
structure and bureaucratic incentives to restrict data access are 
exacerbated by policy and guidance to protect information. The result 
is a strong conservative bias in labeling and a reluctance to share.'' 
In your opinion, what are the initial steps that the Department needs 
to take to address these cultural barriers, and who should be 
responsible for leading this effort?
    Mr. Halvorsen. Representative, when I was the DOD CIO and I believe 
it is still true, that DOD is making good strides in this area and this 
is currently being led by the Deputy Chief Management Officer and fully 
supported by the SD and DSD. I would encourage Congress and DOD to work 
with industry to look at the current laws and rules that restrict 
government employee direct interaction with industry and restrict the 
sharing of information with industry. Industry should also be 
challenged to share data more openly with DOD and the government in 
general. One specific recommendation I would make is for DOD supported 
by legislation to include industry more directly in developing policy/
instructions with respect to business systems. While DOD CIO I 
explicitly involved industry in developing the cloud policy and 
security guidelines. This was challenged by many groups inside and out 
of DOD but the resultant document was well received and is allowing in 
this area much better communications between industry and government.
    Ms. Stefanik. What is the DOD role as it relates to developmental 
test and evaluation for IT systems that are predominantly commercially 
based and where DOD is integrating, but not actually developing new 
capability? Is there a particular balance in developmental testing 
between what DOD is responsible for and what industry should be 
responsible for that can reduce duplication in effort, cost and 
schedule performance?
    Mr. Greer. Yes, there is a balance between what testing DOD is 
responsible for and what performance certifications can be provided by 
industry and accepted by the test community. DOD tests are conducted in 
mission relevant environments to ensure overall capability is effective 
and suitable and to assess resilience to CYBER threats. Component level 
testing of individual COTS hardware components may often be conducted 
at the supplier level and accepted by the Government.
    DOD is in the process of acquiring multiple defense business 
systems (DBS) based upon commercial enterprise resource planning (ERP) 
systems. The first priority is to conduct business process re-
engineering to try to change DOD processes to fit the commercially 
offered systems and minimize customization of the ERP. In most cases, 
the unique nature of the DOD identifies gaps between the DOD process 
and the ERP capabilities. In this case development of special report, 
interface, conversion, extension, form, and workflow (RICE-FW) software 
objects are required.
    In these cases, developmental testing of the DBS focuses on the 
RICE-FW and integration into the DOD facilities and networks rather 
than the basic capabilities that are proven by use in industry. 
Mission-oriented developmental testing using actual users performing 
mission tasks in a test environment allows integrated testing with the 
operational testers, early identification of human factors integration 
problems, and identification of software defects in the RICE-FW objects 
with sufficient time for correction prior to operational testing.
    Additionally, DOD's roles in developmental test and evaluation of 
IT systems integration are in the realm of cybersecurity. DOD must 
ensure that the new IT system does not introduce a vulnerability to the 
DOD network and that the combined cyber defenses are integrated. Just 
because a commercially based IT capability meets industry, it does not 
mean that it is secure. The IT capability may meet standards of 
performance (certification) but it must be tested for effectiveness 
(penetration).
    DOD should be responsible for cybersecurity testing and the vendor 
must correct those deficiencies. This should be in a developmental test 
environment that allows time for the vendor to make corrections. After 
corrections are made, the IT capability is ready for operational 
cybersecurity testing. Cybersecurity is an area where the test-fix-test 
methodology is still relevant.
    Ms. Stefanik. In addition to investments in IT to improve its use 
of data to inform decision-making, DOD also needs to address the 
cultural barriers that hinder effective information sharing throughout 
the Department. For example, RAND found in 2015 that ``institutional 
structure and bureaucratic incentives to restrict data access are 
exacerbated by policy and guidance to protect information. The result 
is a strong conservative bias in labeling and a reluctance to share.'' 
In your opinion, what are the initial steps that the Department needs 
to take to address these cultural barriers, and who should be 
responsible for leading this effort?
    Mr. Greer. Cultural barriers are often difficult to overcome. A 
good method of breaking down barriers is to encourage more transparency 
with regard to ongoing developmental efforts. Sharing visibility into 
IT related projects, investments, and infrastructure may help increase 
re-use of development that would otherwise be unknown across 
organizational boundaries, reducing duplication of effort. Establishing 
a community of interest (COI) for IT related investments may be 
beneficial in fostering this kind of exchange of ideas and information. 
This is a leadership challenge, as is all cultural change. At a 
minimum, the Director, Developmental Test and Evaluation should get 
back statutory authority for oversight of ACAT ID and MAIS programs and 
to ensure access to all data in DOD and have direct access to the 
Defense Acquisition Executive with regard to his independent 
assessments of system performance and issues regarding his ability to 
make those assessments in time to effectively inform decisions.
    I believe that an organization such as the Deputy Assistant 
Secretary of Defense for Developmental Test and Evaluation (DASD 
(DT&E)) helps break down these barriers, where they exist. For example, 
the staff specialists in the DASD (DT&E) engage in multiple programs 
across all the Services. Similarities exist in some programs and the 
staff specialists, by their close association with the program office 
testers, identify lessons learned and share this information among 
other staff specialists and other programs. Where appropriate, the 
staff specialists have encouraged and facilitated direct contact 
between programs to delve into the details of the lessons learned.
    Ms. Stefanik. The 2016 Annual Report on Developmental Test & 
Evaluation stated that the Department's current knowledge management 
capabilities and processes used to gain, collect, and analyze the 
information necessary to conduct acquisition assessments and 
evaluations are deficient and ineffective for today's world. In your 
opinion, what investments in analytical tools and technologies should 
the Department prioritize in order to begin to remedy this situation?
    Mr. Greer. There is a need to transition to an Enterprise 
environment for Weapons Systems RDT&E and Cyber engineering to store, 
access, and share program RDT&E data. Establishment of a classified 
environment with the right controls in place as well as mechanisms to 
increase cross-program security collaboration is needed. With this, 
comes the need to pair an ``RDT&E Warfighting System Cyber Assessment'' 
CIO function that is focused on the tactical weapons systems impacts of 
Cyber to work alongside the ``traditional'' CIO function for business 
systems, email, common databases and promulgation of policy. 
Warfighting acquisition programs and Cyber technical work must be 
staffed by the appropriate mix of Government experts from the Systems 
Engineering and RDT&E community vice the traditional CIO community or 
corporate operations workforce. In the current complex Cyber threat 
environment, Defense Department needs have evolved far beyond 
traditional IT/IA and business systems best practices. Our ability to 
operate in the Cyber Warfare environment of the future hinges on agile 
changes to our policies, organizational structures, workforce, and 
infrastructure. Another Enterprise Solution that would benefit 
acquisition programs is a Shared Data Warehouse and Repository. 
Currently, data is stored and process on separate servers in a single 
program office. This data is isolated from many groups that can use it. 
A DOD Enterprise Solution to data storage and data warehouse tools 
would provide the necessary access for DOD acquisition programs to use 
and share data across various platforms. As mentioned in another 
question is an urgent need to gain access to industry acquisition data, 
which in most cases today is industry owned. The right Contract and 
Legal clauses to obtain access to this data freely and use it in 
government owned facilities and programs would provide the capabilities 
necessary to perform data analytics to conduct acquisition assessments 
and evaluations for all programs.
                                 ______
                                 
                   QUESTIONS SUBMITTED BY MR. SHUSTER
    Mr. Shuster. Can you tell us how many DIUx projects have 
transitioned to programs of record or other major programs within the 
Department of Defense? What is the process for transitioning prototypes 
or pilots to the services or other agencies of the Department? Is the 
contractor required to lead these efforts? What outreach does DIUx do 
within the Department of Defense to alert them to the potential 
innovations available from commercial providers?
    Mr. Halvorsen. Sir I cannot with any specifics and would recommend 
that this question be referred to DOD. I can say that I am a believer 
in the concept and programs that expand DOD presence in technology hubs 
like Silicon Valley are worth pursuing. I would also suggest that 
measuring the effectiveness is a little premature. DOD has not had this 
type of program in place much longer then a year and both DOD and the 
technology hubs are learning how to make this work. In addition to this 
program DOD has been pursuing employee exchange programs with industry. 
These programs though limited to date have been well received by DOD 
and participating companies.
                                 ______
                                 
                   QUESTIONS SUBMITTED BY MR. FRANKS
    Mr. Franks. Can you give a specific example of a good, well-
performing, or successful IT acquisition? What were the characteristics 
that made it successful? How long did it take?
    Mr. Halvorsen. Sir, the Next Generation Department of Navy IT 
services contract is a good example of a successful IT procurement. 
This was the follow on to the Navy Marine Corps Internet (NMCI) 
contract. The contract yielded about $2B dollars in savings to the DON 
and as importantly continued commercial like services for the DON. 
Planning for this took over a year but was shorter then contracts of 
like size and scope. It involved close coordination with industry 
providers, a very detailed understanding of the capabilities desired, 
and a willingness to understand trade space as related to price and 
performance. I no longer have access to the details on this contract, 
but I am sure DOD could provide. There are still some issues with this 
contract but overall I give it very high marks. The negotiation of the 
Windows 10 upgrade license agreement is another good example of the DOD 
applying good practices and working well with an industry partner, in 
this case MICROSOFT. This negotiation was truly an enterprise effort in 
the department with DOD and the services working closely with MICROSOFT 
to interpret the action as an upgrade covered by existing agreements 
and not a major systems change. This enabled DOD to not have any bill 
for the actual license, while the business case for MICROSOFT was 
having DOD be better positioned to transfer to the cloud and 
elimination of costs in supporting older systems. When completed the 
transition to windows 10 will improve security, position DOD for cloud 
transition and result in cost savings. The biggest cost associated with 
the transition to Windows 10 is equipment modernization, which had 
fallen behind in DOD, but would have to be done at some point. Doing 
the modernization in conjunction with the windows 10 upgrade, allows 
for some modernization delay while gaining 80% or more of the new 
windows 10 capability. This is the type of DOD-industry partnership 
that is needed.
    Mr. Franks. When you consider the critical need of modernizing the 
network, particularly to harden against enemy attacks and infiltration, 
shouldn't an immediate and complete overhaul of the IT procurement 
process be a priority?
    Mr. Halvorsen. Yes and this review needs to focus on both the 
requirements/capability process as well as the acquisition process. The 
current requirements process often results in very detailed and 
technically restrictive solutions being acquired that don't get DOD the 
latest solutions or the best value. Many times because of the length of 
the process and over specification, DOD is buying legacy the day the 
contract awards. The review needs to look at successful commercial 
process and encourage partnership with industry. This will present some 
problems, especially with small business, but I believe creative 
solutions can be applied to offer small business an opportunity to 
partipate in major enterprise acquisitions. DOD also needs to increase 
the dialog between the mission/business owners, the acquisition 
workforce and the industry providers. We need to look at the current 
rules restraining DOD employees from direct discussions with industry, 
many of these rules have not been updated in 30 years.
    Mr. Franks. Can you give a specific example of a good, well-
performing, or successful IT acquisition? What were the characteristics 
that made it successful? How long did it take?
    Mr. Greer. I can think of several IT acquisitions that were 
executed efficiently--the examples are described in more detail below:
    1) The Army Logistics Modernization Program (LMP) Increment 2 was a 
highly successful Defense Business System program that came to the 
Limited Deployment Decision milestone in 2015 with a 98% test case 
success rate during developmental testing. All failed test cases had 
work-arounds and corrective action plans acceptable to the user, and 
zero open severity 1 or 2 software problem reports. LMP success may be 
attributable to:
    The LMP program management office (PMO) was highly experienced, 
knowledgeable of the system, and effectively planned and executed the 
complex program consisting of three waves of deployment with multiple 
functional areas. The PMO also established a close working relationship 
with the functional user to perform business process re-engineering, 
clarify requirements, and resolve issues quickly and effectively.
    Increment 2 was a continuation of Increment 1; with the same system 
integrator using the well established and understood change management 
processes. Developmental testing (DT) included various events to 
specifically address performance, interoperability, and cybersecurity. 
Most importantly, the test strategy included mission-oriented 
developmental testing (known in the PMO as Business Operations Test 
(BOT)) that used actual users from the depots performing operational 
mission threads for their test cases. This identified human factors 
improvements that led to better user performance. The DT actual users 
became operational test (OT) trainers that resulted in a very high 
success rate for all the tasks and transactions that were tested during 
Initial Operational Test and Evaluation. The overall schedule from 
milestone B to milestone C was approximately 22 months.
    2) The Marine Corps' ACAT IAC Common Aviation Command and Control 
System (CAC2S) Phase 2 was a very successful integrated acquisition. 
The operational test community worked well with the developmental test 
(DT) community, and accepted much of the DT data that satisfied OT 
requirements, reducing the overall acquisition time by 6-months. USD/
AT&L nominated the CAC2S acquisition effort as a Defense Acquisition 
University integrated acquisition case study. Time between MS B and MS 
C was 34-months.
    3) Defense Agencies Initiative (DAI) has been a successful Defense 
Business System Program. It enables compliance with Congressional 
direction for all DOD organizations to be auditable. The Defense 
Agencies Initiative (DAI) transforms the budget, finance, and 
accounting operations of most DOD Defense Agencies in order to achieve 
accurate and reliable financial information in support of financial 
accountability and effective and efficient decision-making throughout 
the Defense Agencies in support of the missions of the warfighter. 
Defense Agencies Initiative (DAI) is a critical DOD effort to modernize 
the Defense Agencies' financial management capabilities.
    Mr. Franks. We've all heard of Moore's Law and know how rapidly 
technology advances and evolves. Do you know if there are any 
requirements for new systems we are building or purchasing to be 
capable of growing and evolving alongside technology--perhaps 
``upgradeable'' systems which won't be obsolete the moment they become 
operational or shortly thereafter?
    Mr. Greer. The rapid development pace of IT does mean that the 
Government will need to start driving requirements into new systems for 
Non-Proprietary deliverables, interfaces, and use of open systems 
architectures. Additionally, use of system virtualization and cloud 
technologies may mitigate IT obsolescence issues. Ensuring system 
designs, interfaces, and computing hardware are non-proprietary and 
open to the Government mitigates the possibility of ``vendor lock in'' 
and provides multiple flexible upgrade paths. The DISA Defense 
Enterprise Office Solution (DEOS) is a program that provides e-mail, 
voice and data conferencing, office products (e.g., word processing, 
etc.) and other capabilities with a cloud-based solution, potentially 
across the entire DOD enterprise. The acquisition strategy for DEOS 
contains a provision, which DISA calls ``Evergreen'', that will require 
the system integrator to provide upgrades, at no charge, whenever DEOS 
capability upgrades are offered commercially.
    DODI 5000.02 supports growing and evolving alongside technology 
through the use of a modular open systems approach (MOSA). It requires 
programs to identify how a MOSA will or will not be used in their 
acquisition strategy. (Sec 801 PL 113-291). This approach integrates 
technical requirements with contracting mechanisms and legal 
considerations to support a more rapid evolution of capabilities and 
technologies throughout the product life cycle through the use of 
architecture modularity, open systems standards, and appropriate 
business practices. A modular design coupled with an appropriately open 
business model provides a valuable mechanism for continuing competition 
and incremental upgrades, and to facilitate reuse across the joint 
force.

                                  [all]