    [House Hearing, 115 Congress]
    [From the U.S. Government Publishing Office]

                             [H.A.S.C. No. 115-37]

                       CREATING A FLEXIBLE AND EFFECTIVE
                             INFORMATION TECHNOLOGY
                           MANAGEMENT AND ACQUISITION
                        SYSTEM: ELEMENTS FOR SUCCESS IN
                          A RAPIDLY CHANGING LANDSCAPE

                                   __________

                                    HEARING

                                   BEFORE THE

               SUBCOMMITTEE ON EMERGING THREATS AND CAPABILITIES

                                     OF THE

                          COMMITTEE ON ARMED SERVICES

                            HOUSE OF REPRESENTATIVES

                         ONE HUNDRED FIFTEENTH CONGRESS

                                 FIRST SESSION
                                   __________

                                  HEARING HELD

                                 APRIL 26, 2017

    [GRAPHIC(S) NOT AVAILABLE IN TIFF FORMAT]

                                       ______

                              U.S. GOVERNMENT PUBLISHING OFFICE

     25-823                         WASHINGTON : 2017
     -----------------------------------------------------------------------
       For sale by the Superintendent of Documents, U.S. Government Publishing
       Office Internet: bookstore.gpo.gov Phone: toll free (866) 512-1800;
              DC area (202) 512-1800 Fax: (202) 512-2104 Mail: Stop IDCC,
                               Washington, DC 20402-0001

               SUBCOMMITTEE ON EMERGING THREATS AND CAPABILITIES

                    ELISE M. STEFANIK, New York, Chairwoman

    BILL SHUSTER, Pennsylvania           JAMES R. LANGEVIN, Rhode Island
    BRAD R. WENSTRUP, Ohio               RICK LARSEN, Washington
    RALPH LEE ABRAHAM, Louisiana         JIM COOPER, Tennessee
    LIZ CHENEY, Wyoming, Vice Chair      JACKIE SPEIER, California
    JOE WILSON, South Carolina           MARC A. VEASEY, Texas
    FRANK A. LoBIONDO, New Jersey        TULSI GABBARD, Hawaii
    TRENT FRANKS, Arizona                BETO O'ROURKE, Texas
    DOUG LAMBORN, Colorado               STEPHANIE N. MURPHY, Florida
    AUSTIN SCOTT, Georgia
                     Kevin Gates, Professional Staff Member
                  Lindsay Kavanaugh, Professional Staff Member
                              Neve Schadler, Clerk

                                C O N T E N T S

                                  ----------
                                                                       Page

                  STATEMENTS PRESENTED BY MEMBERS OF CONGRESS

    Langevin, Hon. James R., a Representative from Rhode Island,
      Ranking Member, Subcommittee on Emerging Threats and
      Capabilities...................................................     2
    Stefanik, Hon. Elise M., a Representative from New York,
      Chairwoman, Subcommittee on Emerging Threats and Capabilities..     1

                                   WITNESSES

    Greer, Edward, Former Deputy Assistant Secretary of Defense for
      Developmental Test and Evaluation..............................     7
    Halvorsen, Hon. Terry, Former Department of Defense Chief
      Information Officer and Department of the Navy Chief
      Information Officer............................................     5
    Levine, Hon. Peter, Former Department of Defense Deputy Chief
      Management Officer and Under Secretary of Defense for Personnel
      and Readiness..................................................     3

                                    APPENDIX

    Prepared Statements:

        Greer, Edward................................................    46
        Halvorsen, Hon. Terry........................................    36
        Levine, Hon. Peter...........................................    27
        Stefanik, Hon. Elise M.......................................    25

    Documents Submitted for the Record:

        [There were no Documents submitted.]

    Witness Responses to Questions Asked During the Hearing:

        Mr. Scott....................................................    65

    Questions Submitted by Members Post Hearing:

        Mr. Franks...................................................    73
        Mr. Shuster..................................................    73
        Ms. Stefanik.................................................    71

    CREATING A FLEXIBLE AND EFFECTIVE INFORMATION TECHNOLOGY MANAGEMENT AND
        ACQUISITION SYSTEM: ELEMENTS FOR SUCCESS IN A RAPIDLY CHANGING
                                   LANDSCAPE

                                  ----------

                      House of Representatives,
                           Committee on Armed Services,
             Subcommittee on Emerging Threats and Capabilities,
                             Washington, DC, Wednesday, April 26, 2017.
        The subcommittee met, pursuant to call, at 2:03 p.m., in
    room 2118, Rayburn House Office Building, Hon. Elise M.
    Stefanik (chairwoman of the subcommittee) presiding.

     OPENING STATEMENT OF HON. ELISE M. STEFANIK, A REPRESENTATIVE
    FROM NEW YORK, CHAIRWOMAN, SUBCOMMITTEE ON EMERGING THREATS AND
                              CAPABILITIES

        Ms. Stefanik. The subcommittee will come to order. I would
    like to welcome everyone to this hearing of the Emerging
    Threats and Capabilities Subcommittee of the House Armed
    Services Committee on the important topic of information
    technology [IT].
        Several of us from the committee recently had a discussion
    with Eric Schmidt, who is the executive chairman of Alphabet,
    as well as the chairman of the Department of Defense [DOD] new
    Defense Innovation Board. That discussion was very helpful in
    provoking us to think about how the Department could do a
    better job incorporating best practices from industry. And
    while it is unrealistic to think that DOD will ever operate
    exactly like industry, it is nonetheless imperative that we
    achieve every ounce of efficiency out of our back office and IT
    operations in order to fully invest in combat capability. The
    challenge is finding that correct balance, and today's hearing
    brings us one step closer.
        Information technology represents over $30 billion of the
    Department's total budget; however, too often it is treated as
    a support tool secondary to platforms, weapons, training, and
    the operations of the Department. But anyone who has seen U.S.
    forces operate over the past 25 years understands that our
    military advantage comes from those network systems providing
    the intelligence, precision strike, information fusion and
    warning capabilities our warfighters have come to rely on.
        This committee has been focused on reforming the operations
    of the Department for the past 2 years, from streamlining
    acquisition regulations to streamlining an overly cumbersome
    bureaucracy. The end goal of these efforts has been to enable
    us to buy and develop systems with greater agility and
    flexibility so that state-of-the-art tools get into the hands
    of our warfighters faster.
        I know our witnesses today will help provide us with a
    better framework for understanding how to think about defense
    management and acquisition practices for information technology
    as we explore the critical questions before us, such as: What
    are the characteristics of well-performing programs that we
    should focus on? What leading indicators should we be
    monitoring to determine if programs are going off the rails and
    before it is too late? And how can we better identify,
    encourage, and reward those program managers that are executing
    information technology programs well?
        Let me now turn to the ranking member, Jim Langevin of
    Rhode Island, for any opening comments he would like to make.
        Jim.
        [The prepared statement of Ms. Stefanik can be found in the
    Appendix on page 25.]

      STATEMENT OF HON. JAMES R. LANGEVIN, A REPRESENTATIVE FROM
    RHODE ISLAND, RANKING MEMBER, SUBCOMMITTEE ON EMERGING THREATS
                            AND CAPABILITIES

        Mr. Langevin. Thank you, Chairwoman Stefanik, and thank you
    to our witnesses for being here today. I definitely look
    forward to hearing what you have to say, given your wealth of
    knowledge in this area.
        So management and acquisition of information technology at
    the Department of Defense is one of the most challenging and
    pressing DOD organizational and administrative issues facing us
    today. For years, Congress, the executive branch, and industry
    have attempted to bring DOD's IT programs and processes into
    the 21st century. Despite attempts like the Joint Information
    Environment and streamlining acquisition processes, DOD's pace
    to improve its IT posture is not progressing with the desired
    speed to achieve serious efficiencies, increase security, and
    take advantage of enhanced capabilities that are readily
    available. For example, the intelligence community has
    developed and is using commercial cloud services, yet
    widespread use of cloud computing has not yet taken place at
    the Department.
        All that said, I will be the first to acknowledge that the
    Department of Defense IT management, modernization, and cyber
    are extraordinarily complex. I will also acknowledge that
    Congress, in an attempt to be helpful, has taken action that
    has seemingly had the opposite effect.
        That is why today's hearing is so important. IT management
    and modernization experts who served at the DOD sit before us.
    I look forward to their concrete recommendations for improving
    DOD's IT posture in an expeditious manner, and I hope they
    provide granularity on legislative, regulatory, and cultural
    barriers, like risk aversity, that are inhibiting the rapid
    change required. And I am certainly glad that they help us
    understand what is working as well as highlight the success
    that the Department has already had in this arena.
        So with that, Madam Chair, I thank you for organizing this
    hearing. I thank our witnesses for being here today. And I
    yield back.
        Ms. Stefanik. Thank you, Jim.
        Today, we welcome three distinguished witnesses, all of
    whom have served as senior officials within the Department of
    Defense, and so they not only understand the government
    challenges with information technology management and
    acquisition, but also how private sector practices can be
    applied or need to be avoided.
        First, the Honorable Peter Levine, former Department of
    Defense Deputy Chief Management Officer [DCMO] and Under
    Secretary of Defense for Personnel and Readiness [P&R]. Next,
    the Honorable Terry Halvorsen, former Department of Defense
    Chief Information Officer [CIO] and Department of the Navy
    Chief Information Officer. And Mr. Ed Greer, former Deputy
    Assistant Secretary of Defense for Developmental Test and
    Evaluation [DT&E].
        Welcome to all of our witnesses. I would like to remind you
    that your testimony will be included in the record, and we ask
    that you summarize key points from that testimony in 5 minutes
    or less for your opening statements.
        And, Mr. Levine, I will start with you. The floor is yours.

     STATEMENT OF HON. PETER LEVINE, FORMER DEPARTMENT OF DEFENSE
    DEPUTY CHIEF MANAGEMENT OFFICER AND UNDER SECRETARY OF DEFENSE
                      FOR PERSONNEL AND READINESS

        Mr. Levine. Thank you. Chairwoman Stefanik, Ranking Member
    Langevin, and members of the subcommittee, I thank you for the
    opportunity to appear here this afternoon to discuss DOD's
    information technology management and acquisition processes.
    What I would like to focus on, with your permission, is DOD
    business systems acquisition.
        For years, DOD has been trying to buy new business systems
    the same way that it buys weapon systems and other big ticket
    items, and the result has been a series of sometimes
    spectacular failures. The fact is that there is no such thing
    as a stand-alone business system and no such thing as a stand-
    alone business system acquisition. The DOD's business systems
    are interconnected not only with each other, but also with the
    business processes that they run.
        When DOD wants to acquire a new financial management system
    or logistic system or HR [human resources] system, it
    inevitably finds links to dozens if not hundreds of other IT
    systems which either feed it data or rely on it for data. Each
    of these systems, no matter how outdated, redundant, or
    ineffective they may seem, has a core of devoted users who are
    likely to resist change.
        DOD tries to ensure that change management and business
    process reengineering take place concurrently with new business
    systems acquisitions, but it has not been easy. Acquisition
    officials don't have the authority or expertise to require
    their customers to do business process reengineering when they
    buy new systems, and management officials responsible for those
    business processes have proven to be incapable of running large
    acquisitions.
        In the last couple of years, Congress has helped this issue
    considerably by streamlining section 2222 of title 10, which
    provides the statutory requirements for DOD business systems
    acquisitions, and by repealing requirements for major automated
    information systems, or MAIS programs, that mimic the
    acquisition process for major weapon systems.
        When I testified with Terry here last year, I promised that
    the Department would take advantage of the new flexibility
    granted by Congress to better coordinate the roles of the DCMO,
    the CIO, and AT&L [Acquisition, Technology, and Logistics] in
    the acquisition of business systems.
        Two weeks after that hearing, I left the position of DCMO
    to become Acting Under Secretary for P&R, but I am pleased to
    say that streamlining effort went on in my absence and that
    just a couple of months ago the Department issued a new DOD
    instruction addressing business systems acquisitions. This new
    instruction appropriately sequences for the first time decision
    points regarding business solutions, IT solutions, and
    acquisition solutions so that we don't have these redundant
    processes going on side by side without referencing each other,
    but we have a coordinated process.
        This is also the first step toward building an IT-specific
    acquisition process as the Defense Science Board [DSB]
    recommended almost a decade ago. Consistent with the DSB's
    recommendations, the new instruction establishes a unique set
    of decision points for business systems acquisitions and
    requires prototyping and incremental delivery of capability.
        I would have to say, though, that the DSB report, which is
    a 2009 report, recommended a unique acquisition system not just
    for the acquisition of business systems, but for the
    acquisition of all IT. It has been a disappointment for those
    who were here with the Congress--I was on the SASC [Senate
    Armed Services Committee] staff at the time that that report
    came out--that that still hasn't been fully implemented for IT.
        Congress felt strongly enough about this that you enacted
    provisions on this twice, first in 2010 and then in 2015,
    telling the Department: Please go look at this DSB report and
    figure out a way to implement the kind of IT-unique acquisition
    system that the DSB is recommending, having looked at
    commercial practices, and figure out how it would best apply to
    the Department. I commend you to look at that report. It is
    still the right thing to do. It is still what the Department
    needs to do.
        So while I think the Department has made good steps in the
    direction of change, there is still a long way to go. And one
    thing I would add before I wrap up and hand over to Terry is
    that the issuance of a new DOD instruction like this one I just
    mentioned is the beginning of a process of change, not the end
    of a process of change.
        One thing that I have seen in the Department is it takes
    every bit as much effort to make sure that the instruction, the
    legislation, the regulation is really implemented, implemented
    in the spirit it was intended, as it does to enact it in the
    first place. So it will be up to the new team at DOD when it
    arrives to ensure that the new processes are appropriately
    implemented in practice, and in particular with regard to
    business systems, that the CMO [Chief Management Officer] will
    have to remain engaged throughout the acquisition life cycle to
    ensure that the Department adopts new and efficient business
    solutions rather than tailoring new business systems to old and
    inefficient processes.
        With that, I thank you again for inviting me here today,
    and I look forward to your questions.
        [The prepared statement of Mr. Levine can be found in the
    Appendix on page 27.]
        Ms. Stefanik. Thank you, Mr. Levine.
        Mr. Halvorsen, you are recognized for 5 minutes.

    STATEMENT OF HON. TERRY HALVORSEN, FORMER DEPARTMENT OF DEFENSE
      CHIEF INFORMATION OFFICER AND DEPARTMENT OF THE NAVY CHIEF
                          INFORMATION OFFICER

        Mr. Halvorsen. Madam Chairman, Ranking Member,
    distinguished members, thank you for the opportunity to testify
    today before this committee. This is an important topic, and I
    would like to echo much of what Peter just said. Obviously, we
    did testify together on this topic last year, and I did testify
    the last 2 years on this topic.
        DOD has made progress. They continue to face critical
    global challenges. I do think they will meet those challenges.
    But it is faced with an added and unprecedented dimension: This
    is arguably the period in history with the fastest developing
    and most complex technology.
        Unlike previous times, the vast majority of this technology
    growth is occurring in the private sector, not originating with
    the government. This means in addition to identifying the right
    capabilities to meet DOD requirements, DOD must be able to
    acquire and integrate this technology with greater agility.
    Today's environment demands more broadly defining capability
    and not providing detailed requirements that dictate solutions.
        At times, the government process today, because of the
    current thinking, delivers legacy solutions. We really do need
    to think about how we talk about capabilities and not dictating
    a series of technical requirements that will be out [of] date
    the day we publish them.
        DOD needs a better understanding of the commercial
    environment to become more effective and efficient working with
    industry and determining how solutions should be implemented.
        With respect to business systems, DOD really needs to ask
    itself: Should it implement whole commercial solutions or some
    degree of hybrid solutions retaining some government
    capability?
        I will echo what Peter said, that this also needs to focus
    on the process, and I strongly recommend that the going-in
    position for business solutions until proven wrong through
    business case analysis is that the government and DOD should be
    adopting commercial solutions off the shelf. The real question
    really is what businesses should DOD be directly in and where
    should it offload those businesses to the commercial sector.
        Regarding systems that are more aligned with the primary
    mission of DOD, such as national security systems, DOD must
    more carefully weigh the mission risk, the mission security
    requirements, and since these systems are more likely to be
    operated by military and civilian members, DOD must really look
    at the workforce implications of training and sustainment.
        The new, changing environment also means DOD will be
    requiring more services from industry as opposed to just buying
    products. To successfully buy services in this exploding
    technical environment will require DOD to form better
    partnerships with industry and for industry to be more open to
    sharing technical data with DOD.
        To facilitate the building of these critical partnerships,
    I believe this committee and others will need to look at the
    laws governing relationships and contact between DOD officials
    and industry members and expand those programs. The laws we
    have today have been written with great intention, but they are
    outdated. Many of them from a financial perspective still use
    numbers from the 1970s. They don't make any sense, and they
    limit the ability for DOD and other government agencies to
    function correctly.
        We also need to look at the way we test commercial products
    for acceptance and security perspective today, and I elaborate
    this in my written testimony. The security accreditation
    process is costing both the government and industry lots of
    money and doing a disservice to, in the case of DOD, our
    service members for how long it takes to get those products
    certified.
        We insist on testing stuff that has commercially been
    accepted and tested in industry. We need some legislation that
    says: Why can't I accept that testing as the official testing?
    Test can the government operate it correctly, 100 percent, that
    testing should be included, but we really need to look at how
    we do the testing. That has become one of the long poles in the
    tent.
        You have to improve efficiency. I will talk a little bit
    about the McKinsey report, about the 125 million. DOD
    definitely took that report seriously, but some of those
    numbers in there were not well thought out in the end. But they
    should be continuing to refer to that report. It does give the
    right topic areas that DOD should be looking at to gain more
    efficiencies in.
        As Peter, I am happy with the DOD CIO, and the DCMO
    relationship. I think it is strong today, and I think it will
    continue to get better. That is important, because it is a
    combination of the process and the technology.
        In the end, this is a little bit about a cultural solution
    more than it is laws and anything else. We need to think
    differently. I think we have unintentionally been building for
    a long time a culture of distrust and one that was based on
    overregulation and a foundational belief that all the players,
    industry, government, and academia, all needed to be protected
    from itself, and we have done a good job at doing that.
        Somewhere we have lost what was good in the systems we had
    prior. I was quoted at the DOD as saying that our secret weapon
    was our commercial capability and our relationships with
    industry. I would amend that to read our secret weapon is our
    commercial capability, our relationship with industry, and the
    combined efforts of the military, civilian, contractor, and
    commercial workforce to make it all work and deliver the
    results.
        Thank you, and I look forward to your questions.
        [The prepared statement of Mr. Halvorsen can be found in
    the Appendix on page 36.]
        Ms. Stefanik. Thank you.
        Mr. Greer, you are recognized for 5 minutes.

    STATEMENT OF EDWARD GREER, FORMER DEPUTY ASSISTANT SECRETARY OF
             DEFENSE FOR DEVELOPMENTAL TEST AND EVALUATION

        Mr. Greer. Chairwoman Stefanik, Ranking Member Langevin,
    and members of the subcommittee, thank you for the opportunity
    to express my views this afternoon.
        In 2013, this same subcommittee hosted a roundtable
    discussion on IT acquisition reform, and it seems like the same
    challenges and the same issues were discussed 3 or 4 years ago.
    So we do need to be mindful of what kind of progress we have
    made.
        Today, I am going to talk about MAIS [major automated
    information systems] challenges, best practices, the role of
    developmental test and evaluation, and business systems versus
    tactical weapon systems.
        MAIS challenges. The challenging nature of MAIS acquisition
    can be attributed to many factors. I will briefly discuss
    three.
        Program complexity. Typical MAIS programs have to be
    integrated into multiple existing and emerging enterprises that
    contain a large number of government and commercial entities.
        Another challenge: unstable requirements. In many cases,
    the changes are driven by advancements in technology, vendors
    updating hardware, operating system upgrades, and so on, or by
    changing world events.
        And the third challenge: build versus buy. While many IT
    companies regret building enterprise software solutions because
    it was much more expensive than expected, there are times when
    custom software solutions is the right answer. A rational
    build-versus-buy decision starts with well-defined requirements
    and a quantifiable outcome.
        Second, the best practices. There are many best practices
    within the commercial sector and within DOD. I would like to
    highlight just a few.
        Executive leadership participation. Robust and continued
    senior-level attention and participation contributed
    significantly to the success of agile MAIS acquisition programs
    as documented in the DOT&E [Director for Operational Test and
    Evaluation] 2016 annual report. When senior leaders are on-
    site, they can add resources as necessary and they can also
    shorten the decision cycle time.
        Another best practice, continuous developmental test and
    evaluation. The program office should have a coherent DT&E
    strategy to find and fix problems as each software component is
    developed and delivered. Deficiencies are much easier to fix
    before integrating into modules and components.
        And the best practice of evolutionary acquisition, which is
    a method intended to reduce cycle time and speed the delivery
    of capabilities to users. Often, in an evolutionary model, the
    development of increments must occur in parallel to deliver
    capability on time.
        The third topic, the role of developmental test and
    evaluation. Conducting DT&E in an agile environment must be
    done early and often. During a major weapon system development
    cycle, 80 percent of the test and evaluation is developmental
    test and evaluation.
        There are DT&E organizations within the services and at OSD
    [Office of the Secretary of Defense]. I have worked in both.
    They serve a different purpose. The DT&E professionals within
    the services are typically funded by the program managers
    [PMs], whether it is acquisition programs, MAIS, or whatnot,
    and they are potentially subject to biased reporting due to the
    pressure from the PMs. I have seen it firsthand. The OSD DT&E
    organization is not funded by the program managers. They are
    mission funded and therefore independent.
        And the last area to talk about is business systems versus
    tactical weapons systems. It is important to make a distinction
    between cyber and IT/IA [information assurance] policies for
    warfighting systems and those pertaining to business systems or
    corporate enablers like email, common business systems, and
    cloud applications. Technical laboratories and communities
    should be held accountable for making recommendations for IT
    consolidation and savings, but should control their own destiny
    in determining the best solutions. There is not a cookie-cutter
    approach.
        To ensure there is an appropriate focus on the cyber
    systems engineering, it is now time to pair a traditional CIO
    function with an RDT&E [research, development, test and
    evaluation] warfighting system cyber assessment function, which
    is focused on the tactical weapon systems and RDT&E
    infrastructure.
        That concludes my remarks, and I look forward to answering
    your questions.
        [The prepared statement of Mr. Greer can be found in the
    Appendix on page 46.]
        Ms. Stefanik. Thank you, Mr. Greer.
        We now turn to members' questions. My question is for all
    three participants in the panel today.
        Industry best practice calls for the use of leading
    indicators as a way to measure the health and performance of a
    sector; however, DOD, as you know, tends to focus more on
    lagging indicators because they are easier to measure, but they
    don't do a very good job of helping decision makers decide if a
    program is performing well or not, at least not until it is too
    late.
        What kinds of leading indicators should we be using to
    measure IT programs as well as performance of our acquisition
    systems? And what are the obstacles to being able to collect
    that sort of information currently?
        Mr. Halvorsen. One of the first things we ought to do is
    concentrate on money. That is what industry does. We do look at
    how is the money being spent, and if you find that you are
    starting to have undocumented rising cost, that is generally a
    pretty good indicator that something is wrong.
        Performance is the next best thing if you look at industry,
    they will look at performance. And it is not the performance of
    how the acquisition process worked. You ask what are one of the
    problems. We grade the acquisition process.
        Now, I am going to say everybody in this room has fault in
    that. I did, and so does the legislation, because it requires
    that I had to report on how I was doing in the process. Most
    acquisitions, the day they were finally declared unfit, still
    had green in checking the acquisition process. That doesn't
    make a lot of sense to me.
        So why aren't we looking at performance in the field? I
    mean, industry, if I am going to grade it, I grade it: Did it
    meet the customer requirements? That ought to be the number
    one, and track the money. If we did those two things, we would
    absolutely have a much better system today. And then get out of
    the rules and laws that require that we have to track and
    produce: Is the acquisition process itself being followed? The
    acquisition process is part of the problem.
        So I really think if we followed--you could take these
    templates right off industry guidelines, we are not different
    in that respect, and track the actual outcome performance
    results, not the process.
        Mr. Levine. I think that when we look at the DOD
    acquisition system not just for IT but for anything, we need to
    recognize that we rely on our contractors for almost all the
    data on their performance. So we need to be careful about what
    kind of data we prescribe, because if we say we want more data,
    then we are adding a contract requirement and a new regulation
    to a system where we don't need a whole lot of new regulation.
        I would add to what Terry said about tracking dollars,
    though. It is important not just to be able to track dollars,
    but to be able to link dollars to performance. And certainly
    with the acquisition system in general, I have always felt that
    earned value management systems are one of the most important
    things that we have, because they don't just track how much
    money you are spending, but track it against measurable
    milestones so you can see whether you are progressing in the
    way you thought that you should be progressing.
        Now, some of our contractors have earned value management
    systems, some of them don't. It depends on whether--you know,
    if they are government contractors, if you are a Lockheed or a
    Boeing or somebody like that, you will have an earned value
    management system. If you are a commercial contractor, you may
    not have one that matches government standards. So we may need
    to figure out how we can work within data that is otherwise
    already provided and collected by commercial industry to
    determine these things without imposing new requirements.
        Mr. Greer. Industry does a very good job at quantifying
    their requirements. They stay rigid with their requirements.
    They developed a product based on quantifiable outcomes. They
    also make it a priority to make sure their senior leaders are
    proactively engaged. They are on-site where the decision time
    is much shorter.
        Proprietary data is an area that, as Peter mentioned, is a
    concern. In some instances, it is difficult to get our hands on
    the data, and when we do, it costs us dearly. So that is an
    issue.
        The last one is a partnership, perhaps, an integrated
    product team with industry so we don't have to worry about
    lagging indicators. We will be right there working with the
    industry developing the leading indicators and monitoring the
    leading indicators.
        Ms. Stefanik. Thank you.
        I now yield to Mr. Langevin.
        Mr. Langevin. Thank you, Madam Chair.
        And I want to thank all of the witnesses for your
    testimony.
        So to carry on on this point, in your opinion, what do you
    think is the single most important action both Congress and the
    new administration can take to enable a rapid modernization of
    IT? I know it is a broad question, but kind of following on
    with our discussion. Please give us your insights.
        And then the other one, I will say, in your opinion, what
    cultural barriers, such as risk aversity or service
    preferences, exist within the Department and what
    recommendations do you have to overcome them?
        Mr. Halvorsen. So the first thing I would say is we need to
    push the authority to acquire down, and we can do that in small
    numbers. So, I mean, here is an interesting thing. In my
    previous job, I was the CIO. I had a $37 billion budget.
    Approved that. It is the only place in the DOD where as a
    single entity, I as CIO, I had to approve that budget, write
    that budget, yet I couldn't authorize directly a million
    dollars if I saw great technology to put right on the table.
    That doesn't seem to make a lot of sense to me. I think that is
    the first thing I would do.
        Smaller numbers, control. Give it a limit of maybe $10
    million total that the CIO could say, ``Yes, that is great
    technology, I want it,'' and waive some of the requirements
    that an institution as big as DOD has to review every piece of
    that against some competition criteria. That does not make
    sense.
        Part of the time, if I can see best practices in industry,
    and that is where industry is going, I ought to be able to say
    as the CIO, ``Industry CIOs do that, they say this is where
    industry is going, I am making the decision,'' and we go. We
    could do that, try that, small amounts of money. I think that
    would really drive some rapid acquisition quickly.
        The second thing, your question about what else do we need
    to do, I would really recommend that this committee establish a
    group, and I think in 90 days they could get some very good
    results. But stack it with both industry and military business
    systems owners, not just the technical guys, because in the
    end, back to our original question, I never had my customers,
    DOD, soldiers, sailors, marines, and civilians are not real
    shallow, and they are not real--they will tell you what the
    problems are really quickly in language you will get to
    understand quickly if you go ask them. We don't ask them
    enough.
        And so I think that is the other problem. Let's go out and
    ask the customers. Bring some customers on this panel, bring
    some industry experts, bring some leadership, 90 days I think
    you could get a really good result on here are some of the
    specific things you could do to really get at the cultural
    issue, because this is cultural.
        Even when it is fixed and even when we have the authority,
    the whole system, which includes the DOD, the way we inspect
    and review, and, frankly, includes the way GAO [Government
    Accountability Office] inspects and reviews, we are using the
    mindset that was 20 years old.
        We have got to change that. That will take time. But I
    think this committee, because it has done it in the past, could
    establish that a committee take a review and put some things in
    legislation and then follow up on it with your personal
    attention, and that would get some good results.
        Mr. Langevin. Thank you.
        Mr. Levine.
        Mr. Levine. I agree with Terry, that I think that the real
    focus ought to be on planning and prioritization so that you
    are spending the money in the right place. We tend to focus in
    a hearing like this on what is wrong with DOD's policies and
    practices and how can we make things better through that and we
    can make things better.
        But if you are really looking for how are you going to
    improve the IT environment at the Department of Defense,
    whether it is in business systems or weapon systems, the issue
    is, we have a lot of low-hanging fruit out there, things that
    need to be improved. We don't necessarily spend our money in
    the right places for the right priorities, and that
    prioritization, I think, is the number one task that we have.
        Mr. Greer. In response to your first question, contracting
    is, in my mind, a major impediment, whether it is acquisition
    weapon systems or IT systems or whatnot. If you look at the
    entire food chain from A to Z in terms of delivering an IT
    agile system, it starts with the statement of work, then it
    goes into a do-loop in the contracting arena because of
    policies. It is not the executors, it is the policymakers,
    whether it is an OSD or the echelon command within the services
    on more pointed here at Congress, legislation that has been
    burdened on the contracting community. Conducting a zero-based
    review of all of the contracting policies certainly might be
    one approach.
        I mentioned earlier about the CIO. My personal view is--and
    I am an engineer, I have the technical background more so than
    the IT side of the house--but certainly the CIO has such a
    broad responsibility. That is another long lead time approval
    cycle. It has been from my experience.
        In terms of cultural barriers, a lot of the government
    organizations do have in-house software development teams. And
    sometimes they make less than informed decisions regarding
    whether or not they should develop in-house software solutions
    or go out to the commercial sector. And I believe that is a
    cultural issue that ought to be addressed.
        Thank you.
        Mr. Langevin. Thank you.
        I yield back. I will have additional questions if we go a
    second round.
        Ms. Stefanik. Mr. Scott.
        Mr. Scott. Thank you, Madam Chair.
        Gentlemen, thank you for your service.
        And, Mr. Greer, you mentioned the proprietary data and how
    much--you said it costs us dearly, I believe is the term that
    you used. And regardless of what weapon system it is or IT
    system it is, I mean, we have seen examples in the past where
    we as the government, we paid for the research, we paid for the
    development, and then in an error in the contracting, even
    though we paid for all of it, some private company ends up
    owning that data.
        I assume that we are not making that mistake anymore? Is
    that something that we have corrected in the contracting?
        Mr. Levine. Congressman, I would be happy to respond to
    that.
        Congress did change the rules on that several years ago,
    about 3 or 4 years ago. I was with the Senate Armed Services
    Committee then. But you changed the rules. In particular, there
    was a rule that said the way that rights and technical data
    work, if it is 100 percent developed through private
    expenditure, the private company owns it; if 100 percent
    developed at government expenditure, the government owns it;
    and if it is mixed expenditure, we get just government purpose
    rights.
        But those rules also said you have to identify in the
    contract up front which data you are going to have ownership of
    as the government. So even if it was developed exclusively at
    government expense, if we failed to identify it up front, we
    didn't own it, and the contractor would come back and tell us,
    ``Well, we own it now because you didn't say in the contract,
    so you are going to have to pay through the nose to use what
    you paid for in the future.''
        We changed the law so that it said, given that we were
    supposed to own it in the first place, if we failed to identify
    it in the contract, then the amount that we will owe you to
    provide us that data is the amount that it takes to produce the
    data. We are not going to have to pay you monopoly rights to
    buy it from you again. So we did make that change--or you did
    make that change--in the Congress a couple of years ago
        Mr. Scott. Okay. I just know that as somebody who
    represents a depot, this is a problem for us in supply chain
    management in some of the older systems out there where we paid
    for the development of a product and we are having to then pay
    for the schematic to manufacture the part because they don't
    have it anymore.
        Mr. Levine. It has been a huge problem. It should be better
    going forward, but of course that doesn't get us all the
    technical data from contracts in the past that we wrote under
    this old system.
        Mr. Scott. Absolutely.
        Mr. Greer. And I might add for just a couple of seconds
    here, even if it is just one piece of small source code within
    a weapon system, that will hold us hostage, and that is what is
    happening today. So you can have all of the lawyers get
    together and come up with compromises, generate legislation,
    but at the end of the day, the major prime contractors are
    still holding the government hostage.
        Mr. Levine. The change that we made a few years ago, the
    change that Congress made a few years ago also attempted to
    address that by saying that you have to segregate the smallest
    piece of technical data that you actually spent your money on
    and isolate that and tell us how we can work around it so that
    you can't tell us that we have to pay for all of the technical
    data or rely on you for all the technical data. So there is an
    attempt to address that as well.
        There is also currently, I believe, Congress----
        Mr. Scott. Is that attempt effective, or do we need
    different language to make it more effective?
        Mr. Levine. I believe it is effective, but it needs to be
    implemented and practiced over a period of time to see where
    the problems are. Congress has established a technical data
    panel to look at these issues. And I suspect that, with all due
    regard to my friends in the contractor community, they are
    going to try to reverse some of these changes. So I would urge
    you to be very alert to the recommendations of that panel and
    make sure that there isn't backsliding on those issues that you
    are concerned about.
        Mr. Scott. If you would provide additional information to
    us on that, I would be very appreciative of it.
        [The information referred to can be found in the Appendix
    on page 65.]
        Mr. Scott. Gentlemen, it took the Army 10 years to decide
    on which new pistol they were going to use. I don't know if it
    was because of the law, the bureaucracy. But the iPhone came
    about about 10 years ago, the original iPhone. Flip phones were
    still popular back then. With technology there is certainly a
    timeline that we cannot operate in. Is it the law or is it the
    bureaucracy that needs to be changed?
        Mr. Halvorsen. Yes. Some of the laws still need to be
    restructured on that. There are some things that still treat
    off-the-shelf technology like a weapon system. We have got to
    stop doing it. We really do need to say off-the-shelf
    technology, commoditized technology, does not need to be
    treated like a weapon system, but then we have got to get
    through the bureaucracy.
        Mr. Scott. Do you have any specific language that you could
    suggest, provide for our committee?
        Mr. Halvorsen. I will shoot you--yes, sir, I will do that.
        [The information referred to can be found in the Appendix
    on page 65.]
        Mr. Scott. I would appreciate that.
        Madam Chair, I am down to about 20 seconds. I will yield
    the remainder of the time.
        Ms. Stefanik. Thank you.
        Ms. Gabbard.
        Ms. Gabbard. Thank you. Thank you, gentlemen.
        I wonder if you can comment on what impact, if any, you
    think that Buy America provisions have on obtaining the best
    possible technology.
        Mr. Levine. It has always been a concern in the acquisition
    community that if you have Buy American provisions that become
    too restrictive, it will limit our ability to get the best
    weapon systems for our warfighters.
        The acquisition community always has a bias toward getting
    the best possible for the best possible price. As long as the
    Buy American provisions are in the area where they are now, the
    acquisition community has been able to work around them; but
    there is concern if you ramp it up too much, it might impede
    their ability to do what they need to do.
        Mr. Halvorsen. I will echo what Peter said. There are some
    cases where, unfortunately, you can't Buy American in much of
    the technical side.
        I think the other thing we want to talk about here is that
    part of the Buy American also addresses some security concerns.
    What we really ought to be looking at, and this is maybe a bad
    way to say it is, where don't we want to buy? Very candidly, I
    don't want to buy stuff, I didn't want to in my last job, that
    was manufactured in certain nation-states. I think that needs
    to be more of the focus here.
        And what I would say, the words here, we might in the
    technology area want to change that to Buy American and allied.
    The other problem we have in the technology communications
    business is we don't go to war alone, the DOD does not, and we
    need to be able to communicate effectively with allies. So that
    is something we have really got to make sure that agency by
    agency we don't put out Buy American rules that would preclude
    us from then communicating very well.
        I also think there is great benefit if we focus this around
    Buy American and allied and how we could do some better
    arrangements for everybody in terms of cost, delivery times,
    and production.
        Mr. Greer. Outsourcing integrated circuitry is a concern
    that I have had for quite a while from a cybersecurity and
    trusted systems point of view. We are second to none when it
    comes to manufacturing integrated circuitry and mass producing
    or producing small quantities. But the outsourcing of the large
    quantities is something that we all have to be concerned about
    only from malware perspective and other reasons.
        Mr. Levine. I would echo what Terry said, though, that
    there is a difference between worrying about your supply chain
    vulnerability, where you are worried about countries of
    concern, and what you do with Buy American, which is to say
    that all countries are off the table.
        Ms. Gabbard. Right. With regard to, you know, you
    mentioning Buy American and allied, what are some of the
    approaches that our allies are taking with regard to some of
    these challenges with workforce and improving their own speed
    of acquisition? Are there lessons we can learn?
        Mr. Halvorsen. There are. So an example, like the U.K.
    [United Kingdom] and Australia, their CIOs are permitted up to
    I think it is 10 million pounds to look at existing technology
    and say: Hey, this would fit. Let's buy a small amount of it.
    Let's test it. Let's put a performance objective around it and
    track the cost performance to Peter's.
        They have to have those, but they can make the commitments
    right on spot to do that. That makes them a lot more attractive
    to new technology, because if you are a startup new technology,
    you are on a 6-month funding cycle. If you don't get that 6
    months, you are walking away. You have to. So that is something
    we could do.
        They also have changed their workforce construction. My
    counterpart in the CIO at the U.K. until just recently--he
    retired March 31--was what was they called a crown contractor.
    To get around the pay gap issues, they were able to negotiate
    not quite a civilian salary, but better than the government
    salary, and they were able to hire him, give him full civilian
    authority to make the decisions. But he came out of both
    industry and the military, he had been a retired two-star, he
    did 5 years with British Telecom. I think that is kind of what
    we want to be able to bring into the government, people who
    have both government and industry experience. You need both to
    succeed at the top level.
        The Canadians are doing the same thing. And the way they
    are doing their supply chain also, those nations which are in
    Five Eye, and there is only one nation that doesn't
    have this authority, and it is probably the leading nation in
    the Five Eye community, which would be us, they are allowed to
    look at a computer, say, check a supply chain.
    ---------------------------------------------------------------------------
         ``Five Eyes'' (or FVEY), refers to an intelligence
    alliance involving Australia, Canada, New Zealand, the United Kingdom,
    and the United States.
    ---------------------------------------------------------------------------
        So there are some U.S. companies that I wouldn't want to
    buy from because when you look at their components, they are
    all from countries I don't want. That is the most important
    thing. So they are allowed to say: Hey, if that is a U.K.
    company, but all of their parts are sourced out of a place we
    don't like, well, then we are going to go buy an allied
    company. We ought to have that kind of flexibility restriction
    to make those decisions when it is in the best interest of
    defense.
        Ms. Gabbard. Thank you.
        Mr. Levine. One thing that the last administration did, if
    we want to look to a domestic example rather than overseas, is
    to have teams of IT experts who came in from Silicon Valley to
    help DOD and other agencies look at specific problems and
    identify solutions. So they would identify commercial
    solutions. Sometimes they would write software themselves to
    address it.
        I believe that the new administration is considering
    similar kinds of steps. But that gets you ahead of a cycle. You
    know, you don't have to take 5 years----
        Ms. Stefanik. We are running out of time, Mr. Levine.
        Mr. Levine. Thank you.
        Ms. Stefanik. Dr. Abraham.
        Dr. Abraham. Thank you, Madam Chair.
        Gentlemen, thank you for your direct testimony. It is
    appreciated. There was some great solutions or possible
    solutions mentioned in you-all's testimony. And as we know,
    industry has to make a profit. They have a board of directors
    that they report to. And, unfortunately, in the DOD or any
    bureaucracy, that word is not even in their dictionary. And,
    you know, that is a problem.
        Again, it all comes down to accountability. And when you
    have a budget, Mr. Halvorsen, of $37 billion or whatever and
    you can't write a check for something that you see, it is just
    idiotcracy at its finest.
        So it is a national security issue. And I understand we
    have to maintain qualitative and quantitative advantage over
    our enemies. But for these programs that we could just
    literally pick off the shelf and save literally hundreds of
    millions of dollars, it just--it bodes poorly for us in the
    government for us to allow this to happen.
        Are there any examples, can you give me some examples of
    some successful industry partnerships?
        Mr. Halvorsen. Well, you know, I hesitate, because some of
    those I was directly involved in. But I will tell you, the
    Microsoft partnership with DOD, the ability that Microsoft let
    us, looking at the license agreement, let me at that point make
    the decision that the entire Department could go to Windows 10,
    get to a single operating system, without having to renegotiate
    the current contract, that was a partnership with Microsoft.
        That what they saw down the road, and I think this is going
    to happen if the government goes to Windows 10, to one of the
    Representative's comments earlier, it will actually facilitate
    then the follow-on move, which I know is being planned, into
    the 365 and public cloud domain.
        So Microsoft saw that as investment. It was a good, open
    dialogue. That was very good.
        Again, I hate to say it, I will go on the record, I am now
    working for Samsung, but one of the things that happened early,
    we put out a capability statement that said we needed a secure
    smartphone. That is all we put out. Samsung came in. We worked
    with them. We actually co-developed what today they feel is
    their KNOX security system. It kept everybody's prices low. We
    were able to field that at the secret and now top secret level
    in less than 24 months. The products, which are rare in my
    history, were delivered on time and at cost. And in the end,
    they actually cost less than we thought they would.
        Dr. Abraham. That leads somewhat to the second question I
    have. Mr. Scott referenced the iPhone 10 years or less. And,
    again, these products are being developed by a different
    generation of entrepreneurs, engineers, IT people. What are we
    doing to attract some millennials into the fold?
        Mr. Halvorsen. Not enough. We are going to have to address
    a couple things on that. One of them is the pay gap. I will
    tell you, I lost my director of security. You know, he went and
    he quadrupled his salary after spending, you know, 32 years
    serving the government. Good.
        That is one thing we are going to have to address. But we
    are also going to have to address some of the other things that
    the millennials want, which is they want their technology.
        One of the things that when I went in my old job and now in
    my new job, I talk to millennials, ``Why don't you look at
    DOD?'' ``Well, because they will take my phones away. I won't
    be able to do the things I want.''
        Now, that is not true as much anymore. We are using
    multifactor phones, those type of things. But there is the
    perception that the government lags greatly in the ability to
    be flexible in the workplace.
        We have got to get that message out. The rules are there. I
    mean, the laws are there. I mean, I could let my people
    telework. I could do those things. It just wasn't the way we
    thought. That is a cultural issue. The laws are there to make
    that happen.
        That is something that maybe this committee and other
    committees ought to be asking some questions about what are we
    doing on the millennials, what are we doing to be more flexible
    in the workplace? There are some good answers to that, but that
    needs to be the holistic plan, not just individual efforts.
        Dr. Abraham. A minute left. Mr. Levine.
        Mr. Levine. If I could just add one thing to that. We don't
    take advantage enough of the things that we do have to offer. I
    think that the most attractive thing for the Department of
    Defense for people who work there is the mission and the
    feeling of commitment to the mission and that they can make a
    difference. And I think that we don't take enough advantage of
    that. We don't sell that well enough. When we treat civil
    servants as bureaucrats who are just too much, there are too
    many of them, we need to get them out of the way, we need to
    fire them and cut their pay, it doesn't create the image that
    people want to see of themselves. And we need to do a better
    job of selling the mission to get the kind of people we want.
        Dr. Abraham. Very good.
        Mr. Greer.
        Mr. Greer. And just one or two points here. In the IT
    community, I believe that we have to raise the level of our
    professional series. We have a lot of nonprofessional IT
    specialists, and they serve a very good purpose, but where we
    are today, what the cyber vulnerabilities and resiliency
    challenges, I think making sure that we hire enough
    professional series in the IT community is important. And it is
    not necessarily the pay that is attracting, like Peter said, it
    is the quality of work for these millennials.
        Dr. Abraham. Thank you, gentlemen.
        I yield back.
        Ms. Stefanik. Mr. Larsen.
        Mr. Larsen. Thank you, Madam Chair.
        Mr. Greer, can I talk a little bit about your build-versus-
    buy conundrum in your testimony, in your written testimony? I
    am sorry for being late, you may have addressed this. But I
    wanted to ask you in your experience on this build-versus-buy
    tension generally are there some criteria that we should be
    thinking about where there is a leverage point within the DOD
    where they ought to build versus they ought to buy in general?
        Mr. Greer. That is a difficult question to answer in a
    short period of time, mainly because there is a hybrid
    solution. There are a lot of commercial software packages like
    SAP that we do purchase, but we use government people to
    program it and code the SAP system. And so that is always an
    issue.
        Mr. Larsen. Sorry. Mr. Halvorsen looks like he is excited
    to answer too. But go ahead.
        Mr. Greer. Well, I am just expressing my views.
        Mr. Larsen. That is great.
        Mr. Greer. Okay. And this is why----
        Mr. Larsen. So then continue.
        Mr. Greer. But in terms of having firm requirements and
    quantifiable outcomes and making that make-buy decision, it
    gets difficult within the government, mainly because of
    cultural issues. There are a lot of software programmers in the
    government, and they inherently lean in the in-house solution
    side.
        Mr. Larsen. So let me ask you this then. Are there specific
    build-versus-buy decisions on cloud computing? Is that treated
    differently than, say, operational software like on the
    Microsoft decision?
        Mr. Halvorsen. It should be, but it is generally not the
    culture. But I want to be very clear on this: If you could buy
    it, buy it. I did not find that to be as difficult for anything
    that wasn't a weapon systems as [inaudible]. If you can buy it,
    we should buy that, and we should put--and we really did try to
    do this.
        I mean, one of the plaques over at my desk was
    ``customization bad.'' You should fight every attempt we take
    to customize that software off the shelf, because every time we
    do it, it is a lifelong contract for contractors without much
    gain. Our business processing systems, back to Peter's point,
    don't differ that much from the best in industry. They
    shouldn't, if they do.
        So one of the first criteria is, if it is a business
    system, you should have to come in with a different business
    case that says why I am not buying that solution, why am I
    building any specific software.
        It is getting that way for many communication systems,
    because we are using off-the-shelf communication. If it is off
    the shelf, really consider why you need to put something extra.
    And in the communications it might be for cybersecurity, but I
    have found if I go to industry, when I did that, if I told them
    I wanted that extra, most of the time they were happy to put it
    in, and most of the time they carried that in their commercial
    product because it was going to sell more as security becomes
    more important.
        Mr. Levine. The track record of the Department is, if we
    are buying a commercial-type business system, if we start
    tailoring it, we end up spending more money on the tailoring
    than we spent on the system in the first place. It just doesn't
    make sense to do anything unique unless you absolutely have to.
        Mr. Larsen. I am sure if we went back to the first hearing
    that I attended on this issue when I was first elected in 2001
    a lot of the answers would be the same.
        On the build versus buy then does that principle, can that
    principle apply to the people? For instance, in Washington
    State, we have the example of the National Guard working with
    private sector and public sector folks on cybersecurity and
    doing red teaming and doing some mapping out of how to approach
    defense. And some of these folks are reservists and they are
    trying to attract folks in for 2 weeks a year, two weekends a
    month, in a sense buying people as opposed to building them
    internally. Does that model work in terms of the IT personnel
    acquisition side, if you are acquiring people?
        Mr. Halvorsen. It certainly needs to be part of your plan.
    It can't fill all the balance. And there are some security
    issues that you might not want to put everybody that way. But
    absolutely we need to be taking advantage of that.
        If you look at our last sets of operations, you look at the
    Reserves we put on board, many of them were with those
    technical skill sets. As the CIO when I was there, I would
    reach out and bring Reserves on who were working for Google,
    working for whatever big company. And it is actually part of
    the deal, I will give DOD great credit, they actually have that
    in their plan.
        Continuing that into other areas I think makes sense with
    caution. It is not as easy to do that with people as it is with
    software, but it is something that we certainly ought to be
    looking at.
        Mr. Greer. Contractor support is an area that we need to
    leverage even more. There is always a need for inherently
    governmental personnel to be able to represent the government.
    But I think we can leverage more on the contractor support
    side, especially as your workload goes up and down. You are
    able to make adjustments much more quicker.
        Mr. Larsen. Yeah. Thank you.
        Ms. Stefanik. Thank you very much to each of our three
    panelists, Mr. Levine, Mr. Halvorsen, and Mr. Greer. And thank
    you to the members for their thoughtful questions. This is a
    critical issue that falls under this subcommittee's
    jurisdiction that we will continue to focus on as we prepare
    and draft this year's NDAA [National Defense Authorization
    Act], and we look forward to continuing to engage with you to
    get your suggestions.
        Thank you very much. This hearing is adjourned.
        [Whereupon, at 2:57 p.m., the subcommittee was adjourned.]

    =======================================================================

                                A P P E N D I X

                                 April 26, 2017

    =======================================================================

                  PREPARED STATEMENTS SUBMITTED FOR THE RECORD

                                 April 26, 2017

    =======================================================================

    [GRAPHIC(S) NOT AVAILABLE IN TIFF FORMAT]

    =======================================================================

                  WITNESS RESPONSES TO QUESTIONS ASKED DURING

                                  THE HEARING

                                 April 26, 2017

    =======================================================================

                 RESPONSES TO QUESTIONS SUBMITTED BY MR. SCOTT

        Mr. Levine. The statutory provision to which I referred in my
    testimony is section 815 of the National Defense Authorization Act for
    Fiscal Year 2012. As I indicated, this provision:
          Ensures that DOD may require the delivery of data
    generated in the performance of a contract that is necessary for the
    purpose of reprocurement, sustainment, modification or upgrade, even if
    such delivery was not required by the original contract. In exchange
    for the delivery of the data (which was generated at government
    expense) the government is required to compensate the contractor only
    for reasonable costs incurred to covert and deliver the data in the
    appropriate form.
          Requires the contractor to provide technical data ``is
    necessary for the segregation of an item or process from, or the
    reintegration of that item'' with the rest of the system. In other
    words, if a contractor wants to withhold data on a widget that was
    developed at contractor expense, it has to tell DOD how to work around
    the widget, so that the rest of the technical data, which DOD owns,
    won't be useless. Since I am no longer with the Department, I am unable
    to give a report on the extent to which this provision has been
    successfully implemented. However, section 813(b) of the National
    Defense Authorization Act for Fiscal Year 2016 establishes a
    Government-Industry Advisory Panel to review statutory requirements
    regarding technical data rights. Some may seek to use this panel to
    unwind the 2012 reforms.   [See page 13.]
        Mr. Halvorsen. Language that stresses DOD should use commercial
    business case analysis to include review of DOD business process when
    procuring what is clearly a commercial solution to include hardware,
    software and IT service contracts. Something like the following:
        ``DOD will apply commercial best practices when conducting business
    process and detailed market analysis when procuring readily available
    IT hardware software and services in support of or in replacing
    business systems. If DOD can demonstrate through business intelligence
    and analysis a clear product/service leader(s) in a specific area DOD
    can pursue sole source or limited partnership procurements. These facts
    can include the ability to support a large enterprise like DOD, similar
    to how other large commercial companies develop procurement
    partnerships.
        An example of this would be using the MICROSOFT based Services
    Cloud supported by MICROSOFT 365 and Azure. This is the leading cloud
    solution for the fortune 500 and clearly dominates the market and would
    be considered a best of breed solution. If DOD can make this case it
    should be allowed to by-pass current acquisition rules and go direct to
    procuring a best of breed solution, negotiating and spending time on
    obtaining the best value (pricing and performance) instead of defending
    and deliberating on multiple of solutions. This doesn't mean the
    MICRSOFT would be the guaranteed provider, it does mean that MICROSOFT
    products/services would be the used to form the service cloud
    architecture and operating system.
        I want to make it clear I am not a MICROSOFT employee and receive
    no compensation from MICROSOFT for these recommendations. My current
    employer Samsung receives no direct benefit as a result of these
    recommendations. These recommendations are aligned with statements and
    testimony I made as the DOD CIO. I use MICROSOFT as it is the most
    compelling example and in my opinion moving to the cloud is a necessary
    and critical step to improving DOD security, greatly improving
    productivity and saving dollars.
        With respect to all of the responses I have provided to the
    Questions for the record, they represent my personal opinions and
    beliefs and are in my own words. I am currently an employee of Samsung
    Electronics of America. The responses I have provided have not been
    endorsed, constrained of approved by Samsung Electronics of America or
    any other entity.   [See page 13.]
        Mr. Greer. There are a few acquisition programs that have had
    difficulty obtaining data/software/interface rights by prime
    contractors. Despite language that stipulates the Government has
    purpose rights if Government funds are spent to develop a deliverable,
    industry still presses back when the Government attempts to exert its
    rights. Industry is interpreting that if a deliverable contains their
    proprietary information or reveals the workings of what they claim as
    their intellectual property (IP), they may limit access to it.
    Additionally, for software intensive deliverables, industry has started
    `bundling' IP with the deliverable to the government, and claiming the
    government has less than purpose rights to the overall product. This is
    leading to complex licensing issues for deliverables the Government
    should have purpose rights to. This has happened on F-35 as well as
    many other programs. Language should be clarified to state the
    Government has no less than purpose rights to the code segments (source
    and executable) and interfaces used in an overarching deliverable
    funded in any part by the Government.
        From an acquisition oversight perspective, with the KC-46A program,
    Boeing made it clear during contract negotiations that they would not
    use Boeing 767 reliability data to meet the government requirement.
    That sounded like the right approach at the time given the KC-46A
    differences in configuration, but in practice it meant that Boeing will
    not share any reliability test data from their Boeing 767-2C (the
    provisioned freighter) either from Boeing DT&E or FAA Amended Type
    Certificate (ATC) certification testing. Although we do get some
    anecdotal failure data from these flights in which the government
    participates, it is purely word of mouth from those attending the
    briefings or flying the tests. We do have reliability data from
    flights, which are used to meet contract specifications. The net result
    is that we have very incomplete reliability data. The DASD (DT&E)'s
    official MS C assessment concluded, ``As a result, the Lead
    Developmental Test Organization (LDTO) used data only from flights in
    which the Air Force participated to estimate the system's progress
    against the reliability growth curve, which presents an incomplete
    picture of the system's reliability.''
        Also, the tension over Boeing 767 proprietary data makes it
    difficult and time-consuming to obtain test data that the government
    feels is necessary to evaluate the system under test when that data
    comes from Boeing DT&E or ATC testing. When it was in Boeing's interest
    to share more quickly and ``freely'' during the boom issues prior to MS
    C, the data flowed more freely. Most of the time, it can take weeks or
    longer to get the data the government believes that it needs.
        To the best of my knowledge, Boeing has provided (or will provide)
    the data required for gov't spec verification, though it has been a
    painful process that runs through Boeing Commercial Aircraft (BCA)
    because they technically own the aircraft during EMD. As best as I can
    tell, we will never get the B-757-2C reliability data from Boeing
    internal and ATC testing. I sat in on a meeting between BCA, Boeing
    Defense, and the program office and can relate that BCA was surprised
    that we wanted the requested data. In the commercial world, BCA simply
    delivers aircraft that meet the customers' specifications. The
    commercial customers don't care how the aircraft was built and tested.
    Maybe part of this equation involves the gov't rethinking what data we
    really need, which drives back to the acquisition process and how the
    gov't buys stuff.
        The Department of Defense has tremendous capabilities to modify
    legacy weapon systems if full data rights were available. Government
    personnel could perform the Non Recurring Engineering development for
    small to medium upgrades in-house saving millions of taxpayer's
    dollars. For example, The NAVAIRSYSCOM's Naval Air Warfare Center,
    Aircraft Center has an Aircraft Prototyping Facility for modifying DOD
    weapon systems. The DOD should be taking advantage of facilities like
    this as opposed to sole sourcing modifications to the prime
    contractors. The Army has similar modification facilities.
        The lack of government access to design data related to items
    (platforms, systems, components, hardware, software, etc.) already
    purchased by the government has been and continues to be a major cost
    driver for any subsequent upgrade done to these items during their
    service lifetime. Although IT systems are certainly a critical element
    of this problem, the problem is pervasive. Asserted data rights
    involving organic government prototype development programs costs time
    and money at a minimum and, in the extreme, can simply stop an effort.
        This problem is further compounded on a broad scale by government
    laboratories and field activities that are reticent to reverse-engineer
    the data out of the government-owned items due to ultra-conservative
    legal advice predicting litigation as well as fear of Congressional
    inquiries. This is in stark contrast to the private sector where
    backing data out of owned items through reverse engineering is
    commonplace, protected by legal precedent and an essential element of
    one company modifying or upgrading another's product. Reverse
    engineering is not always practical but it is often an effective work-
    around particularly in situations where the prime contractor no longer
    produces the item.
        The complexity in system acquisitions and understanding
    intellectual property rights in Modeling & Simulation (M&S)
    acquisitions, requires a clear understanding from Program Managers,
    System Engineers, and Procuring Contracting Officers to M&S Integrated
    Product Teams of the Government's license rights in intellectual
    property comprised of technical data (e.g. technical packages and other
    information relating to the developed product's design, configuration,
    or to operation, maintenance, installation, or training data) and
    computer software (e.g. executable code, source code, code listings,
    algorithms, formulae, design details). The lack of clear understanding
    among various Program Offices highlights the need for specific
    guidelines for contracting for open M&S systems and the data rights
    required. Open System Architecture is a key pillar of the Department of
    Defense's Better Buying Power (BBP) initiative to implement best
    practices to strengthen the Defense Department's buying power, improve
    industry productivity, and provide an affordable, value-added military
    capability to the Warfighter.   [See page 13.]

    =======================================================================

                  QUESTIONS SUBMITTED BY MEMBERS POST HEARING

                                 April 26, 2017

    =======================================================================

                      QUESTIONS SUBMITTED BY MS. STEFANIK

        Ms. Stefanik. In addition to investments in IT to improve its use
    of data to inform decision-making, DOD also needs to address the
    cultural barriers that hinder effective information sharing throughout
    the Department. For example, RAND found in 2015 that ``institutional
    structure and bureaucratic incentives to restrict data access are
    exacerbated by policy and guidance to protect information. The result
    is a strong conservative bias in labeling and a reluctance to share.''
    In your opinion, what are the initial steps that the Department needs
    to take to address these cultural barriers, and who should be
    responsible for leading this effort?
        Mr. Levine. There are both good and bad reasons for restricting
    data. Some restrictions on access to data are appropriate to safeguard
    classified and sensitive unclassified data from unauthorized
    disclosure. Widespread dissemination of such data to those without a
    need to know, even within the Department, could lead to compromise.
        Other restrictions appear to be imposed for more parochial reasons:
    program managers, PEOs, and component acquisition executives may seek
    to retain control over their programs by controlling the flow of data
    to Comptrollers, to CAPE, and even to others in their chain of command.
        Before I left the Department, DOD was seeking to address this issue
    through a coordinated ``big data'' project. The objective of the
    project was to identify data elements are needed by users at all levels
    of the acquisition system, as well as by outside stakeholders in the
    Department, and standardize that data to increase transparency. If
    successful, this project should help address the problem identified by
    the RAND review.
        Ms. Stefanik. In addition to investments in IT to improve its use
    of data to inform decision-making, DOD also needs to address the
    cultural barriers that hinder effective information sharing throughout
    the Department. For example, RAND found in 2015 that ``institutional
    structure and bureaucratic incentives to restrict data access are
    exacerbated by policy and guidance to protect information. The result
    is a strong conservative bias in labeling and a reluctance to share.''
    In your opinion, what are the initial steps that the Department needs
    to take to address these cultural barriers, and who should be
    responsible for leading this effort?
        Mr. Halvorsen. Representative, when I was the DOD CIO and I believe
    it is still true, that DOD is making good strides in this area and this
    is currently being led by the Deputy Chief Management Officer and fully
    supported by the SD and DSD. I would encourage Congress and DOD to work
    with industry to look at the current laws and rules that restrict
    government employee direct interaction with industry and restrict the
    sharing of information with industry. Industry should also be
    challenged to share data more openly with DOD and the government in
    general. One specific recommendation I would make is for DOD supported
    by legislation to include industry more directly in developing policy/
    instructions with respect to business systems. While DOD CIO I
    explicitly involved industry in developing the cloud policy and
    security guidelines. This was challenged by many groups inside and out
    of DOD but the resultant document was well received and is allowing in
    this area much better communications between industry and government.
        Ms. Stefanik. What is the DOD role as it relates to developmental
    test and evaluation for IT systems that are predominantly commercially
    based and where DOD is integrating, but not actually developing new
    capability? Is there a particular balance in developmental testing
    between what DOD is responsible for and what industry should be
    responsible for that can reduce duplication in effort, cost and
    schedule performance?
        Mr. Greer. Yes, there is a balance between what testing DOD is
    responsible for and what performance certifications can be provided by
    industry and accepted by the test community. DOD tests are conducted in
    mission relevant environments to ensure overall capability is effective
    and suitable and to assess resilience to CYBER threats. Component level
    testing of individual COTS hardware components may often be conducted
    at the supplier level and accepted by the Government.
        DOD is in the process of acquiring multiple defense business
    systems (DBS) based upon commercial enterprise resource planning (ERP)
    systems. The first priority is to conduct business process re-
    engineering to try to change DOD processes to fit the commercially
    offered systems and minimize customization of the ERP. In most cases,
    the unique nature of the DOD identifies gaps between the DOD process
    and the ERP capabilities. In this case development of special report,
    interface, conversion, extension, form, and workflow (RICE-FW) software
    objects are required.
        In these cases, developmental testing of the DBS focuses on the
    RICE-FW and integration into the DOD facilities and networks rather
    than the basic capabilities that are proven by use in industry.
    Mission-oriented developmental testing using actual users performing
    mission tasks in a test environment allows integrated testing with the
    operational testers, early identification of human factors integration
    problems, and identification of software defects in the RICE-FW objects
    with sufficient time for correction prior to operational testing.
        Additionally, DOD's roles in developmental test and evaluation of
    IT systems integration are in the realm of cybersecurity. DOD must
    ensure that the new IT system does not introduce a vulnerability to the
    DOD network and that the combined cyber defenses are integrated. Just
    because a commercially based IT capability meets industry, it does not
    mean that it is secure. The IT capability may meet standards of
    performance (certification) but it must be tested for effectiveness
    (penetration).
        DOD should be responsible for cybersecurity testing and the vendor
    must correct those deficiencies. This should be in a developmental test
    environment that allows time for the vendor to make corrections. After
    corrections are made, the IT capability is ready for operational
    cybersecurity testing. Cybersecurity is an area where the test-fix-test
    methodology is still relevant.
        Ms. Stefanik. In addition to investments in IT to improve its use
    of data to inform decision-making, DOD also needs to address the
    cultural barriers that hinder effective information sharing throughout
    the Department. For example, RAND found in 2015 that ``institutional
    structure and bureaucratic incentives to restrict data access are
    exacerbated by policy and guidance to protect information. The result
    is a strong conservative bias in labeling and a reluctance to share.''
    In your opinion, what are the initial steps that the Department needs
    to take to address these cultural barriers, and who should be
    responsible for leading this effort?
        Mr. Greer. Cultural barriers are often difficult to overcome. A
    good method of breaking down barriers is to encourage more transparency
    with regard to ongoing developmental efforts. Sharing visibility into
    IT related projects, investments, and infrastructure may help increase
    re-use of development that would otherwise be unknown across
    organizational boundaries, reducing duplication of effort. Establishing
    a community of interest (COI) for IT related investments may be
    beneficial in fostering this kind of exchange of ideas and information.
    This is a leadership challenge, as is all cultural change. At a
    minimum, the Director, Developmental Test and Evaluation should get
    back statutory authority for oversight of ACAT ID and MAIS programs and
    to ensure access to all data in DOD and have direct access to the
    Defense Acquisition Executive with regard to his independent
    assessments of system performance and issues regarding his ability to
    make those assessments in time to effectively inform decisions.
        I believe that an organization such as the Deputy Assistant
    Secretary of Defense for Developmental Test and Evaluation (DASD
    (DT&E)) helps break down these barriers, where they exist. For example,
    the staff specialists in the DASD (DT&E) engage in multiple programs
    across all the Services. Similarities exist in some programs and the
    staff specialists, by their close association with the program office
    testers, identify lessons learned and share this information among
    other staff specialists and other programs. Where appropriate, the
    staff specialists have encouraged and facilitated direct contact
    between programs to delve into the details of the lessons learned.
        Ms. Stefanik. The 2016 Annual Report on Developmental Test &
    Evaluation stated that the Department's current knowledge management
    capabilities and processes used to gain, collect, and analyze the
    information necessary to conduct acquisition assessments and
    evaluations are deficient and ineffective for today's world. In your
    opinion, what investments in analytical tools and technologies should
    the Department prioritize in order to begin to remedy this situation?
        Mr. Greer. There is a need to transition to an Enterprise
    environment for Weapons Systems RDT&E and Cyber engineering to store,
    access, and share program RDT&E data. Establishment of a classified
    environment with the right controls in place as well as mechanisms to
    increase cross-program security collaboration is needed. With this,
    comes the need to pair an ``RDT&E Warfighting System Cyber Assessment''
    CIO function that is focused on the tactical weapons systems impacts of
    Cyber to work alongside the ``traditional'' CIO function for business
    systems, email, common databases and promulgation of policy.
    Warfighting acquisition programs and Cyber technical work must be
    staffed by the appropriate mix of Government experts from the Systems
    Engineering and RDT&E community vice the traditional CIO community or
    corporate operations workforce. In the current complex Cyber threat
    environment, Defense Department needs have evolved far beyond
    traditional IT/IA and business systems best practices. Our ability to
    operate in the Cyber Warfare environment of the future hinges on agile
    changes to our policies, organizational structures, workforce, and
    infrastructure. Another Enterprise Solution that would benefit
    acquisition programs is a Shared Data Warehouse and Repository.
    Currently, data is stored and process on separate servers in a single
    program office. This data is isolated from many groups that can use it.
    A DOD Enterprise Solution to data storage and data warehouse tools
    would provide the necessary access for DOD acquisition programs to use
    and share data across various platforms. As mentioned in another
    question is an urgent need to gain access to industry acquisition data,
    which in most cases today is industry owned. The right Contract and
    Legal clauses to obtain access to this data freely and use it in
    government owned facilities and programs would provide the capabilities
    necessary to perform data analytics to conduct acquisition assessments
    and evaluations for all programs.
                                     ______

                       QUESTIONS SUBMITTED BY MR. SHUSTER
        Mr. Shuster. Can you tell us how many DIUx projects have
    transitioned to programs of record or other major programs within the
    Department of Defense? What is the process for transitioning prototypes
    or pilots to the services or other agencies of the Department? Is the
    contractor required to lead these efforts? What outreach does DIUx do
    within the Department of Defense to alert them to the potential
    innovations available from commercial providers?
        Mr. Halvorsen. Sir I cannot with any specifics and would recommend
    that this question be referred to DOD. I can say that I am a believer
    in the concept and programs that expand DOD presence in technology hubs
    like Silicon Valley are worth pursuing. I would also suggest that
    measuring the effectiveness is a little premature. DOD has not had this
    type of program in place much longer then a year and both DOD and the
    technology hubs are learning how to make this work. In addition to this
    program DOD has been pursuing employee exchange programs with industry.
    These programs though limited to date have been well received by DOD
    and participating companies.
                                     ______

                       QUESTIONS SUBMITTED BY MR. FRANKS
        Mr. Franks. Can you give a specific example of a good, well-
    performing, or successful IT acquisition? What were the characteristics
    that made it successful? How long did it take?
        Mr. Halvorsen. Sir, the Next Generation Department of Navy IT
    services contract is a good example of a successful IT procurement.
    This was the follow on to the Navy Marine Corps Internet (NMCI)
    contract. The contract yielded about $2B dollars in savings to the DON
    and as importantly continued commercial like services for the DON.
    Planning for this took over a year but was shorter then contracts of
    like size and scope. It involved close coordination with industry
    providers, a very detailed understanding of the capabilities desired,
    and a willingness to understand trade space as related to price and
    performance. I no longer have access to the details on this contract,
    but I am sure DOD could provide. There are still some issues with this
    contract but overall I give it very high marks. The negotiation of the
    Windows 10 upgrade license agreement is another good example of the DOD
    applying good practices and working well with an industry partner, in
    this case MICROSOFT. This negotiation was truly an enterprise effort in
    the department with DOD and the services working closely with MICROSOFT
    to interpret the action as an upgrade covered by existing agreements
    and not a major systems change. This enabled DOD to not have any bill
    for the actual license, while the business case for MICROSOFT was
    having DOD be better positioned to transfer to the cloud and
    elimination of costs in supporting older systems. When completed the
    transition to windows 10 will improve security, position DOD for cloud
    transition and result in cost savings. The biggest cost associated with
    the transition to Windows 10 is equipment modernization, which had
    fallen behind in DOD, but would have to be done at some point. Doing
    the modernization in conjunction with the windows 10 upgrade, allows
    for some modernization delay while gaining 80% or more of the new
    windows 10 capability. This is the type of DOD-industry partnership
    that is needed.
        Mr. Franks. When you consider the critical need of modernizing the
    network, particularly to harden against enemy attacks and infiltration,
    shouldn't an immediate and complete overhaul of the IT procurement
    process be a priority?
        Mr. Halvorsen. Yes and this review needs to focus on both the
    requirements/capability process as well as the acquisition process. The
    current requirements process often results in very detailed and
    technically restrictive solutions being acquired that don't get DOD the
    latest solutions or the best value. Many times because of the length of
    the process and over specification, DOD is buying legacy the day the
    contract awards. The review needs to look at successful commercial
    process and encourage partnership with industry. This will present some
    problems, especially with small business, but I believe creative
    solutions can be applied to offer small business an opportunity to
    partipate in major enterprise acquisitions. DOD also needs to increase
    the dialog between the mission/business owners, the acquisition
    workforce and the industry providers. We need to look at the current
    rules restraining DOD employees from direct discussions with industry,
    many of these rules have not been updated in 30 years.
        Mr. Franks. Can you give a specific example of a good, well-
    performing, or successful IT acquisition? What were the characteristics
    that made it successful? How long did it take?
        Mr. Greer. I can think of several IT acquisitions that were
    executed efficiently--the examples are described in more detail below:
        1) The Army Logistics Modernization Program (LMP) Increment 2 was a
    highly successful Defense Business System program that came to the
    Limited Deployment Decision milestone in 2015 with a 98% test case
    success rate during developmental testing. All failed test cases had
    work-arounds and corrective action plans acceptable to the user, and
    zero open severity 1 or 2 software problem reports. LMP success may be
    attributable to:
        The LMP program management office (PMO) was highly experienced,
    knowledgeable of the system, and effectively planned and executed the
    complex program consisting of three waves of deployment with multiple
    functional areas. The PMO also established a close working relationship
    with the functional user to perform business process re-engineering,
    clarify requirements, and resolve issues quickly and effectively.
        Increment 2 was a continuation of Increment 1; with the same system
    integrator using the well established and understood change management
    processes. Developmental testing (DT) included various events to
    specifically address performance, interoperability, and cybersecurity.
    Most importantly, the test strategy included mission-oriented
    developmental testing (known in the PMO as Business Operations Test
    (BOT)) that used actual users from the depots performing operational
    mission threads for their test cases. This identified human factors
    improvements that led to better user performance. The DT actual users
    became operational test (OT) trainers that resulted in a very high
    success rate for all the tasks and transactions that were tested during
    Initial Operational Test and Evaluation. The overall schedule from
    milestone B to milestone C was approximately 22 months.
        2) The Marine Corps' ACAT IAC Common Aviation Command and Control
    System (CAC2S) Phase 2 was a very successful integrated acquisition.
    The operational test community worked well with the developmental test
    (DT) community, and accepted much of the DT data that satisfied OT
    requirements, reducing the overall acquisition time by 6-months. USD/
    AT&L nominated the CAC2S acquisition effort as a Defense Acquisition
    University integrated acquisition case study. Time between MS B and MS
    C was 34-months.
        3) Defense Agencies Initiative (DAI) has been a successful Defense
    Business System Program. It enables compliance with Congressional
    direction for all DOD organizations to be auditable. The Defense
    Agencies Initiative (DAI) transforms the budget, finance, and
    accounting operations of most DOD Defense Agencies in order to achieve
    accurate and reliable financial information in support of financial
    accountability and effective and efficient decision-making throughout
    the Defense Agencies in support of the missions of the warfighter.
    Defense Agencies Initiative (DAI) is a critical DOD effort to modernize
    the Defense Agencies' financial management capabilities.
        Mr. Franks. We've all heard of Moore's Law and know how rapidly
    technology advances and evolves. Do you know if there are any
    requirements for new systems we are building or purchasing to be
    capable of growing and evolving alongside technology--perhaps
    ``upgradeable'' systems which won't be obsolete the moment they become
    operational or shortly thereafter?
        Mr. Greer. The rapid development pace of IT does mean that the
    Government will need to start driving requirements into new systems for
    Non-Proprietary deliverables, interfaces, and use of open systems
    architectures. Additionally, use of system virtualization and cloud
    technologies may mitigate IT obsolescence issues. Ensuring system
    designs, interfaces, and computing hardware are non-proprietary and
    open to the Government mitigates the possibility of ``vendor lock in''
    and provides multiple flexible upgrade paths. The DISA Defense
    Enterprise Office Solution (DEOS) is a program that provides e-mail,
    voice and data conferencing, office products (e.g., word processing,
    etc.) and other capabilities with a cloud-based solution, potentially
    across the entire DOD enterprise. The acquisition strategy for DEOS
    contains a provision, which DISA calls ``Evergreen'', that will require
    the system integrator to provide upgrades, at no charge, whenever DEOS
    capability upgrades are offered commercially.
        DODI 5000.02 supports growing and evolving alongside technology
    through the use of a modular open systems approach (MOSA). It requires
    programs to identify how a MOSA will or will not be used in their
    acquisition strategy. (Sec 801 PL 113-291). This approach integrates
    technical requirements with contracting mechanisms and legal
    considerations to support a more rapid evolution of capabilities and
    technologies throughout the product life cycle through the use of
    architecture modularity, open systems standards, and appropriate
    business practices. A modular design coupled with an appropriately open
    business model provides a valuable mechanism for continuing competition
    and incremental upgrades, and to facilitate reuse across the joint
    force.

                                      [all]