[House Hearing, 114 Congress]
[From the U.S. Government Publishing Office]


                    A REVIEW OF RECOMMENDATIONS FOR 
                     NSF PROJECT MANAGEMENT REFORM

=======================================================================

                              JOINT HEARING

                               BEFORE THE

               SUBCOMMITTEE ON RESEARCH AND TECHNOLOGY &
                       SUBCOMMITTEE ON OVERSIGHT

              COMMITTEE ON SCIENCE, SPACE, AND TECHNOLOGY
                        HOUSE OF REPRESENTATIVES

                    ONE HUNDRED FOURTEENTH CONGRESS

                             FIRST SESSION

                               __________

                            February 4, 2016

                               __________

                           Serial No. 114-59

                               __________

 Printed for the use of the Committee on Science, Space, and Technology
 
 
 [GRAPHIC NOT AVAILABLE IN TIFF FORMAT]


       Available via the World Wide Web: http://science.house.gov
       
       
                               ___________
                               
                               
                       U.S. GOVERNMENT PUBLISHING OFFICE
20-829PDF                      WASHINGTON : 2017                       
_______________________________________________________________________________________              
For sale by the Superintendent of Documents, U.S. Government Publishing Office, 
http://bookstore.gpo.gov. For more information, contact the GPO Customer Contact Center, 
U.S. Government Publishing Office. Phone 202-512-1800, or 866-512-1800 (toll-free).
E-mail, [email protected].  
             
              
              
              COMMITTEE ON SCIENCE, SPACE, AND TECHNOLOGY

                   HON. LAMAR S. SMITH, Texas, Chair
FRANK D. LUCAS, Oklahoma             EDDIE BERNICE JOHNSON, Texas
F. JAMES SENSENBRENNER, JR.,         ZOE LOFGREN, California
    Wisconsin                        DANIEL LIPINSKI, Illinois
DANA ROHRABACHER, California         DONNA F. EDWARDS, Maryland
RANDY NEUGEBAUER, Texas              SUZANNE BONAMICI, Oregon
MICHAEL T. McCAUL, Texas             ERIC SWALWELL, California
MO BROOKS, Alabama                   ALAN GRAYSON, Florida
RANDY HULTGREN, Illinois             AMI BERA, California
BILL POSEY, Florida                  ELIZABETH H. ESTY, Connecticut
THOMAS MASSIE, Kentucky              MARC A. VEASEY, Texas
JIM BRIDENSTINE, Oklahoma            KATHERINE M. CLARK, Massachusetts
RANDY K. WEBER, Texas                DONALD S. BEYER, JR., Virginia
BILL JOHNSON, Ohio                   ED PERLMUTTER, Colorado
JOHN R. MOOLENAAR, Michigan          PAUL TONKO, New York
STEPHEN KNIGHT, California           MARK TAKANO, California
BRIAN BABIN, Texas                   BILL FOSTER, Illinois
BRUCE WESTERMAN, Arkansas
BARBARA COMSTOCK, Virginia
GARY PALMER, Alabama
BARRY LOUDERMILK, Georgia
RALPH LEE ABRAHAM, Louisiana
DRAIN LAHOOD, Illinois
                                 ------                                

                Subcommittee on Research and Technology

                 HON. BARBARA COMSTOCK, Virginia, Chair
FRANK D. LUCAS, Oklahoma             DANIEL LIPINSKI, Illinois
MICHAEL T. MCCAUL, Texas             ELIZABETH H. ESTY, Connecticut
RANDY HULTGREN, Illinois             KATHERINE M. CLARK, Massachusetts
JOHN R. MOOLENAAR, Michigan          PAUL TONKO, New York
BRUCE WESTERMAN, Arkansas            SUZANNE BONAMICI, Oregon
DAN NEWHOUSE, Washington             ERIC SWALWELL, California
GARY PALMER, Alabama                 EDDIE BERNICE JOHNSON, Texas
RALPH LEE ABRAHAM, Louisiana
LAMAR S. SMITH, Texas
                                 ------                                

                       Subcommittee on Oversight

                 HON. BARRY LOUDERMILK, Georgia, Chair
F. JAMES SENSENBRENNER, JR.,         DON BEYER, Virginia
    Wisconsin                        ALAN GRAYSON, Florida
BILL POSEY, Florida                  ZOE LOFGREN, California
THOMAS MASSIE, Kentucky              EDDIE BERNICE JOHNSON, Texas
BILL JOHNSON, Ohio
DAN NEWHOUSE, Washington
LAMAR S. SMITH, Texas
                           
                           C O N T E N T S

                            February 4, 2016

                                                                   Page
Witness List.....................................................     2

Hearing Charter..................................................     3

                           Opening Statements

Statement by Representative Barbara Comstock, Chairwoman, 
  Subcommittee on Research and Technology, Committee on Science, 
  Space, and Technology, U.S. House of Representatives...........     7
    Written Statement............................................     8

Statement by Representative Daniel Lipinski, Ranking Minority 
  Member, Subcommittee on Research and Technology, Committee on 
  Science, Space, and Technology, U.S. House of Representatives..    11
    Written Statement............................................    13

                               Witnesses:

Ms. Cynthia Heckmann, Project Director, National Academy of 
  Public Administration
    Oral Statement...............................................    16
    Written Statement............................................    20

Dr. Richard Buckius, Chief Operating Officer, National Science 
  Foundation
    Oral Statement...............................................    31
    Written Statement............................................    33

Ms. Allison Lerner, Inspector General, National Science 
  Foundation
    Oral Statement...............................................    38
    Written Statement............................................    40
Discussion.......................................................    50

             Appendix I: Answers to Post-Hearing Questions

Ms. Cynthia Heckmann, Project Director, National Academy of 
  Public Administration..........................................    58

Dr. Richard Buckius, Chief Operating Officer, National Science 
  Foundation.....................................................    62

Ms. Allison Lerner, Inspector General, National Science 
  Foundation.....................................................    66

            Appendix II: Additional Material for the Record

Statement submitted by Representative Barry Loudermilk, Chairman, 
  Subcommittee on Oversight, Committee on Science, Space, and 
  Technology, U.S. House of Representatives......................    74

Statement submitted by Representative Donald S. Beyer, Jr., 
  Ranking Minority Member, Subcommittee on Oversight, Committee 
  on Science, Space, and Technology, U.S. House of 
  Representatives................................................    75

Statement submitted by Representative Lamar S. Smith, Chairman, 
  Committee on Science, Space, and Technology, U.S. House of 
  Representatives................................................    77

Statement submitted by Representative Eddie Bernice Johnson, 
  Ranking Member, Committee on Science, Space, and Technology, 
  U.S. House of Representatives..................................    80

 
                      A REVIEW OF RECOMMENDATIONS
                   FOR NSF PROJECT MANAGEMENT REFORM

                              ----------                              


                       THURSDAY, FEBRUARY 4, 2016

                  House of Representatives,
  Subcommittee on Research and Technology &
                         Subcommittee on Oversight,
               Committee on Science, Space, and Technology,
                                                   Washington, D.C.

    The Subcommittees met, pursuant to call, at 9:41 a.m., in 
Room 2318, Rayburn House Office Building, Hon. Barbara Comstock 
[Chairwoman of the Subcommittee on Research and Technology] 
presiding.
[GRAPHICS NOT AVAILABLE IN TIFF FORMAT]

    Chairwoman Comstock. The Committee on Science, Space, and 
Technology will come to order.
    Without objection, the Chair is authorized to declare 
recesses of the Committee at any time.
    Welcome to today's hearing entitled ``A Review of 
Recommendations for NSF Project Management Reform.'' I now 
recognize myself for five minutes for an opening statement.
    Today, we will be reviewing recommendations made by the 
National Academy of Public Administration (NAPA) on how to 
improve the National Science Foundation's management of 
cooperative agreements to support the development and 
construction of large-scale research projects.
    NSF and the National Science Board commissioned the study 
in response to concerns raised by this committee, the NSF 
Inspector General, and others regarding NSF's management and 
oversight of cooperative agreements and proper stewardship of 
federal funds. NAPA assembled a committee that conducted an 
eightmonth review of NSF's practices and looked at how other 
science funding agencies like NASA and the Department of Energy 
Office of Science manage similar projects.
    One of our committee's most important responsibilities is 
to ensure that federal science agencies spend taxpayer dollars 
as effectively and efficiently as possible. Every dollar wasted 
on mismanagement is a dollar that could be spent on 
groundbreaking basic research or training future scientists.
    Our committee has held two hearings over the last year on 
NSF's management of major projects like the National Ecological 
Observatory Network (NEON) and the LSST telescope. These 
hearings have revealed that NSF needs to do more to ensure that 
taxpayer funding is not wasted on mismanagement and abuse.
    The NAPA study committee has provided 13 thoughtful 
recommendations for NSF. I look forward to hearing testimony on 
those ideas. I also look forward to hearing from NSF on how the 
Foundation plans to respond, and from the Inspector General on 
her thoughts about how some of these changes could prevent 
problems found in past audits of cooperative agreements.
    Together, I hope we can work towards ensuring taxpayer 
dollars are well-managed. American leadership in science and 
innovation is the key to our nation's future economic 
prosperity, as well as our security. We want to be strong 
advocates for federal support of basic research that advances 
science in the national interest, but we can only invest more 
in research when taxpayers have faith and confidence that their 
money is being spent wisely.
    [The prepared statement of Chairwoman Comstock follows:]
    [GRAPHICS NOT AVAILABLE IN TIFF FORMAT]
    
    Chairwoman Comstock. I thank you. And I now recognize the 
Ranking Member of the Research and Technology Subcommittee, the 
gentleman from Illinois, Mr. Lipinski, for an opening 
statement.
    Mr. Lipinski. Thank you, Chairwoman Comstock. I want to 
thank you and Chairman Loudermilk for holding this hearing to 
review the recommendations for the National Science 
Foundation's project management reform.
    In 1995, the NSF created an agency-wide budgetary account 
to promote effective planning and management in the 
Foundation's support for major research equipment and 
facilities construction, or MREFC. This account supports the 
acquisition, construction, and commissioning of major multiuser 
research facilities, which typically cost between $100-$500 
million and may take a decade to complete from planning to the 
start of full operations.
    In the early 2000s, this committee and the scientific 
community raised concerns over management and oversight of the 
MREFC account. In response, the NSF took a number of steps to 
strengthen their processes and cost controls. Over the next 
decade, the agency shepherded through many successful projects, 
which today remain important cutting-edge research facilities 
for the scientific committee.
    In 2010, the NSF Office of Inspector General began raising 
new concerns about policies and practices for awarding and 
managing large construction projects. Since then, this 
committee has held a number of oversight hearings related to 
the MREFC account and specific projects. During this period, 
disagreement arose between NSF and the OIG over what 
constitutes appropriate and/or necessary policies and practices 
with no apparent resolution in sight. This committee proposed 
draft legislation, some of it on a bipartisan basis, in an 
effort to forge a path forward.
    Then, as now, I feel that while this committee's 
prerogative is to exercise its oversight and legislative 
authority--this is very important--we must also be aware of the 
unintended consequences of micromanagement.
    Afterwards, serious structural problems were discovered 
with the management of the National Ecological Observatory 
Network, or NEON. Since December of 2014, and largely in 
response to the failures with NEON, NSF has taken a number of 
additional steps to strengthen its business practices. But the 
OIG has continued to raise red flags.
    Early last year, the National Science Board and NSF 
leadership commissioned a third-party independent review of the 
Foundation's use of cooperative agreements to support large 
projects, which would include benchmarking NSF's practices 
against those of other agencies with large scientific 
facilities.
    The result was a report by the National Association of 
Public Administration that is the subject of today's hearing. 
By all accounts, the NAPA review is thorough, thoughtful, and 
balanced. The experts on the NAPA panel came down solidly in 
support of NSF continuing to use cooperative agreements rather 
than contracts to support MREFC projects. So in my view, we can 
probably dispense with that debate.
    With respect to specific cost control policies and 
practices, NAPA largely agreed with NSF that their policies are 
not inconsistent with OMB guidelines but still urged them to 
implement stronger preventative cost controls.
    NAPA also offered several recommendations for NSF to 
strengthen its project management procedures across the agency 
and to rethink roles and responsibilities for planning, 
management, and oversight of MREFC projects.
    I understand that the IG supports the NAPA recommendations 
overall but would still urge NSF to take steps above and beyond 
those recommended in a NAPA report. I hope we will have a 
chance to discuss all this during the hearing.
    Before I conclude, I want to make two points: First, I'm 
hopeful that this report will serve not just to strengthen 
NSF's businesses practices for large projects but also to 
smooth the way towards greater trust between OIG and NSF 
management, both of whom, I'm sure, have the best interest of 
the scientific community and taxpayers in mind.
    Second, I want to highlight the very last section of the 
NAPA report entitled ``The Cost of Increased Oversight.'' It 
would be irresponsible for us to ignore the reality that it 
will not be possible for NSF to implement NAPA's 
recommendations, let alone the IG's, without increased funding 
to support such oversight. I hope that the NSF requests enough 
funding in their budget to implement the NAPA recommendations.
    In addition, I hope that all my colleagues would join me 
this spring in urging our appropriators to fully fund the 
agency's request for its management account.
    With that, I thank the witnesses for being here this 
morning, and I yield back.
    [The prepared statement of Mr. Lipinski follows:]
    [GRAPHICS NOT AVAILABLE IN TIFF FORMAT]
    
    Chairwoman Comstock. Thank you, Mr. Lipinski.
    And since this morning we had the National Prayer 
Breakfast, we have a number of our colleagues who are late or 
unavailable, so my apologies for, first, the delay on all of 
our parts, but that's what's going on. The traffic was pretty 
bad out there, too, as you probably experienced.
    But now, let me introduce our witnesses. Our first witness 
today is Ms. Cynthia Heckmann, Project Director of the National 
Academy of Public Administration's review on the NSF's use of 
cooperative agreements to support the development, 
construction, and operations of state-of-the-art, large-scale, 
multiuser research facilities.
    Prior to joining NAPA, Ms. Heckmann had an extensive career 
at the Government Accountability Office, as well as career 
experience about the executive branch and in state government. 
Ms. Heckmann received her master's of public administration 
from Northeastern University and her bachelor of arts from 
Simmons College.
    Our second witness today is Dr. Richard Buckius. Dr. 
Buckius is the Chief Operating Officer for the National Science 
Foundation. He assumed the position of COO in October 2014, 
having previously been a Senior Policy Advisor for NSF. He is 
an author and coauthor of numerous publications on the topics 
of radiation heat transfer, numerical fluid mechanics, and 
combustion. He received his bachelor's, master's, and Ph.D. in 
mechanical engineering at the University of California 
Berkeley.
    Our last witness today is Ms. Allison Lerner. Ms. Lerner is 
the Inspector General for the National Science Foundation. 
Before joining NSF in April 2009, Ms. Lerner served in many 
leadership positions at the Department of Commerce, including 
Counsel to the Inspector General. She has received several 
national awards for excellence and was selected to be a member 
of the Government Accountability and Transparency Board by the 
President in June 2011. Ms. Lerner received her law and 
undergraduate degrees from the University of Texas.
    I now recognize Ms. Heckmann for five minutes to present 
her testimony.

               TESTIMONY OF MS. CYNTHIA HECKMANN,

                       PROJECT DIRECTOR,

           NATIONAL ACADEMY OF PUBLIC ADMINISTRATION

    Ms. Heckmann. Good morning, Chairwoman Comstock, Ranking 
Member Lipinski, and Members of the Subcommittee. Thank you for 
the invitation to discuss recommendations from the Academy's 
report on NSF's use of cooperative agreements to support the 
development, construction, and operations of state-of-the-art, 
large-scale, multiuser research facilities.
    The NSF Director and National Science Board jointly 
requested this review with a focus on the agency's largest 
cooperative agreements of 100 million or more under the MREFC 
account. The Academy's panel and study team's review focused on 
agency's policies and practices governing the lifecycle of 
these projects, including issues raised by the Inspector 
General and Congressional concerns. We also looked at 
comparable agencies with large capital investment projects most 
analogous to NSF's for promising practices that might be 
transferable.
    Overall, we found that the cooperative agreements are the 
appropriate mechanism to support these efforts. We also 
acknowledge the many recent efforts by NSF has undertaken to 
implement new policies and practices to address Inspector 
General and Congressional concerns.
    Our recommendations are intended to support NSF and the 
board's commitment to improving core business practices and the 
agency's key performance goal of ensuring program integrity and 
responsible stewardship of major research facilities. We've 
included suggested implementation steps for each 
recommendation, and where appropriate in our text, highlighted 
other agencies' promising practices. Our recommendations 
provide a number of actions and options to strengthen oversight 
and enhance agency governance, practices, and processes.
    For today, we are grouping the recommendations as follows: 
One, policies on cost estimating and cost analysis. NSF has a 
strengthened its methodological approach to cost estimating and 
analysis and updated its policies in the Large Facilities 
Manual to reflect these efforts. This includes a tightened 
control environment and improved cost surveillance strategies.
    However, we identified some additional opportunities to 
bolster cost analysis requirements for award recipients and 
improve internal agency processes for detecting potential 
issues in proposals. In terms of cost estimating, we recommend 
that NSF change current language in the manual, making it clear 
that award recipients are expected, not just encouraged, to 
follow guidance in the GAO's Cost Estimating and Assessment 
Guide and Schedule Assessment Guide when developing those cost 
and schedule estimates.
    In terms of cost analysis, we recommend that NSF require 
that decisions not to act on recommendations from the pre-award 
cost analyses conducted by their cost analysis and audit 
resolution branch be reviewed by the Large Facilities Office 
and forwarded to the Chief Financial Officer for a final 
determination. These decisions should be documented in writing 
and shared with the MREFC panel prior to the release of any 
award funds.
    Two, managing budgeting contingency: The use of contingency 
is a commonly accepted business and project management risk 
mitigation practice for construction projects. NSF's position 
is that contingency funds should be retained and managed by the 
award recipients. However, the common federal agency practice 
is for the agency to hold either all or a majority percentage 
of the contingency funds. By holding at least a percentage of 
these funds, NSF could further strengthen internal controls and 
accountability. Therefore, we recommend NSF adopt the practice 
of retaining control of a portion of the award recipient's 
contingency funds and distributing them with other incremental 
funds as needed.
    Three, management fee policies and processes: NSF includes 
a management fee in several of its cooperative agreements to 
cover business expenses related to construction or operations 
that would otherwise be non-reimbursable under governing cost 
principles. However, most of the examples provided to us 
appeared to be costs that would either be covered under 
indirect costs or contingency.
    Despite recent changes NSF has made, it's been its policy 
to provide clearer guidance on appropriate and inappropriate 
uses of management fee. The IG continues to have concerns. 
Furthermore, OMB's efforts in the uniform guidance have not 
resulted in greater clarity on some of the most controversial 
aspects of management fee. We recommend NSF eliminate the 
practice of including management fee in cooperative agreements 
on future projects.
    Four, governing for effective stewardship: Governing 
bodies, agency internal coordinating committees, and advisory 
review panels play important roles in governance, development, 
and execution of MREFC projects. We've offered recommendations 
for each of these groups to address ambiguity and further build 
on the agency's capacity to effectively manage these projects.
    NSF and NSB roles and responsibilities, starting with them, 
the statutory joint leadership model of an appointed director 
and board is quite unique among federal agencies. The current 
working relationship appears to be working well. It's not 
always been the case, and we've recommended that they establish 
and publish a joint duties and responsibilities document to 
institutionalize the role.
    The MREFC panel is only engaged right now in design review 
phases of an MREFC project. For more consistent and informed 
oversight, the panel should be involved throughout the 
lifecycle of a project's development and implementation, really 
reviewing project status on a set schedule.
    In terms of advisory committees and review panels, the 
panels need to include experts with requisite project and 
financial management knowledge and experience, and we are 
recommending that NSF explicitly identify the requirements and 
add them to the selection criteria for those external 
reviewers. We also recommend that NSF establish a FACA 
committee for the director to use as a sounding board.
    Large Facilities Office, the role and the placement of this 
office has been subject to debate. We concluded that the 
organizational placement is not as important as project 
management roles and responsibilities. Its current placement is 
also logical within the office of the CFO, as its focus is on 
strengthening project management assistance. We did make 
recommendations in terms of hiring two additional FTEs and 
directing the MREFC charter be revised to change the status of 
the LFO head from nonvoting to a full voting member.
    In terms of planning and portfolio management process, we 
have a recommendation in terms of the annual facilities plan 
where there's some confusion over what its role is and ask that 
NSF evaluate it in terms of its agency's current planning and 
strategic planning processes.
    Finally, project management skills are needed for effective 
oversight. NSF has strong--the folks have strong scientific 
credentials, not as likely to have the same corollary skills 
and experience in project management.
    We have three recommendations for project management. First 
is to identify skill requirements for internal staff and 
develop corollary project management training for the staff; 
second, to require award recipient project managers be 
certified in project management and specified minimum project 
experience requirement threshold in the actual terms and 
conditions; and lastly, establish formal communities of 
practice to share best practices and lessons learned, a 
requirement for all MREFC projects.
    In closing, let me just reiterate NSF has undertaken a wide 
range of actions to improve the oversight of these MREFC 
projects. The efforts are a work in progress. They appear 
headed in the right direction. Given the initiatives underway 
and the culture change needed to socialize these changes, time 
will be needed. And from a continuous improvement perspective, 
time will be needed to see what is working and what requires 
further adjustments.
    Thank you for the opportunity to share these views.
    [The prepared statement of Ms. Heckmann follows:]
    [GRAPHICS NOT AVAILABLE IN TIFF FORMAT]
    
    Chairwoman Comstock. Okay. Thank you, Ms. Heckmann.
    And now, Dr. Buckius.

               TESTIMONY OF DR. RICHARD BUCKIUS,

                    CHIEF OPERATING OFFICER,

                  NATIONAL SCIENCE FOUNDATION

    Dr. Buckius. Madam Chairwoman, Members of the Committee, 
thank you for the opportunity to discuss NSF's oversight of 
major research facility infrastructure projects and NSF's 
response to the recent NAPA report.
    The National Science Foundation supports fundamental 
research in frontiers of knowledge across all fields of science 
and engineering. As part of that mission, NSF supports a broad 
array of 28 major transformative research facilities that are 
geographically distributed observatories, telescopes, mobile 
platforms such as research vessels and aircraft. In a total, 
the operational cost of NSF's entire portfolio of research 
infrastructure requires approximately 1.2 billion per year.
    I'd like to start by thanking NAPA for its rigorous review 
of NSF's use of cooperative agreements to support large-scale 
investments in science and technology. The members of this 
committee, the Inspector General, and the experts at NAPA have 
all been exceptionally helpful to the Foundation in identifying 
areas where NSF can improve and make our oversight of critical 
facilities even stronger.
    We appreciate the panel's overall conclusion that 
cooperative agreements are the appropriate mechanism for the 
agency to use for construction and operations of large 
facilities. In using this funding mechanism, the Foundation is 
committed to improving the rigor and oversight of its processes 
and deploying appropriate levels of internal projects, 
programmatic, and financial management expertise.
    In order to respond to the report, the Director has created 
an implementation team to address each of the recommendations. 
And I will divide this conversation into two broad areas: 
first, business practices; and second, oversight, 
accountability, and stewardship.
    Let me start by saying that NSF will implement all 13 
recommendations in some form. In the case of the business 
practices, NSF will provide stronger requirements of cost 
estimating and adjudication of cost analysis findings, as 
recommended by the panel, and will revamp the processes of 
obligating and allocating contingency based on the project's 
level of risk. The panel's comparison with other agencies is 
extremely useful, and we will follow up with these agencies for 
more information detailing the process of partial withholding 
of contingency, while also ensuring NSF's continued compliance 
with uniform guidance.
    With regard to management fee, the Foundation is continuing 
to implement the stringent policy we put in place last year, 
and we are currently considering the alternatives set forth in 
the panel's report. As I've previously testified before this 
committee, NSF acknowledges that some awardees should have 
shown better judgment in the use of their management fee even 
if they were not in violation of any law or regulation 
governing the use of these funds. The Foundation has learned a 
number of lessons about management fees, and we have designed 
the policy around the lessons learned.
    While many of the panel's recommendations are implementable 
within a relatively short time, I would note that we believe 
the management fee topic will likely take a more thorough 
analysis on the part of NSF and some of the other 
recommendations.
    The panel's holistic view of NSF's oversight, 
accountability, and stewardship of large research facilities is 
welcomed by the Foundation. We are considering all the 
recommendations on the roles and responsibilities of the 
National Science Board, MREFC Panel, and the Office of the 
Director as a single endeavor. We plan to enhance the role of 
independent expertise in project and financial management, as 
well as the cross-agency sharing of best practices.
    The National Science Board examined the panel's 
recommendations over the course of the last few days, and this 
is our first NSB meeting since the release of the report. We 
look forward to working together with the board to strengthen 
our oversight, accountability, and stewardship, as recommended 
by the panel.
    NSF is committed to developing project management skills, 
experience, and training for both foundation staff and MREFC 
project managers.
    I'd like to clarify for the Committee that any changes NSF 
undertakes can and will apply to existing--not just new--
cooperative agreements. As the Foundation improves its 
processes, we can revise, as appropriate, existing agreements 
to strengthen oversight.
    In closing, I'd like to reiterate how much NSF welcomes the 
NAPA report and its recommendations. It is only with strong 
support from the inspector general, external experts like NAPA, 
and Congress that complete oversight of taxpayer resources can 
be achieved, and we appreciate all these efforts.
    Thank you again for the opportunity to testify, and I will 
be pleased to answer any of your questions.
    [The prepared statement of Dr. Buckius follows:]
    [GRAPHICS NOT AVAILABLE IN TIFF FORMAT]
    
    Chairwoman Comstock. I now recognize Ms. Lerner for five 
minutes.

                TESTIMONY OF MS. ALLISON LERNER,

                       INSPECTOR GENERAL,

                  NATIONAL SCIENCE FOUNDATION

    Ms. Lerner. Thank you, Madam Chairwoman, and Members of the 
Subcommittee. I appreciate the opportunity to discuss my 
office's views on the National Academy of Public 
Administration's report on NSF's use of cooperative agreements 
to support large-scale investment in research.
    Since 2010, my office has issued 28 reports containing more 
than 80 recommendations that relate to NSF's management of 
cooperative agreements for large-facility projects. Beginning 
with our audits of over $1.1 billion in proposal costs for 
three construction projects, the issues we identified have 
occurred at multiple facilities and ultimately contributed to 
the decision to procure the NAPA report. I commend Drs. Cordova 
and David for their vision in commissioning this effort, as 
well as the NAPA panel and staff who conducted the review and 
prepared the outstanding final product.
    The report sets forth practical recommendations that, if 
implemented by the agency in a timely fashion, will 
significantly improve NSF's ability to ensure accountability 
over these high-risk, high-dollar projects, and thus go a long 
way toward addressing many of the issues my office has raised.
    Our office supports all of the NAPA report's 
recommendations, and I'll briefly discuss six of them. First, 
the report recommended that NSF require review of exceptions to 
recommendations from pre-award cost analyses conducted by the 
Foundation's in-house analyst, or CAAR. Because our work has 
identified instances in which CAAR's important concerns have 
not been addressed in a timely fashion, we share NAPA's belief 
that actions necessary to ensure that the critical issues CAAR 
identifies are brought to the attention of senior officials and 
panelists and resolved. A similar process should be followed 
for other important internal analyses and for the findings of 
the incurred cost audits NSF will soon be procuring.
    We wholeheartedly agree with, NAPA's recommendation that 
NSF should retain control over a portion of awardees' 
contingency funds. Our previous audits have found that 
construction budgets for NSF's large-facility projects 
contained tens of millions of dollars in unsupported 
contingency costs. The risk of misuse of these funds is 
heightened because NSF does not require awardees to track 
expenditure of funds in their accounting systems, which makes 
it all but impossible for us to audit these significant 
expenditures.
    With respect to the panel's recommendation to eliminate the 
use of management fee, our September 2015 Alert Memo documented 
positive steps NSF has taken to strengthen its fee policy and 
noted improvements that are needed to determine if management 
fee is necessary for an awardee's financial viability. If NSF 
decides to continue providing such fees, it should consider and 
address the issues noted in our Alert Memo.
    OIG also agrees with three recommendations that focus on 
the need for NSF to strengthen business and project management 
skills within the agency and at its awardees. The culture 
change needed to implement these recommendations is clearly 
warranted because, as the NEON project has illustrated, 
deficiencies in a project's business processes have the 
potential to undermine its scientific goals if the project must 
be de-scoped due to cost or schedule overruns.
    Finally, I'd like to note three areas that were not covered 
by the NAPA recommendations but which are still critical to 
NSF's ability to manage large-facility projects. First, our 
recent work has found that NSF does not require the earned 
value management systems for its large-facility projects to be 
certified or the data they contain to be verified. The poor 
quality of the information in these reports for the NEON 
project was one of the reasons why the cost overrun for that 
project was undetected for so long. NSF is currently evaluating 
what actions it should take to ensure the quality of EVM data.
    Second, our office has repeatedly recommended that NSF 
require annual incurred cost submissions for projects over $50 
million. Such submissions provide critical information about 
how an awardee has spent federal funds and are essential tools 
for both NSF and the OIG. In two very recent reviews, however, 
auditors found the submissions for two large construction 
projects were not adequate to initiate an audit because so much 
important information was missing from them. NSF is currently 
seeking OMB clearance related to such submissions.
    Finally, many of our reports have stressed the need for 
annual incurred cost audits of large-facility recipients. Such 
audits provide NSF with the best evidence of how awardees are 
actually expending federal funds. While we commend NSF for 
deciding to require incurred cost audits at project completion, 
by waiting until the end to obtain an audit, NSF will miss 
opportunities to identify and correct problematic expenditures 
in the project's early days. Given the level of risk we've 
identified with these projects, annual or at least biannual 
audits are clearly warranted.
    In closing, NSF's swift and decisive implementation of the 
NAPA report's recommendation will have a significant positive 
impact on the Foundation's ability to manage high-dollar, high-
risk, large-facility projects. I look forward to continuing to 
work with Congress, the Foundation, and the National Science 
Board toward this goal.
    And I'd be happy to answer any questions.
    [The prepared statement of Ms. Lerner follows:]
    [GRAPHICS NOT AVAILABLE IN TIFF FORMAT]
    
    Chairwoman Comstock. Thank you. I now recognize myself for 
five minutes for questions. I thank the witnesses.
    And I did want to let you know we got word that our 
colleagues are stuck on a bus trying to get back from the 
prayer breakfast, so again, our apologies for our other 
colleagues who aren't here this morning. I guess no good deed 
goes unpunished, right?
    Okay, Ms. Lerner, do you believe that the recommendations 
made by NAPA on how NSF should handle cost proposal analyses 
and audits goes far enough to fix cost problems that you have 
found in your audit work of large-scale projects?
    Ms. Lerner. Thank you. We are pleased that NSF is committed 
to undertaking one of the eight actions noted in GAO's cost 
estimating guide. We're concerned, however, that for the large, 
high-dollar, high-risk projects, that some of those options, as 
a GAO notes in the guide itself--are not rigorous enough to 
provide the information that's necessary. And we would 
recommend that, for the high-dollar, high-risk construction 
projects, that either a pre-award audit be done or an 
independent cost estimate be obtained and that some of the less 
rigorous options like a sufficiency review, which was used for 
the LSST project, not be pursued.
    Chairwoman Comstock. Thank you. And, Ms. Heckmann, the IG 
testified that one issue the NAPA report did not make a 
recommendation on that she considers critical--she was talking 
about in her testimony is the use of an earned value management 
system for tracking and measuring project cost and schedule 
performance. The NAPA report did note the Department of Energy 
and NASA have well-developed EVM policies in place. Why did 
this study panel stop short of making a recommendation on using 
a certified EVM system for projects?
    Ms. Heckmann. At the time of our review, we understood that 
the National Science Foundation was in the midst of reviewing 
its EVM policies and processes and working to standardize the 
processes and be sure to adopt the standards that are federal 
standards. So what we did in terms of looking at other 
agencies' practices was point out the promising practices that 
we felt were transferable rather than really focusing a great 
deal of time on the EVM. It was clear to us that they were 
committed to using EVM.
    What we also felt was important was that their staff--
again, getting back to the project management perspective--
needed to understand what EVM really is and how to read EVM 
charts, et cetera, and how to really do the oversight that's 
necessary for an EVM. So the bottom line there is that we saw 
it as kind of a work in progress, and we were not at a point to 
really make a recommendation further. We would really need to 
see what in fact they have done.
    Chairwoman Comstock. Okay. Thank you.
    And, Doctor, would you like to address that also?
    Dr. Buckius. Ms. Heckmann is exactly right. We are in the 
midst of evaluating all aspects of EVM. I'd go so far to say, 
though, that it's important, I think, to look at the 
validation. That certification is one aspect, and we're going 
to consider that, but the validation is the key because that's 
where you get the answers. And so, I think the recommendation 
is well-placed and we're taking it very seriously.
    Chairwoman Comstock. All right. Thank you.
    Ms. Lerner, did you want to add anything to that or----
    Ms. Lerner. [Nonverbal response.]
    Chairwoman Comstock. Okay. Thank you.
    And I will now yield to my colleague, Mr. Lipinski.
    Mr. Lipinski. Thank you. As they sometimes say, you need to 
spend money to make money. You also have to spend some money in 
order to save money when we're talking about making some of 
these changes and having the proper oversight when it comes to 
MREFC.
    There are a lot of good recommendations in the NAPA report, 
and we know some of them will cost more than others to 
implement. And the Inspector General would like to see the NSF 
go much further on some fronts than NAPA recommended. So I just 
want to ask all three of you, given the cost of implementing 
some of the recommended reforms, how do you think NSF should 
prioritize the various recommendations and ensure that the 
enhanced oversight does not impede NSF's scientific mission? We 
sure would like to see them all done as, you know, quickly as 
possible, but we know that there's probably going to have to be 
some prioritization done in terms of which ones are done first. 
So I'll start with Ms. Heckmann.
    Ms. Heckmann. Thank you. I--we think it's really critical 
that they develop the project management expertise internally 
to be able to really perform the stewardship responsibilities 
that are necessary. So in terms of the role of the LFO and what 
it is doing to ensure that there's consistency across the 
organization and the practices, as well as providing the 
support and the assurance that's necessary for stewardship, we 
feel that those are very critical.
    Obviously, in the area of contingency and management fees, 
management fee--I agree with Dr. Buckius--will take longer. 
It's a more complicated issue and there's no great guidance out 
there that's really--kind of sets the stage for where to go in 
the future. I mean, that will take some time. And in fact we 
made a recommendation--we--in the text of our report, we decide 
that if there's some specific requirements that NSF projects 
have that do not really fit the current definitions, it may 
make sense to really identify those and seek special 
legislative authority there.
    Bottom line, in terms of balance, it is a balancing act. 
Budgets are tight. There's been a lot of initiatives underway. 
NSF will need to really step back and do detailed workload and 
workforce analyses to determine what the next steps are. But, 
you know, dealing with contingency issues, dealing with the--
how they manage--the MREFC panel I think is one that's a very 
easy fix, wouldn't require a lot of time, effort, or cost, and 
would really help them in terms of ensuring that they have 
processes in place as well for monitoring and really shoring up 
their oversight processes.
    Mr. Lipinski. Okay. Let me go to Ms. Lerner, and then we'll 
go to Dr. Buckius.
    Ms. Lerner. I certainly agree that there are--the human 
capital investment that needs to be made in ensuring that NSF 
and its awardees have necessary program management expertise is 
critical. In terms of financial investments, I do believe that 
investments in strong pre-award and post-award audits will pay 
dividends in the information that they provide NSF managers 
about how the funds plan to be used and are actually being 
used. When you look at the cost of some of these projects which 
are, you know, 350 to almost $500 million, investing a couple 
hundred thousand dollars in an audit seems a very appropriate 
thing to do because, especially since it's one of the only ways 
NSF post-award can actually see how its funds are being used.
    Mr. Lipinski. Thank you. Dr. Buckius?
    Dr. Buckius. Well, thank you for this question. It is a 
balance. So we've got 13 recommendations here. A number of them 
can be implemented with relatively low cost. Modifying text and 
ensuring that our awardees are responding, I think, you know--I 
would argue that's moderate.
    Regarding the expertise, I think we've already made some 
steps in that direction by our most recent hires under the 
existing LFO allocation--Large Facility Office, excuse me--we 
are allocating two more positions to that. It's already been 
done. They're going to be posted.
    Our attitude is, in the case of LFO, they've been 
understaffed, I would argue, for a number of years, and so it's 
time for us to actually make that happen. This comes out of our 
AOAM, administration operations account, and I think you'll 
read in the '16 and '17 budget it's a strained account already, 
but this is necessary.
    The comments on the MREFC and--are interesting because they 
also don't require a lot of cost from the point of view of 
people. They're going to require a lot of cost in terms of 
time, though, because we now believe--and I think the NAPA 
report clearly showed--we need to do complete lifecycle 
analysis. We've been doing gate analysis and we've been 
focusing on when things move from one gate to another. We need 
to ensure continuous oversight. So that'll be time, time is 
money, and so that'll cost us in the long run, but I think it's 
absolutely necessary.
    So as you've noted, it's going to be a balance to ensure 
that we actually get the right oversight, right care, and 
obviously to balance our budgets that we have to do. Thank you 
for the question.
    Mr. Lipinski. Thank you. I'm out of time so I'll yield 
back.
    Chairwoman Comstock. We are short of members if you'd like 
a second round.
    Mr. Lipinski. Well, let me very quickly--Dr. Buckius, I 
don't know if you'll be able to answer this, you know, five 
days before the budget request is released, but can we expect 
to see an increased request for the NSF management account in 
order to pay for some of this reform?
    Dr. Buckius. The request we got in front of you will help 
us a lot in order to make sure that we're making progress. I'm 
thinking long-term '18, '19, I think that those are the areas 
that we're going to have to spend a lot more care regarding 
these kinds of people issues. 'Seventeen is on the mark.
    Mr. Lipinski. Thank you. One other thing if I could ask 
you, Dr. Buckius, I understand NSF is taking under 
consideration the NAPA recommendation to eliminate management 
fee and is initiating an evaluation of alternatives. Is there 
any kind of preview you can give us at this early date? Do you 
have any time frame in mind, including the analysis?
    Dr. Buckius. Sure. Let me--this is the issue that we talked 
about in my last testimony. So we implemented what I would call 
an extremely strict policy on management fees. We write--or we 
ask now for the positive; that is, what are you going to do 
with it, and we also indicate what you can't do with it. We 
implemented this in June of '15. We're in the midst of now 
assessing what the impact has been by that strict policy. Our 
deadline there is May, and we're going to work towards May in 
order to make the assessment of how that particular policy that 
was implemented in June has impacted the folks that actually 
are using it.
    The NAPA report also makes a number of recommendations, as 
you just heard from Ms. Heckmann. Some of those I think we 
really need to consider and see how we can actually use other 
mechanisms in order to be able to get these kinds of resources 
in the hands of the folks that we--so remember the whole goal 
here. The whole goal is to provide agencies, organizations the 
opportunity to compete for these awards. We need competing 
proposals if we're going to fund these kinds of entities. And 
so the management fee permits them to do certain things that 
they need to do.
    So we need to assess, as the NAPA report has indicated, how 
we can figure--how--what other mechanisms we can possibly use 
other than management fee. So our goal is to try to do that 
over the--so we've got these two issues. Our goal is to try to 
do that over the next months. We have to work with our National 
Science Board, though, to ensure that they're on board.
    Mr. Lipinski. Very good. Thank you. I yield back again.
    Chairwoman Comstock. Okay. And I now recognize Ms. 
Bonamici.
    Ms. Bonamici. Thank you very much, Madam Chair.
    I want to start--even though Chairman Smith isn't here yet, 
I just wanted to start by thanking him on the record for 
organizing a trip with Dr. Cordova a little more than a year 
ago to Antarctica and some of the--with some of the NSF staff 
and Dr. Cordova. And I come to this discussion with that 
background because--going on that trip, which was really an 
enlightening experience and fascinating--understanding not only 
the importance of NSF investments and facilities but also the 
challenges. And if you want to go to a place where there are 
challenges with NSF facilities, it's a place like Antarctica. 
So that brought to the other committee members who went on that 
trip, I think, a perspective of appreciation of some of the 
challenges but also the importance.
    So Ms. Heckmann described in her testimony NSF will need to 
make some hard funding decisions that address the demand for 
more rigorous accountability systems balanced against the 
mission to advance science. Obviously, there are many good 
recommendations in the NAPA report, and some will cost more 
than others to implement, as we've heard, and the IG would like 
to see NSF go farther in some areas than NAPA recommends. I 
don't think anyone would argue that it's important to have 
appropriate oversight, but obviously, there need to be 
priorities set.
    So I want to ask you, Dr. Buckius--are we saying your name 
right? Close?
    Dr. Buckius. You can call me Richard. It's Buckius.
    Ms. Bonamici. Buckius, thank you. To expand a little bit, 
we've had some discussion about this already, but given the 
cost of implementing some of the recommended reforms, how 
should NSF go through the prioritization process of all the 
various recommendations while you're also trying to ensure that 
the enhanced oversight does not impede NSF scientific missions? 
So could you talk a little bit about that process and how 
you're making the decisions, the priority decisions?
    Dr. Buckius. Yes, so let me try--first of all, your trip to 
Antarctica probably demonstrated a lot of the facility issues 
that we deal with----
    Ms. Bonamici. Right.
    Dr. Buckius. --on a regular basis. I mean it's--that 
particular facility has almost everything exaggerated from the 
point of view of difficulties.
    Ms. Bonamici. Right.
    Dr. Buckius. And yet the science that we're doing there and 
the leadership for this country in Antarctica is tremendous. So 
our approach has been on a number of these audit 
recommendations is to look at the risk and try to balance the 
audits related to risk so that we aren't going in and proposing 
to audit, say, incurred cost audits on every single proposal. I 
mean, that just isn't--it won't ever happen. It's impossible. 
So what we do is we try to assess the level of risk in a 
project, and then we try to look at those that have the largest 
dollar value. So we've set limits in the case of incurred costs 
at 100 million, and then assess the risk of those, and then go 
and take a look at what we need to do. We propose to do them at 
the end of the award.
    If necessary, though, if we see a risky issue, we can jump 
in and audit those earlier. So that's the approach we want to 
take so that we don't burden ourselves overly with an audit 
process that might not give us the necessary information that 
we need.
    So remember, though, we are--say in the comment that Ms. 
Lerner made regarding EVM, we annually look at this, okay? We 
look at them monthly, so it's not like we're not paying 
attention to it. Her recommendation, which is the one we have 
to look at, on certification and validation, we probably can do 
some better in the validation setting. And so that's kind of 
our goal. Our goal is to try to go in and look at the most 
risky issues, try to look at the biggest projects, and invest 
in those areas. So that's our objective.
    Ms. Bonamici. Thank you. And could you also expand a little 
bit on the timeline and that process of developing the 
comprehensive organizational chart to incorporate the 
recommendations? You mentioned LFO and additional investment 
needed there. But what is the process of going through that 
timeline? Give us a little more details about that and what if 
any outside expertise will help you through this process of the 
timeline and developing the recommendations?
    Dr. Buckius. I would argue we meet on this subject every 
week as a group with the Director, maybe every other week, 
depending upon travel. So this is an urgent issue. We are going 
to implement as many of these recommendations as we can as soon 
as we can.
    Now, let's talk a little bit about the FACA committee. We 
think that's an excellent recommendation. We are trying to 
figure out how to do that in a stepwise process. So we're at 
this point talking about trying to use a subcommittee of one of 
our current ACs so that we can get that moving ASAP with the 
idea in the long run of creating a separate committee.
    So again, the idea--we want to implement these as fast as 
we possibly can because we realize this is going to be helpful 
to all of us. At the same time, in order to set up a FACA 
committee, it takes a fair amount of work, okay, in order to be 
able to get that all constructed. And so we want to do that, 
but we've got to figure out ways to get there first.
    The LFO issue--Matt Hawkins is our LFO head, and I'm sure 
he would tell you the same thing--we were understaffed, I would 
argue, 3, four years ago in that area, okay? And so, like 
you've heard, I was here in the last year-and-a-half or so, and 
we're staffing that up, okay? It has to be a priority, and so 
we're going to do it. And so, like I said, we've already got 
two more postings coming on in that. And these are the kind of 
investments that the agency has to make. And so the cost is one 
thing. Implementation is as fast as we can.
    Now, the management fee is the one that's probably going to 
be the slowest because we've got to take a look at how we're 
going to ensure that we have competitors. And we just don't 
want to have some unintended consequences by implementation of 
a policy that would actually alienate or give us the inability 
to have proposers in these areas.
    Ms. Bonamici. Terrific. Well, thank you. And I see that I'm 
out of time, but I just want to say that we appreciate the 
progress made to date and appreciate the Director's and your 
willingness to come back and report to us, so thank you.
    I yield back. Thank you, Madam Chair.
    Chairwoman Comstock. Thank you. And I'm just going to ask 
an additional question.
    Ms. Lerner, in Dr. Buckius's testimony, he notes that NSF 
manages 28 major research facilities, and that NEON is only one 
facility within this portfolio that NSF needs to consider when 
looking at its policies and procedures related to proper 
oversight. This implies that many of the oversight management 
issues identified are isolated to NEON. In your audits and 
reviews of all the major research facilities in NSF's 
portfolio, have you found similar issues in the oversight and 
management of other projects?
    Ms. Lerner. As I noted at the start of my remarks, you 
know, we've done 28 reports with 80 recommendations, and those 
findings are not limited to NEON. We've also found issues and 
concerns at the LSST project, at the DKIST project when it was 
called ATST, and with the OOI project, and most recently, with 
incurred cost submissions for the AUI project. So the issues 
and the challenges that we've identified are not limited.
    Chairwoman Comstock. Okay. Well,the bus apparently is still 
en route, so again, my apologies. If my colleagues have any 
additional question, I'd be happy to yield to them.
    But we'll keep the record open for additional questions. I 
appreciate your testimony here this morning and helping us in 
moving forward on these issues. And we look forward to 
continuing to work with you.
    And, yes, the record will remain open for two weeks for 
additional written comments and written questions from our 
Members who both are here and those who weren't able to join 
us. So thank you. And the meeting is now adjourned.
    Dr. Buckius. Thank you.
    [Whereupon, at 10:30 a.m., the Subcommittees were 
adjourned.]

                               Appendix I

                              ----------                              


                   Answers to Post-Hearing Questions

[GRAPHICS NOT AVAILABLE IN TIFF FORMAT]


                              Appendix II

                              ----------                              


                   Additional Material for the Record

[GRAPHICS NOT AVAILABLE IN TIFF FORMAT]
                                 
                                 [all]