[House Hearing, 114 Congress]
[From the U.S. Government Publishing Office]
A REVIEW OF RECOMMENDATIONS FOR
NSF PROJECT MANAGEMENT REFORM
=======================================================================
JOINT HEARING
BEFORE THE
SUBCOMMITTEE ON RESEARCH AND TECHNOLOGY &
SUBCOMMITTEE ON OVERSIGHT
COMMITTEE ON SCIENCE, SPACE, AND TECHNOLOGY
HOUSE OF REPRESENTATIVES
ONE HUNDRED FOURTEENTH CONGRESS
FIRST SESSION
__________
February 4, 2016
__________
Serial No. 114-59
__________
Printed for the use of the Committee on Science, Space, and Technology
[GRAPHIC NOT AVAILABLE IN TIFF FORMAT]
Available via the World Wide Web: http://science.house.gov
___________
U.S. GOVERNMENT PUBLISHING OFFICE
20-829PDF WASHINGTON : 2017
_______________________________________________________________________________________
For sale by the Superintendent of Documents, U.S. Government Publishing Office,
http://bookstore.gpo.gov. For more information, contact the GPO Customer Contact Center,
U.S. Government Publishing Office. Phone 202-512-1800, or 866-512-1800 (toll-free).
E-mail, [email protected].
COMMITTEE ON SCIENCE, SPACE, AND TECHNOLOGY
HON. LAMAR S. SMITH, Texas, Chair
FRANK D. LUCAS, Oklahoma EDDIE BERNICE JOHNSON, Texas
F. JAMES SENSENBRENNER, JR., ZOE LOFGREN, California
Wisconsin DANIEL LIPINSKI, Illinois
DANA ROHRABACHER, California DONNA F. EDWARDS, Maryland
RANDY NEUGEBAUER, Texas SUZANNE BONAMICI, Oregon
MICHAEL T. McCAUL, Texas ERIC SWALWELL, California
MO BROOKS, Alabama ALAN GRAYSON, Florida
RANDY HULTGREN, Illinois AMI BERA, California
BILL POSEY, Florida ELIZABETH H. ESTY, Connecticut
THOMAS MASSIE, Kentucky MARC A. VEASEY, Texas
JIM BRIDENSTINE, Oklahoma KATHERINE M. CLARK, Massachusetts
RANDY K. WEBER, Texas DONALD S. BEYER, JR., Virginia
BILL JOHNSON, Ohio ED PERLMUTTER, Colorado
JOHN R. MOOLENAAR, Michigan PAUL TONKO, New York
STEPHEN KNIGHT, California MARK TAKANO, California
BRIAN BABIN, Texas BILL FOSTER, Illinois
BRUCE WESTERMAN, Arkansas
BARBARA COMSTOCK, Virginia
GARY PALMER, Alabama
BARRY LOUDERMILK, Georgia
RALPH LEE ABRAHAM, Louisiana
DRAIN LAHOOD, Illinois
------
Subcommittee on Research and Technology
HON. BARBARA COMSTOCK, Virginia, Chair
FRANK D. LUCAS, Oklahoma DANIEL LIPINSKI, Illinois
MICHAEL T. MCCAUL, Texas ELIZABETH H. ESTY, Connecticut
RANDY HULTGREN, Illinois KATHERINE M. CLARK, Massachusetts
JOHN R. MOOLENAAR, Michigan PAUL TONKO, New York
BRUCE WESTERMAN, Arkansas SUZANNE BONAMICI, Oregon
DAN NEWHOUSE, Washington ERIC SWALWELL, California
GARY PALMER, Alabama EDDIE BERNICE JOHNSON, Texas
RALPH LEE ABRAHAM, Louisiana
LAMAR S. SMITH, Texas
------
Subcommittee on Oversight
HON. BARRY LOUDERMILK, Georgia, Chair
F. JAMES SENSENBRENNER, JR., DON BEYER, Virginia
Wisconsin ALAN GRAYSON, Florida
BILL POSEY, Florida ZOE LOFGREN, California
THOMAS MASSIE, Kentucky EDDIE BERNICE JOHNSON, Texas
BILL JOHNSON, Ohio
DAN NEWHOUSE, Washington
LAMAR S. SMITH, Texas
C O N T E N T S
February 4, 2016
Page
Witness List..................................................... 2
Hearing Charter.................................................. 3
Opening Statements
Statement by Representative Barbara Comstock, Chairwoman,
Subcommittee on Research and Technology, Committee on Science,
Space, and Technology, U.S. House of Representatives........... 7
Written Statement............................................ 8
Statement by Representative Daniel Lipinski, Ranking Minority
Member, Subcommittee on Research and Technology, Committee on
Science, Space, and Technology, U.S. House of Representatives.. 11
Written Statement............................................ 13
Witnesses:
Ms. Cynthia Heckmann, Project Director, National Academy of
Public Administration
Oral Statement............................................... 16
Written Statement............................................ 20
Dr. Richard Buckius, Chief Operating Officer, National Science
Foundation
Oral Statement............................................... 31
Written Statement............................................ 33
Ms. Allison Lerner, Inspector General, National Science
Foundation
Oral Statement............................................... 38
Written Statement............................................ 40
Discussion....................................................... 50
Appendix I: Answers to Post-Hearing Questions
Ms. Cynthia Heckmann, Project Director, National Academy of
Public Administration.......................................... 58
Dr. Richard Buckius, Chief Operating Officer, National Science
Foundation..................................................... 62
Ms. Allison Lerner, Inspector General, National Science
Foundation..................................................... 66
Appendix II: Additional Material for the Record
Statement submitted by Representative Barry Loudermilk, Chairman,
Subcommittee on Oversight, Committee on Science, Space, and
Technology, U.S. House of Representatives...................... 74
Statement submitted by Representative Donald S. Beyer, Jr.,
Ranking Minority Member, Subcommittee on Oversight, Committee
on Science, Space, and Technology, U.S. House of
Representatives................................................ 75
Statement submitted by Representative Lamar S. Smith, Chairman,
Committee on Science, Space, and Technology, U.S. House of
Representatives................................................ 77
Statement submitted by Representative Eddie Bernice Johnson,
Ranking Member, Committee on Science, Space, and Technology,
U.S. House of Representatives.................................. 80
A REVIEW OF RECOMMENDATIONS
FOR NSF PROJECT MANAGEMENT REFORM
----------
THURSDAY, FEBRUARY 4, 2016
House of Representatives,
Subcommittee on Research and Technology &
Subcommittee on Oversight,
Committee on Science, Space, and Technology,
Washington, D.C.
The Subcommittees met, pursuant to call, at 9:41 a.m., in
Room 2318, Rayburn House Office Building, Hon. Barbara Comstock
[Chairwoman of the Subcommittee on Research and Technology]
presiding.
[GRAPHICS NOT AVAILABLE IN TIFF FORMAT]
Chairwoman Comstock. The Committee on Science, Space, and
Technology will come to order.
Without objection, the Chair is authorized to declare
recesses of the Committee at any time.
Welcome to today's hearing entitled ``A Review of
Recommendations for NSF Project Management Reform.'' I now
recognize myself for five minutes for an opening statement.
Today, we will be reviewing recommendations made by the
National Academy of Public Administration (NAPA) on how to
improve the National Science Foundation's management of
cooperative agreements to support the development and
construction of large-scale research projects.
NSF and the National Science Board commissioned the study
in response to concerns raised by this committee, the NSF
Inspector General, and others regarding NSF's management and
oversight of cooperative agreements and proper stewardship of
federal funds. NAPA assembled a committee that conducted an
eightmonth review of NSF's practices and looked at how other
science funding agencies like NASA and the Department of Energy
Office of Science manage similar projects.
One of our committee's most important responsibilities is
to ensure that federal science agencies spend taxpayer dollars
as effectively and efficiently as possible. Every dollar wasted
on mismanagement is a dollar that could be spent on
groundbreaking basic research or training future scientists.
Our committee has held two hearings over the last year on
NSF's management of major projects like the National Ecological
Observatory Network (NEON) and the LSST telescope. These
hearings have revealed that NSF needs to do more to ensure that
taxpayer funding is not wasted on mismanagement and abuse.
The NAPA study committee has provided 13 thoughtful
recommendations for NSF. I look forward to hearing testimony on
those ideas. I also look forward to hearing from NSF on how the
Foundation plans to respond, and from the Inspector General on
her thoughts about how some of these changes could prevent
problems found in past audits of cooperative agreements.
Together, I hope we can work towards ensuring taxpayer
dollars are well-managed. American leadership in science and
innovation is the key to our nation's future economic
prosperity, as well as our security. We want to be strong
advocates for federal support of basic research that advances
science in the national interest, but we can only invest more
in research when taxpayers have faith and confidence that their
money is being spent wisely.
[The prepared statement of Chairwoman Comstock follows:]
[GRAPHICS NOT AVAILABLE IN TIFF FORMAT]
Chairwoman Comstock. I thank you. And I now recognize the
Ranking Member of the Research and Technology Subcommittee, the
gentleman from Illinois, Mr. Lipinski, for an opening
statement.
Mr. Lipinski. Thank you, Chairwoman Comstock. I want to
thank you and Chairman Loudermilk for holding this hearing to
review the recommendations for the National Science
Foundation's project management reform.
In 1995, the NSF created an agency-wide budgetary account
to promote effective planning and management in the
Foundation's support for major research equipment and
facilities construction, or MREFC. This account supports the
acquisition, construction, and commissioning of major multiuser
research facilities, which typically cost between $100-$500
million and may take a decade to complete from planning to the
start of full operations.
In the early 2000s, this committee and the scientific
community raised concerns over management and oversight of the
MREFC account. In response, the NSF took a number of steps to
strengthen their processes and cost controls. Over the next
decade, the agency shepherded through many successful projects,
which today remain important cutting-edge research facilities
for the scientific committee.
In 2010, the NSF Office of Inspector General began raising
new concerns about policies and practices for awarding and
managing large construction projects. Since then, this
committee has held a number of oversight hearings related to
the MREFC account and specific projects. During this period,
disagreement arose between NSF and the OIG over what
constitutes appropriate and/or necessary policies and practices
with no apparent resolution in sight. This committee proposed
draft legislation, some of it on a bipartisan basis, in an
effort to forge a path forward.
Then, as now, I feel that while this committee's
prerogative is to exercise its oversight and legislative
authority--this is very important--we must also be aware of the
unintended consequences of micromanagement.
Afterwards, serious structural problems were discovered
with the management of the National Ecological Observatory
Network, or NEON. Since December of 2014, and largely in
response to the failures with NEON, NSF has taken a number of
additional steps to strengthen its business practices. But the
OIG has continued to raise red flags.
Early last year, the National Science Board and NSF
leadership commissioned a third-party independent review of the
Foundation's use of cooperative agreements to support large
projects, which would include benchmarking NSF's practices
against those of other agencies with large scientific
facilities.
The result was a report by the National Association of
Public Administration that is the subject of today's hearing.
By all accounts, the NAPA review is thorough, thoughtful, and
balanced. The experts on the NAPA panel came down solidly in
support of NSF continuing to use cooperative agreements rather
than contracts to support MREFC projects. So in my view, we can
probably dispense with that debate.
With respect to specific cost control policies and
practices, NAPA largely agreed with NSF that their policies are
not inconsistent with OMB guidelines but still urged them to
implement stronger preventative cost controls.
NAPA also offered several recommendations for NSF to
strengthen its project management procedures across the agency
and to rethink roles and responsibilities for planning,
management, and oversight of MREFC projects.
I understand that the IG supports the NAPA recommendations
overall but would still urge NSF to take steps above and beyond
those recommended in a NAPA report. I hope we will have a
chance to discuss all this during the hearing.
Before I conclude, I want to make two points: First, I'm
hopeful that this report will serve not just to strengthen
NSF's businesses practices for large projects but also to
smooth the way towards greater trust between OIG and NSF
management, both of whom, I'm sure, have the best interest of
the scientific community and taxpayers in mind.
Second, I want to highlight the very last section of the
NAPA report entitled ``The Cost of Increased Oversight.'' It
would be irresponsible for us to ignore the reality that it
will not be possible for NSF to implement NAPA's
recommendations, let alone the IG's, without increased funding
to support such oversight. I hope that the NSF requests enough
funding in their budget to implement the NAPA recommendations.
In addition, I hope that all my colleagues would join me
this spring in urging our appropriators to fully fund the
agency's request for its management account.
With that, I thank the witnesses for being here this
morning, and I yield back.
[The prepared statement of Mr. Lipinski follows:]
[GRAPHICS NOT AVAILABLE IN TIFF FORMAT]
Chairwoman Comstock. Thank you, Mr. Lipinski.
And since this morning we had the National Prayer
Breakfast, we have a number of our colleagues who are late or
unavailable, so my apologies for, first, the delay on all of
our parts, but that's what's going on. The traffic was pretty
bad out there, too, as you probably experienced.
But now, let me introduce our witnesses. Our first witness
today is Ms. Cynthia Heckmann, Project Director of the National
Academy of Public Administration's review on the NSF's use of
cooperative agreements to support the development,
construction, and operations of state-of-the-art, large-scale,
multiuser research facilities.
Prior to joining NAPA, Ms. Heckmann had an extensive career
at the Government Accountability Office, as well as career
experience about the executive branch and in state government.
Ms. Heckmann received her master's of public administration
from Northeastern University and her bachelor of arts from
Simmons College.
Our second witness today is Dr. Richard Buckius. Dr.
Buckius is the Chief Operating Officer for the National Science
Foundation. He assumed the position of COO in October 2014,
having previously been a Senior Policy Advisor for NSF. He is
an author and coauthor of numerous publications on the topics
of radiation heat transfer, numerical fluid mechanics, and
combustion. He received his bachelor's, master's, and Ph.D. in
mechanical engineering at the University of California
Berkeley.
Our last witness today is Ms. Allison Lerner. Ms. Lerner is
the Inspector General for the National Science Foundation.
Before joining NSF in April 2009, Ms. Lerner served in many
leadership positions at the Department of Commerce, including
Counsel to the Inspector General. She has received several
national awards for excellence and was selected to be a member
of the Government Accountability and Transparency Board by the
President in June 2011. Ms. Lerner received her law and
undergraduate degrees from the University of Texas.
I now recognize Ms. Heckmann for five minutes to present
her testimony.
TESTIMONY OF MS. CYNTHIA HECKMANN,
PROJECT DIRECTOR,
NATIONAL ACADEMY OF PUBLIC ADMINISTRATION
Ms. Heckmann. Good morning, Chairwoman Comstock, Ranking
Member Lipinski, and Members of the Subcommittee. Thank you for
the invitation to discuss recommendations from the Academy's
report on NSF's use of cooperative agreements to support the
development, construction, and operations of state-of-the-art,
large-scale, multiuser research facilities.
The NSF Director and National Science Board jointly
requested this review with a focus on the agency's largest
cooperative agreements of 100 million or more under the MREFC
account. The Academy's panel and study team's review focused on
agency's policies and practices governing the lifecycle of
these projects, including issues raised by the Inspector
General and Congressional concerns. We also looked at
comparable agencies with large capital investment projects most
analogous to NSF's for promising practices that might be
transferable.
Overall, we found that the cooperative agreements are the
appropriate mechanism to support these efforts. We also
acknowledge the many recent efforts by NSF has undertaken to
implement new policies and practices to address Inspector
General and Congressional concerns.
Our recommendations are intended to support NSF and the
board's commitment to improving core business practices and the
agency's key performance goal of ensuring program integrity and
responsible stewardship of major research facilities. We've
included suggested implementation steps for each
recommendation, and where appropriate in our text, highlighted
other agencies' promising practices. Our recommendations
provide a number of actions and options to strengthen oversight
and enhance agency governance, practices, and processes.
For today, we are grouping the recommendations as follows:
One, policies on cost estimating and cost analysis. NSF has a
strengthened its methodological approach to cost estimating and
analysis and updated its policies in the Large Facilities
Manual to reflect these efforts. This includes a tightened
control environment and improved cost surveillance strategies.
However, we identified some additional opportunities to
bolster cost analysis requirements for award recipients and
improve internal agency processes for detecting potential
issues in proposals. In terms of cost estimating, we recommend
that NSF change current language in the manual, making it clear
that award recipients are expected, not just encouraged, to
follow guidance in the GAO's Cost Estimating and Assessment
Guide and Schedule Assessment Guide when developing those cost
and schedule estimates.
In terms of cost analysis, we recommend that NSF require
that decisions not to act on recommendations from the pre-award
cost analyses conducted by their cost analysis and audit
resolution branch be reviewed by the Large Facilities Office
and forwarded to the Chief Financial Officer for a final
determination. These decisions should be documented in writing
and shared with the MREFC panel prior to the release of any
award funds.
Two, managing budgeting contingency: The use of contingency
is a commonly accepted business and project management risk
mitigation practice for construction projects. NSF's position
is that contingency funds should be retained and managed by the
award recipients. However, the common federal agency practice
is for the agency to hold either all or a majority percentage
of the contingency funds. By holding at least a percentage of
these funds, NSF could further strengthen internal controls and
accountability. Therefore, we recommend NSF adopt the practice
of retaining control of a portion of the award recipient's
contingency funds and distributing them with other incremental
funds as needed.
Three, management fee policies and processes: NSF includes
a management fee in several of its cooperative agreements to
cover business expenses related to construction or operations
that would otherwise be non-reimbursable under governing cost
principles. However, most of the examples provided to us
appeared to be costs that would either be covered under
indirect costs or contingency.
Despite recent changes NSF has made, it's been its policy
to provide clearer guidance on appropriate and inappropriate
uses of management fee. The IG continues to have concerns.
Furthermore, OMB's efforts in the uniform guidance have not
resulted in greater clarity on some of the most controversial
aspects of management fee. We recommend NSF eliminate the
practice of including management fee in cooperative agreements
on future projects.
Four, governing for effective stewardship: Governing
bodies, agency internal coordinating committees, and advisory
review panels play important roles in governance, development,
and execution of MREFC projects. We've offered recommendations
for each of these groups to address ambiguity and further build
on the agency's capacity to effectively manage these projects.
NSF and NSB roles and responsibilities, starting with them,
the statutory joint leadership model of an appointed director
and board is quite unique among federal agencies. The current
working relationship appears to be working well. It's not
always been the case, and we've recommended that they establish
and publish a joint duties and responsibilities document to
institutionalize the role.
The MREFC panel is only engaged right now in design review
phases of an MREFC project. For more consistent and informed
oversight, the panel should be involved throughout the
lifecycle of a project's development and implementation, really
reviewing project status on a set schedule.
In terms of advisory committees and review panels, the
panels need to include experts with requisite project and
financial management knowledge and experience, and we are
recommending that NSF explicitly identify the requirements and
add them to the selection criteria for those external
reviewers. We also recommend that NSF establish a FACA
committee for the director to use as a sounding board.
Large Facilities Office, the role and the placement of this
office has been subject to debate. We concluded that the
organizational placement is not as important as project
management roles and responsibilities. Its current placement is
also logical within the office of the CFO, as its focus is on
strengthening project management assistance. We did make
recommendations in terms of hiring two additional FTEs and
directing the MREFC charter be revised to change the status of
the LFO head from nonvoting to a full voting member.
In terms of planning and portfolio management process, we
have a recommendation in terms of the annual facilities plan
where there's some confusion over what its role is and ask that
NSF evaluate it in terms of its agency's current planning and
strategic planning processes.
Finally, project management skills are needed for effective
oversight. NSF has strong--the folks have strong scientific
credentials, not as likely to have the same corollary skills
and experience in project management.
We have three recommendations for project management. First
is to identify skill requirements for internal staff and
develop corollary project management training for the staff;
second, to require award recipient project managers be
certified in project management and specified minimum project
experience requirement threshold in the actual terms and
conditions; and lastly, establish formal communities of
practice to share best practices and lessons learned, a
requirement for all MREFC projects.
In closing, let me just reiterate NSF has undertaken a wide
range of actions to improve the oversight of these MREFC
projects. The efforts are a work in progress. They appear
headed in the right direction. Given the initiatives underway
and the culture change needed to socialize these changes, time
will be needed. And from a continuous improvement perspective,
time will be needed to see what is working and what requires
further adjustments.
Thank you for the opportunity to share these views.
[The prepared statement of Ms. Heckmann follows:]
[GRAPHICS NOT AVAILABLE IN TIFF FORMAT]
Chairwoman Comstock. Okay. Thank you, Ms. Heckmann.
And now, Dr. Buckius.
TESTIMONY OF DR. RICHARD BUCKIUS,
CHIEF OPERATING OFFICER,
NATIONAL SCIENCE FOUNDATION
Dr. Buckius. Madam Chairwoman, Members of the Committee,
thank you for the opportunity to discuss NSF's oversight of
major research facility infrastructure projects and NSF's
response to the recent NAPA report.
The National Science Foundation supports fundamental
research in frontiers of knowledge across all fields of science
and engineering. As part of that mission, NSF supports a broad
array of 28 major transformative research facilities that are
geographically distributed observatories, telescopes, mobile
platforms such as research vessels and aircraft. In a total,
the operational cost of NSF's entire portfolio of research
infrastructure requires approximately 1.2 billion per year.
I'd like to start by thanking NAPA for its rigorous review
of NSF's use of cooperative agreements to support large-scale
investments in science and technology. The members of this
committee, the Inspector General, and the experts at NAPA have
all been exceptionally helpful to the Foundation in identifying
areas where NSF can improve and make our oversight of critical
facilities even stronger.
We appreciate the panel's overall conclusion that
cooperative agreements are the appropriate mechanism for the
agency to use for construction and operations of large
facilities. In using this funding mechanism, the Foundation is
committed to improving the rigor and oversight of its processes
and deploying appropriate levels of internal projects,
programmatic, and financial management expertise.
In order to respond to the report, the Director has created
an implementation team to address each of the recommendations.
And I will divide this conversation into two broad areas:
first, business practices; and second, oversight,
accountability, and stewardship.
Let me start by saying that NSF will implement all 13
recommendations in some form. In the case of the business
practices, NSF will provide stronger requirements of cost
estimating and adjudication of cost analysis findings, as
recommended by the panel, and will revamp the processes of
obligating and allocating contingency based on the project's
level of risk. The panel's comparison with other agencies is
extremely useful, and we will follow up with these agencies for
more information detailing the process of partial withholding
of contingency, while also ensuring NSF's continued compliance
with uniform guidance.
With regard to management fee, the Foundation is continuing
to implement the stringent policy we put in place last year,
and we are currently considering the alternatives set forth in
the panel's report. As I've previously testified before this
committee, NSF acknowledges that some awardees should have
shown better judgment in the use of their management fee even
if they were not in violation of any law or regulation
governing the use of these funds. The Foundation has learned a
number of lessons about management fees, and we have designed
the policy around the lessons learned.
While many of the panel's recommendations are implementable
within a relatively short time, I would note that we believe
the management fee topic will likely take a more thorough
analysis on the part of NSF and some of the other
recommendations.
The panel's holistic view of NSF's oversight,
accountability, and stewardship of large research facilities is
welcomed by the Foundation. We are considering all the
recommendations on the roles and responsibilities of the
National Science Board, MREFC Panel, and the Office of the
Director as a single endeavor. We plan to enhance the role of
independent expertise in project and financial management, as
well as the cross-agency sharing of best practices.
The National Science Board examined the panel's
recommendations over the course of the last few days, and this
is our first NSB meeting since the release of the report. We
look forward to working together with the board to strengthen
our oversight, accountability, and stewardship, as recommended
by the panel.
NSF is committed to developing project management skills,
experience, and training for both foundation staff and MREFC
project managers.
I'd like to clarify for the Committee that any changes NSF
undertakes can and will apply to existing--not just new--
cooperative agreements. As the Foundation improves its
processes, we can revise, as appropriate, existing agreements
to strengthen oversight.
In closing, I'd like to reiterate how much NSF welcomes the
NAPA report and its recommendations. It is only with strong
support from the inspector general, external experts like NAPA,
and Congress that complete oversight of taxpayer resources can
be achieved, and we appreciate all these efforts.
Thank you again for the opportunity to testify, and I will
be pleased to answer any of your questions.
[The prepared statement of Dr. Buckius follows:]
[GRAPHICS NOT AVAILABLE IN TIFF FORMAT]
Chairwoman Comstock. I now recognize Ms. Lerner for five
minutes.
TESTIMONY OF MS. ALLISON LERNER,
INSPECTOR GENERAL,
NATIONAL SCIENCE FOUNDATION
Ms. Lerner. Thank you, Madam Chairwoman, and Members of the
Subcommittee. I appreciate the opportunity to discuss my
office's views on the National Academy of Public
Administration's report on NSF's use of cooperative agreements
to support large-scale investment in research.
Since 2010, my office has issued 28 reports containing more
than 80 recommendations that relate to NSF's management of
cooperative agreements for large-facility projects. Beginning
with our audits of over $1.1 billion in proposal costs for
three construction projects, the issues we identified have
occurred at multiple facilities and ultimately contributed to
the decision to procure the NAPA report. I commend Drs. Cordova
and David for their vision in commissioning this effort, as
well as the NAPA panel and staff who conducted the review and
prepared the outstanding final product.
The report sets forth practical recommendations that, if
implemented by the agency in a timely fashion, will
significantly improve NSF's ability to ensure accountability
over these high-risk, high-dollar projects, and thus go a long
way toward addressing many of the issues my office has raised.
Our office supports all of the NAPA report's
recommendations, and I'll briefly discuss six of them. First,
the report recommended that NSF require review of exceptions to
recommendations from pre-award cost analyses conducted by the
Foundation's in-house analyst, or CAAR. Because our work has
identified instances in which CAAR's important concerns have
not been addressed in a timely fashion, we share NAPA's belief
that actions necessary to ensure that the critical issues CAAR
identifies are brought to the attention of senior officials and
panelists and resolved. A similar process should be followed
for other important internal analyses and for the findings of
the incurred cost audits NSF will soon be procuring.
We wholeheartedly agree with, NAPA's recommendation that
NSF should retain control over a portion of awardees'
contingency funds. Our previous audits have found that
construction budgets for NSF's large-facility projects
contained tens of millions of dollars in unsupported
contingency costs. The risk of misuse of these funds is
heightened because NSF does not require awardees to track
expenditure of funds in their accounting systems, which makes
it all but impossible for us to audit these significant
expenditures.
With respect to the panel's recommendation to eliminate the
use of management fee, our September 2015 Alert Memo documented
positive steps NSF has taken to strengthen its fee policy and
noted improvements that are needed to determine if management
fee is necessary for an awardee's financial viability. If NSF
decides to continue providing such fees, it should consider and
address the issues noted in our Alert Memo.
OIG also agrees with three recommendations that focus on
the need for NSF to strengthen business and project management
skills within the agency and at its awardees. The culture
change needed to implement these recommendations is clearly
warranted because, as the NEON project has illustrated,
deficiencies in a project's business processes have the
potential to undermine its scientific goals if the project must
be de-scoped due to cost or schedule overruns.
Finally, I'd like to note three areas that were not covered
by the NAPA recommendations but which are still critical to
NSF's ability to manage large-facility projects. First, our
recent work has found that NSF does not require the earned
value management systems for its large-facility projects to be
certified or the data they contain to be verified. The poor
quality of the information in these reports for the NEON
project was one of the reasons why the cost overrun for that
project was undetected for so long. NSF is currently evaluating
what actions it should take to ensure the quality of EVM data.
Second, our office has repeatedly recommended that NSF
require annual incurred cost submissions for projects over $50
million. Such submissions provide critical information about
how an awardee has spent federal funds and are essential tools
for both NSF and the OIG. In two very recent reviews, however,
auditors found the submissions for two large construction
projects were not adequate to initiate an audit because so much
important information was missing from them. NSF is currently
seeking OMB clearance related to such submissions.
Finally, many of our reports have stressed the need for
annual incurred cost audits of large-facility recipients. Such
audits provide NSF with the best evidence of how awardees are
actually expending federal funds. While we commend NSF for
deciding to require incurred cost audits at project completion,
by waiting until the end to obtain an audit, NSF will miss
opportunities to identify and correct problematic expenditures
in the project's early days. Given the level of risk we've
identified with these projects, annual or at least biannual
audits are clearly warranted.
In closing, NSF's swift and decisive implementation of the
NAPA report's recommendation will have a significant positive
impact on the Foundation's ability to manage high-dollar, high-
risk, large-facility projects. I look forward to continuing to
work with Congress, the Foundation, and the National Science
Board toward this goal.
And I'd be happy to answer any questions.
[The prepared statement of Ms. Lerner follows:]
[GRAPHICS NOT AVAILABLE IN TIFF FORMAT]
Chairwoman Comstock. Thank you. I now recognize myself for
five minutes for questions. I thank the witnesses.
And I did want to let you know we got word that our
colleagues are stuck on a bus trying to get back from the
prayer breakfast, so again, our apologies for our other
colleagues who aren't here this morning. I guess no good deed
goes unpunished, right?
Okay, Ms. Lerner, do you believe that the recommendations
made by NAPA on how NSF should handle cost proposal analyses
and audits goes far enough to fix cost problems that you have
found in your audit work of large-scale projects?
Ms. Lerner. Thank you. We are pleased that NSF is committed
to undertaking one of the eight actions noted in GAO's cost
estimating guide. We're concerned, however, that for the large,
high-dollar, high-risk projects, that some of those options, as
a GAO notes in the guide itself--are not rigorous enough to
provide the information that's necessary. And we would
recommend that, for the high-dollar, high-risk construction
projects, that either a pre-award audit be done or an
independent cost estimate be obtained and that some of the less
rigorous options like a sufficiency review, which was used for
the LSST project, not be pursued.
Chairwoman Comstock. Thank you. And, Ms. Heckmann, the IG
testified that one issue the NAPA report did not make a
recommendation on that she considers critical--she was talking
about in her testimony is the use of an earned value management
system for tracking and measuring project cost and schedule
performance. The NAPA report did note the Department of Energy
and NASA have well-developed EVM policies in place. Why did
this study panel stop short of making a recommendation on using
a certified EVM system for projects?
Ms. Heckmann. At the time of our review, we understood that
the National Science Foundation was in the midst of reviewing
its EVM policies and processes and working to standardize the
processes and be sure to adopt the standards that are federal
standards. So what we did in terms of looking at other
agencies' practices was point out the promising practices that
we felt were transferable rather than really focusing a great
deal of time on the EVM. It was clear to us that they were
committed to using EVM.
What we also felt was important was that their staff--
again, getting back to the project management perspective--
needed to understand what EVM really is and how to read EVM
charts, et cetera, and how to really do the oversight that's
necessary for an EVM. So the bottom line there is that we saw
it as kind of a work in progress, and we were not at a point to
really make a recommendation further. We would really need to
see what in fact they have done.
Chairwoman Comstock. Okay. Thank you.
And, Doctor, would you like to address that also?
Dr. Buckius. Ms. Heckmann is exactly right. We are in the
midst of evaluating all aspects of EVM. I'd go so far to say,
though, that it's important, I think, to look at the
validation. That certification is one aspect, and we're going
to consider that, but the validation is the key because that's
where you get the answers. And so, I think the recommendation
is well-placed and we're taking it very seriously.
Chairwoman Comstock. All right. Thank you.
Ms. Lerner, did you want to add anything to that or----
Ms. Lerner. [Nonverbal response.]
Chairwoman Comstock. Okay. Thank you.
And I will now yield to my colleague, Mr. Lipinski.
Mr. Lipinski. Thank you. As they sometimes say, you need to
spend money to make money. You also have to spend some money in
order to save money when we're talking about making some of
these changes and having the proper oversight when it comes to
MREFC.
There are a lot of good recommendations in the NAPA report,
and we know some of them will cost more than others to
implement. And the Inspector General would like to see the NSF
go much further on some fronts than NAPA recommended. So I just
want to ask all three of you, given the cost of implementing
some of the recommended reforms, how do you think NSF should
prioritize the various recommendations and ensure that the
enhanced oversight does not impede NSF's scientific mission? We
sure would like to see them all done as, you know, quickly as
possible, but we know that there's probably going to have to be
some prioritization done in terms of which ones are done first.
So I'll start with Ms. Heckmann.
Ms. Heckmann. Thank you. I--we think it's really critical
that they develop the project management expertise internally
to be able to really perform the stewardship responsibilities
that are necessary. So in terms of the role of the LFO and what
it is doing to ensure that there's consistency across the
organization and the practices, as well as providing the
support and the assurance that's necessary for stewardship, we
feel that those are very critical.
Obviously, in the area of contingency and management fees,
management fee--I agree with Dr. Buckius--will take longer.
It's a more complicated issue and there's no great guidance out
there that's really--kind of sets the stage for where to go in
the future. I mean, that will take some time. And in fact we
made a recommendation--we--in the text of our report, we decide
that if there's some specific requirements that NSF projects
have that do not really fit the current definitions, it may
make sense to really identify those and seek special
legislative authority there.
Bottom line, in terms of balance, it is a balancing act.
Budgets are tight. There's been a lot of initiatives underway.
NSF will need to really step back and do detailed workload and
workforce analyses to determine what the next steps are. But,
you know, dealing with contingency issues, dealing with the--
how they manage--the MREFC panel I think is one that's a very
easy fix, wouldn't require a lot of time, effort, or cost, and
would really help them in terms of ensuring that they have
processes in place as well for monitoring and really shoring up
their oversight processes.
Mr. Lipinski. Okay. Let me go to Ms. Lerner, and then we'll
go to Dr. Buckius.
Ms. Lerner. I certainly agree that there are--the human
capital investment that needs to be made in ensuring that NSF
and its awardees have necessary program management expertise is
critical. In terms of financial investments, I do believe that
investments in strong pre-award and post-award audits will pay
dividends in the information that they provide NSF managers
about how the funds plan to be used and are actually being
used. When you look at the cost of some of these projects which
are, you know, 350 to almost $500 million, investing a couple
hundred thousand dollars in an audit seems a very appropriate
thing to do because, especially since it's one of the only ways
NSF post-award can actually see how its funds are being used.
Mr. Lipinski. Thank you. Dr. Buckius?
Dr. Buckius. Well, thank you for this question. It is a
balance. So we've got 13 recommendations here. A number of them
can be implemented with relatively low cost. Modifying text and
ensuring that our awardees are responding, I think, you know--I
would argue that's moderate.
Regarding the expertise, I think we've already made some
steps in that direction by our most recent hires under the
existing LFO allocation--Large Facility Office, excuse me--we
are allocating two more positions to that. It's already been
done. They're going to be posted.
Our attitude is, in the case of LFO, they've been
understaffed, I would argue, for a number of years, and so it's
time for us to actually make that happen. This comes out of our
AOAM, administration operations account, and I think you'll
read in the '16 and '17 budget it's a strained account already,
but this is necessary.
The comments on the MREFC and--are interesting because they
also don't require a lot of cost from the point of view of
people. They're going to require a lot of cost in terms of
time, though, because we now believe--and I think the NAPA
report clearly showed--we need to do complete lifecycle
analysis. We've been doing gate analysis and we've been
focusing on when things move from one gate to another. We need
to ensure continuous oversight. So that'll be time, time is
money, and so that'll cost us in the long run, but I think it's
absolutely necessary.
So as you've noted, it's going to be a balance to ensure
that we actually get the right oversight, right care, and
obviously to balance our budgets that we have to do. Thank you
for the question.
Mr. Lipinski. Thank you. I'm out of time so I'll yield
back.
Chairwoman Comstock. We are short of members if you'd like
a second round.
Mr. Lipinski. Well, let me very quickly--Dr. Buckius, I
don't know if you'll be able to answer this, you know, five
days before the budget request is released, but can we expect
to see an increased request for the NSF management account in
order to pay for some of this reform?
Dr. Buckius. The request we got in front of you will help
us a lot in order to make sure that we're making progress. I'm
thinking long-term '18, '19, I think that those are the areas
that we're going to have to spend a lot more care regarding
these kinds of people issues. 'Seventeen is on the mark.
Mr. Lipinski. Thank you. One other thing if I could ask
you, Dr. Buckius, I understand NSF is taking under
consideration the NAPA recommendation to eliminate management
fee and is initiating an evaluation of alternatives. Is there
any kind of preview you can give us at this early date? Do you
have any time frame in mind, including the analysis?
Dr. Buckius. Sure. Let me--this is the issue that we talked
about in my last testimony. So we implemented what I would call
an extremely strict policy on management fees. We write--or we
ask now for the positive; that is, what are you going to do
with it, and we also indicate what you can't do with it. We
implemented this in June of '15. We're in the midst of now
assessing what the impact has been by that strict policy. Our
deadline there is May, and we're going to work towards May in
order to make the assessment of how that particular policy that
was implemented in June has impacted the folks that actually
are using it.
The NAPA report also makes a number of recommendations, as
you just heard from Ms. Heckmann. Some of those I think we
really need to consider and see how we can actually use other
mechanisms in order to be able to get these kinds of resources
in the hands of the folks that we--so remember the whole goal
here. The whole goal is to provide agencies, organizations the
opportunity to compete for these awards. We need competing
proposals if we're going to fund these kinds of entities. And
so the management fee permits them to do certain things that
they need to do.
So we need to assess, as the NAPA report has indicated, how
we can figure--how--what other mechanisms we can possibly use
other than management fee. So our goal is to try to do that
over the--so we've got these two issues. Our goal is to try to
do that over the next months. We have to work with our National
Science Board, though, to ensure that they're on board.
Mr. Lipinski. Very good. Thank you. I yield back again.
Chairwoman Comstock. Okay. And I now recognize Ms.
Bonamici.
Ms. Bonamici. Thank you very much, Madam Chair.
I want to start--even though Chairman Smith isn't here yet,
I just wanted to start by thanking him on the record for
organizing a trip with Dr. Cordova a little more than a year
ago to Antarctica and some of the--with some of the NSF staff
and Dr. Cordova. And I come to this discussion with that
background because--going on that trip, which was really an
enlightening experience and fascinating--understanding not only
the importance of NSF investments and facilities but also the
challenges. And if you want to go to a place where there are
challenges with NSF facilities, it's a place like Antarctica.
So that brought to the other committee members who went on that
trip, I think, a perspective of appreciation of some of the
challenges but also the importance.
So Ms. Heckmann described in her testimony NSF will need to
make some hard funding decisions that address the demand for
more rigorous accountability systems balanced against the
mission to advance science. Obviously, there are many good
recommendations in the NAPA report, and some will cost more
than others to implement, as we've heard, and the IG would like
to see NSF go farther in some areas than NAPA recommends. I
don't think anyone would argue that it's important to have
appropriate oversight, but obviously, there need to be
priorities set.
So I want to ask you, Dr. Buckius--are we saying your name
right? Close?
Dr. Buckius. You can call me Richard. It's Buckius.
Ms. Bonamici. Buckius, thank you. To expand a little bit,
we've had some discussion about this already, but given the
cost of implementing some of the recommended reforms, how
should NSF go through the prioritization process of all the
various recommendations while you're also trying to ensure that
the enhanced oversight does not impede NSF scientific missions?
So could you talk a little bit about that process and how
you're making the decisions, the priority decisions?
Dr. Buckius. Yes, so let me try--first of all, your trip to
Antarctica probably demonstrated a lot of the facility issues
that we deal with----
Ms. Bonamici. Right.
Dr. Buckius. --on a regular basis. I mean it's--that
particular facility has almost everything exaggerated from the
point of view of difficulties.
Ms. Bonamici. Right.
Dr. Buckius. And yet the science that we're doing there and
the leadership for this country in Antarctica is tremendous. So
our approach has been on a number of these audit
recommendations is to look at the risk and try to balance the
audits related to risk so that we aren't going in and proposing
to audit, say, incurred cost audits on every single proposal. I
mean, that just isn't--it won't ever happen. It's impossible.
So what we do is we try to assess the level of risk in a
project, and then we try to look at those that have the largest
dollar value. So we've set limits in the case of incurred costs
at 100 million, and then assess the risk of those, and then go
and take a look at what we need to do. We propose to do them at
the end of the award.
If necessary, though, if we see a risky issue, we can jump
in and audit those earlier. So that's the approach we want to
take so that we don't burden ourselves overly with an audit
process that might not give us the necessary information that
we need.
So remember, though, we are--say in the comment that Ms.
Lerner made regarding EVM, we annually look at this, okay? We
look at them monthly, so it's not like we're not paying
attention to it. Her recommendation, which is the one we have
to look at, on certification and validation, we probably can do
some better in the validation setting. And so that's kind of
our goal. Our goal is to try to go in and look at the most
risky issues, try to look at the biggest projects, and invest
in those areas. So that's our objective.
Ms. Bonamici. Thank you. And could you also expand a little
bit on the timeline and that process of developing the
comprehensive organizational chart to incorporate the
recommendations? You mentioned LFO and additional investment
needed there. But what is the process of going through that
timeline? Give us a little more details about that and what if
any outside expertise will help you through this process of the
timeline and developing the recommendations?
Dr. Buckius. I would argue we meet on this subject every
week as a group with the Director, maybe every other week,
depending upon travel. So this is an urgent issue. We are going
to implement as many of these recommendations as we can as soon
as we can.
Now, let's talk a little bit about the FACA committee. We
think that's an excellent recommendation. We are trying to
figure out how to do that in a stepwise process. So we're at
this point talking about trying to use a subcommittee of one of
our current ACs so that we can get that moving ASAP with the
idea in the long run of creating a separate committee.
So again, the idea--we want to implement these as fast as
we possibly can because we realize this is going to be helpful
to all of us. At the same time, in order to set up a FACA
committee, it takes a fair amount of work, okay, in order to be
able to get that all constructed. And so we want to do that,
but we've got to figure out ways to get there first.
The LFO issue--Matt Hawkins is our LFO head, and I'm sure
he would tell you the same thing--we were understaffed, I would
argue, 3, four years ago in that area, okay? And so, like
you've heard, I was here in the last year-and-a-half or so, and
we're staffing that up, okay? It has to be a priority, and so
we're going to do it. And so, like I said, we've already got
two more postings coming on in that. And these are the kind of
investments that the agency has to make. And so the cost is one
thing. Implementation is as fast as we can.
Now, the management fee is the one that's probably going to
be the slowest because we've got to take a look at how we're
going to ensure that we have competitors. And we just don't
want to have some unintended consequences by implementation of
a policy that would actually alienate or give us the inability
to have proposers in these areas.
Ms. Bonamici. Terrific. Well, thank you. And I see that I'm
out of time, but I just want to say that we appreciate the
progress made to date and appreciate the Director's and your
willingness to come back and report to us, so thank you.
I yield back. Thank you, Madam Chair.
Chairwoman Comstock. Thank you. And I'm just going to ask
an additional question.
Ms. Lerner, in Dr. Buckius's testimony, he notes that NSF
manages 28 major research facilities, and that NEON is only one
facility within this portfolio that NSF needs to consider when
looking at its policies and procedures related to proper
oversight. This implies that many of the oversight management
issues identified are isolated to NEON. In your audits and
reviews of all the major research facilities in NSF's
portfolio, have you found similar issues in the oversight and
management of other projects?
Ms. Lerner. As I noted at the start of my remarks, you
know, we've done 28 reports with 80 recommendations, and those
findings are not limited to NEON. We've also found issues and
concerns at the LSST project, at the DKIST project when it was
called ATST, and with the OOI project, and most recently, with
incurred cost submissions for the AUI project. So the issues
and the challenges that we've identified are not limited.
Chairwoman Comstock. Okay. Well,the bus apparently is still
en route, so again, my apologies. If my colleagues have any
additional question, I'd be happy to yield to them.
But we'll keep the record open for additional questions. I
appreciate your testimony here this morning and helping us in
moving forward on these issues. And we look forward to
continuing to work with you.
And, yes, the record will remain open for two weeks for
additional written comments and written questions from our
Members who both are here and those who weren't able to join
us. So thank you. And the meeting is now adjourned.
Dr. Buckius. Thank you.
[Whereupon, at 10:30 a.m., the Subcommittees were
adjourned.]
Appendix I
----------
Answers to Post-Hearing Questions
[GRAPHICS NOT AVAILABLE IN TIFF FORMAT]
Appendix II
----------
Additional Material for the Record
[GRAPHICS NOT AVAILABLE IN TIFF FORMAT]
[all]