Federal Research: Information on the Advanced Technology Program's 1997
Award Selection (Testimony, 02/26/98, GAO/T-RCED-98-92).

GAO discussed the National Institute of Standards and Technology's
(NIST) Advanced Technology Program's (ATP) fiscal year (FY) 1997 award
selection process, focusing on identifying the information that ATP used
to determine whether to accept or reject competing project proposals.

GAO noted that: (1) according to program officials, for the FY 1997
competition, NIST made the determination of whether the applicant could
probably find funding elsewhere based on information gathered throughout
the proposal review process; (2) this included questioning the
applicants during the oral review phase if doubt remained as to whether
the applicants could have found project funding elsewhere; (3) for the
FY 1997 competition, there was no requirement that applicants report
that they could not find funding elsewhere; (4) however, in December
1997, ATP revised its requirements such that in the future applicants
must indicate on the proposal application their efforts to find private
funding; (5) likewise, program officials told GAO that information
acquired during the proposal review was used to determine if program
support was important to the project from a national economic
perspective; (6) specifically, according to ATP officials, one of the
five selection criteria for evaluating program proposals, "Potential Net
Broad-based Economic Benefits," relates to whether or not funding a
project would create a serious national economic concern; (7) according
to the guidance to applicants for preparing project proposals, the
review process would include a review of the proposal by panels of
outside experts in business and economics to determine the proposed
project's potential for broad-based benefits and its commercial
viability; and (8) however, program officials neither defined what they
meant by a serious national economic concern nor how the ATP reviews
resulted in a determination that a delay in project progress would not
be a serious national economic concern.

--------------------------- Indexing Terms -----------------------------

 REPORTNUM:  T-RCED-98-92
     TITLE:  Federal Research: Information on the Advanced Technology 
             Program's 1997 Award Selection
      DATE:  02/26/98
   SUBJECT:  Research and development
             Funds management
             Research program management
             Economic analysis
             Technical proposal evaluation
IDENTIFIER:  NIST Advanced Technology Program
             
******************************************************************
** This file contains an ASCII representation of the text of a  **
** GAO report.  Delineations within the text indicating chapter **
** titles, headings, and bullets are preserved.  Major          **
** divisions and subdivisions of the text, such as Chapters,    **
** Sections, and Appendixes, are identified by double and       **
** single lines.  The numbers on the right end of these lines   **
** indicate the position of each of the subsections in the      **
** document outline.  These numbers do NOT correspond with the  **
** page numbers of the printed product.                         **
**                                                              **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced.  Tables are included, but    **
** may not resemble those in the printed version.               **
**                                                              **
** Please see the PDF (Portable Document Format) file, when     **
** available, for a complete electronic file of the printed     **
** document's contents.                                         **
**                                                              **
** A printed copy of this report may be obtained from the GAO   **
** Document Distribution Center.  For further details, please   **
** send an e-mail message to:                                   **
**                                                              **
**                                            **
**                                                              **
** with the message 'info' in the body.                         **
******************************************************************


Cover
================================================================ COVER


Before the Subcommittee on Technology
Committee on Science
House of Representatives

For Release
on Delivery
Expected at
10 a.m.  EST
Thursday
February 26, 1998

FEDERAL RESEARCH - INFORMATION ON
THE ADVANCED TECHNOLOGY PROGRAM'S
1997 AWARD SELECTION

Statement of Susan D.  Kladiva,
Associate Director,
Energy, Resources, and Science Issues,
Resources, Community, and Economic
Development Division

GAO/T-RCED-98-92

GAO/RCED-98-92T


(141165)


Abbreviations
=============================================================== ABBREV

  ATP -
  NIST -

============================================================ Chapter 0

Madam Chairwoman and Members of the Subcommittee: 

We are pleased to be here today to discuss the Advanced Technology
Program (ATP), which is administered by the National Institute of
Standards and Technology (NIST) within the Department of Commerce. 
ATP is a competitive, cost-sharing program designed for the federal
government to work in partnership with industry to foster the
development and broad dissemination of challenging, high-risk
technologies that offer the potential for significant, broad-based
economic benefits for the nation.  On September 29, 1997, NIST
provided the Committee with a list of projects that had been selected
to receive awards under the fiscal year 1997 ATP competitions.  In a
cover sheet attached to this list, NIST stated that ATP rejected
project proposals when it concluded that (1) the applicants could
probably find funding elsewhere or (2) a delay in project progress
would not be a serious national economic concern.  In our report
released today, we identified the information that ATP used to make
these determinations.\1

That report highlights the following: 

  -- According to program officials, for the fiscal year 1997
     competition, NIST made the determination of whether the
     applicants could probably find funding elsewhere based on
     information gathered throughout the proposal review process. 
     This included questioning the applicants during the oral review
     phase if doubt remained as to whether the applicants could have
     found project funding elsewhere.  For the fiscal year 1997
     competition, there was no requirement that applicants report
     that they could not find funding elsewhere.  However, in
     December 1997, ATP revised its requirements such that in the
     future applicants must indicate on the proposal application
     their efforts to find private funding. 

  -- Likewise, program officials told us that information acquired
     during the proposal review was used to determine if program
     support was important to the project from a national economic
     perspective.  Specifically, according to ATP officials, one of
     the five selection criteria for evaluating program proposals,
     "Potential Net Broad-based Economic Benefits," relates to
     whether or not funding a project would create a serious national
     economic concern.  According to the guidance to applicants for
     preparing project proposals, the review process would include a
     review of the proposal by panels of outside experts in business
     and economics to determine the proposed project's potential for
     broad-based benefits and its commercial viability.  However,
     program officials neither defined what they meant by a serious
     national economic concern nor how the ATP reviews resulted in a
     determination that a delay in project progress would not be a
     serious national economic concern. 


--------------------
\1 Federal Research:  Information on the Advanced Technology
Program's 1997 Award Selection (GAO/RCED-98-82R, Feb.  24, 1998)


   BACKGROUND
---------------------------------------------------------- Chapter 0:1

ATP's fiscal year 1997 competitions consisted of one general
competition and six focused program competitions in the following
areas:  motor vehicle manufacturing technology, information
infrastructure for health care, digital data storage, technologies
for the integration of manufacturing applications, component-based
software, and tissue engineering.  A total of 64 industry-generated
projects were selected out of 570 applications, with an industry cost
share of $142 million and ATP investment of $162 million over the
life of the projects. 

The ATP Proposal Preparation Kit applicants used to prepare project
proposals contains background material on the program, guidance for
prepararing the proposal application, and all of the required forms. 
According to the ATP regulation, projects are selected for funding
based on the following five selection criteria:  (1) scientific and
technical merit, (2) the potential net broad-based economic benefits,
(3) adequacy of plans for eventual commercialization, (4) level of
commitment and organizational structure, and (5) experience and
qualifications. 

As part of the selection process, ATP uses peer reviewers to assess
the proposed technology's scientific and technical merit and its
potential for yielding broad-based economic benefits to the nation. 
The peer reviewers' comments are documented on worksheets, and ATP
uses these comments to determine which proposals have the highest
merit.  Applicants may also be asked to make oral presentations of
their proposals at NIST. 

Federal funding for ATP peaked at a high of $341 million in fiscal
year 1995.  Currently, ATP's fiscal year 1998 budget stands at $192.5
million.  New awards were capped at $62 million during fiscal year
1997 and at $82 million during fiscal year 1998.  For fiscal year
1999, the President's budget proposes $269 million for the program. 


   ATP'S EFFORTS TO DETERMINE THE
   AVAILABILITY OF PRIVATE SECTOR
   FUNDS
---------------------------------------------------------- Chapter 0:2

According to ATP officials, NIST determined whether the applicants
could probably find funding elsewhere based on information gathered
throughout the fiscal year 1997 competition proposal review process. 
For example, the technical reviewer worksheets contained sections for
optional evaluations of the degree to which program support is
necessary and the pace with which domestic and foreign competitors
are developing essentially the same or competing technologies.  In
addition, the business reviewer worksheets directed reviewers to
evaluate why applicants could not fund the project 100 percent on
their own.  ATP officials also said that applicants were questioned
during the oral review phase if doubt remained in this area. 

For the fiscal year 1997 competition, there was no requirement that
applicants report that they could not find funding elsewhere. 
However, in December 1997, NIST revised its ATP Proposal Preparation
Kit to request that future applicants describe what efforts were
made, before applying for ATP funding, to secure private capital to
wholly support their project.  According to ATP officials, the
information provided in this portion of the application will be used
in future competitions together with information gathered elsewhere
in the application to aid in making an overall funding decision on
the applicant's proposal. 


   ATP'S EFFORTS TO DETERMINE
   WHETHER PROJECTS MAY POSE A
   SERIOUS NATIONAL ECONOMIC
   CONCERN
---------------------------------------------------------- Chapter 0:3

For the fiscal year 1997 competition, applicants were required to
provide detailed information on a variety of economic factors.  For
example, applicants were asked to describe the project's potential
incremental economic benefits to the nation if funded by the ATP
compared with the benefits from research and development if privately
funded at a lower level; provide supporting evidence on market size
and commercialization pathways to the broad-based benefits; and
identify spillover benefits to other fields of activity or entire
industries. 

ATP's proposal review indicated that ATP should not fund projects
unless there is strong evidence that the funding can bring about
important national economic benefits beyond what would likely result
without ATP involvement.  The business reviewer worksheets contained
a section for an evaluation of the potential broad-based economic
benefits for the proposed projects.  Reviewers were asked to evaluate
the proposed project in terms of (1) the potential to improve U.S. 
economic growth and productivity, (2) timeliness, (3) the degree to
which ATP support is necessary, and (4) cost-effectiveness (or
probable benefits relative to costs). 

In summary, for the fiscal year 1997 competition, ATP did not have a
requirement in place for applicants to document whether they sought
funding elsewhere, but relied on the review process to make this
determination.  For future competitions, ATP has included a question
on the application form regarding the applicant's efforts to find
private funding.  There is no guidance as to the detail expected in
the response.  Regarding the determination of whether projects would
pose a serious national economic concern if not selected, ATP
collects a great deal of economic information from applicants in its
proposal process; however, there is no explicit reference to serious
national economic concern in any of the associated guidance for
evaluating the potential broad-based economic benefits of proposed
projects. 

Madam Chairwoman, this concludes my prepared remarks.  I would be
happy to respond to any questions you or the members of the
Subcommittee may have. 


*** End of document. ***