    Environmental Research: STAR Grants Focus on Agency Priorities, but
    Management Enhancements Are Possible (Letter Report, 09/11/2000,
    GAO/RCED-00-170).

    Pursuant to a congressional request, GAO provided information on the
    Environmental Protection Agency's (EPA) Science to Achieve Results
    (STAR) grants program, focusing on: (1) whether funding amounts awarded
    for the grants align with EPA's strategic goals, EPA's Office of
    Research and Development's (ORD) research priorities, and program office
    priorities; (2) the extent to which the completed focused grants have
    provided research that is being used by EPA's program offices; and (3)
    ways in which ORD could enhance its management of the program to help
    ensure that it meets its objectives.

    GAO noted that: (1) STAR grant funding has generally been aligned with
    EPA's ORD's, and the program offices' broadly defined priorities; (2)
    since the STAR program began in 1995 through March 2000, about
    two-thirds of the approximately $415 million in funding for STAR grants
    has addressed EPA's multipurpose strategic goal of ensuring that EPA
    uses sound science in addressing environmental hazards and improving
    environmental protection; (3) although STAR grants were generally
    aligned with the agency's strategic priorities and those of ORD and the
    program offices, EPA's program officials varied in the extent to which
    they believe the grants' results are useful to them; (4) according to
    the officials, the grants varied in their usefulness to the program
    offices largely because the STAR program's goals--meeting program office
    needs specifically and advancing environmental science generally--may be
    in conflict; (5) ORD has not consistently tracked the STAR grants to
    ensure that they are completed on time and that the results are
    available for use by EPA's program office; (6) according to several
    program officials, ORD could improve its communication with the program
    offices during the grant process; (7) ORD has not consistently obtained
    information from cognizant program officials when designing the requests
    for grant applications or reviewing grant proposals for relevancy during
    the grant selection process, nor has ORD adequately communicated the
    results of the completed grants; (8) ORD recognizes these concerns and
    has several initiatives under way; (9) these include establishing a Web
    site to improve communication and involving more program officials in
    grant application design and review; (10) because the STAR program is
    relatively new, ORD's primary focus has been on establishing a framework
    and processes for the program and not on setting program criteria to
    measure its overall effectiveness; and (11) recent reports by two
    independent scientific organizations agree that while the program is
    well structured to achieve its goals, communications between ORD and
    EPA's program offices should be improved.

    --------------------------- Indexing Terms -----------------------------

     REPORTNUM:  RCED-00-170
         TITLE:  Environmental Research: STAR Grants Focus on Agency
    	     Priorities, but Management Enhancements Are Possible
          DATE:  09/11/2000
       SUBJECT:  Strategic planning
    	     Research grants
    	     Agency missions
    	     Grant administration
    	     Program evaluation
    	     Environmental research
    IDENTIFIER:  EPA Science to Achiever Results
    	     Grant Program

    ******************************************************************
    ** This file contains an ASCII representation of the text of a  **
    ** GAO Testimony.                                               **
    **                                                              **
    ** No attempt has been made to display graphic images, although **
    ** figure captions are reproduced.  Tables are included, but    **
    ** may not resemble those in the printed version.               **
    **                                                              **
    ** Please see the PDF (Portable Document Format) file, when     **
    ** available, for a complete electronic file of the printed     **
    ** document's contents.                                         **
    **                                                              **
    ******************************************************************
    GAO/RCED-00-170
    A

    Report to the Chairman, Subcommittee on VA, HUD, and Independent Agencies,
    Committee on Appropriations, House of

    Representatives

    September 2000 ENVIRONMENTAL RESEARCH

    STAR Grants Focus on Agency Priorities, but Management Enhancements Are
    Possible

    GAO/ RCED- 00- 170

    Letter 3 Appendixes Appendix I: Scope and Methodology 22

    Appendix II: EPA Funding for Environmental Research 24 Appendix III: STAR
    Program Requests for Grant Applications

    by ORD's Priority Topics, Fiscal Years 1995 Through 2000 26

    Appendix IV: Comments From the Environmental Protection Agency 28

    Tables Table 1: STAR Grants by EPA Strategic Goals, Fiscal Years 1995- 99 11
    Figures Figure 1: EPA's Process for Awarding STAR Grants 9

    Figure 2: Usefulness of 25 Completed Focused Grants to EPA Program Officials
    15 Figure 3: Percentage of STAR Funding, by Type of Grant, Fiscal

    Years 1995 and 1998 25

    Abbreviations

    EPA Environmental Protection Agency ORD Office of Research and Development
    STAR Science to Achieve Results

    Resources, Community, and Economic Development Division

    Lett er

    B- 142370 September 11, 2000 The Honorable James T. Walsh Chairman,
    Subcommittee on VA, HUD, and Independent Agencies Committee on
    Appropriations House of Representatives

    Dear Mr. Chairman: The Environmental Protection Agency (EPA) has
    significantly changed the way it conducts scientific research to fulfill its
    mission of protecting human health and the environment. Responding to
    criticisms of its research quality and focus, EPA established the Science to
    Achieve Results (STAR) grant program in 1995, which is administered by EPA's
    Office of Research and Development (ORD). The three objectives of the
    program are to (1)

    ensure that the agency involves the best non- EPA scientists in its research
    efforts, (2) provide useful research support to the agency's program
    offices, and (3) train a cadre of environmental scientists for the future.
    The STAR program accounted for about 20 percent of the funds ORD obligated
    for research for fiscal year 1999. ORD awards STAR grants to scientists on
    the basis of 10 broad strategic goals as set out by EPA's annual performance

    plan, 1 ORD's six priority topics for scientific research at the agency, and
    the specific research priorities of each program office. STAR grants are
    available for (1) basic exploratory environmental research, (2) research
    focused on specific environmental topics (STAR focused grants), and (3)
    fellowships to graduate students to develop the nation's capacity for
    addressing future environmental concerns. As of January 2000, about 800
    exploratory and focused grants had been awarded and 53 of them

    completed; in addition, about 600 fellowships had been awarded and about 200
    completed. You asked us to review the status of the STAR grants program.
    Specifically, we are providing information on (1) whether funding amounts
    awarded for the grants align with EPA's strategic goals, ORD's research
    priorities, and

    1 The Government Performance and Results Act of 1993 requires each federal
    agency to prepare an annual plan that includes the performance goals to be
    achieved. See table 1 for a complete listing of EPA's strategic goals.

    program office priorities; (2) the extent to which the completed focused
    grants have provided research that is being used by EPA's program offices;
    and (3) ways in which ORD could enhance its management of the program to
    help ensure that it meets its objectives. Because the STAR program is new
    and relatively few grants have been completed, it is too early to
    definitively judge the program's long- term effectiveness. To address your
    questions, we analyzed program data; interviewed EPA

    officials knowledgeable about STAR grants; and reviewed studies by
    independent scientific groups- specifically, a March 2000 report on the STAR
    program by a joint subcommittee of EPA's Science Advisory Board and ORD's
    Board of Scientific Counselors and a draft report on EPA's research
    management by the National Research Council. 2 To further examine the extent
    to which STAR grants were being used by EPA's program offices, we compiled a
    list of the 53 focused and exploratory grants completed as of December 6,
    1999, from the National Center for

    Environmental Research's Web site, as identified by STAR program managers.
    We discussed the usefulness of 25 completed grants that were focused on
    specific environmental topics with potential users of the

    research in EPA, as identified by the STAR program managers. We limited our
    review to focused grants because they were more likely to be of more
    immediate use to the program offices than those grants related to
    exploratory research. Because providing useful research to the program

    offices is only one of the program's three objectives, our observations on
    whether the research support is being used by the program offices should not
    be the only measure of judging the program's overall effectiveness. We also
    recognize that scientific research can have long- term benefits but may

    not be immediately useful. Our detailed scope and methodology are discussed
    in appendix I. Results in Brief STAR grant funding has generally been
    aligned with EPA's, ORD's, and the

    program offices' broadly defined priorities. Since the STAR program began in
    1995 through March 2000, about two- thirds of the approximately $415 million
    in funding for STAR grants has addressed EPA's multipurpose 2 A Joint SAB/
    BOSC Report: Review of the Science to Achieve Results ( STAR) Program,

    Joint Subcommittee of the Science Advisory Board and the Board of Scientific
    Counselors, March 2000; and Strengthening Science at the U. S. Environmental
    Protection Agency: Research Management and Peer Review Practices, National
    Research Council (draft report).

    strategic goal of ensuring that EPA uses sound science in addressing
    environmental hazards and improving environmental protection. Another 21
    percent has addressed the strategic goal aimed at achieving clean air, and
    the remaining grant funding has addressed six of EPA's other strategic
    goals. STAR grant funds also generally have been aligned with one or more

    of ORD's six research priorities, such as achieving safe drinking water, and
    with the program offices' main priorities. Some program office research
    needs, such as reducing acid rain, were not addressed by STAR grants because
    EPA determined they had a lower research priority and because STAR funding
    was limited. Although STAR grants were generally aligned with the agency's
    strategic

    priorities and those of ORD and the program offices, EPA's program officials
    varied in the extent to which they believe the grants' results are useful to
    them. Of the 25 completed focused grants we examined, 6 were being used and
    4 were not expected to be used at all, according to program officials. For
    the remaining 15 completed grants, the officials were

    uncertain whether the grants would prove useful or not to their programs,
    but they stated they might be useful in the future- by serving as building
    blocks for research, for example. According to the officials, the grants
    varied in their usefulness to the program offices largely because the STAR

    program's goals- meeting program office needs specifically and advancing
    environmental science generally- may be in conflict. ORD could enhance its
    management of the program to help ensure it meets its objectives in several
    areas. First, ORD has not consistently tracked the STAR grants to ensure
    that they are completed on time and that the results are available for use
    by EPA's program offices. Second, according to several EPA program
    officials, ORD could improve its communication with the

    program offices during the grant process. ORD has not consistently obtained
    information from cognizant program officials when designing the requests for
    grant applications or reviewing grant proposals for relevancy during the
    grant selection process, nor has ORD adequately communicated the results of
    the completed grants. ORD recognizes these concerns and has several
    initiatives under way. These include establishing a Web site to

    improve communication and involving more program officials in grant
    application design and review. Because the STAR program is relatively new,
    ORD's primary focus has been on establishing a framework and processes for
    the program and not on setting program criteria to measure its overall
    effectiveness. Recent reports by two independent scientific organizations
    agree that while the program is well structured to achieve its goals,
    communications between ORD and EPA's program offices should be

    improved. Accordingly, we are making several recommendations to EPA for
    enhancing its management of the STAR program. We believe these will help
    improve the program's potential usefulness and effectiveness.

    We provided a draft of this report to EPA for its review and comment. EPA
    generally agreed with the report's conclusions that the grants aligned with
    EPA's, ORD's, and the program offices' broadly defined priorities and that

    funding for STAR grants falls within priority research areas. EPA also
    agreed with two of the three recommendations in this report and said it is
    taking steps to implement them. EPA did not agree with the basis for the
    third recommendation regarding the need for improved tracking and
    availability of final grant reports to program offices. Specifically, EPA
    stated that its review of grant files showed that more final reports were
    completed than indicated by our analysis. However, at the time of our
    review, a STAR program official charged with posting reports to the Web

    site represented that our list of final reports, compiled from those posted
    on EPA's Web site and available to program officials, was complete. Because
    it now appears that a significant number of completed final

    reports had not been posted to EPA's Web site, which was not represented to
    us at the time of our review, we believe the recommendation is still
    warranted. Finally, EPA identified several aspects of the draft report that
    it believed needed modification; for example, it stated that we negatively

    characterized the data on the usefulness of STAR grants to the program
    offices. We clarified and presented additional information to address EPA's
    comments and made technical changes as appropriate.

    Background Credible scientific research is critical to EPA's decisions on
    environmental regulations. EPA relies on agency- funded research as well as
    on research

    funded by others, including government agencies, such as the National
    Science Foundation and National Institutes of Health, and private sources.
    Prior to the STAR program, EPA mainly funded research through

    cooperative agreements with outside entities, issued through its
    laboratories, to supplement the research EPA scientists performed. In
    addition, ORD funded a relatively small number of grants, managed by ORD
    headquarters staff, for longer- term exploratory research.

    During the early 1990s, EPA's independent Science Advisory Board, the
    National Academy of Sciences, and other groups reviewed the status of
    scientific research at EPA. The groups recommended that EPA make changes to
    strengthen the quality of its scientific research, such as better balancing
    its short- and long- term research and taking steps to train the

    next generation of scientists. EPA responded, in part, by establishing the
    STAR program in 1995. In doing so, EPA shifted much of its funding for
    external research away from noncompetitively awarded cooperative agreements
    administered by ORD's laboratories and placed more emphasis on competitively
    awarded, peer- reviewed grants. EPA believed that this realignment would
    allow ORD scientists to spend less time administering contracts and more
    time conducting research, while the STAR program

    would involve top- quality scientists from outside EPA in helping to meet
    the agency's research needs. The STAR program has three principal
    objectives. First, through the

    program, EPA attempts to make the nation's academic research community an
    integral part of its research program, thereby ensuring that the agency has
    the highest quality science for its use. Second, EPA seeks to supplement and
    complement its internal research program, thereby supporting its mission by
    meeting the scientific and technical needs of the program offices. Finally,
    EPA attempts to support higher education in selected scientific fields,
    thereby developing a stronger scientific community for the future.

    To achieve these objectives, the STAR program relies on three types of
    grants: exploratory grants, focused grants, and graduate fellowships.
    Exploratory grants are for fairly broad areas of environmental science, such
    as environmental chemistry and physics. Focused grants are targeted

    more specifically at particular research needs, such as developing standards
    for safe drinking water, that have been identified in ORD's strategic plan.
    Focused grants can be awarded to environmental research
    “centers,” such as the Airborne Particulate Matter Centers.
    These centers comprise multiple scientists- usually from different
    disciplines- who are concerned with longer- term and cross- issue research.
    Finally, fellowships are given to graduate students in specified
    environmental disciplines, such

    as toxicology, chemistry, and economics. Fellowships for master's and
    doctoral students are for 2 and 3 years, respectively. For fiscal year 1999,
    ORD obligated about $91 million in STAR grants for externally conducted
    research. This represented about 20 percent of ORD's overall obligations of
    about $458 million for research. Other resources within and outside of EPA

    also provide funding for environmental research. Additional information on
    overall environmental research funding and funding levels for the three
    types of STAR grants is provided in appendix II. EPA solicits STAR grants
    through annual requests for grant applications.

    Proposals for exploratory and focused grants are reviewed for their

    scientific and technical merit by panels of non- EPA scientists, which
    reject those that are not considered scientifically sound. Next, panels of
    EPA officials from ORD and other offices review and rank the proposals for
    relevance to their research needs. ORD then selects proposals to be awarded,
    taking into consideration both scientific merit and relevance to research
    needs. ORD follows a similar process for fellowship proposals.

    EPA's process for awarding STAR grants is illustrated in figure 1.

    Figure 1: EPA's Process for Awarding STAR Grants

    EPA's strategic plan (10 strategic goals) ORD's strategic plan (6 priority
    topics)

    Topic- specific research plan (developed by research coordinating teams)

    Extramural Intramural research needs

    research needs RFAs forwarded via ORD Federal Register, Internet,

    Requests for implementation

    and university and plans scientific organizations Applications written

    (developed by laboratories) External peer Scientific

    review and community

    ranking of proposals (based on technical merit)

    Internal relevancy review (performed by agencywide panel) Proposal selection
    (based on research needs and to complement intramural research)

    STAR Grant Funding Is Because EPA's, ORD's, and the program offices'
    priorities are broadly

    Aligned With EPA's, defined, most STAR grant funding has generally been
    aligned with them. Since the STAR program began in 1995, ORD has funded
    grants aligned

    ORD's, and Program with 8 of EPA's 10 strategic goals. Most of the grant
    funds have been aligned

    Offices' Broad Goals with EPA's strategic goal of ensuring that the agency
    uses sound science in addressing environmental issues, followed by the goal
    aimed at achieving

    clean air. Furthermore, all of the funding for STAR grants awarded falls
    within one or more of ORD's six priority research topics, and grants have
    generally been awarded for the highest priority needs of each program
    office. However, some lower- priority research topics, such as reducing acid
    rain, are not being funded by STAR grants or other EPA research because of
    resource constraints.

    STAR Grant Funding Falls Since 1995, when the STAR program began, and
    through March 2000, ORD Within EPA's Strategic Goals

    has awarded about $415 million in exploratory and focused grants. Once all
    of the awards are made for fiscal year 1999, total exploratory and focused
    grant awards will amount to about $430 million. EPA has awarded these grants
    in keeping with eight program- related strategic goals that could benefit
    from STAR grant research. Table 1 shows how the $415 million in grant awards
    corresponds to EPA's strategic goals. The table does not include fellowship
    grant awards, which amounted to about $39 million for fiscal years 1995
    through 1999.

    Table 1: STAR Grants by EPA Strategic Goals, Fiscal Years 1995- -99 --
    Dollars in millions

    EPA's strategic goal Award amounts

    Clean air $87.0

    Clean and safe water 16. 5

    Safe food 2.5

    Preventing pollution and reducing risk in communities, homes, 3.0

    workplaces and ecosystems Better waste management, restoration of
    contaminated waste 9.7

    sites, and emergency response Reduction of global and cross- border
    environmental risks 15. 6

    Expansion of Americans' right to know about their environment 3.6

    Sound science, improved understanding of environmental risk,

    277.1

    and greater innovation to address environmental problems Total $415. 0

    Note: Individual grant amounts are associated with one strategic goal. STAR
    grants are not applicable to 2 of EPA's 10 strategic goals, “a
    credible deterrent to pollution and greater compliance with the law”
    and “effective management.”

    Source: GAO's analysis of EPA's data.

    STAR grants applicable to the sound science and clean air goals represented
    about 67 and 21 percent of the funds awarded, respectively. Since sound
    science is a multipurpose goal, grants funded under it frequently apply to
    another goal as well.

    STAR Grants Align With In addition to EPA's strategic goals, ORD has
    established six broad priority ORD and Program Office research topics: safe
    drinking water, high- priority air pollutants, emerging Priorities issues,
    improving ecological risk assessment, improving human health risk
    assessment, and pollution prevention and new technology. These are

    generally consistent with EPA's strategic goals. Three of ORD's priority
    topics- improving the assessment of risks to human health, emerging issues,
    and improving ecological risk assessment- have been the focus of the
    majority of STAR grants awarded. Appendix III shows how the research areas
    of the awarded grants are aligned with the six research topics. ORD takes
    the needs of program and regional offices into consideration

    when writing the requests for grant applications and awarding STAR grants.
    ORD's research coordination teams include program office and regional
    representatives who identify EPA's research needs and determine

    the research that will be conducted internally or externally. According to
    the science advisers in the offices of Air and Radiation and Solid Waste and
    Emergency Response who have served on the teams, ORD considers program
    offices' research priorities and ensures that the offices' top research
    needs are covered by the STAR grants. For example, because EPA considers
    particulate matter in the air to be a costly potential health risk, the
    Office of Air and Radiation has particulate matter research as a top
    priority, and ORD has awarded STAR grants for research on particulate matter
    under its topic of high- priority air pollutants. A program official in the
    Office of Air and Radiation, who is responsible for developing the

    standards, is incorporating STAR grant results into the new standards.
    Funding for STAR grants has generally been consistent with the highest
    research priorities of the program offices. Some lower- priority research
    needs are not benefiting from STAR grants, however, because of resource
    constraints. According to program officials, the likelihood that the grants
    will address a research need depends on where that need ranks in the

    program offices' and ORD's priorities. For example, officials concerned with
    acid rain issues, which is a lower priority for the Office of Air and
    Radiation, told us that to date no STAR grants have included research that

    can be used to meet their needs. Similarly, no STAR grants have targeted
    research on stratospheric ozone or asthma mitigation. Officials responsible
    for these areas believe they could benefit from STAR research, but their
    research needs have a relatively low priority within the Office of Air and

    Radiation. Although ORD recently decided to fund a combustion research
    project that the Office of Solid Waste and Emergency Response will use, an
    official from that office told us that the STAR program generally has
    focused on few of its research needs because waste issues are currently a
    low priority for ORD.

    According to program officials, the approximately $100 million in annual
    funding for STAR grants is not sufficient to cover all of the program
    offices' research needs that are not otherwise met by other sources, such as
    ORD's

    laboratories. To compensate, the officials stated, they seek out research
    being conducted by other organizations, some in other countries, on similar
    topics that may meet their research needs. Recently, the Science Advisory

    Board's Research Strategies Advisory Committee completed its annual review
    of the agency's budget request for fiscal year 2001 science and technology
    funds. The committee agreed that EPA is not adequately funding research on
    certain environmental issues, such as hazardous waste cleanup. ORD officials
    concur that budget constraints limit grant awards and that consequently some
    research cannot be funded through the

    program. They told us that they consider research being done by scientists
    at EPA and other agencies, such as the National Science Foundation, when
    awarding STAR grants to ensure that program offices' research needs get the
    best coverage possible.

    Usefulness of the Although STAR grants are generally aligned with EPA's
    strategic priorities Focused Grants to

    and those of ORD and the program offices, program officials believed that
    the usefulness of the completed focused grants to their programs varied.
    EPA's Program Offices

    They stated that they were using 6 of the grants, did not expect to use 4 of
    Varies, Largely

    them, and were uncertain about the extent to which the remaining 15 Because
    of Competing grants might be useful in the future. According to the
    officials, the grants varied in their usefulness to the programs largely
    because the STAR

    Program Objectives program's goals- advancing environmental science
    generally and meeting

    and the Nature of program needs specifically- may be in conflict.

    Grants Usefulness of the

    For the 25 completed grants we examined, 6 were being used by EPA Completed
    Focused Grants

    program officials because they met a research need. For example, one to
    EPA's Program Offices

    STAR grant created an animal model emulating the development of autism
    Varies

    in humans. The goal of the research was to assess how a woman's exposure to
    certain chemicals during pregnancy could affect the fetus. According to
    officials in ORD and the Office of Prevention, Pesticides, and Toxic

    Substances, the STAR research was the first proving a relationship between
    chemicals and the development of autism. The officials are using the results
    as they develop guidelines for assessing risks to children's health.

    In contrast, program officials did not expect to use the research results
    from four grants at all. For example, a grant researching the development of
    methods to prevent wastes by auto suppliers did not provide useful results,
    according to officials in the Office of Prevention, Pesticides, and Toxic
    Substances. The officials stated that they had begun similar research 2 to 3
    years before the grant was issued and that they had researched the issue
    more extensively than the grantees did. They stated, however, that the
    grantees benefited by increasing their technical expertise in conducting
    this type of research.

    Program officials believed that the extent to which the remaining 15
    completed focused grants might prove useful in the future varied. For

    example, six grants, totaling over $750,000, that researched methods for
    estimating the benefit of environmental policies were not currently useful
    but were expected to be useful to the agency in the long term. Program
    officials told us that the research topic- estimating values for lives,
    ecosystems, and other entities affected by environmental policies- is an
    important one for the agency. These officials generally agreed that the
    grants contributed to an improved understanding of the overall issue of
    environmental valuation and would probably provide some basis for future
    work on the issue. One of the program officials also stated that it was
    probably prudent to fund these grants in order to advance knowledge and

    understanding of this issue, even though the grants are not providing
    research results that can be used immediately. On the other hand, the future
    usefulness of a grant investigating wet cleaning processes- instead of the
    standard dry cleaning process- is less certain, according to officials in
    the Office of Prevention, Pesticides, and Toxic Substances. The officials

    stated that the research was well done and illustrated the environmental
    benefits of wet cleaning over dry cleaning. However, they believe that the
    garment cleaning industry is unlikely to adopt the new technology because
    cleaning companies have already made costly changes in the infrastructure of
    their plants to comply with EPA regulations. The officials plan to keep

    the grant results as a reference on the garment cleaning industry in the
    event that future regulatory changes make alternative cleaning approaches
    more viable.

    Figure 2 shows how program officials perceived the usefulness of the 25
    completed focused grants.

    Figure 2: Usefulness of 25 Completed Focused Grants to EPA Program Officials

    Number of grants 6 Useful

    15 4 Not useful Usefulness uncertain

    Note: These focused grants were completed as of December 1999.

    The STAR Program's The STAR program's objectives may be contradictory in
    that the program is

    Competing Goals and expected to focus on achieving long- term gains in
    environmental science as

    Reliance on Grants Limit the well as being responsive to the more immediate
    needs of the program

    Usefulness of the Research offices. According to some program officials,
    they understood that the to EPA Program Officials

    foremost goal of the STAR program was to advance environmental science
    rather than to meet the programs' needs. As a result, STAR grants may be
    limited in the extent to which they are useful to the program offices.
    Furthermore, the STAR program's reliance on grants to accomplish
    environmental research limits its ability to directly respond to the
    research needs of EPA's program offices. The principal purpose of any grant
    is to

    assist grantees in furthering a public purpose rather than to obtain
    services directly for the government's benefit. According to several EPA
    program officials, program offices' needs do not always match up with grant
    recipients' research objectives; therefore, some of the STAR grant topics
    are not directly applicable. In addition, one program official told us that

    STAR researchers are sometimes reluctant to answer questions posed by
    program officials because the terms of the grant do not require them to be
    responsive to the needs of EPA's program offices.

    Research for STAR grants may also not be completed in a time frame that is
    useful to the program offices. STAR grants range from 1 to 5 years. Grantees
    can request a 1- year extension, which ORD officials must automatically
    approve in accordance with OMB's Circular A- 110, according to ORD
    officials, and ORD may authorize additional extensions. Thus, even

    though it may be several years before STAR grants produce results, the
    grantees are still in compliance with the terms of their grants. Program
    officials also differed in whether grant timing fit their needs. For
    example, an official in the Office of Air and Radiation told us that
    projecting research needs that far in advance was possible, while another in
    the Office of Solid Waste and Emergency Response stated that many of his
    office's research

    needs involve cleanup work at Superfund sites and require results in a
    shorter amount of time than the grants can usefully provide.

    ORD Could Enhance ORD could enhance its management of the STAR program to
    help ensure

    Its Management of the that the results of the grants are readily useful to
    the EPA program offices

    and that the program meets its objectives. Areas needing further STAR
    Program to Help improvements include (1) tracking grants to ensure that they
    are Ensure That It Meets

    completed on time and have produced the research intended; (2) improving Its
    Objectives

    communications with EPA's program offices when designing and reviewing
    grants for relevancy and disseminating grant results; and (3) establishing
    criteria to measure program effectiveness. Reports by two independent
    scientific organizations agree that while the program is generally well
    structured for achieving its goals, communications between ORD and EPA's

    program offices should be improved. ORD Could Further Improve

    ORD has not been adequately tracking grants and has not ensured that Its
    Tracking of Grant

    exploratory and focused grants were completed on time. As of December
    Results

    1999, 144 of the nearly 200 grantees for exploratory and focused grants had
    missed their deadlines, including extensions, for submitting final reports.
    ORD officials recently noted that they plan to do a better job of tracking

    grants. In that regard, in June 1999, ORD hired a contractor to track and
    identify interim and final reports. ORD has identified a number of
    additional final reports since December 1999. According to a STAR

    program manager, however, ORD cannot ensure that research will produce
    results in the agreed time or with the intended results. Similarly, ORD has
    not adequately tracked the fellowship grants to determine whether they have
    been effective. It had not analyzed whether any of the approximately

    200 fellowships completed as of December 1999 had resulted in college
    graduates taking jobs in environmental occupational fields. Recognizing

    the need to determine whether fellowship recipients are continuing in the
    environmental sciences, ORD began in early 2000 to track the recipients of
    completed fellowships. In that effort, ORD has obtained follow- up

    information for about 60 of the 200 completed fellowships. ORD Could Better
    ORD could also improve its communication with the program offices
    Communicate With Program

    during the grant process, according to several EPA program officials. For
    Offices About STAR Grants example, ORD has not consistently obtained
    information from cognizant program office officials when designing requests
    for grant applications or

    reviewing grant proposals for relevancy during the grant selection process.
    While a number of program officials have participated in these processes,
    others told us that they have not been invited to do so. One official, who
    had not been involved in the pre- award process, believes that, had she been
    involved, she might have influenced ORD to select a grant that could have
    met her and others' research needs. She acknowledged that ORD has recently
    recognized that more participation from the program offices prior to
    awarding the grants could increase their usefulness. The draft report by

    the National Research Council agreed that some program officials had
    expressed concern that, in practice, they have little influence on ORD's
    research priorities. Furthermore, ORD had not adequately communicated grant
    results to program officials. For 13 of the 25 grants we examined, program
    officials learned about completed research as a result of ORD outreach and
    notification. For the remaining grants, officials learned about 1 grant's
    final results through a professional conference and were not aware of the
    completed research for the remaining 11 grants until we contacted them for
    this review. In addition, most of the program officials were not aware of
    interim research results for the grants we reviewed. One program official
    explained that, if he had been aware of the interim research results on air
    pollution in large buildings for one STAR grant, he could have notified the

    researchers of problems with their methodology and corrections could have
    been made. Instead, the usefulness of the completed research is
    questionable, and unless the methodology issue can be resolved, the

    official will be unable to use the results. ORD has taken steps to
    communicate the results of STAR program research, including posting STAR
    research report summaries on its Web site, providing program offices with
    lists of grants that might contain useful research, and sponsoring workshops
    on grant topics. While the program officials acknowledged that ORD makes
    STAR information available, the

    format of this information makes it difficult for program officials to
    identify research relevant to their needs. For example, program officials
    stated that interim and summaries of final research reports are organized on
    ORD's Web site by grant topic, a format that makes identifying specific
    relevant research difficult because they have to sift through numerous
    lengthy reports. For the ORD workshops on grant topics, not all grant topics
    are covered. For example, the indoor air research topics were not covered in
    any workshop, according to a program official. ORD managers acknowledge
    ongoing difficulties in conveying research results to the program offices
    and say they have been increasing their efforts to do so.

    According to both the National Research Council's draft report and the March
    2000 report by the joint subcommittee of EPA's Science Advisory Board and
    ORD's Board of Scientific Counselors, now that the STAR

    program is starting to produce results, EPA should place greater emphasis on
    communicating those results to potential users in the agency. By
    strengthening communication between STAR program managers and the program
    offices, ORD could better ensure that STAR results are being rapidly and
    effectively transferred to the officials who can use them. The

    joint subcommittee report also recognizes that ORD has initiated several
    efforts to improve communication, such as developing reports on the status
    of research topics.

    ORD Has Not Focused on Because the STAR program is relatively new, ORD's
    primary focus has been Developing Measures to on establishing a framework
    and processes for the program, not on setting Help Ensure the Program

    criteria for measuring its overall effectiveness. For example, ORD has not
    Meets Its Goals

    set goals for measuring whether the grants are meeting the research needs of
    EPA's program offices. Such measures would allow STAR managers to determine
    if the program as a whole is achieving its expected results of involving the
    best scientists in EPA research, providing useful research for

    the program offices, and training a cadre of environmental scientists for
    the future. According to ORD officials, most of their efforts to date have
    focused on establishing a high- quality research program. They now believe
    it is time to place a greater emphasis on evaluating the program's benefits.

    Consequently, ORD asked the joint subcommittee of EPA's Science Advisory
    Board and ORD's Board of Scientific Counselors for assistance in developing
    measures for determining if the STAR program is meeting its overall goals.

    The March 2000 joint subcommittee report focused on evaluating whether the
    STAR program is structured appropriately to achieve its goals. Overall,

    the report found that the program is structured and managed to generate
    high- quality science, conducted by well- qualified scientists, on topics
    that are relevant to the environmental problems identified in EPA's
    strategic plan. The report concluded that the STAR program is generally well
    planned, organized, and managed for achieving its goals. However, it
    recommended that ORD seek expert assistance in developing a monitoring and
    evaluation system for the program. Conclusions Initial results indicate that
    the STAR program is yielding some research that is useful to EPA's program
    offices and that it is well- structured and

    managed to generate high- quality science. However, for the STAR program to
    better meet one of its major goals- that of supporting the research needs of
    EPA's program offices- ORD must improve its tracking of grants to ensure
    that the research is being completed as agreed and that it is communicating
    the research results throughout EPA. Reports by two

    independent scientific groups agree that the STAR program's research results
    should be better communicated. EPA recognizes this concern and has taken
    several actions to improve communications, but more could be done.

    ORD has focused the majority of its efforts to date on the near- term
    objectives of establishing the STAR program and on awarding the grants.
    However, ORD has not established criteria to help ensure that the program is
    meeting its major objectives. Now that the program has begun to

    produce results, it is appropriate for EPA to focus greater attention on
    managing on- going research, disseminating research results, and evaluating
    program results to date. Recommendations To enhance the effectiveness of the
    STAR program, we recommend that the

    Administrator of EPA direct the Assistant Administrator of ORD to take the
    following actions:

    Track and monitor the grants to ensure that interim and final research
    results are delivered on time and are made available as soon as possible for
    use by the program offices. Take the additional steps needed to better
    disseminate and communicate STAR research results to the appropriate program
    officials. This would require continuing and expanding the efforts

    already under way to consult with program offices in determining the most
    effective communication methods. Develop program criteria to evaluate the
    effectiveness of each type of grant- exploratory grants, focused grants, and
    fellowships. In addition,

    the criteria should assist EPA in drawing an overall conclusion on whether
    these grants satisfy the program's overall objectives. Agency Comments We
    provided a draft of this report to EPA for its review and comment. EPA's

    comments and our detailed responses are in appendix IV. EPA agreed with the
    report's conclusions that the grants generally align with EPA's, ORD's, and
    the program offices' broadly defined priorities and the funding for STAR
    grants falls within priority research areas. However, EPA suggested changes
    in five other areas. First, EPA stated that our report negatively

    characterized the data on the usefulness of STAR grants to the program
    offices, noting that no benchmarks exist for an appropriate level of
    usefulness. We disagree. Rather, our report simply portrays the views of EPA
    program officials, one of the intended beneficiaries of the program, on

    the grants' usefulness. We agree that no benchmarks exist for the usefulness
    of grants, although benchmarks would be helpful to evaluate the program.
    Second, EPA stated that there is no contradiction in using grants to support
    the program offices and to further advance environmental research. We
    recognize that the STAR program can contribute to both of these goals.
    However, as the draft report stated, program officials noted

    that these goals may conflict and that the primary purpose of grants is to
    assist grantees in furthering a public purpose rather than in obtaining
    services directly for the government's benefit. Third, EPA stated that the

    report misrepresented the relative amount of STAR grant funding to overall
    ORD funding. We changed the report to more accurately reflect the proportion
    of ORD's funding accounted for by the STAR grant program. Fourth, EPA stated
    that it did not agree with our conclusions as to the number of grants that
    had been completed and made available to program offices. Specifically, EPA
    stated that, by counting only those final reports posted to its Web site, we
    did not include a substantial number of additional completed grant reports
    that were in its files. Therefore, EPA disagreed with our conclusion and
    recommendation that tracking of STAR

    grants should be improved. While EPA states that additional research reports
    were in its files, at the time of our review, a STAR program official
    confirmed that all final reports were posted to the Web site. If they were
    not posted, they were not readily available to the program offices.

    Therefore, we believe our conclusions and recommendation are warranted.
    Finally, EPA stated that it has been taking steps to implement two of the

    report's recommendations to better disseminate and communicate STAR research
    results and to evaluate the program. We acknowledged these efforts in the
    report.

    We are sending copies of this report to appropriate congressional
    committees; interested Members of Congress; the Honorable Carol M. Browner,
    Administrator, Environmental Protection Agency; and other interested
    parties. We will also make copies available to others on request.

    Please call me at (202) 512- 6111 if you have any questions about this
    report. Key contributors to this report were John Wanska, Karla Springer,
    John C. Johnson, and Roger Bothun.

    Sincerely yours, Peter F. Guerrero Director, Environmental

    Protection Issues

    Appendi Appendi xes x I

    Scope and Methodology To obtain information on the Environmental Protection
    Agency's (EPA) Science To Achieve Results (STAR) grant program, we
    interviewed officials in, and collected and analyzed information from, the
    Office of Research and Development's (ORD) National Center for Environmental
    Research and the Grants Administration Division; the Office of Air and
    Radiation; the Office of Solid Waste and Emergency Response; and the Science
    Advisory Board's Research Strategies Advisory Committee. We also reviewed
    two recent reports by independent scientific groups. The March 2000 report
    by a joint

    subcommittee of EPA's Science Advisory Board and ORD's Board of Scientific
    Counselors objectives were to examine whether (1) the STAR program is
    structured appropriately to achieve its stated purpose, (2) the program is
    integrated effectively with the agency's strategic plans and programs, and
    (3) efforts to communicate with the external scientific and regulatory
    communities regarding STAR research opportunities and

    outputs were adequate. The draft report by the National Research Council
    assessed the overall structure and management of EPA's research program and
    evaluated scientific peer review procedures used by the agency. To determine
    whether the grants awarded met EPA's strategic goals and ORD's priorities,
    we interviewed ORD officials to obtain an understanding of how grants are
    categorized by subject area and how these subjects were aligned with EPA's
    strategic goals and priorities. We also obtained information on the grant
    award process and reviewed documentation on the process. The documents
    reviewed included EPA's and ORD's strategic

    plans and various listings of the grants awarded. The listings identified
    the year of the award, the research topic, and the dollars awarded. Using
    this information, we compared the grants awarded to EPA's goals and ORD's

    research priorities to determine how the grants were aligned with specific
    goals and priorities. To assess the extent to which the completed focused
    grants have provided research that is being used by EPA's program offices,
    we obtained information on those grants completed as of December 6, 1999,
    from the National Center for Environmental Research's Web site. At the
    suggestion of STAR program managers, we considered a grant to be completed
    if its final report was posted on the Web site and the December 6, 1999,
    date provided STAR program staff time to ensure that all available final
    research

    reports had been posted on the site. We excluded completed exploratory
    grants, since those grants were less likely, according to STAR managers,
    than focused grants to yield results that the programs could use in the
    nearterm. We also obtained from a STAR manager a list of staff in the
    program offices and regions who were most likely to benefit from the results
    of the

    completed grants. We contacted those officials, and selected others who we
    learned were also potential users of the results, to discuss their views on
    the research from the completed grants. Officials we interviewed were from
    ORD; the offices of Prevention, Pesticides, and Toxic Substances, Water, Air
    and Radiation, Solid Waste and Emergency Response, International Activities,
    and Policy and Reinvention; and regions IX and X.

    In addition, we used data from EPA's automated Grants Information Control
    System to compile lists of grants that should have been completed as of
    December 6, 1999. Exploratory and focused grant recipients are required by
    their grants to provide a final report to ORD within 90 days of the
    expiration of the grant's project period. Therefore, to determine which

    exploratory and focused grants should have been completed and had final
    reports submitted to STAR managers, we considered only those with project
    periods ending at least 90 days prior to December 6, 1999. As a condition of
    the grant, fellowship recipients must submit their theses or other
    publications to ORD after the end of the grant period, as soon as those
    papers are available. Because that deadline is less definitive, to determine
    which fellowships should have been completed, we used

    December 31, 1999, as the cutoff date and asked STAR managers about any
    actions they were taking to ensure that the fellowships were effective. We
    did not independently verify the accuracy of the grants data. However, we
    discussed the issue of data accuracy with an official in the Grants
    Administration Division, who told us that the data fields we used were
    likely to be accurate because they are relatively easy for the grant project
    managers to keep updated. We conducted our work between September 1999 and
    August 2000 in accordance with generally accepted government auditing
    standards.

    Appendi x II

    EPA Funding for Environmental Research For fiscal year 1999, EPA obligated
    about $458 million for ORD, whose primary purpose is to provide research for
    the agency. About $218 million of this amount was for internal research ORD
    conducted, about $91 million was for the external STAR program, and about
    $149 million was for other external research, such as contracts. In addition
    to the ORD funding, EPA's program offices fund research projects. In fiscal
    year 1999, about $118 million was available for these projects, but the
    agency does not know how much of that was spent on research- related
    efforts. ORD is currently trying to identify all the research that EPA is
    conducting; however, ORD does not

    plan to identify the amount of funding for research conducted outside of ORD
    because of difficulties in defining and accounting for research. Estimates
    of the total federal funding for environmental research range from about $2
    billion to $3.2 billion for fiscal year 2000. 1 The relative funding levels
    for the three types of STAR grants- exploratory, focused, and fellowships-
    has changed from fiscal year 1995 to fiscal year 1998. In fiscal year 1995,
    the year the STAR program began, EPA awarded approximately $58 million, with
    focused grants accounting for the largest share of funding, about $29
    million, or 50 percent, of the total. In fiscal year

    1998, ORD awarded approximately $148 million in STAR grants, and focused
    grants constituted 83 percent of the total amount as shown in figure 3. 2 1
    In April 2000, the American Association for the Advancement of Science
    estimated about

    $2 billion in total federal environmental research for fiscal year 1999; in
    1999, the National Science Foundation estimated about $3. 2 billion.

    2 Fiscal year 1998 was the last year for which all of the STAR grants were
    awarded. ORD is still in the process of awarding grants for requests for
    proposals from fiscal year 1999 and has not yet awarded any grants for the
    fiscal year 2000 applications.

    Figure 3: Percentage of STAR Funding, by Type of Grant, Fiscal Years 1995
    and 1998

    Fiscal year 1995 Fiscal year 1998 50%

    7%

    Focused

    Fellowships

    10% 9%

    Exploratory Fellowships

    Noncenter

    41%

    Exploratory

    46%

    Center Noncenter

    37% 83%

    Focused

    Source: GAO's analysis of EPA's data.

    Over half of the focused grants awarded for fiscal year 1998 were for
    research centers at universities, a practice that ORD has recently begun to
    emphasize. There were no center grants for fiscal year 1995. According to a
    STAR program manager, many of the research issues that the agency faces

    currently, and expects to face in the future, require multidisciplinary
    work, and grants to centers are more suited to this type of research.
    Fellowship grants accounted for about $10 million in fiscal year 1998 and
    supported approximately 300 fellowships. ORD expects to continue funding
    fellowships at this level. Awards for exploratory grants have decreased
    significantly. According a to STAR program manager, exploratory grants are
    proving to be the least useful of the STAR grants. While the manager
    believed some of the broad research done with exploratory grants is
    beneficial to the agency, ORD has gradually diminished the exploratory

    share in favor of more focused grants.

    STAR Program Requests for Grant Applications by ORD's Priority Topics,
    Fiscal

    Appendi x I II

    Years 1995 Through 2000 ORD's priority topics Research to

    Research to improve

    improve Pollution

    Safe Highpriority ecological

    human health prevention

    STAR Requests for Application by drinking air

    Emerging risk

    risk and new

    subject matter water

    pollutants issues assessment

    assessment technology

    Air

    Indoor air quality X X

    Health effects of particulate matter X X X X

    Air pollution chemistry and physics X X X X

    Air toxics X X X

    Mercury fate and transport X X X X Water

    Drinking water X X X

    Risk- based decisions for contaminated sediments X X X

    Water and watersheds X X

    Health effects of arsenic X X Ecology

    Ecological assessment and indicators X X

    Global climate change X X

    Regional scale assessment and analysis X X

    Ecology and oceanography of harmful algal blooms X X X

    Ecosystem restoration X Health

    Exposure of children to pesticides X X

    Endocrine disrupters X X X

    Children's environmental health and disease prevention research centers X X

    Human health risk assessment X X X

    Role of inter- individual variability in human susceptibility X X X

    Children's vulnerability to toxic substances in environment X X X X X

    Exposure to waste combustion products X X X X

    Chemical mixtures in environmental health X X X X X Other

    Analytical and monitoring methods X X X X X

    (Continued From Previous Page)

    ORD's priority topics Research to

    Research to improve

    improve Pollution

    Safe Highpriority ecological

    human health prevention

    STAR Requests for Application by drinking air

    Emerging risk

    risk and new

    subject matter water

    pollutants issues assessment

    assessment technology

    Environmental fate and treatment of toxics and hazardous wastes X X X

    Environmental statistics X X X X

    High- performance computing X X X X

    Technology for a sustainable environment X X X

    Decision- making and valuation for environmental policy X X

    General solicitation - exploratory research X X X X X X

    Socioeconomic projects related to pollution prevention X X

    Program on bio- remediation X X X

    Futures: detecting the early signals X Total 10 12 22 18 24 11

    Source: EPA's classification of grants awarded by research priorities.

    Comments From the Environmental

    Appendi x V I Protection Agency Note: GAO's comments supplementing those in
    the report text appear at the end of this appendix.

    See comment 1.

    See comment 2.

    See comment 3. See comment 4.

    See comment 5.

    GAO's Comments. 1. We do not agree that our report negatively characterizes
    the number of grants useful to program officials. Our report states the
    number of completed focused grants that are useful, not useful, or of
    uncertain usefulness based on our discussions with EPA program officials. We
    recognize that there is no benchmark for the number of grants that should

    be useful to program officials, although one would be helpful to determine
    the effectiveness of the STAR program. Furthermore, while we agree that STAR
    research may be useful to parties outside of EPA, we did not go

    beyond interviewing EPA program and regional officials about the usefulness
    of STAR focused grants because it is a stated goal of these grants to
    address the research needs of these officials.

    2. We disagree that there may not be a contradiction in using a grants
    program to simultaneously support the agency's mission and to advance the
    understanding of complex environmental issues. We recognize that EPA has
    made efforts to direct STAR grants into research areas that are relevant to
    the program offices, and our review has shown that, consequently, some
    research results have proven useful in meeting program needs. However, the
    principal purpose of any grant is to assist grantees in furthering a public
    purpose rather than obtaining services directly for the government's

    benefit. Furthermore, program officials stated that the needs of the program
    offices have not always matched the grant recipients' research objectives
    and that the grant research has often been designed to contribute to broad
    scientific issues. Therefore, as the program officials we

    interviewed pointed out, the goals of the program may conflict. 3. We agree
    that the amounts comparing STAR program obligations with total ORD research
    obligations for fiscal year 1999 should be revised. We revised the report to
    reflect that ORD obligated $91 million for the STAR

    program out of a total of $458 million it obligated for all research. Thus,
    the STAR program accounts for about 20 percent of all research obligated by
    ORD. We do not agree that the STAR program's obligated amounts should be
    compared to ORD's overall budget because that amount may include funding for
    activities other than research.

    4. We disagree with EPA's assertion that our conclusion and recommendation
    regarding the need to improve the tracking process is incorrect and should
    be revised accordingly. While EPA states that additional research reports
    were available in the files at the time of our

    review, these were not posted to the Web site and were therefore not readily
    available to the program offices. We worked with STAR program

    managers to develop the methodology for counting completed grants as those
    with final reports posted to the Web site, as discussed in appendix I.
    Furthermore, we provided the officials with additional time to identify all
    completed reports in their files and to post them to the Web site. As a
    result, we believe that no changes are needed to our report.

    5. Our report acknowledges the efforts EPA has begun to develop criteria for
    evaluating the STAR program's effectiveness and to better disseminate and
    communicate research results.

    (160505) Lett er

    (160505) Lett er

    Ordering Information The first copy of each GAO report is free. Additional
    copies of reports are $2 each. A check or money order should be made out to

    the Superintendent of Documents. VISA and MasterCard credit cards are
    accepted, also. Orders for 100 or more copies to be mailed to a single
    address are discounted 25 percent.

    Orders by mail: U. S. General Accounting Office P. O. Box 37050 Washington,
    DC 20013

    Orders by visiting: Room 1100 700 4th St. NW (corner of 4th and G Sts. NW)
    U. S. General Accounting Office Washington, DC

    Orders by phone: (202) 512- 6000 fax: (202) 512- 6061 TDD (202) 512- 2537

    Each day, GAO issues a list of newly available reports and testimony. To
    receive facsimile copies of the daily list or any list from the past 30
    days, please call (202) 512- 6000 using a touchtone phone. A recorded menu
    will provide information on how to obtain these lists.

    Orders by Internet: For information on how to access GAO reports on the
    Internet, send an e- mail message with “info” in the body to:

    info@ www. gao. gov or visit GAO's World Wide Web home page at: http:// www.
    gao. gov

    To Report Fraud,

    Contact one:

    Waste, or Abuse in Web site: http:// www. gao. gov/ fraudnet/ fraudnet. htm

    Federal Programs

    e- mail: fraudnet@ gao. gov 1- 800- 424- 5454 (automated answering system)

    GAO United States General Accounting Office

    Page 1 GAO/ RCED- 00- 170 STAR Grant Program

    Contents

    Page 2 GAO/ RCED- 00- 170 STAR Grant Program

    Page 3 GAO/ RCED- 00- 170 STAR Grant Program United States General
    Accounting Office

    Washington, D. C. 20548 Page 3 GAO/ RCED- 00- 170 STAR Grant Program

    B- 142370 Page 4 GAO/ RCED- 00- 170 STAR Grant Program

    B- 142370 Page 5 GAO/ RCED- 00- 170 STAR Grant Program

    B- 142370 Page 6 GAO/ RCED- 00- 170 STAR Grant Program

    B- 142370 Page 7 GAO/ RCED- 00- 170 STAR Grant Program

    B- 142370 Page 8 GAO/ RCED- 00- 170 STAR Grant Program

    B- 142370 Page 9 GAO/ RCED- 00- 170 STAR Grant Program

    B- 142370 Page 10 GAO/ RCED- 00- 170 STAR Grant Program

    B- 142370 Page 11 GAO/ RCED- 00- 170 STAR Grant Program

    B- 142370 Page 12 GAO/ RCED- 00- 170 STAR Grant Program

    B- 142370 Page 13 GAO/ RCED- 00- 170 STAR Grant Program

    B- 142370 Page 14 GAO/ RCED- 00- 170 STAR Grant Program

    B- 142370 Page 15 GAO/ RCED- 00- 170 STAR Grant Program

    B- 142370 Page 16 GAO/ RCED- 00- 170 STAR Grant Program

    B- 142370 Page 17 GAO/ RCED- 00- 170 STAR Grant Program

    B- 142370 Page 18 GAO/ RCED- 00- 170 STAR Grant Program

    B- 142370 Page 19 GAO/ RCED- 00- 170 STAR Grant Program

    B- 142370 Page 20 GAO/ RCED- 00- 170 STAR Grant Program

    B- 142370 Page 21 GAO/ RCED- 00- 170 STAR Grant Program

    Page 22 GAO/ RCED- 00- 170 STAR Grant Program

    Appendix I

    Appendix I Scope and Methodology

    Page 23 GAO/ RCED- 00- 170 STAR Grant Program

    Page 24 GAO/ RCED- 00- 170 STAR Grant Program

    Appendix II

    Appendix II EPA Funding for Environmental Research

    Page 25 GAO/ RCED- 00- 170 STAR Grant Program

    Page 26 GAO/ RCED- 00- 170 STAR Grant Program

    Appendix III

    Appendix III STAR Program Requests for Grant Applications by ORD's Priority
    Topics, Fiscal Years 1995 Through 2000

    Page 27 GAO/ RCED- 00- 170 STAR Grant Program

    Page 28 GAO/ RCED- 00- 170 STAR Grant Program

    Appendix IV

    Appendix IV Comments From the Environmental Protection Agency

    Page 29 GAO/ RCED- 00- 170 STAR Grant Program

    Appendix IV Comments From the Environmental Protection Agency

    Page 30 GAO/ RCED- 00- 170 STAR Grant Program

    Appendix IV Comments From the Environmental Protection Agency

    Page 31 GAO/ RCED- 00- 170 STAR Grant Program

    Appendix IV Comments From the Environmental Protection Agency

    Page 32 GAO/ RCED- 00- 170 STAR Grant Program

    Appendix IV Comments From the Environmental Protection Agency

    Page 33 GAO/ RCED- 00- 170 STAR Grant Program

    United States General Accounting Office Washington, D. C. 20548- 0001

    Official Business Penalty for Private Use $300

    Address Correction Requested Bulk Rate

    Postage & Fees Paid GAO Permit No. GI00
    *** End of document. ***