Grants Management: EPA Needs to Strengthen Efforts to Address	 
Persistent Challenges (29-AUG-03, GAO-03-846).			 
                                                                 
The Environmental Protection Agency (EPA) has long faced problems
managing its grants, which constitute over one-half of the	 
agency's annual budget, or about $4 billion. EPA uses grants to  
implement its programs to protect human health and the		 
environment and awards grants to thousands of recipients,	 
including state and local governments, tribes, universities, and 
nonprofit organizations. EPA's ability to efficiently and	 
effectively accomplish its mission largely depends on how well it
manages its grant resources. As requested, GAO determined (1)	 
major challenges EPA faces in managing its grants and how it has 
addressed these challenges in the past, (2) extent to which EPA's
recently issued policies and grants management plan address these
challenges, and (3) promising practices, if any, that could	 
assist EPA in addressing these challenges.			 
-------------------------Indexing Terms------------------------- 
REPORTNUM:   GAO-03-846 					        
    ACCNO:   A08267						        
  TITLE:     Grants Management: EPA Needs to Strengthen Efforts to    
Address Persistent Challenges					 
     DATE:   08/29/2003 
  SUBJECT:   Federal grants					 
	     Financial management				 
	     Grant administration				 
	     Grant award procedures				 
	     Best practices					 
	     Budget administration				 

******************************************************************
** This file contains an ASCII representation of the text of a  **
** GAO Product.                                                 **
**                                                              **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced.  Tables are included, but    **
** may not resemble those in the printed version.               **
**                                                              **
** Please see the PDF (Portable Document Format) file, when     **
** available, for a complete electronic file of the printed     **
** document's contents.                                         **
**                                                              **
******************************************************************
GAO-03-846

                                       A

Letter

August 29, 2003 The Honorable Don Young Chairman, Committee on
Transportation and Infrastructure House of Representatives The Honorable
Anne M. Northup House of Representatives The Environmental Protection
Agency (EPA) has faced persistent challenges in managing its grants, which
constitute over one- half of the agency*s budget, or about $4 billion
annually. To support its mission of protecting human health and the
environment, in fiscal year 2002, EPA awarded grants to a variety of
recipients, including state and local governments, tribes, universities,
and nonprofit organizations. As of September 30, 2002, there were 4,100
grant recipients. Given the size and diversity of EPA*s programs, its
ability to efficiently and effectively accomplish its mission largely
depends on how well it manages its grant resources and builds
accountability into its efforts.

Congressional hearings in 1996, 1999, and 2003 have focused on EPA*s
problems in effectively managing its grants. We and EPA*s Inspector
General have reported on a number of weaknesses throughout the grants
management process* from awarding grants to measuring grant results. 1
EPA*s efforts to address its grants management problems have not fully

resolved them. Because EPA has faced persistent problems in managing
grants, we designated its grants management as a major management
challenge in January 2003. 2 Late in 2002, EPA launched new efforts to
address some of its grants management problems. It issued two new
policies* one to promote

competition in awarding grants and one to improve its oversight of grants.
Furthermore, in April 2003, EPA issued a 5- year grants management plan to
address its long- standing grants management problems.

1 See, for example, U. S. General Accounting Office, Environmental
Protection Agency: Problems Persist in Effectively Managing Grants, GAO-
03- 628T (Washington, D. C: June 11, 2003).

2 U. S. General Accounting Office, Major Management Challenges and Program
Risks: Environmental Protection Agency, GAO- 03- 112 (Washington, D. C.:
January 2003).

You asked us to determine the (1) major challenges EPA faces in managing
its grants and how it has addressed these challenges in the past, (2)
extent to which EPA*s recently issued policies and grants management plan
address these challenges, and (3) promising practices, if any, that could
assist EPA in addressing these challenges.

To identify the challenges EPA faces in managing its grants and to examine
how it has addressed these challenges in the past, we (1) analyzed 93
reports on EPA*s grants management, including our reports, EPA*s

Inspector General reports, and EPA*s internal management reviews conducted
from 1996 through 2003, (2) systematically reviewed and recorded
information from the 1,232 records of calendar year 2002 in- depth reviews
of grantee performance* from financial management to progress in achieving
grant objectives, and (3) interviewed EPA officials and reviewed documents
obtained from them. 3 To determine the extent to

which EPA*s recently issued policies and grants management plan address
these challenges, we (1) reviewed the new policies and plan and
interviewed EPA officials responsible for key aspects of the plan, (2)
attended EPA*s grants management training courses, and (3) observed five
EPA in- depth reviews of grantees. To identify what promising practices,
if any, could assist EPA in addressing these challenges, we interviewed
officials in six federal agencies, including the five largest grants-
awarding agencies, and in private sector organizations and reviewed
documentation

from them. We also reviewed two GAO reports that describe effective
training and performance management systems for federal agencies. Our
detailed scope and methodology are discussed in appendix I.

Results in Brief EPA continues to face four key grants management
challenges, despite past efforts to address them. These challenges are (1)
selecting the most

qualified grant applicants, (2) effectively overseeing grantees, (3)
measuring the results of grants, and (4) effectively managing grant staff
and resources. In recent years, EPA took a series of actions to address
these challenges by, among other things, issuing policies, conducting
training, and developing a new data system for grants management. However,
these

actions had mixed results because of the complexity of the problems,
weaknesses in design and implementation, and insufficient management

3 Federal financial assistance includes grants, cooperative agreements,
loans, loan guarantees, scholarships, and other forms of assistance. For
this report, we focused on both grants and cooperative agreements, and for
simplicity, refer to both as *grants.*

attention. For example, EPA*s efforts to improve oversight included
indepth monitoring of grantees but did not include using a statistical
approach

to identifying grantees for reviews, collecting standard information from
the reviews, and analyzing the results to identify systemic grants
management problems. EPA, therefore, could not be assured that it was
identifying and resolving grantee problems and using its resources to more
effectively

target its oversight efforts. EPA*s 2002 competition and oversight
policies and 2003 grants management plan focus on the major grants
management challenges we identified but will require strengthening,
enhanced accountability, and a sustained commitment to succeed.
Specifically, the new competition policy should encourage EPA to select
the most qualified applicants by requiring competition for more grants.
However, effective implementation of the

policy will require a major cultural shift for EPA managers and staff
because the competitive process will require significant planning and take
more time than awarding grants noncompetitively. Furthermore, the new
oversight policy, while now requiring more in- depth monitoring to
identify problems, still does not address the problem we noted in EPA*s
earlier polices* the need for EPA to develop information on systemic
problems with grantees. Finally, EPA*s new grants management plan does
offer, for the first time, a comprehensive road map with goals,
objectives, and milestones for addressing grants management challenges.
However, while the plan recognizes the importance of accountability, it
does not completely address how EPA will hold all managers and staff
accountable for successfully fulfilling their grants responsibilities. For
example, the plan does not call for including grants management
performance standards in managers* and supervisors* performance
agreements. Without holding all managers and staff accountable for grants
management, EPA cannot ensure the sustained commitment required for the
plan*s success.

Although information on promising grants management practices is limited,
the federal agencies and other organizations we contacted identified some
practices for each of EPA*s four major challenges that may further assist
EPA in improving its grants management. For example, one federal agency
takes into account applicants* potential to achieve results when selecting
grantees. A private foundation conducts preaward reviews of some
applicants to learn about the organization and assess its fiscal health.
In addition, we have provided a guide for federal agencies to use in
developing a comprehensive performance management system to hold managers
and staff accountable for achieving desired agency results. This

guide could be useful in helping EPA ensure accountability for grants
management performance.

We are making recommendations to the EPA Administrator to strengthen
grants management, specifically in overseeing grantees, measuring
environmental outcomes, incorporating accountability for grants management
responsibilities, considering promising practices, and reporting on the
progress of its efforts in its annual report to Congress.

We provided a draft of this report to EPA for its review and comment. EPA
stated that it agreed with our recommendations and will implement them as
part of its 5- year grants management plan. EPA*s comments are presented
in appendix V.

Background EPA administers and oversees grants primarily through the
Office of Grants and Debarment, 10 program offices in headquarters, 4 and
program offices

and grants management offices in EPA*s 10 regional offices. Figure 1 shows
EPA*s key offices involved in grants activities for headquarters and the
regions. (See app. II for a map of EPA*s regional office locations.)

4 According to EPA officials, two headquarters* offices, EPA*s Office of
General Counsel and the Office of the Chief Financial Officer conduct
limited grant activity.

Figure 1: EPA*s Key Offices Involved in Grant Activities

The management of EPA*s grants program is a cooperative effort involving
the Office of Administration and Resources Management*s Office of Grants
and Debarment, program offices in headquarters, and grants management and
program offices in the regions. The Office of Grants and Debarment
develops grant policy and guidance. It also carries out certain types of
administrative and financial functions for the grants approved by the
headquarters program offices, such as awarding grants and overseeing the
financial management of these grants. On the programmatic side,
headquarters program offices establish and implement national policies for
their grant programs, and set funding priorities. They are also
responsible for the technical and programmatic oversight of their grants.
In the regions, grants management offices carry out certain administrative
and financial

functions for the grants, such as awarding grants approved by the regional
program offices, 5 while the regional program staff provide technical and
programmatic oversight of their grantees.

As of June 2003, 109 grants specialists in the Office of Grants and
Debarment and the regional grants management offices were largely
responsible for administrative and financial grant functions. Furthermore,
1, 835 project officers were actively managing grants in headquarters and

regional program offices. These project officers are responsible for the
technical and programmatic management of grants. Unlike grant specialists,
however, project officers generally have other primary responsibilities,
such as using the scientific and technical expertise for which they were
hired.

In fiscal year 2002, EPA took 8,070 grant actions 6 totaling about $4.2
billion. 7 These awards were made to six main categories of recipients as
shown in figure 2.

5 Program offices in Regions 4, 5, 6, 9, and 10 award grants directly. 6
Grant actions include new awards, increase and decrease amendments. The
8,070 grant actions involving funding were composed of 4,374 new grants,
2,772 increase amendments, and 924 decrease amendments. In addition, EPA
awarded 1,620 no cost extensions, which did not involve funding, in fiscal
2002.

7 GAO did not verify EPA*s budget data.

Figure 2: Percentage of EPA Grant Dollars Awarded by Recipient Type,
Fiscal Year 2002

EPA offers two types of grants* nondiscretionary and discretionary: 
Nondiscretionary grants support water infrastructure projects, such as

the drinking water and clean water state revolving fund programs, and
continuing environmental programs, such as the Clean Air Program for
monitoring and enforcing Clean Air Act regulations. For these grants,
Congress directs awards to one or more classes of prospective recipients
who meet specific eligibility criteria; the grants are often awarded on
the basis of formulas prescribed by law or agency regulation. In fiscal
year 2002, EPA awarded about $3.5 billion in nondiscretionary grants. EPA
awarded these grants primarily to states or other governmental entities. 
Discretionary grants fund a variety of activities, such as environmental

research and training. EPA has the discretion to independently determine
the recipients and funding levels for these grants. In fiscal year 2002,
EPA awarded about $719 million in discretionary grants. EPA awarded
discretionary grants primarily to nonprofit organizations, universities,
and government entities. The grant process has the following four phases:

 Preaward. EPA reviews the application paperwork and makes an award
decision.  Award. EPA prepares the grant documents and instructs the
grantee on technical requirements, and the grantee signs an agreement to
comply with all requirements.

 Postaward. After awarding the grant, EPA provides technical assistance,
oversees the work, and provides payments to the grantee; the grantee
completes the work, and the project ends.

 Closeout of the award. EPA ensures that all technical work and
administrative requirements have been completed; EPA prepares closeout
documents and notifies the grantee that the grant is completed.

EPA*s grantees are subject to the same type of financial management
oversight as the recipients of other federal assistance. Specifically, the
Single Audit Act 8 requires grantees to have an audit of their financial
statements and federal awards or a program specific audit if they spend
$300, 000 or more in federal awards in a fiscal year. 9 Grantees submit
these audits to a central clearinghouse operated by the Bureau of the
Census, which then forwards the audit findings to the appropriate agency
for any necessary action. However, the act does not cover all grants and
all aspects of grants management and, therefore, agencies must take
additional steps to ensure that federal funds are spent appropriately.

In addition, EPA conducts in- depth reviews to analyze grantees*
compliance with grant regulations and specific grant requirements. 10
Furthermore, to determine how well offices and regions oversee grantees,

EPA conducts three types of internal management reviews that address
grants management: (1) Management Oversight Reviews, which assess grants
management activities in regional offices; (2) Management

Effectiveness Reviews, which are self- assessments of grants management
activities performed by EPA*s headquarters and regional offices; and (3)

8 The Single Audit Act Amendments of 1996, Pub. L. No. 104- 156, 110 Stat.
1396 (codified at 31 U. S. C. S:S: 7501- 7507). 9 The Office of Management
and Budget, as authorized by the act, increased this amount to $500, 000
in federal awards as of June 23, 2003. 10 EPA refers to these in- depth
reviews as advance monitoring.

Post- Award Validation Reviews, which assess the effectiveness of
initiatives EPA has taken to improve oversight efforts.

EPA has faced persistent problems in managing its grants. EPA*s Inspector
General testified before Congress in 1996 and again in 1999 that EPA did
not fulfill its obligation to properly monitor grants. Acknowledging these
problems, EPA identified oversight, including grant closeouts, as a
material weakness* an accounting and internal control system weakness that
the

EPA Administrator must report to the President and Congress. 11 EPA*s
fiscal year 1999 Federal Managers* Financial Integrity Act report
indicated that this oversight material weakness had been corrected, but
the Inspector General testified that the weakness continued. In 2002, the
Inspector General again recommended that EPA designate grants management
as a material weakness. The Office of Management and Budget (OMB) also

recommended in 2002 that EPA designate grants management as a material
weakness. In its fiscal year 2002 Annual Report, 12 EPA ultimately decided
to maintain this issue as an agency- level weakness, which is a lower
level of risk than a material weakness. EPA reached this decision because
it believes its ongoing corrective action efforts will help to resolve
outstanding grants management challenges. However, in adding EPA*s grants
management to our list of EPA*s major management challenges in January
2003, we signaled our concern that EPA has not yet taken sufficient action
to ensure that it can manage its grants effectively. Four Key Grants

We identified four key challenges that EPA continues to face in managing
Management

its grants despite efforts to address them. These challenges are (1)
selecting the most qualified grant applicants, (2) effectively overseeing
Challenges Persist

grantees, (3) measuring the results of grants, and (4) effectively
managing despite EPA*s Past grant staff and resources. In recent years, 13
EPA took a series of actions to

Efforts to Address address these challenges by, among other things,
issuing policies on

competition and oversight, conducting training for project officers and
Them

nonprofit organizations, and developing a new data system for grants
management. However, these actions had mixed results because of the 11 See
31 U. S. C. S: 3512. 12 U. S. Environmental Protection Agency, Fiscal Year
2002 Annual Report, EPA- 190- R- 03- 001 (Washington, D. C.: Jan. 31,
2003). 13 EPA took these actions through early 2002.

complexity of the problems, weaknesses in design and implementation, and
insufficient management attention.

EPA Has Not Always EPA has not always ensured that the most qualified
applicants receive

Selected the Most Qualified grant awards. For discretionary grants, the
agency requires project officers Applicants despite Issuing a

to evaluate grant applications and recommend funding those that will most
Competition Policy

effectively contribute to EPA*s program objectives and priorities. But
project officers have not always performed all of the steps required to
review grant applications. According to a 2003 Inspector General report,
14 project officers did not always, among other things, (1) ensure a link

between the project funded and EPA*s mission, (2) assess the probability
of success prior to the award, and (3) determine the reasonableness of
proposed project costs. As a result, EPA may have lost the opportunity to
fund other projects that could have better achieved its mission.

The Federal Grant and Cooperative Agreement Act of 1977 15 encourages
agencies to use competition in awarding grants. To encourage competition,
EPA issued a grants competition policy in 1995. However, EPA*s policy did

not result in meaningful competition throughout the agency, according to
EPA officials. 16 Furthermore, EPA*s own internal management reviews and a
2001 Inspector General report found that EPA has not always encouraged
competition. 17 Finally, EPA has not always engaged in widespread
solicitation when it

could be beneficial to do so. Widespread solicitation would provide
greater assurance that EPA receives proposals from a variety of eligible
and highly qualified applicants who otherwise may not have known about
grant

14 EPA Office of Inspector General, EPA Must Emphasize Importance of Pre-
Award Reviews for Assistance Agreements, Report No. 2003- P- 0007
(Washington, D. C.: Mar. 31, 2003).

15 Federal Grant and Cooperative Agreement Act of 1977, Pub. L. No. 95-
224, 92 Stat. 3 (codified as amended at 31 U. S. C. S:S: 6301- 6308). 16
According to a 2003 National Research Council report, however, EPA*s
Science To Achieve Results (STAR) program effectively awards grants for
extramural research through its peerreviewed, competitive award process.
See National Research Council, The Measure of STAR: Review of the U. S.
Environmental Protection Agency*s Science To Achieve Results (STAR)
Research Grants Program. (Washington, D. C.: 2003).

17 EPA Office of the Inspector General, EPA*s Competitive Practices for
Assistance Awards,

Report No. 2001- P- 00008 (Philadelphia, PA: May 21, 2001).

opportunities. According to a 2001 EPA Inspector General report, 18
program officials indicated that widespread solicitation was not necessary
because *word gets out* to eligible applicants. Applicants often sent
their proposals directly to these program officials, who funded them using
*uniquely qualified* as the justification for a noncompetitive award. This
procedure created the appearance of preferential treatment by not offering
the same opportunities to all potential applicants. In addition, the
agency provided incomplete or inconsistent public information on its grant
programs in the Catalog of Federal Domestic Assistance. Therefore,
potential applicants may not have been adequately informed of funding
opportunities. EPA Has Not Always

EPA has not always effectively overseen grant recipients despite efforts
to Effectively Overseen Grant

improve monitoring. To address oversight problems, EPA issued a series of
Recipients despite Past

policies starting in 1998. However, these oversight policies have had
mixed Actions to Improve

results in addressing this challenge. Oversight

EPA Has Not Always Effectively EPA*s oversight efforts have not always
been proactive or well

Overseen Grant Recipients documented, or ensured compliance with grant
requirements and

objectives. According to EPA*s 2001 internal management reviews, project
officers had minimal contact with grantees, limiting their oversight to
reviewing grantees* progress reports. However, project officers did not
proactively ensure the recipient submitted the reports on time or that the
reports included the required financial information. According to our
analysis of EPA*s 2002 in- depth reviews, 15 percent of the reviews
identified a problem with grantees* submitting complete progress reports
on time.

Furthermore, EPA*s guidance emphasizes the importance of documenting
oversight efforts, in part, because a lack of documentation raises
questions about the extent of the monitoring conducted. However, project
officers and grants specialists have not always documented their
monitoring activities. According to a 2002 EPA internal management review,
for example, one grants management office developed a form to make it
easier

to ensure monitoring activities were completed, but the form was missing
from 50 percent of the grant files reviewed. Even when staff used the 18
EPA Office of Inspector General, Report No. 2001- P- 00008.

monitoring form, they did not always complete it. Furthermore, project
officers did not always document that they had monitored the required five
key areas under the policy, such as ensuring compliance with the terms and
conditions of the grant award. 19 Finally, EPA has not effectively ensured
that grantees comply with the

following grant requirements and objectives:  Adequate financial and
internal controls. In 2001, 20 we reported that

EPA*s oversight of nonprofit grantees* costs did not ensure that grantees
used funds for costs allowed under OMB*s guidance. 21 For example, EPA did
not include transaction testing, which can identify unallowable
expenditures, such as lobbying.  Compliance with grant regulations. In
2002, according to the Inspector General, EPA did not monitor grantees*
procurements to determine if

the grantees were using a competitive process to obtain the best products,
at the best price, from the most qualified firms. 22 In addition, in 1999
and again in 2002, the Inspector General reported conflict- ofinterest
problems because grant recipients had awarded contracts to parties who had
helped them prepare their grants and, therefore, had advance knowledge
about grantees* plans to award contracts. 23 Our review of EPA*s 2002 in-
depth reviews also found that a number of

19 The monitoring of key areas is required under EPA Order 5700.4, Interim
Grantee Compliance Assistance Initiative Policy, (February 2002). The
other four key areas include the (1) progress of work, (2) meeting of all
programmatic and statutory and regulatory requirements are met, (3)
equipment purchased under the award is managed and accounted for, and (4)
financial expenditures. 20 U. S. General Accounting Office, Environmental
Protection: EPA*s Oversight of Nonprofit

Grantees* Costs Is Limited, GAO- 01- 366 (Washington, D. C.: Apr. 6,
2001). 21 OMB has issued three circulars defining allowable costs for
different types of grantees: A21, A- 87, and A- 122. 22 EPA Office of
Inspector General, Procurements Made by Assistance Agreement Recipients
Should Be Competitive, Report No. 2002- P- 00009 (Philadelphia, PA: Mar.
28, 2002).

23 EPA Office of Inspector General, Report of Audit on the Center for
Chesapeake Communities, Report No. E6DEP8- 03- 0014- 9100117
(Philadelphia, PA: Mar. 31, 1999) and EPA Office of Inspector General,
Assistance Agreement X993795- 01 Awarded by EPA to the Lake Wallenpaupack
Watershed Management District, Report No. 2002- M- 00007 (Philadelphia,
PA: Jan. 18, 2002).

compliance problems persist. (See table 7 in app. III.) For example, EPA
found accounting problems in 37 of 189 administrative reviews and
procurement problems in 70 of 189 of these reviews.

 Meeting grant objectives. EPA has not fully ensured that recipients are
meeting grant objectives. For example, in 2000, we reported that EPA had
not adequately tracked its Science To Achieve Results (STAR) research
grants to ensure that the grantees had submitted the results of their
scientific investigations* the objective of the grant* in a timely manner.
24 We found that 144 of the nearly 200 grants we reviewed had missed their
deadline for submitting final reports, even after some

extensions had been given. EPA*s Past Actions to Improve

To address oversight problems, EPA issued three oversight policies* in
Oversight Have Had Mixed 1998, 1999, and February 2002* designed to
promote proactive and well Results

documented oversight of grantees to ensure compliance with grant
regulations and objectives. For the first time, these policies required
the following:  Baseline monitoring of all grantees at least once during
the life of the

grant. EPA staff are responsible for contacting the grantee, reviewing the
grantee*s reports, assessing compliance with terms and conditions, and
monitoring payments to the grantee.

 In- depth reviews for at least 5 to 10 percent of grant recipients. Both
grants management offices and program offices conduct these reviews either
at the grantee*s location (on- site) or at EPA*s office or another
location (off- site). The grant management offices conduct administrative
reviews, which are designed to evaluate grantees* financial and
administrative capacity. The program offices conduct programmatic reviews,
which are designed to assess the grantees* activities in five key areas:
assessing progress of work, reviewing financial expenditures, meeting the
grant*s terms and conditions,

meeting all programmatic and statutory and regulatory requirements, and
verifying that equipment purchased under the award is managed and
accounted for. The policy required that programmatic staff use a suggested
protocol in conducting their in- depth reviews. Furthermore,

24 U. S. General Accounting Office, Environmental Research: STAR Grants
Focus on Agency Priorities, but Management Enhancements Are Possible, GAO/
RCED- 00- 170 (Washington, D. C.: Sept. 11, 2000).

the policy included suggested criteria for selecting grantees to be
reviewed.  Annual monitoring plans submitted by headquarters and regional
offices that outline the offices* proposed oversight activities. The
policy also suggested activities to be included in these plans, such as
monitoring grantees* progress of work, documenting their efforts, and
closing out grants in a timely manner.  Grantee compliance database*
developed by the Office of Grants and

Debarment* that the grants management offices would use to store the in-
depth reviews.

We found that EPA*s design and implementation of its actions to improve
grantee oversight have had mixed results. In particular, we found both
improvements and weaknesses in EPA*s in- depth reviews and its ability to

track grantee compliance. On the positive side, EPA*s policies required
headquarters and regional offices to conduct a minimum number of in- depth
reviews. According to our analysis of EPA*s in- depth reviews, the agency
conducted 1,232 reviews in 2002. Table 1 shows that these reviews were
conducted almost equally on- site and off- site, and for a variety of
different types of grantees.

Tabl e 1: Number of In- depth Reviews by Type of Grantee and Review, 2002
Type of review

Percentage of Type of grantee Off- site On- site Tot al revi ews total
reviews

Nonprofits 110 134 244 20 State 140 198 338 27 Native American tribes 190
137 327 27

Local government 82 44 126 10 University 105 55 160 13 Other 17 20 37 3

Tot al 644 588 1,232 100 Source: GAO analysis of EPA*s in- depth reviews.

EPA also more evenly distributed, and increased the number of, on- site
reviews among its program and regional offices between 2001 and 2002.
According to the Inspector General, EPA conducted 466 on- site, in- depth
reviews in 2001, but 265 of these, or 57 percent, were conducted by only
two of EPA*s offices. 25 However, according to our analysis of the 2002
indepth reviews, on- site reviews were more evenly distributed: 588 on-
site reviews, with the two EPA offices that conducted the most reviews
representing approximately 28 percent of the total. (See table 6 in app.
III.)

Furthermore, about half of EPA*s in- depth reviews found problems with
grantees. (See table 11 in app. III.) According to our analysis, in 2002,
EPA grants specialists and project officers identified 1, 250 problems in
21 areas. (See table 7 in app. III.) Tables 2 and 3 show the most
frequently identified problems for the 189 administrative and 1,017
programmatic reviews. For example, 73 of 189 administrative reviews found
problems with grantees* written procedures, while 308 of the 1,017
programmatic reviews identified technical issues. Tabl e 2: Most
Frequently Identified Problems, by Problem Area for Administrative

Reviews, 2002 Number of reviews Type of problem with reported problem

Written procedures 73 Procurement 70 Personnel/ payroll 51 Accounting 37
Source: GAO analysis of EPA*s in- depth reviews.

25 EPA Office of Inspector General, Additional Efforts Needed to Improve
EPA*s Oversight of Assistance Agreements, Report No. 2002- P- 00018
(Washington, D. C.: Sept. 30, 2002).

Tabl e 3: Most Frequently Identified Problems, by Problem Area for
Programmatic Reviews, 2002 Number of reviews with Type of problem reported
problem

Technical issues 308 Progress reports 167 Personnel/ payroll 92 Quality
assurance 71 Source: GAO analysis of EPA*s in- depth reviews.

The differences in types of problems frequently identified, as shown in
tables 2 and 3, reflect differences in the focus of administrative and
programmatic reviews. Table 4 describes the nature of these problems. (See
table 12 in app. III for a description of all the problems.)

Tabl e 4: Description of Most Frequently Identified Problems in EPA*s In-
depth Reviews Problem Types of problems included in EPA*s in- depth
reviews

Accounting Any failure of a grantee*s financial management system or
shortcomings in the procedures it used to ensure the proper accounting of
federal funds. For example, EPA found cases in which a grantee:  could
not compare budgeted amounts to actual expenditures,  did not properly
reconcile report balances to the general ledger, or  did not separately
track funds for different grants. Personnel/

Problems varied depending on the type of review conducted. Administrative
reviews included cases payroll in which a grantee did not track the amount
of time its employees spent on specific grant activities.

Programmatic reviews included cases in which grantees did not have
sufficient staff resources to perform the grant activities.

Procurement Grantees lacked documentation to support sole- source
contracts, and grantees did not report their efforts to encourage
procurement from minority- and woman- owned businesses.

Progress reports A grantee*s progress report was missing, late, or did not
include all the necessary information. Quality assurance A grantee needed
to revise its quality assurance plan, which is required to ensure the
quality of data collected during the grant work.

Technical issues A grantee was behind in the progress of his or her work.
Written procedures A grantee*s written policies or procedures were either
missing or inadequate. Source: GAO analysis of EPA*s in- depth reviews.

In response to these problems, EPA required or recommended that grantees
take corrective actions for about half, or 640, of the 1,250 problems
identified. (See table 13 in app. III.) For example, EPA found that one
grantee had not adequately separated duties to provide checks and

balances with the grantee*s organization. That is, the executive director
was also the project manager, payment authorizer, contract executor, and
payroll officer. EPA recommended that the organization separate these
responsibilities to ensure proper handling of federal funds.

EPA also took more significant actions against some grantees. These
significant actions include the following:

 Suspension order* EPA requires a grantee to stop work and take action to
minimize its grant- related expenditures until it resolves EPA*s areas of
concern.

 Payment holds* EPA decides it will not make any more payments to a
grantee until the grantee has resolved an area of concern.

 Disallowance of costs* EPA disallows a grantee*s expenditures when it
determines that the grantee did not spend its grant funds in accordance
with federal cost principles or its particular grant agreement. 
Debarment* EPA declares that a particular organization or individual is

not eligible to receive grants for a specific period of time. 
Termination* EPA cancels the grant. In our analysis of EPA*s 2002 in-
depth reviews, we identified 47 significant actions against its grantees.
The most frequent action taken was withholding payment: 15 of the 47
significant actions. In no cases did EPA debar a grantee. Nonprofit
organizations and tribes each received 21 of these actions, states
received 5, and local governments and universities

received none. (See table 16 in app. III.) EPA*s significant actions
included the following, for example:

 Region 8 issued a payment hold against one of its tribal grantees after
questioning costs because the recipient appeared to be performing work
outside the scope of the grant agreement. The region determined that

the remaining $27,459 in the grantee*s account should be withheld until
the tribe submitted documentation to demonstrate that the questioned costs
were allowable.

 Region 3 disallowed costs for one of its nonprofit grantees. The region
disallowed $51,085 based on a conflict- of- interest, lack of competition
in

the subagreement, and lack of supporting documentation in accounting
records.

 The Grants Administration Division terminated a grant for a nonprofit
grantee. EPA took this action because the grantee actually had no paid
employees of its own but instead used the employees of a for- profit
company to conduct the work. This arrangement created an apparent conflict
of interest.

 Region 4 withheld $40,000 of funding from a state because it found that
the state had not agreed to fully implement its responsibilities as
defined by the grant.

EPA data also showed that the agency took other significant actions in
2002 that did not result from in- depth reviews. According to our
analysis, EPA took 24 other significant actions because of problems
identified during an investigation by EPA*s Inspector General or during
oversight of grantees in

actions other than in- depth reviews. On another positive note, EPA
improved its closeout of grants after it developed specific procedures for
closing out nonconstruction grants and identified a strategy for closing
out construction grants. 26 This strategy assessed impediments to closing
out grants and actions to address these

impediments. EPA had successfully resolved its backlog by 2002. As a
result, EPA has eliminated this backlog as a material weakness, and thus
it better ensures that grant commitments have been met. Despite these
positive steps, EPA*s actions did not address the need to

develop information on systemic problems with grantees to better target
its oversight efforts* a weakness in policy design. EPA could not develop
systemic information for three reasons. First, EPA did not use a
statistical approach to selecting grantees for in- depth reviews. In
particular, EPA allowed each of its offices to determine how many in-
depth reviews the office would conduct* from at least 5 to 10 percent of
grantees* and what criteria the office would use to select grantees for
in- depth reviews. The policy*s flexibility in selecting grantees was
premised on the belief that

26 Construction grants are grants used for designing and constructing
facilities such as wastewater treatment plants. Nonconstruction grants are
grants such as those awarded to states for conducting air quality
monitoring or grants to universities for conducting environmental
research.

those closest to grantees, such as a program or regional office, could
best decide on which grantees to select for oversight. However, if EPA
used a statistical approach to grantee selection, it could then project
the results of the reviews to all EPA grantees. 27 Since EPA did not use
such a sampling

plan in conducting its 2002 reviews, neither EPA nor we could use data
from these reviews to determine the overall compliance of grantees.
Furthermore, without a statistical approach, EPA cannot determine whether
5, 10, or any other percentage, adequately provides it with a
comprehensive assessment of its grantees.

Second, EPA does not have a standard reporting format to ensure
consistency and clarity in reporting review results. Review results
include such items as problems identified and corrective actions required
to address these problems. As we found in our analysis of the 2002 in-
depth reviews, EPA officials across the agency report in various formats
that do not always clearly present the results of the review. For example,
some EPA officials provided a narrative report on the results of their
reviews,

while others completed a protocol that they used in conducting their
review. We found 349 instances where the project officer or grants
management specialist did not clearly explain whether he or she had
discovered a problem or not.

Finally, EPA did not develop data elements for its grantee compliance
database or a plan for using that data to identify and act on systemic
problems. We found that valuable information could be collected from the
2002 in- depth reviews for assessing such issues as the (1) types of
grantees having problems, (2) types of problem areas needing further
attention, (3) types of reviews* on- site or off- site* that provide the
best insights into certain problems areas, and (4) corrective actions
required or

recommended to resolve problems. With such information, EPA could, over
time, identify problem areas and develop trends to assess the
effectiveness of corrective actions in order to better target its
oversight efforts. In particular, according to our analysis of EPA*s 2002
in- depth reviews, administrative reviews identify more problems when
conducted on site, while the number of problems identified by programmatic
reviews does not differ by on- site or off- site reviews. (See table 10 in
app. III.) However, nearly half of the programmatic reviews,

27 A probability sample is one in which each member of the population has
a known, nonzero probability of selection.

which constituted more than 80 percent of the 2002 reviews, were conducted
on site. (See table 5 in app. III.) Since on- site reviews are resource
intensive because of travel costs and staff used, a systematic analysis
could enable EPA to better target its resources. Similarly, EPA could
incorporate other information into its grantee compliance database, such
as Inspector General reports, to identify problem areas, and target
oversight resources. In addition, EPA could use the database to track the
resolution of problems.

The weaknesses in the policies are compounded by the fact that EPA did not
always effectively implement its oversight policies. Specifically, EPA
used its database to collect and store in- depth reviews, but not all EPA
offices were required under the policies to enter the results of their
reviews into the database, and not all of the offices required to enter
information into the database did so. Moreover, EPA officials acknowledged
the agency did not have an effective method to verify the number of in-
depth reviews it reported to Congress. We found that our total number of
in- depth reviews* 1,232* was about the same as EPA*s total* 1,208* but we
found discrepancies, on an office- by- office basis, in the number of
reviews conducted. (See table 6 in app. III.) For example, in one office
we identified 176 more reviews than EPA did, and in another office we
identified 76 fewer reviews than EPA did. Finally, we found that EPA*s in-
depth reviews did not always indicate whether these reviews had used a
protocol. For example, in 294 of the 893 nonmandated reviews, it was
unclear whether the grants specialist or project officer used a protocol.
28 (See tables 14 and 15 in app. III.)

The Inspector General has also reported on problems with EPA*s
implementation of its actions to address oversight problems. In 2002, the
Inspector General found, among other things, inconsistent performance of
monitoring responsibilities, inadequate preparation of monitoring plans,
and inadequate internal management reviews. 29 For example, the Inspector

General found that EPA did not use the internal management reviews to draw
overall conclusions on the effectiveness of oversight and did not quantify
its review results. Furthermore, some senior resource officials did

28 Nonmandated reviews were defined in our analysis as reviews conducted
for reasons other than statutory or periodic programmatic requirements. Of
the 893 nonmandated reviews, for 91 of the administrative reviews, 191 of
the programmatic reviews, and 12 of the

joint reviews, it was unclear if a protocol was used. 29 EPA Office of
Inspector General, Report No. 2002- P- 00018.

not emphasize the importance of grants oversight. 30 According to these
officials, the minimal control they have over the regional program
offices* priorities contributes to problems in effective grants management
oversight. The Inspector General found that EPA lacked an official who is
clearly responsible for monitoring the regional program offices*
priorities for overseeing grants.

EPA*s Efforts to Address the Planning for grants to achieve environmental
results* and measuring

Complex Problem of results* is a difficult, complex challenge. As pointed
out in an earlier

report, 31 while it is important to measure outcomes of environmental
Measuring Environmental

activities rather than just the activities (outputs) themselves, agencies
face Results Have Not challenges in doing this. Output measures are
inherently easier to develop Consistently Ensured That and report on than
environmental results measures. In contrast, defensible Grantees Achieve
Them outcome measures are substantially more difficult to develop because
of

the (1) absence of baseline data against which environmental improvements
can be measured, (2) inherent problems in quantifying certain results, (3)
difficulty of linking program activities with environmental results
because multiple conditions influence environmental results, and (4)
resources needed for measurement.

Nonetheless, EPA has focused on grants* environmental results in its
partnership with the states under the National Environmental Performance
Partnership System. Under this system, states may enter into *Performance
Partnership Agreements* with their EPA regional offices. The partners use

a common set of national environmental indicators (* core performance
measures*). In developing these performance measures, EPA and state
officials have sought to move beyond counting the number of actions
conducted and toward environmental outcomes.

However, EPA has awarded some discretionary grants before considering how
the results of the grantees* work would contribute to achieving
environmental results. In 2001, we reported that EPA program officials
treated EPA*s strategic goals and objectives as a clerical tool for

30 Senior resource officials are typically deputy assistant administrators
in headquarters offices and assistant regional administrators, and are in
charge of strengthening agencywide fiscal resource management while also
ensuring compliance with laws and regulations.

31 U. S. General Accounting Office, Managing for Results: EPA Faces
Challenges in Developing Results- Oriented Performance Goals and Measures,
GAO/ RCED- 00- 77 (Washington, D. C.: Apr. 28, 2000).

categorizing grants after the funds were already awarded rather than as a
tool to guide grant selection, although the grants generally were aligned
with EPA*s strategic goals and objectives. 32 By assessing the relevance
of these grants to EPA*s strategic plan after awarding the grant, EPA
cannot ensure that it is awarding grants that will best help it accomplish
its mission.

EPA has also not developed environmental measures and outcomes for all of
its grant programs. In 2000, we reported that EPA did not have program
criteria to measure the effectiveness of its STAR program. 33 Instead, EPA
focused on the procedures and processes for awarding grants. As a result,
EPA was uncertain about what the program was achieving. Similarly, OMB
recently evaluated four EPA grant programs to assess their effectiveness
in achieving and measuring results. 34 According to this evaluation, these
four programs lacked outcome- based measures* measures that demonstrated

the impact of the programs on improving human health and the environment.
It concluded that one of EPA*s major challenges was demonstrating program
effectiveness in achieving public health and environmental results.
Furthermore, as we reported in 1999, the National Environmental
Performance Partnership System was hampered by the technical problems we
noted earlier, such as the absence of baseline data against which
environmental improvements could be measured. 35 Finally, EPA often does
not require grantees to submit work plans that

explain how a project will achieve measurable environmental results. The
grantee work plan should describe the project, its objectives, the method
the grantee will use to accomplish the objectives, and expected outcomes.

The project officer uses the work plan to evaluate performance under the
32 U. S. General Accounting Office, Environmental Protection: Information
on EPA Project Grants and Use of Waiver Authority, GAO- 01- 359
(Washington, D. C.: Mar. 9, 2001). 33 GAO/ RCED- 00- 170.

34 The four EPA programs assessed were the Drinking Water State Revolving
Fund, Leaking Underground Storage Tanks, Nonpoint Source Grants, and
Tribal General Assistance programs. OMB evaluated these programs using its
Program Assessment Rating Tool (PART), a questionnaire that evaluated four
critical areas of performance: purpose and design, strategic planning,
management, results and accountability. These assessments were included in
the President*s 2004 budget submission. 35 U. S. General Accounting
Office, Environmental Protection: Collaborative EPA- State Effort Needed
to Improve New Performance Partnership System, GAO/ RCED- 99- 171
(Washington, D. C.: June 21, 1999).

grant agreement. In 2002, EPA*s Inspector General reported that EPA
approved some grantees* work plans without determining the projects* long-
term human health and environmental outcomes. 36 In fact, for almost half
of the 42 discretionary grants the Inspector General reviewed, EPA did not
even attempt to measure the projects* outcomes. Instead, EPA funded grants
on the basis of work plans that focused on short- term procedural

results, such as meetings or conferences. In some cases, it was unclear
what the grant had accomplished. In 2003, the Inspector General again
found the project officers had not negotiated environmental outcomes in
work plans. The Inspector General found that 42 percent of the grant work
plans reviewed* both discretionary and nondiscretionary grants* lacked

negotiated environmental outcomes. 37 EPA Has Not Always

EPA has not always appropriately allocated the workload for staff
Effectively Managed Its managing grants, provided them with adequate
training, or held them

Grants Staff and Resources accountable. Additionally, EPA has not always
provided staff with the

despite Some Past Efforts resources, support, and information necessary to
manage the agency*s

grants. To address these problems, EPA has taken a number of actions, such
as conducting additional training and developing a new electronic grants
management system. However, implementation weaknesses have meant that EPA
has not completely resolved its resource management problems.

EPA Has Not Always Effectively In terms of staff, EPA faces workload
problems, according to its internal

Managed Its Grant Resources management reviews. EPA has not had enough
staff in some locations to

adequately manage the number of grants it awards. Furthermore, workload
was sometimes unevenly distributed. For example, in one program office, 29
project officers had more than 4 grants to manage, while 11 had more than
20. Additionally, in one regional office, project officers and grant
specialists did not promptly provide required approvals for certain grants
management tasks, in part because their competing workloads prevented them
from processing approvals on time. Finally, in 2002, the Inspector General
reported that there was considerable uncertainty over the size of

36 EPA Office of Inspector General, Surveys, Studies, Investigations, and
Special Purpose Grants, Report No. 2002- P- 00005 (Philadelphia, PA: Mar.
21, 2002). 37 EPA Office of Inspector General, EPA Must Emphasize
Importance of Pre- Award Reviews for Assistance Agreements, Report No.
2003- P- 00007 (Washington, D. C.: Mar. 31, 2003).

the workload that grants specialists and project officers could manage
effectively. 38 These workload problems could, in some instances, result
in poor grants management.

EPA requires all project officers to certify their grants management
skills by attending a 3- day training course; this course is based on a
training manual that EPA periodically updates. Even though EPA has this
requirement, it has not always provided grants staff with the training
necessary to properly manage all aspects of grants. According to EPA*s
internal management reviews from 1997 through 2002, some staff managing
grants had not completed the basic training required to manage grants.

Other staff may have completed the basic training but need additional
training to refresh their skills or to become familiar with all of their
grants management responsibilities and requirements. For example, in some
instances, project officers were not familiar with the five key areas they
were to review when monitoring grantees, such as the financial aspects of
a grantee*s performance. According to a senior EPA official, with such a
small percentage of some project officers spending time on grants

management duties, it is difficult for EPA to ensure that its 1, 835
project officers who are actively managing grants have the level of
training they need. Furthermore, the agency has not always held its staff*
such as project

officers* accountable for fulfilling grants management responsibilities.
According to the Inspector General and internal management reviews, EPA
has not clearly defined project officers* grants management
responsibilities in their position descriptions and performance
agreements. Without specific standards for grants management in
performance agreements, it is difficult for EPA to hold staff accountable.
It is, therefore, not surprising that, according to the Inspector General,
project officers faced no

consequences for failing to effectively perform grants management duties.
Compounding the problem, agency leadership has not always emphasized the
importance of project officers* grants management duties. 39 It may be
difficult to hold project officers accountable because many

project officers spend only a small proportion of their time managing
grants. For example, a recent EPA workload assessment noted that

38 EPA Office of Inspector General, Report No. 2002- P- 00018. 39 EPA
Office of Inspector General, Report No. 2003- P- 00007.

approximately 35 percent of the agency*s project officers manage only one
grant, instead devoting the bulk of their time to their scientific and
technical responsibilities.

EPA has also not adequately managed its resources to support grant staff
in performing their duties. Some EPA internal management reviews noted a
lack of resource commitment* time and money* to conduct grant management
activities and develop staff. For example, in some cases, grantee
oversight, particularly the on- site reviews, was limited by the scarcity
of such resources as travel funds.

Finally, staff did not always have the information they needed to
effectively manage grants. According to several EPA internal management
reviews, staff lacked accessible or useable reference material* such as
policy and

guidance documents, and other information resources, such as reports of
grantee expenditures. Additionally, we and others have reported that EPA
did not use information from performance evaluations or information
systems to better manage its grants. For example, one EPA region did not
analyze the results of its own internal surveys, which were designed to
assess the effectiveness of its grants management operations.

Efforts to Improve the EPA has taken actions to address concerns that all
of its grants

Management of Grant Resources management staff receive training. EPA
developed a database to document

Have Had Mixed Results which project officers had received the required 3-
day basic training and

began to require that all project officers take an on- line refresher
course every 3 years. While these steps were primarily designed to improve
the training of project officers, grants specialists also have received
some training, and EPA has recently pilot- tested a standard training
course specifically designed for them. EPA managers and staff have also
received other types of training to enhance grants management skills. For
example, EPA has piloted a 1- day course on grants management for managers
and

supervisors. The Office of Grants and Debarment also provides training on
specific aspects of grants management* such as postaward monitoring* as
requested by program or regional offices. Finally, in response to concerns
that some grantees did not have the skills required to manage grants, EPA
developed a 1- day grants management training course for nonprofit
grantees.

However, these actions have not been fully successful. In 2002, EPA*s
Inspector General reported that the agency did not have adequate internal
controls in place to ensure that project officers complied with the
training

requirements. 40 Specifically, one region did not track the names and
dates of project officers who received training, the agencywide database
on training for project officers was inaccurate and had limited
functionality, and the on- line refresher course did not have the controls
necessary to

prevent staff from obtaining false certifications. EPA*s training efforts
have also needed strengthening to better ensure that staff receive and
retain the skills necessary to manage grants. In particular, while offices
can request specific training from the Office of Grants and Debarment, EPA
does not have a process to ensure that all staff receive the training
needed to implement new policies and regulatory requirements in a timely
manner. For instance, when EPA issued new post- award monitoring policies
to improve its grants oversight, project officers did not receive training
on these new requirements unless their program or regional office
specifically requested that training or it was time for the officers to
take the refresher course. EPA also does not measure the impact of its
training programs by determining the baseline level of skills that
employees possess before taking training and determining what, if any,
skills have improved as a result of the training. Nor does EPA link its

training activities with improved performance on the job, especially in
targeted areas such as monitoring contacts with grantees and documentation
of files. As a result, EPA cannot hold either the trainers or the trainees
accountable for improving grants management skills and performance.

EPA has also taken steps to improve another critical resource* its primary
data system for managing grants. In 1997, it began developing the
Integrated Grants Management System (IGMS), which, according to an EPA
official, will facilitate oversight by allowing electronic management

throughout the life of a grant. EPA is developing the system through a
series of modules (databases). EPA estimates that once IGMS becomes fully
operational, it will save roughly $1.6 million annually. EPA believes IGMS
could help resolve some of the long- standing problems in grants
management by implementing controls to prevent the submission of certain
documents without required elements and to provide electronic reminders of
when certain activities or documents are due. Additionally, EPA designed
the system to reduce the potential for data entry errors. In 2001, EPA
began implementing the module for the application and award phases of a
grant. Using IGMS, EPA expects to electronically review the

40 EPA Office of Inspector General, Report No. 2002- P- 00018.

grantee*s application, prepare and review EPA*s documents, and approve the
award. In April 2003, EPA began using IGMS* postaward module. This module
allows project officers to electronically enter project milestones into
the system, communicate with other staff involved in overseeing grants,
receive reports from grantees, and initiate closeout activities. EPA
expects that grants specialists and regional program staff will use IGMS
to electronically manage grants by September 2004 and headquarters program
staff by the end of 2006.

Despite this promising development, the implementation period for IGMS has
been lengthy. This delay was due in part to Y2K concerns, the complexity
of the system for grants management, and the addition of other major
business processes into the system. Consequently, EPA has had to plan a
lengthy implementation schedule in order to ensure that it can properly
roll out the system to all of the programs and regions with its available
training and user- support resources. Therefore, while those offices that
have received IGMS are using it to manage grants, EPA cannot yet determine
whether IGMS has solved some of the agency*s resource management problems.

New Policies and Plan EPA*s recently issued policies on competition and
oversight and a 5- year

Show Promise but plan to address its long- standing grants management
problems are promising and focus on the major management challenges, but
these

Require Strengthening, policies and plan require strengthening, enhanced
accountability, and Enhanced

sustained commitment to succeed. In particular, EPA*s policy generally
Accountability, and

requiring competition for discretionary grants should encourage selection
of the most qualified applicants. However, successful implementation of
Sustained Commitment this policy requires a more planned, rigorous
approach to awarding some

to Succeed grants than in the past, which is a major cultural shift for
EPA managers

and staff. Similarly, EPA*s new oversight policy should increase the
agency*s monitoring of individual grantees, but it does not enable the
agency to identify and address systemic problems. Finally, EPA*s new
grants management plan takes significant steps toward improving grant
management, but the plan will be difficult to implement if EPA does not
demonstrate that it is committed to carrying out the plan over the 5- year
period and if it does not hold managers and staff accountable for their

grants management performance.

Competition Policy Should In September 2002, EPA issued a policy to
promote competition in grant

Encourage the Selection of awards by requiring that most discretionary
grants be competed.

the Most Qualified Discretionary grants totaled about $719 million of the
$4.2 billion in grants

awarded in fiscal year 2002. The policy applies to most discretionary
grant Applicants if Successfully

programs or individual grants of more than $75,000. 41 It exempts
individual Implemented

grants only if they meet certain criteria, such as national security
interests. Exemptions require detailed, written justification and
approval. The policy also promotes widespread solicitation for competed
grants by establishing specific requirements for announcing funding
opportunities in, for example, the Federal Register and on Web sites.
Because the competition policy represents a major change in the way EPA

awards discretionary grants, the agency created the position of a
seniorlevel competition advocate for grants. This advocate oversees the
policy*s implementation and compliance and evaluates its effectiveness.
Among other things, the competition advocate is responsible for developing
implementation guidance and coordinating training to ensure successful
implementation. The competition advocate ensures compliance, in part, by
reviewing any requests for programs to be exempted from competition. The
advocate will also have to work with headquarters and regional offices

as well as staff and management officials at all levels throughout EPA to
successfully implement the policy. The competition policy faces
implementation barriers because it

represents a major cultural shift for EPA staff and managers, who have had
limited experience with competition, according to the Office of Grants and
Debarment. The policy requires EPA officials to take a more planned,
rigorous approach to awarding grants. That is, EPA staff must determine
the evaluation criteria and ranking of these criteria for a grant, develop
the grant announcement, and generally publish it at least 60 days before
the

application deadline. Staff must also evaluate applications* potentially
from a larger number of applicants than in the past* and notify applicants
of their decisions. These activities will require significant planning and
take more time than awarding grants noncompetitively. Office of Grants and
Debarment officials anticipate a learning curve as staff implement the
policy and will evaluate the policy*s effectiveness, including the
threshold dollar level in 2005. While the policy has been in effect for a
number of

41 In certain circumstances exempted programs may conduct a competition;
when they do, they must be conducted in a manner consistent with the
competition policy.

months, it is too early to tell if the policy has resulted in increased
competition over the entire fiscal year. EPA officials believe that
preliminary results indicate that the policy is increasing the use of
competition.

Oversight Policy Makes EPA*s December 2002 oversight policy makes
important improvements, but

Important Improvements it does not fully address the weaknesses in
previous oversight policies that

but Requires Strengthening we noted earlier. The new policy, among other
things, requires the

to Identify Systemic following:

Problems  Offices to increase baseline monitoring from at least once
during the life

of the grant to annually throughout the life of the grant;  Offices to
increase the percentage of in- depth monitoring to at least 10

percent of grant recipients;  Grant management specialists conducting on-
site, in- depth reviews to

include *transaction testing* for unallowable costs, such as lobbying and
entertainment expenditures. Transaction testing consists of reviewing
grantees* accounting ledgers and underlying documentation;

 Offices to identify their criteria for selecting recipients for in-
depth reviews and submit these criteria as part of their annual plans; 
Offices to enter in- depth reviews into the grantee compliance database;

and  The Office of Grants and Debarment*s Grants Administration Division
to

have a work group recommend database enhancements to make the database
more useful for program offices. This work group plans to report its
findings in August 2003.

While the new policy makes these refinements in oversight, it still does
not enable EPA to identify systemic problems in grants management that we
noted earlier. Specifically, the policy does not do the following:

 Incorporate a statistical approach to selecting grantees for review.
EPA*s new policy annually selects 10 percent of grant recipients for
indepth review. EPA officials believe that a single, higher percentage,
rather than a range, will eliminate variation among its offices and
provide better coverage. However, without a statistical approach, EPA

cannot evaluate whether this 10 percent is appropriate. With such an
approach, EPA could adjust the number and allocation of in- depth reviews
to match the level of risk associated with each type of grant recipient
and project the results of the reviews to all EPA grantees.

 Require a standard reporting format for in- depth reviews. EPA needs the
clear and precise information from its in- depth reviews that a standard
reporting format would provide* and which was absent from the 2002 in-
depth reviews we analyzed. Standard information is important so that EPA
can use the information to guide its grants oversight efforts agencywide.
 Maximize use of information in the grantee compliance database. While
EPA has recognized the importance of the information in the

database by forming a database work group and collecting a limited amount
of data from in- depth reviews, it has not yet developed a plan for using
data from all its oversight efforts* in- depth reviews, corrective
actions, and other compliance efforts* to fully identify systemic

problems and then inform grants management officials about oversight areas
that need to be addressed.

New Plan Focuses on the According to EPA*s Assistant Administrator for
Administration and Four Major Management

Resources Management, the agency*s April 2003 5- year grants management
Challenges but Will Require

plan is the most critical component of EPA*s efforts to improve its grants
management. 42 The plan has five goals and accompanying objectives, as
Strengthening, Sustained follows: Commitment, and Enhanced

Accountability  Promote competition in awarding grants. Identify annual
funding

priorities, encourage a large and diverse group of applicants, promote the
importance of competition within the agency, and provide adequate support
for the grant competition advocate.

 Strengthen EPA*s grants oversight. Improve internal management reviews
of EPA*s offices, improve and expand reviews of grant recipients, develop
approaches to prevent or limit grants management weaknesses, establish
clear lines of accountability for grants oversight, and provide high-
level coordination, planning, and priority setting.

42 For further details on the grants management plan see app. IV.

 Support the identification and achievement of environmental outcomes.
Ensure that grantees include expected environmental outcomes and
performance measures in grant work plans and improve reporting on progress
made in achieving environmental outcomes.

 Enhance the skills of EPA personnel involved in grants management.
Update training materials and courses and improve delivery of training to
project officers and grants specialists.  Leverage technology to improve
program performance. Enhance and

expand information systems that support grants management and oversight,
among other things.

In addition to the goals and objectives, the plan establishes performance
measures, targets, and action steps with completion dates for 2003 through
2006. For example, to strengthen EPA*s grants oversight, the plan calls
for conducting consolidated management reviews 43 of headquarters and
regional offices to review the operations of grants management activities
and for completing reviews for three regional offices and four
headquarters offices in 2003. These reviews will examine EPA offices*
oversight of grantees throughout the life of the grant. (See app. IV.)

We believe the grants management plan is comprehensive in that it focuses
on the four major management challenges* grantee selection, oversight,
environmental results, and resources* that we identified in our work. For
the first time, EPA plans a coordinated, integrated approach to improving
grants management. The plan is also a positive step because it (1)
identifies objectives, milestones, and resources to achieve the plan*s
goals; (2) provides an accompanying annual tactical plan that outlines
specific tasks for each goal and objective, identifies the person
accountable for completing the task, and sets an expected completion date;
(3) attempts to build accountability into grants management by
establishing performance measures for each of the plan*s five goals; (4)
recognizes the need for greater involvement of senior officials in
coordinating grants management throughout the agency by establishing a
senior- level grants management

43 EPA stated that its internal grants management reviews consist of (1)
comprehensive management reviews of headquarters and regional offices
conducted by the Office of Grants and Debarment; (2) grants performance
measure reviews performed by the Office of Grants and Debarment, using
performance measures and documentation in the central databases; and (3)
grants management self- assessments conducted by EPA offices.

council to coordinate, plan, and set priorities for grants management; and
(5) establishes best practices for grants management offices.

EPA has already begun implementing several of the actions in the plan or
meant to support the plan; these actions address previously identified
problems. For example, EPA now ensures that grants are adequately

solicited by posting its available grants on the federal grants Web site
http:// www. fedgrants. gov. To enhance oversight, it initiated the
consolidated grants management reviews of headquarters and regional
offices. To better manage for results, EPA formed an Environmental
Outcomes Work Group to address concerns about focusing on

environmental outcomes. Finally, for training, EPA (1) instituted
verification procedures to ensure that only certified project officers
manage grants; (2) drafted its long- term plan for grants management
training; (3) updated the project officer training manual; and (4)
launched a new training course for grants managers. In addition, EPA is
implementing an office award program to promote effective grants
management.

Although EPA*s plan lays out a road map for improving grants management,
it has two weaknesses. First, in terms of measuring results, while the
plan supports identifying and achieving environmental outcomes, the
suggested protocol for in- depth reviews does not call for project
officers to ask grantees about their progress in using measures to achieve
environmental outcomes. Second, EPA*s plan may not fully address long-
standing training problems. Specifically, it is too early to tell whether
EPA*s effort to develop a long- term training plan will provide (1) a
process to ensure that all staff

members receive the training necessary to implement new policies and
regulations in a timely manner or (2) a mechanism to evaluate the impact
of its training efforts on improving staff*s grants management knowledge
and skills, the extent to which staff*s performance improves over time,
and demonstrated results in improving grants management performance. The
impact of training could be measured by indicators such as the proportion
of financial or other problems identified early on by ongoing monitoring,
thereby avoiding potentially greater losses in the long run. Furthermore,
the plan*s performance measures for evaluating training are
outputoriented* how many staff are trained* rather than outcome- oriented*
how much skills have improved.

Successful implementation of the new plan requires more than addressing
these two weaknesses, however. It requires all staff* senior management,
project officers, and grants specialists* to be fully committed to, and
accountable for, grants management. Recognizing the importance of

commitment and accountability, EPA*s 5- year grants management plan has as
one of its objectives the establishment of clear lines of accountability
for grants oversight. The plan, among other things, calls for (1) ensuring
that performance standards established for grants specialists and project
officers adequately address grants management responsibilities in 2004;
(2) clarifying and defining the roles and responsibilities of senior
resource officials, grant specialists, project officers, and others in
2003; and (3) analyzing project officers* and grants specialists* workload
in 2004. In implementing this plan, however, EPA faces challenges to
enhancing

accountability. Although the plan calls for ensuring that project
officers* performance standards adequately address their grants management
responsibilities, agencywide implementation may be difficult. Currently,

project officers do not have uniform performance standards, according to
officials in EPA*s Office of Human Resources and Organizational Services.
Instead, each supervisor sets standards for each project officer, and
these standards may not include grants management responsibilities. It
could

take up to a year to establish and implement a uniform performance
standard, according to these officials. Instead, the Assistant
Administrator for the Office of Administration and Resources Management is
planning to issue guidance in October 2003 on including grants management
responsibilities in individual performance agreements for the next
performance cycle beginning in January 2004. Once individual project
officers* performance standards are established for the approximately
1,800 project officers, strong support by managers at all levels, as well
as regular communication on performance expectations and feedback, will be
key to ensuring that staff with grants management duties successfully meet
their responsibilities.

Although EPA*s current performance management system can accommodate
development of performance standards tailored to each project officer*s
specific grants management responsibilities, the current system provides
only two choices for measuring performance* satisfactory or
unsatisfactory, which may make it difficult to make meaningful
distinctions in performance. Such an approach may not provide enough
meaningful information and dispersion in ratings to recognize and reward
top performers, help everyone attain their maximum potential, and deal
with poor performers.

Furthermore, it is difficult to implement performance standards that will
hold project officers accountable for grants management because these
officers have a variety of responsibilities and some project officers
manage

few grants, and because grants management responsibilities often fall into
the category of *other duties as assigned.* To address this issue, EPA
officials are considering, among other options, whether the agency needs
to develop a smaller cadre of well- trained project officers to oversee
grantees, rather than rely on approximately 1,800 project officers with
different levels of grants management responsibilities and skills. Some
EPA officials believe that having a cadre may help the agency more
effectively implement revised grants management performance standards
because

fewer officers with greater expertise would oversee a larger percentage of
the grants.

EPA will also have difficulty achieving the plan*s goals if all managers
and staff are not held accountable for grants management. The plan does
not call for including grants management standards in managers* and
supervisors* agreements. In contrast, senior grants managers in the Office
of Grants and Debarment as well as other Senior Executive Service managers
have performance standards that address grants management
responsibilities. 44 However, middle- level managers and supervisors also
need to be held accountable for grants management because they oversee
many of the staff that have important grants management responsibilities.
According to Office of Grants and Debarment officials, they are working on
developing performance standards for all managers and supervisors with
grants responsibilities.

Further complicating the establishment of clear lines of accountability,
the Office of Grants and Debarment does not have direct control over many
of the managers and staff who perform grants management duties*
particularly the approximately 1,800 project officers in headquarters and
regional program offices. The division of responsibilities between the
Office of Grants and Debarment and program and regional offices will
continue to present a challenge to holding staff accountable and improving
grants management, and will require the sustained commitment of EPA*s
senior managers.

44 The senior managers include the Director of the Office of Grants and
Debarment, the Director of the Grants Administration Division, and the
Grants Competition Advocate.

Promising Practices Although information on successful grants management
practices is Could Assist EPA in

limited, we identified promising practices in the five federal agencies
that award the largest amount of grant dollars, as well as in other
organizations, Addressing Its Grants

that may help EPA address its grants management challenges. These
Management

practices include (1) screening grantees and assessing some grantees*
Challenges

financial and internal controls before awarding a grant; (2) using an
information system to track and manage in- depth grantee reviews, identify
problems, and analyze trends; and (3) incorporating a results orientation
throughout the grants process, including the preaward phase and the final
reporting of results achieved. In addition, we have developed a guide to
assist federal agencies in developing effective training programs and
performance management systems. 45 Screen and Select the Best

According to some grants management officials we spoke with, screening
Applicants

and reviewing the adequacy of grantees* financial and internal controls
before awarding a grant helps to avoid problem grantees. For example,

 The Robert Wood Johnson Foundation conducts preaward desk reviews of
some applicants to learn about the organization and assess its fiscal
health. Specifically, for grants under $200,000, the foundation may check
a Web site that contains financial information on nonprofit organizations.
For grants larger than $200, 000, especially when the grantee is newly
formed or has never received money from the foundation, staff may request
more detailed information* such as an

organizational chart, copies of audit reports, accounting policies,
investment policies, and a conflict- of- interest statement. In rare
cases, the foundation*s financial staff may visit the applicant to assess
its systems.

 The Department of Commerce (Commerce) Office of Inspector General
conducts a three- step screening process to identify deficiencies that
could hinder the ability of nonprofit applicants to properly control
federal funds: (1) a name check, through the Federal Bureau of
Investigation, to determine whether the individuals connected with a

45 U. S. General Accounting Office, Human Capital: A Guide for Assessing
Strategic Training and Development Efforts in the Federal Government, GAO-
03- 893G (Washington, D. C.: July 2003) and U. S. General Accounting
Office, Results- Oriented Cultures: Creating a Clear Linkage Between
Individual Performance and Organizational Success, GAO- 03- 488
(Washington, D. C.: Mar. 14, 2003).

proposed award have been convicted of embezzlement or other crimes; (2) a
review of the single audit clearinghouse database to determine if the
applicant had a single audit performed and subsequent follow- up to
determine if any findings were resolved; and (3) a review of a credit
report to determine whether the applicant is financially viable. For the
last two reporting periods* April 2002 to March 2003* Commerce*s Inspector
General screened 1,657 proposed awards; delayed 67 until concerns were
satisfactorily resolved; and established special

conditions for 42 awards to adequately safeguard federal funds.  The
Department of Health and Human Services (HHS) uses an internal

*alert list,* accessible to grants management staff throughout the
department, to help identify potential grantees who (1) previously had not
complied with grant regulations or the terms and conditions of a

grant, (2) failed to have had a single audit performed, or (3) have
significant single audit findings that need correction. For these
potential grantees, HHS may incorporate special terms and conditions into
their agreements and more closely monitor them throughout the grant.
Applicants may be put on the alert list by agencies within the

department or at the recommendation of the Office of Inspector General.
According to HHS officials, the alert list should be used for problem
grantees so that the department has the necessary controls in place to
safeguard federal funds rather than be used to prohibit grantees from
receiving grant funding.

 HHS*s Substance Abuse and Mental Health Services Administration (SAMHSA)
uses the alert list and also conducts pre- award reviews to assess
applicants* ability to manage grant funds. Like the Department of
Commerce, SAMHSA screens some nonprofit grantees to ensure they have the
financial systems needed to manage grant funds. SAMHSA

examines the database for the single audit clearinghouse and its own audit
database to determine if the grant applicant has received a single audit
and whether the audit had significant findings. If the applicant has no
significant findings, SAMHSA assumes that the applicant has adequate
financial systems in place. However, if an audit found significant
problems that the applicant did not adequately address, SAMHSA may
designate the applicant as high risk and add it to the HHS alert list.
SAMHSA may still award grant funds but set restrictions to safeguard the
funds. For example, the grantee may be required to draw funds on a
reimbursement basis, that is, after it submits documentation on its
expenses. Grantees who have not received a single audit must

provide copies of their policies and procedures. SAMHSA assesses this
information to determine if it is sufficient for managing federal funds.

Use an Information System The Federal Transit Administration, within the
Department of to Manage In- Depth

Transportation, uses an information system to track and manage its
Reviews, Identify Problems,

reviews of grantees. This system contains information on problems and
Analyze Trends

identified in the reviews and a schedule of corrective actions that the
grantee needs to take. Using this system, officials can track
correspondence between the agency and grantee and document the resolution
of problems. In addition, officials can use this system to analyze trends,
such as the compliance of individual grantees over time, the compliance of
all grantees or a group of grantees (such as those in a region),
compliance issues found in different types of reviews, and areas

where grantees may require additional technical assistance. This
information system could serve as a model for enhancing EPA*s grantee
compliance database.

Incorporate a Results Several agencies consider grant results in awarding
and reviewing

Orientation throughout the grantees, including the following:

Grants Process  Rural Development, within the U. S. Department of
Agriculture, takes

outcomes into account by requiring applicants to provide information on
the results they expect to achieve and by selecting grantees, in part, on
the basis of this information. For instance, when job creation is a goal
of the grant, the Rural Business Enterprise Grant Program gives applicants
points for jobs they expect to create or save with a grant and

requires them to provide written commitments from businesses that plan on
creating jobs as a result of the grant. Rural Development then gives
points to applicants based on the grant dollars per job they expect to
create.

 HHS*s Health Resources and Services Administration incorporates
information on the results its grantees have achieved into its annual
performance report. The administration collects data directly from its
grantees and surveys a sample of the people its grantees serve to
determine the results the grantees have achieved. Because of the agency*s
success in managing its health center grants to achieve

results, 46 OMB rated the program *effective* in its governmentwide
performance and assessment- rating tool. Out of 79 grant programs
evaluated, this was one of only two rated *effective,* OMB*s highest
rating category. 47 According to OMB, the health center program uses
performance information to improve annual administrative and clinical
outcomes. The agency also conducts its own evaluations and has a
contractor regularly evaluate the program; these evaluations indicate the
program is effective at achieving its goal of extending high- quality
health care to underserved populations.

Train and Hold Staff We have developed a guide that can assist federal
agencies in planning, Accountable for Grants

designing, implementing, and evaluating effective training and development
programs. 48 This guide may help EPA address a training Management
weakness in the plan that we identified* EPA*s lack of a mechanism to
determine the extent to which training improves staff*s grant management
skills. For example, the guide suggests evaluating the impact of training
based on such measures as changes in employee skills, knowledge, or
abilities; changes in on- the- job behaviors; the impact of the training
on program or organizational results, including, in some cases, assessing
the return on investment by comparing training costs with benefits.

In terms of accountability, we developed key practices that federal
agencies can use to establish effective performance management systems. 49
These systems can be used to drive internal change and achieve desired

results. The key practices have helped public sector organizations in the
United States and abroad create a clear linkage between individual
performance and organizational success. They are also critical for
ensuring individual accountability for results throughout the
organization. Key

46 The health center program provides grants to health centers to provide
medical care to uninsured, underserved, and vulnerable populations in
rural and urban areas. 47 OMB*s PART tool rates programs on a four- point
scale from *effective* to *moderately effective* to *adequate* to
*ineffective.* A fifth rating possibility comes into play with

*results not demonstrated* if adequate measures or data to gauge the
program*s performance were not available. The other grant program rated
*effective* was the Medicare Integrity Program, which funds a variety of
efforts to fight fraud and abuse in the Medicare program.

48 GAO- 03- 893G. 49 GAO- 03- 488.

practices of an effective performance management system include, for
example:  Align individual performance expectations with organizational
goals.

An explicit alignment helps individuals see the connection between their
daily activities and organizational goals.

 Use competencies to provide a fuller assessment of performance.

Competencies define the skills and supporting behaviors staff need to
effectively contribute to organizational results.

 Make meaningful distinctions in performance. Effective performance
management systems strive to provide candid and constructive feedback and
the necessary objective information and documentation to award top
performers and deal with poor performers.

Conclusions If EPA is to better achieve its environmental mission, it must
more effectively manage its grants programs* which account for more than
half of its annual budget. EPA*s new policies and 5- year grants
management plan

show promise, but they are missing several critical elements necessary for
the agency to address past grants management weaknesses. Specifically,
EPA*s new oversight policy lacks (1) statistical methods to identify
grantees for review, (2) a standard reporting format to ensure consistency
and clarity in report results, and (3) a plan for integrating and
analyzing compliance information from multiple sources. These actions
would help EPA identify systemic problems with its grantees and better
target its oversight resources. EPA*s plan does not adequately address the
need to assess grantees* progress towards achieving environmental
outcomes. If project officers tracked grantees* progress during in- depth
reviews, they would better help the agency manage grants for results.
Finally, in our view, EPA*s plan does not go far enough to ensure that all
managers and staff with grant management responsibilities are held
accountable for fulfilling their grants management responsibilities. Until
EPA holds all managers and staff accountable for these responsibilities,
it cannot be assured that it will achieve the objectives of its grants
management plan.

EPA recognizes the importance of continuously improving its grants
management practices. To this end, we believe EPA could benefit from
considering the practical approaches successfully implemented by other
federal agencies and organizations. It could also benefit from using our
guide to establish effective training programs to develop relevant staff

skills and a performance management system to hold staff accountable for
using these skills to carry out their grants management responsibilities.

Given EPA*s uneven performance in improving its grants management, it is
too early to tell if its new policies and comprehensive 5- year grants
management plan will succeed more than past initiatives. While the plan
shows promise, we believe that congressional oversight is important to
ensure that EPA*s Administrator, managers, and staff implement the plan in
a sustained, coordinated fashion to meet the plan*s ambitious targets and
time frames. To help facilitate this oversight, EPA*s annual report to
Congress could serve as a vehicle for EPA to report on its achievements in
improving grants management.

Recommendations for To ensure that EPA*s recent efforts to address its
grants management

Executive Action challenges are successful, we recommend that the
Administrator of EPA

provide sufficient resources and commitment to meeting the agency*s grants
management plan*s goals, objectives, and performance targets within the
specified time frames. Furthermore, to strengthen EPA*s efforts we
recommend

 incorporating appropriate statistical techniques in selecting grantees
for in- depth reviews;

 requiring EPA staff to use a standard reporting format for in- depth
reviews so that the results can be entered into the grant databases and
analyzed agencywide;

 developing a plan, including modifications to the grantee compliance
database, to use data from its various oversight efforts* in- depth
reviews, significant actions, corrective actions taken, and other
compliance information* to fully identify systemic problems, inform grants
management officials of areas that need to be addressed, and take
corrective action as needed;

 modifying its in- depth review protocols to include questions on the
status of grantees* progress in measuring and achieving environmental
outcomes;

 incorporating accountability for grants management responsibilities
through performance standards that address grants management for all
managers and staff in headquarters and the regions responsible for

grants management and holding managers and staff accountable for meeting
these standards; and

 evaluating the promising practices identified in this report and
implementing those that could potentially improve EPA grants management.

To better inform Congress about EPA*s achievements in improving grants
management, we recommend that the Administrator of EPA report on the
agency*s accomplishments in meeting the goals and objectives developed in
the grants management plan and other actions to improve grants management,
beginning with its 2003 annual report to Congress.

Agency Comments and We provided a draft of this report to EPA for its
review and comment. In

Our Evaluation response, EPA stated that it agreed with our
recommendations and that it will implement them as part of its 5- year
grants management plan. EPA

further pointed out that it has demonstrated commitment to grants
management as evidenced by its (1) issuance of directives from the Deputy
Administrator/ Acting Administrator requiring senior managers to hold
employees accountable and to include compliance with grants

management policies as part of midyear performance discussions; (2)
ongoing review of performance standards, which will ensure that
performance agreements of all employees involved in grants management
adequately reflect their grants management responsibilities; (3)
requirement for assistant administrators and regional administrators to
address in the assurance letters under the Federal Managers* Financial
Integrity Act the steps they are taking to address grants management

weaknesses; and (4) development of a tactical action plan, which outlines
commitments and milestone dates under the grants management plan and
identifies who is responsible for completing those commitments. EPA also
provided the Office of Grants and Debarment*s Fiscal Year 2003 Federal
Managers* Financial Integrity Act assurance letter, which describes in
detail the actions the office has taken.

In technical comments, EPA added that it had redeployed resources to the
Office of Grants and Debarment and anticipates receiving new resources in
fiscal year 2004 to conduct grants management oversight. Other technical
comments have been incorporated into the report, as appropriate.

EPA*s written comments are presented in appendix V.

As agreed with your offices, unless you publicly announce the contents of
this report earlier, we plan no further distribution until 30 days from
the report date. At that time, we will send copies of this report to the
congressional committees with jurisdiction over EPA and its activities;
the Honorable Marianne Lamont Horinko, Acting EPA Administrator; and the
Honorable Joshua B. Bolten Director, Office of Management and Budget. We
will also make copies available to others upon request. In addition, the
report will be available at no charge on the GAO Web site at htpp:// www.
gao. gov.

If you have any questions about this report, please contact me at (202)
512- 3841. Key contributors to this report are listed in appendix VI.

John B. Stephenson Director, Natural Resources and Environment

Appendi Appendi xes x I

Objectives, Scope, and Methodology This appendix details the methods we
used to determine the (1) major challenges the Environmental Protection
Agency (EPA) faces in managing its grants and how it has addressed these
challenges in the past, (2) extent to which EPA*s recently issued policies
and grants management plan address these challenges, and (3) promising
practices, if any, that could assist EPA in addressing these challenges.

To identify the challenges EPA faces in managing its grants and how it has
addressed these challenges in the past, we identified and analyzed 93
reports written on EPA*s grants, including our reports, EPA*s Inspector

General reports, and EPA*s internal management reviews conducted from 1996
through 2003. To analyze the findings from these reviews, we summarized
the findings from each report and then grouped the findings using the
long- table method of analysis. From those groups, we identified four
broad grants management challenges. We next interviewed and obtained
documents from EPA officials to determine the actions they have taken in
the past to address these challenges. We also obtained the records of
1,232 in- depth grantee reviews conducted in 2002 and analyzed each one
using a data collection instrument. We conducted inter- rater reliability
tests to ensure that information from in- depth reviews was entered
consistently by each reviewer. In the event a document item was unclear,
reviewers flagged the case and consulted to determine the appropriate
response. We requested that EPA send documents for all evaluative reviews
conducted in

calendar year 2002. Since EPA does not have information on the exact
number of reviews it conducts each year, we were not able to verify
whether we received information on the total population of evaluative
reviews conducted in 2002. Because we could not verify the total number of
reviews conducted and because some regions and program offices conduct a
small number of reviews, we requested documents for all indepth evaluative
reviews in 2002 (rather than a sample) to ensure coverage of all 10
regions and 10 program offices.

To determine the extent to which EPA*s recently issued policies and grants
management plan address these challenges, we reviewed the new policies and
plan and interviewed EPA officials responsible for implementing key
aspects of the plan. We also attended EPA*s grants management training
course for project officers both before and after they revised their
project

officer training manual and EPA*s training course for nonprofit
organizations. We also observed several of EPA*s in- depth reviews of
grantees: one administrative on- site review conducted by EPA
headquarters, two on- site joint reviews conducted by EPA*s Region 1
office, and two joint desk reviews conducted by EPA*s Region 1 office. In
addition,

we interviewed officials in EPA*s Office of Human Resources and
Organizational Services to discuss the agency*s performance management
system.

To identify what promising practices, if any, could assist EPA in
addressing its grants management challenges, we interviewed and obtained
documentation from grants management officials, and other officials, in
the

federal government and from private sector organizations. For the federal
government, we met with officials from the five largest granting agencies:
the Departments of Agriculture, Education, Health and Human Services,
Housing, and Transportation. At the recommendation of these and other
officials, we interviewed other agency officials, including those from
Agriculture*s Rural Development, the Department of Commerce, Health

and Human Service*s Health Resources and Services Administration and
Substance Abuse and Mental Health Services Administration, and
Transportation*s Federal Transit Administration. We also interviewed
officials from the Council on Foundations, a membership organization, and
two foundations: the Robert Wood Johnson Foundation and the Ford
Foundation. We asked these officials for examples of their grant practices
that might address the types of challenges EPA faces in managing grants.
We also reviewed two GAO reports that describe effective training and
performance management systems for federal agencies.

We conducted our work from June 2002 through June 2003 in accordance with
generally accepted government auditing standards.

EPA*S 10 Regions and Regional Office

Appendi x II

Locations Note: Region 2 also includes Puerto Rico and the Virgin Islands.
Region 9 also includes Guam, American Samoa, Trust Territories, and the
Commonwealth of the Northern Mariana Islands.

Results of Our Analysis of EPA*S 2002 Appendi x III

In- depth Reviews This appendix presents tables that provide detail on the
in- depth reviews that the Environmental Protection Agency (EPA) conducted
in 2002. Indepth reviews include both on- site reviews conducted at the
grantee*s location and off- site reviews conducted at an EPA office or
another location. Tables 5 and 6 show the number of different types of
reviews and how many reviews each EPA office conducted. Tables 7 to 12
show the number and types of problems EPA found and how many problems were
found for each type of grantee and size of grant. Table 13 shows the
number of times EPA required or recommended action to correct a problem.
Tables 14 and 15 show how many reviews EPA conducted with or without a

protocol. Tables 16 and 17 show how many and what types of significant
actions EPA took against grantees. Analysis of Number

and Types of EPA*s In- depth Reviews

Tabl e 5: Number of Reviews by Type of Reviews Percentage of Type of
review Off- site review On- site review Tot al total

Administrative 100 89 189 15 Programmatic 526 491 1,017 83 Joint 18 8 26 2

Tot al 644 588 1,232 100

Source: GAO*s analysis of EPA*s in- depth reviews.

Tabl e 6: Number of Off- site and On- site Reviews and Number of Reviews
EPA Indicated It Performed, by Program and Regional Office Off- site
review On- site review Program/ regional office GAO EPA GAO EPA

Office of Grants and Debarment 42 45 30 43 Office of the Administrator 43
66 9 10 Office of Air and Radiation 6 6 17 23 Office of Enforcement and
Compliance Assurance 1 0 2 6 Office of Environmental Information 2 2 0 1
Office of International Affairs 0 0 3 3 Office of Prevention, Pesticides
and Toxic Substances 5 6 3 2 Office of Research and Development 72 137 43
54 Office of Solid Waste and Emergency Response 5 0 2 7 Office of Water 22
17 24 20 Region 1 11 13 16 18 Region 2 3 6 12 13 Region 3 34 62 55 35
Region 4 107 137 104 107 Region 5 12 0 28 62 Region 6 63 64 59 60 Region 7
26 32 36 33 Region 8 21 20 54 25 Region 9 145 3 57 23 Region 10 24 13 34
34

Tot al 644 629 588 579

Source: GAO*s analysis of EPA*s in- depth reviews. Note: The columns with
a GAO heading represent the review totals we found in the documents EPA
provided. The columns with an EPA heading represent the review totals EPA
officials reported to Congress.

Analysis of Problems Identified in EPA*s In- depth Reviews

Tabl e 7: Number of Problems Identified by Type of Review Administrative
Programmatic Joint Problem (N= 189) (N= 1,017) (N= 26) Tot al

Accounting 37 26 1 64

Administrative 16 23 1 40

Approval of modifications 2 4 0 6

Conflict of interest 5 0 0 5

Cost sharing 10 4 0 14

Financial expenditures 20 35 0 55

Indirect costs 12 2 5 19

Internal controls 13 1 0 14

Lobbying 0 0 0 0

Missing required audit 4 2 2 8

Personnel/ payroll 51 92 5 148

Procurement 70 41 12 123

Program income 1 1 0 2

Progress reports 13 167 4 184

Property management 12 0 0 12

Quality assurance 0 71 1 72

Subagreements 1 9 0 10

Technical issues 20 308 6 334

Terms and conditions of work 3 30 2 35

Tr ave l 16 3 0 19

Written procedures 73 11 2 86 Tot al 379 830 41 1,250

Source: GAO*s analysis of EPA*s in- depth reviews.

Table 8: Number of Problems by Type of Grantee Native American Local
Nonprofits States

tribes government

Universities Other Problem (N= 244) (N= 338)

(N= 327) (N= 126) (N= 160) (N= 37) Total

Accounting 17 17 27 2 1 0 64

Administrative 12 13 14 0 0 1 40

Approval of modifications 1 3 0 1 1 0 6

Conflict of interest 4 0 1 0 0 0 5

Cost sharing 1 3 7 2 1 0 14

Financial expenditures 13 18 20 2 2 0 55

Indirect costs 7 0 12 0 0 0 19

Internal controls 10 1 3 0 0 0 14

Lobbying 0 0 0 0 0 0 0

Missing required audit 2 2 4 0 0 0 8

Personnel/ payroll 27 43 58 11 7 2 148

Procurement 37 18 39 19 7 3 123

Program income 1 1 0 0 0 0 2

Progress reports 28 26 88 24 14 4 184

Property management 2 0 9 0 1 0 12

Quality assurance 5 16 18 26 6 1 72

Subagreements 1 8 0 1 0 0 10

Technical issues 48 117 119 33 12 5 334

Terms and conditions of work 8 16 8 1 2 0 35

Travel 4 1 14 0 0 0 19

Written procedures 40 8 27 6 5 0 86 Total 268 311 468 128 59 16 1,250

Source: GAO*s analysis of EPA*s in- depth reviews.

Tabl e 9: Number of Problems by Dollar Amount of Grants

Dollars in thousands

Dollar amount not identified a

1- 99 100- 299

300- 999 1,000 + Problem

(N= 537) (N= 222) (N= 200) (N= 128) (N= 145) Tot al

Accounting $30 $7 $5 $4 $18 $64

Administrative 19 5 3 5 8 40

Approval of modifications 3 0 1 0 2 6

Conflict of interest 1 0 1 1 2 5

Cost sharing 10 1 1 1 1 14

Financial expenditures 28 8 4 1 14 55

Indirect costs 10 0 1 3 5 19

Internal controls 4 0 3 2 5 14

Lobbying 0 0 0 0 0 0

Missing required audit 4 0 1 0 3 8

Personnel/ payroll 68 13 25 18 24 148

Procurement 53 12 25 16 17 123

Program income 0 0 1 1 0 2

Progress reports 80 39 34 16 15 184

Property management 3 0 2 3 4 12

Quality assurance 39 8 17 4 4 72

Subagreements 4 0 3 0 3 10

Technical issues 139 60 68 23 44 334

Terms and conditions of work 21 4 1 2 7 35

Tr avel 9 3 2 2 3 19

Written procedures 41 7 16 10 12 86 Tot al $566 $167 $214 $112 $191 $1,
250

Source: GAO*s analysis of EPA*s in- depth reviews. a We could not identify
the value of the grant award for these reviews.

Tabl e 10: Number of Problems by Type of Review Off- site On- site
Administrative

Programmatic Joint

Administrative Programmatic

Joint Problem (N= 100) (N= 526) (N= 18) (N= 89) (N= 491) (N= 8) Tot al

Accounting 2 9 1 35 17 0 64

Administrative 6 9 0 10 14 1 40

Approval of modifications 1 1 0 1 3 0 6

Conflict of interest 0 0 0 5 0 0 5

Cost sharing 1 3 0 9 1 0 14

Financial expenditures 2 17 0 18 18 0 55

Indirect costs 2 0 3 10 2 2 19

Internal controls 3 0 0 10 1 0 14

Lobbying 0 0 0 0 0 0 0

Missing required audit 1 0 2 3 2 0 8

Personnel/ payroll 11 39 3 40 53 2 148

Procurement 26 14 8 44 27 4 123

Program income 1 0 0 0 1 0 2

Progress reports 3 88 3 10 79 1 184

Property management 2 0 0 10 0 0 12

Quality assurance 0 48 1 0 23 0 72

Subagreements 0 4 0 1 5 0 10

Technical issues 5 165 5 15 143 1 334

Terms and conditions of work 0 19 1 3 11 1 35

Tr avel 0 0 0 16 3 0 19

Written procedures 27 5 2 46 6 0 86 Tot al 93 421 29 286 409 12 1,250

Source: GAO*s analysis of EPA*s in- depth reviews.

Tabl e 11: Number of Problems Per In- depth Review Number of problems

Number of reviews Percentage Identified (N= 1, 232) of reviews

0 623 51 1 286 23 2 166 13 3 83 7 4 36 3 5 17 1 6 9 1 7 4 0 8 4 0 9 1 0 10
2 0 11 1 0

Tot al 1,232 100

Source: GAO*s analysis of EPA*s in- depth reviews. Note: Percentage does
not add up to 100 due to rounding.

Tabl e 12: Description of Problems Identified in EPA*s Reviews Problem
Description of problems included in EPA*s in- depth reviews

Accounting Any failure of a grantee*s financial management system or
shortcomings in the procedures it used to ensure the proper accounting of
federal funds. For example, EPA found cases in which a grantee:  could
not compare budgeted amounts with actual expenditures,  did not properly
reconcile report balances to the general ledger, or  did not separately
track funds for different grants. Administrative Cases in which a
grantee*s record keeping system was inadequate. Approval of modifications
Any instance in which a grantee had begun performing tasks outside its
original scope of work without

seeking prior approval from EPA. Conflict of interest Cases in which a
grantee, using grant funds, entered into a contract with a closely
affiliated organization. Cost sharing Cases in which a grantee failed to
appropriately track and document its cost- sharing expenditures. Financial
expenditures Cases in which the grantee did not sufficiently document
expenses to determine the eligibility of costs or charged ineligible costs
to the grant.

Indirect costs Cases in which a grantee did not have an approved indirect
cost rate, or indirect cost allocation plan. Internal controls Cases in
which a grantee did not adequately segregate financial responsibilities.
Lobbying No instances of lobbying problems. Missing required audit Cases
in which a grantee did not have its required audit performed or had not
submitted a copy of its audit

to EPA. Personnel/

Problems varied depending on the type of review conducted. Administrative
reviews included cases in payroll which a grantee did not track the amount
of time its employees spent on specific grant activities. Programmatic
reviews included cases in which grantees did not have sufficient staff
resources to perform the grant activities.

Procurement Cases in which grantees lacked documentation to support sole-
source contracts and did not report their efforts to encourage procurement
from minority- and woman- owned businesses. Program income In one case, a
grantee generated income through the use of grant funds but did not manage
the funds in accordance with the grant agreement, and in another the grant
agreement did not allow the grantee to

generate such income. Progress reports Instances in which a grantee*s
progress report was missing or late, or did not include all the necessary
information.

Property management Cases in which the grantee did not properly control
property, such as equipment. Quality assurance Instances in which a
grantee needed to revise its quality assurance plan. Quality assurance
plans are

required to ensure the quality of data collected during the grant work.
Subagreements Cases in which a grantee did not properly monitor
subgrantees, or when a subgrantee*s files were

incomplete. Technical issues Cases in which a grantee was behind in the
progress of its work. Terms and conditions Cases in which a grantee was
not meeting the terms and conditions of a grant agreement. Terms and
conditions vary depending on the grant agreement and in some cases overlap
with the other problem

categories. Tr avel Cases in which grantee lacked documentation to support
travel expenditures or did not obtain written

approval from the appropriate official prior to incurring travel expenses.
Written procedures Cases in which a grantee*s written policies or
procedures were either missing or inadequate. Source: GAO*s analysis of
EPA*s in- depth reviews.

Analysis of Corrective Actions Identified in EPA*s In- depth Reviews

Tabl e 13: Number of Problems and Corrective Actions Number of

Percentage of Number of corrective problems with Problem problems

actions corrective action

Accounting 64 51 80 Administrative 40 25 63 Approval of modifications 6 5
83

Conflict of interest 5 2 40 Cost sharing 14 10 71 Financial expenditures
55 35 64 Indirect costs 19 13 68 Internal controls 14 13 93 Lobbying 0 0 0
Missing required audit 8 7 88 Personnel/ payroll 148 65 44 Procurement 123
81 66 Program income 2 0 0 Progress reports 184 74 40 Property management
12 10 83 Quality assurance 72 18 25 Subagreements 10 8 80 Technical issues
334 124 37 Terms and conditions of work 35 14 40

Tr ave l 19 17 89 Written procedures 86 68 79

Tot al 1,250 640 51 Source: GAO*s analysis of EPA*s in- depth reviews.

Analysis of Protocols Used for EPA*s In- depth Reviews

Tabl e 14: Number of Mandated and Nonmandated Reviews Number Percentage

Mandated 339 28 Nonmandated 893 72

Tot al 1,232 100

Source: GAO*s analysis of EPA*s in- depth reviews. Note: Mandated reviews
were defined in the GAO analysis as evaluations conducted as a result of
statutory or periodic programmatic requirements.

Tabl e 15: Number of Nonmandated Reviews With or Without a Review
Protocol, by Program and Regional Office, and by Type of Review

Administrative Programmatic Joint Protocol

Unclear/ no Protocol

Unclear/ no Protocol Unclear/ no Program/ regional office used protocol
used protocol used protocol

Office of Grants and Debarment 35 37 0 0 0 0 Office of the Administrator 0
0 51 1 0 0 Office of Air and Radiation 0 0 22 1 0 0 Office of Enforcement
and

0 0 3 0 0 0 Compliance Assurance Office of Environmental Information 0 0 2
0 0 0

Office of International Affairs 0 0 0 3 0 0 Office of Prevention,
Pesticides and

0 0 8 0 0 0 Toxic Substances Office of Research and Development 0 0 68 47
0 0

Office of Solid Waste and Emergency 0 0 7 0 0 0 Response Office of Water 0
0 38 8 0 0

Region 1 8 3 1 1 3 3 Region 2 1 4 0 7 1 0 Region 3 16 0 17 11 0 1 Region 4
0 0 112 29 0 0 Region 5 8 4 0 11 0 0 Region 6 8 1 28 33 0 0 Region 7 12 1
27 7 7 0 Region 8 8 4 51 2 0 0 Region 9 2 32 12 27 1 1 Region 10 0 5 40 3
2 7

Tot al 98 91 487 191 14 12

Source: GAO*s analysis of EPA*s in- depth reviews.

Analysis of Significant Actions Taken Against Grantees as a Result of In-
depth Reviews

Table 16: Number of Significant Actions Taken Against Grantees by Type of
Grantee Native American

Local Nonprofits States

tribes government

Universities Other Action taken (N= 244) (N= 338)

N= 327) (N= 126) (N= 160) (N= 37) Total

Agreement termination 1 0 0 0 0 0 1

Annulment 0 0 0 0 0 0 0

Disallowance of costs 5 1 1 0 0 0 7

Grantee debarment 0 0 0 0 0 0 0

High risk designation/ special condition 3 0 1 0 0 0 4

Payment hold 4 4 7 0 0 0 15

Referral for investigation 2 0 2 0 0 0 4

Stop work orders 3 0 2 0 0 0 5

Threat to take action 3 0 6 0 0 0 9

Other 0 0 2 0 0 0 2 Total 21 5 21 0 0 0 47

Source: GAO*s analysis of EPA*s in- depth reviews.

Table 17: Number of Significant Actions Per In- depth Review Number of
significant actions Number of reviews Percentage of reviews 0 1,202 98 1
20 2 2 4 0 3 5 0 4 1 0

Total 1,232 100

Source: GAO*s analysis of EPA*s in- depth reviews.

Summary of EPA*s Grants Management Plan,

Appendi x IV

2003- 2008 Goal 1: Enhance the

Objectives:

Skills of EPA Personnel 1. 1 Update and enhance training materials and
course curricula to Involved in Grants

emphasize high priority areas. Management

1.2 Improve delivery and availability of training programs. 1.3 Provide
training to managers and supervisors. Table 18 shows the performance
measures for goal 1.

Tabl e 18: Goal 1 Performance Measures Performance measure Baseline Target

Percentage of grants managed by certified project officers 85% 2003: 100%

Percentage of grants managed by project 2004: 30% officers who have taken
enhanced project

2005: 60% officer refresher course 0%

2006: 100% Source: EPA.

Action steps:

2003:  Issue Project Officer Training Manual (fifth edition), which will
focus

on core competencies needed to manage grants.  Conduct project officer
training with special emphasis on the core

competency areas that were enhanced in the fifth edition of the training
manual.

 Issue first edition of the Grants Specialist Training Manual.

 Conduct grants specialist training focusing on core competency areas. 
Develop long- term Grants Management Training Plan.  Issue guidance on
the proper use of amendments.

 Ensure that project officers receive basic and refresher grants
management training on a timely basis (ongoing).  Develop a pilot grants
management training program for managers and

supervisors. 2004:  Enhance and update on- line reference materials for
grant specialists in

core competencies.  Implement on- line training for grants specialists in
core competency

areas.  Enhance on- line training for refresher project officers course
to include

additional materials on preapplication review, competition, post- award
monitoring, environmental outcomes, and other new areas covered in the
Project Officer Training Manual, as well as verification of completion at
each step.

 Conduct grants management classroom training for managers and
supervisors.

2005:  Implement on- line basic project officers course, including
verification of

completion at each step.  Conduct grants management on- line training for
managers and

supervisors. 2006:  Establish certification program for grants
specialists based on an

examination of skills and core competencies required to manage grants.
Goal 2: Promote

Objectives: Competition in the 2.1 Improve the agency*s process for
identifying annual funding priorities Award of Grants

and planning for competition.

2. 2 Encourage a large and diverse group of grant applicants. 2.3 Promote
agencywide understanding of the importance of competition. 2.4 Provide
adequate support to the Grants Competition Advocate. Table 19 shows the
performance measures for goal 2.

Tabl e 19: Goal 2 Performance Measures Performance measure Baseline Target

Percentage of new grants subject to the 27% 2003: 30% competition order
that are competed 2004: 60%

2005: 85% Percentage of new grants to nonprofit

24% 2003: 30% recipients subject to the competition order

2004: 55% that are competed

2005: 75% Source: EPA.

Action steps:

2003:  Provide guidance to national program managers on Catalog of
Federal

Domestic Assistance (CFDA) descriptions that publicize funding priorities
and opportunities.

 Include national program managers funding priorities in the CFDA on an
annual basis to ensure they are well advertised and linked to the agency*s
Government Performance and Results Act goals.

 Develop guidelines and make postcompetition award information available
to the public through the Internet.

 Develop a central grants competition Web page linked to program office
Web sites to make grant solicitations easily available to the public.

 Post all grant solicitations on the Fed Biz Opps Web site.  Develop
standard agencywide grants competition training materials and

incorporate them into project officer and grant specialist training.

 Provide full staffing for the Grants Competition Advocate. 2004: 
Develop a structured annual planning process for grant competitions. 
Expand public awareness of funding opportunities by improving the

accuracy and specificity of program descriptions in the CFDA.  Research
innovative grants competition strategies performed by federal, state, or
local governments and determine suitability for adapting them for the
agency*s use. 2005:  Implement a structured annual planning process for
grant competitions.

Goal 3: Leverage

Objectives:

Technology to Improve 3.1 Continue deployment of the Integrated Grant
Management System

Program Performance (IGMS).

3.2 Integrate IGMS with federal- wide e- grant initiatives. 3. 3 Enhance
and expand information systems that support grants oversight. Table 20
shows performance measures for goal 3.

Table 20: Goal 3 Performance Measures Performance measure Baseline Target

Percentage of grants funding Regions 53% Regions 2003: 65% packages that
are submitted 2004: 85% electronically

HQ 0% HQ 2006: 25% Average number of days to process

60 days Regions 2003: 57 days a grant 2004: 51 days HQ To be established

Percentage of award transactions Regions 0% Regions 2004: 100% transmitted
electronically into the HQ 100% Integrated Financial Management System
(IFMS)

Number of electronic applications Target to be established received in
IGMS from e- grants portal

Source: EPA.

Action steps:

2003:  Complete deployment of IGMS in Regions 2, 4, 5, and 8 and of pilot
in headquarters component.

 Develop an interface between IGMS and the IFMS so that commitment notice
information need be entered only once.

 Complete analysis of changes required to IGMS and Grants Information
Control System to enable these systems to interface with federal e- grants
portal.

 Develop and implement an IGMS postaward module.  Correct
inconsistencies and inaccuracies in the naming of grantees,

grant specialists, and project officers so that accurate reports of
workload and recipients counts are available.

 Enhance the IGMS electronic grant file reporting views for the reports
needed to monitor grant specialist and project officer workloads.

 Expand the project officer database to track certification and
recertification activities.

 Establish a tracking system to identify competition status of grants.
2004:  Enable electronic transmission of award data from IGMS to the
IFMS.  Develop an interface to federal e- grants portal to enable IGMS to
accept

applications.  Develop a project officer interface to simplify use of
IGMS.  Ensure that agency information technology systems make grants

information readily available to EPA personnel.  Create an agency Web
site to allow grant specialists and project officers

to access best practices and other tools for postaward management. 2005: 
Expand e- grants portal to enable IGMS to accept reporting.  Modify IGMS
data elements to e- grants standards for reporting.  Develop an
administrative review checklist in IGMS to ensure that grant

packages are complete, comprehensive and in compliance with EPA orders and
policies.

2006:  Complete deployment of IGMS in headquarters.

Goal 4: Strengthen

Objectives:

EPA Oversight of 4. 1 Improve grants management reviews of EPA offices.

Grants 4.2 Improve and expand external reviews of grant recipients.

4.3 Develop approaches to prevent or limit grants management weaknesses.
4.4 Establish clear lines of accountability for grants oversight. 4.5
Provide high- level coordination, planning, and priority setting for
grants management.

Table 21 shows the performance measures for goal 4.

Tabl e 21: Goal 4 Performance Measures Performance measure Baseline Target

Percentage of grants awarded in the fourth quarter of the agency*s fiscal
year 67% 2004: 57%

Number of comprehensive internal reviews Regions 2003: 3 of EPA grants
management operations 0 HQ 2003: 4

Percentage of active recipients who receive advanced monitoring 5% 2003:
10% Percentage of active recipients who have on- site reviews conducted by
program

2004: 5% increase office and/ or grants management offices Establish in
2003 over baseline Percentage of offices that submit postaward

monitoring plans on time 60% 2003: 100% Percentage of eligible grants
closed out FY 2001: 89%

FY 2001: 99% FY 2002: 51% FY 2002: 90%

Source: EPA.

Action steps:

2003:  Combine the management oversight and postaward validation grants

management review protocols of offices.  Conduct cradle- to- grave
management reviews of headquarters and

regional offices starting in 2003.  Issue revised guidance on grants
management self- assessments.

 Require all offices to conduct baseline monitoring of all active
grantees (ongoing).

 Increase the level of advanced monitoring of grantees by offices,
including desk reviews and on- site reviews, to a minimum of 10 percent
annually of active recipients.

 Ensure that all offices record their activities in the agency*s
compliance database including documentation of the results of compliance
reviews.

 Ensure that all offices submit timely annual postaward monitoring plans.

 Ensure that all offices, on an annual basis, define and identify
categories of at- risk grantees requiring technical and/ or compliance
assistance.

 Complete development, with the EPA*s Office of Inspector General, of an
instructional video for nonprofit recipients on how to manage their
grants.

 Provide training courses for nonprofit recipients.  Develop grants
management tribal training manual and provide training to tribes. 
Develop guidance on required procurement processes under grants, including
competition, cost and price analyses, and avoidance of

conflicts of interest, and make available to grantees.  Clarify roles and
responsibilities of senior resource officials.  Issue grants policy
document defining roles and responsibilities of

grants management officers, program office officials, and project
officers.

 Establish senior- level Grants Management Council.  Update Grants
Management Office business plans in 2003 and annually

thereafter.

2004:  Establish a preaward financial systems review program for at- risk

grantees.  Establish grants clearinghouse/ hotline for grant recipients.
 Review and update EPA*s indirect cost rate policies.  Ensure that
performance standards established for grant specialists and

project officers adequately address grants management responsibilities. 
Develop workload analysis of project officers and grant specialists. 
Establish standard operation procedures and best practices for grants

management offices. 2005:  Provide guidance to grants management offices
and program offices on

how resources should be allocated to ensure effective and efficient grants
management oversight.

2006:  Develop grants management resource tool to replace the Assistance
Administration Manual.

Goal 5: Support

Objectives:

Identifying and 5. 1 Include expected environmental outcomes and
performance measures Achieving

in grant work plans. Environmental

5.2 Improve reporting on progress made in achieving environmental Outcomes
outcomes.

Table 22 shows the performance measures for goal 5.

Tabl e 22: Goal 5 Performance Measures Performance measure Baseline Target

Percentage of grant work plans, decision Establish in FY 2004: 70%
memoranda, and terms of condition that

2003 2005: 80% include a discussion of how grantees plan

2006: 100% to measure and report on environmental progress

Source: EPA.

Action steps:

2003:  Develop a tutorial for grantees on how to develop performance

measures for work plans.  Issue grants policy guidance to ensure that all
grant work plans,

decision memoranda, and/ or terms of condition include environmental
outcomes and how to measure them. 2004:  Require a discussion of expected
environmental outcomes and

performance measures in grant solicitations. 2005:  Establish reporting
on environmental outcomes as a criterion for

approval of interim and final reports.  Incorporate success in reporting
on outcomes into the criteria for

awarding new grants.

Comments from the Environmental

Appendi x V Protection Agency

Appendi x VI

GAO Contacts and Staff Acknowledgments GAO Contacts Andrea Wamstad Brown
(202) 512- 3319 Bruce Skud (617) 788- 0588 Staff

In addition to those named above, Carl Barden, Christopher Murray, Paul
Acknowledgments Schearf, Rebecca Shea, Carol Herrnstadt Shulman, Kelli Ann
Walther, and Amy Webbink made key contributions to this report.

Related GAO Products

Environmental Protection Agency: Problems Persist in Effectively Managing
Grants. GAO- 03- 628T. Washington, D. C.: June 11, 2003.

Federal Assistance: Grant System Continues to Be Highly Fragmented. GAO-
03- 718T. Washington, D. C.: April 29, 2003.

Results- Oriented Cultures: Creating a Clear Linkage between Individual
Performance and Organizational Success. GAO- 03- 488. Washington, D. C.:
March 14, 2003.

Major Management Challenges and Risks: Environmental Protection Agency.
GAO- 03- 112. Washington D. C.: January 2003.

Results- Oriented Cultures: Using Balanced Expectations to Manage Senior
Executive Performance. GAO- 02- 966. Washington, D. C.: September 27,
2002.

Environmental Protection: Grants Awarded for Continuing Environmental
Programs and Projects. GAO- 01- 860R. Washington, D. C.: June 29, 2001.

Environmental Protection: EPA*s Oversight of Nonprofit Grantees* Costs Is
Limited. GAO- 01- 366. Washington, D. C.: April 6, 2001.

Environmental Protection: Information on EPA Project Grants and Use of
Waiver Authority. GAO- 01- 359. Washington, D. C.: March 9, 2001.

Environmental Research: STAR Grants Focus on Agency Priorities, but
Management Enhancements Are Possible. GAO/ RCED- 00- 170. Washington, D.
C.: September 11, 2000.

Environmental Protection: Grants for International Activities and Smart
Growth. GAO/ RCED- 00- 145R. Washington, D. C.: May 31, 2000.

Environmental Protection: Collaborative EPA- State Effort Needed to
Improve Performance Partnership System. GAO/ T- RCED- 00- 163. Washington,
D. C.: May 2, 2000.

Managing for Results: EPA Faces Challenges in Developing ResultsOriented
Performance Goals and Measures. GAO/ RCED- 00- 77. Washington, D. C.:
April 28, 2000.

Environmental Protection: Collaborative EPA- State Effort Needed to
Improve New Performance Partnership System. GAO/ RCED- 99- 171.
Washington, D. C.: June 21, 1999.

Environmental Protection: Factors Contributing to Lengthy Award Times for
EPA Grants. GAO/ RCED- 99- 204. Washington, D. C.: July 14, 1999.

Environmental Protection: EPA*s Progress in Closing Completed Grants and
Contracts. GAO/ RCED- 99- 27. Washington, D. C.: November 20, 1998.

Dollar Amounts of EPA*s Grants and Agreements. GAO/ RCED- 96- 178R.
Washington D. C.: May 29, 1996.

(360223)

Report to Congressional Requesters

August 2003 GRANTS MANAGEMENT EPA Needs to Strengthen Efforts to Address
Persistent Challenges

GAO- 03- 846

Contents Letter 1

Results in Brief 2 Background 4 Four Key Grants Management Challenges
Persist despite EPA*s Past

Efforts to Address Them 9 New Policies and Plan Show Promise but Require
Strengthening,

Enhanced Accountability, and Sustained Commitment to Succeed 27 Promising
Practices Could Assist EPA in Addressing Its Grants

Management Challenges 35 Conclusions 39 Recommendations for Executive
Action 40 Agency Comments and Our Evaluation 41

Appendixes

Appendix I: Objectives, Scope, and Methodology 43

Appendix II: EPA*S 10 Regions and Regional Office Locations 45

Appendix III: Results of Our Analysis of EPA*S 2002 In- Depth Reviews 46
Analysis of Number and Types of EPA*s In- depth Reviews 46 Analysis of
Problems Identified in EPA*s In- depth Reviews 48 Analysis of Corrective
Actions Identified in EPA*s In- depth Reviews 54

Analysis of Protocols Used for EPA*s In- depth Reviews 55 Analysis of
Significant Actions Taken Against Grantees as a Result of In- depth
Reviews 57

Appendix IV: Summary of EPA*s Grants Management Plan, 2003- 2008 58 Goal
1: Enhance the Skills of EPA Personnel Involved in Grants

Management 58 Goal 2: Promote Competition in the Award of Grants 59 Goal
3: Leverage Technology to Improve Program Performance 61 Goal 4:
Strengthen EPA Oversight of Grants 63 Goal 5: Support Identifying and
Achieving Environmental Outcomes 66

Appendix V: Comments from the Environmental Protection Agency 68

Appendix VI: GAO Contacts and Staff Acknowledgments 70 GAO Contacts 70
Staff Acknowledgments 70

Related GAO Products 71 Tables Table 1: Number of In- depth Reviews by
Type of Grantee and

Review, 2002 14 Table 2: Most Frequently Identified Problems, by Problem
Area for

Administrative Reviews, 2002 15 Table 3: Most Frequently Identified
Problems, by Problem Area for

Programmatic Reviews, 2002 16 Table 4: Description of Most Frequently
Identified Problems in

EPA*s In- depth Reviews 16 Table 5: Number of Reviews by Type of Reviews
46 Table 6: Number of Off- site and On- site Reviews and Number of Reviews
EPA Indicated It Performed, by Program and

Regional Office 47 Table 7: Number of Problems Identified by Type of
Review 48 Table 8: Number of Problems by Type of Grantee 49 Table 9:
Number of Problems by Dollar Amount of Grants 50 Table 10: Number of
Problems by Type of Review 51 Table 11: Number of Problems Per In- depth
Review 52 Table 12: Description of Problems Identified in EPA*s Reviews 53
Table 13: Number of Problems and Corrective Actions 54 Table 14: Number of
Mandated and Nonmandated Reviews 55 Table 15: Number of Nonmandated
Reviews With or Without a

Review Protocol, by Program and Regional Office, and by Type of Review 56
Table 16: Number of Significant Actions Taken Against Grantees by Type of
Grantee 57

Table 17: Number of Significant Actions Per In- depth Review 57 Table 18:
Goal 1 Performance Measures 58 Table 19: Goal 2 Performance Measures 60
Table 20: Goal 3 Performance Measures 62 Table 21: Goal 4 Performance
Measures 64 Table 22: Goal 5 Performance Measures 67

Figures Figure 1: EPA*s Key Offices Involved in Grant Activities 5 Figure
2: Percentage of EPA Grant Dollars Awarded by Recipient

Type, Fiscal Year 2002 7

Abbreviations

CFDA Catalog of Federal Domestic Assistance EPA Environmental Protection
Agency HHS Department of Health and Human Services IFMS Integrated
Financial Management System IGMS Integrated Grants Management System OMB
Office of Management and Budget PART Program Assessment Rating Tool SAMHSA
Substance Abuse and Mental Health Services Administration STAR Science To
Achieve Results

This is a work of the U. S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed in
its entirety without further permission from GAO. However, because this
work may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this material
separately.

a

GAO United States General Accounting Office

EPA continues to face four key management challenges, despite past efforts
to address them. These challenges are (1) selecting the most qualified
applicants, (2) effectively overseeing grantees, (3) measuring the results
of grants, and (4) effectively managing grants staff and resources. In
recent years, EPA has taken a series of actions to address these
challenges by, among other things, issuing policies, conducting training,
and developing a new data system for grants management. However, these
past actions had mixed results because of the complexity of the problems,
weaknesses in design and implementation, and insufficient management
emphasis.

EPA*s 2002 competition and oversight policies and 2003 grants management
plan focus on the major challenges GAO identified, but will require
strengthening, enhanced accountability, and a sustained commitment to
succeed. For example, EPA*s new oversight policy mandates more in- depth
monitoring of grantees but it does not build in a process for analyzing
the results of the in- depth monitoring to address systemic grantee
problems. Such analysis could better target EPA*s oversight efforts. In
addition, its 5- year grants management plan does offer, for the first
time, a comprehensive road map with objectives, goals, and milestones for
addressing grants management challenges. However, the plan does not
completely address how EPA will hold all managers and staff accountable
for successfully fulfilling their management responsibilities. Therefore,
EPA cannot ensure the sustained commitment required for the plan*s
success. Although information on promising grants management practices is
limited,

the federal agencies and other organizations GAO contacted identified some
practices for each of EPA*s four major challenges that may further assist
EPA in improving its grants management. For example, one federal agency
takes into account applicants* potential to achieve results when selecting
grantees. A private foundation conducts preaward reviews of some
applicants to learn about the organization and assess its fiscal health.
In addition, GAO has developed a guide for federal agencies to use to hold
managers and staff accountable for achieving desired agency results. This
guide could be useful in helping EPA ensure accountability for grants
management performance. The Environmental Protection

Agency (EPA) has long faced problems managing its grants, which constitute
over one- half of

the agency*s annual budget, or about $4 billion. EPA uses grants to
implement its programs to protect human health and the

environment and awards grants to thousands of recipients, including state
and local governments, tribes,

universities, and nonprofit organizations. EPA*s ability to efficiently
and effectively

accomplish its mission largely depends on how well it manages its grant
resources. As requested, GAO determined (1) major challenges EPA faces in
managing its grants and how it has addressed these challenges in the past,
(2) extent to which EPA*s recently

issued policies and grants management plan address these challenges, and
(3) promising practices, if any, that could assist EPA in addressing these

challenges. The Administrator of EPA should ensure that the agency (1)
meets goals, targets, and time frames in its grants management plan, and
(2) takes specific actions to strengthen its efforts and report annually
to Congress on its grant management accomplishments.

In commenting on a draft of this report, EPA stated that it agreed with
GAO*s recommendations and will implement them as part of its 5- year
grants management plan.

www. gao. gov/ cgi- bin/ getrpt? GAO- 03- 846. To view the full product,
including the scope and methodology, click on the link above. For more
information contact John B. Stephenson at (202) 512- 3841 or stephensonj@
gao. gov. Highlights of GAO- 03- 846, a report to

congressional requesters

August 2003

GRANTS MANAGEMENT

EPA Needs to Strengthen Efforts to Address Persistent Challenges

Page i GAO- 03- 846 Grants Management

Contents

Page ii GAO- 03- 846 Grants Management

Contents

Page iii GAO- 03- 846 Grants Management

Page 1 GAO- 03- 846 Grants Management United States General Accounting
Office Washington, D. C. 20548

Page 1 GAO- 03- 846 Grants Management

A

Page 2 GAO- 03- 846 Grants Management

Page 3 GAO- 03- 846 Grants Management

Page 4 GAO- 03- 846 Grants Management

Page 5 GAO- 03- 846 Grants Management

Page 6 GAO- 03- 846 Grants Management

Page 7 GAO- 03- 846 Grants Management

Page 8 GAO- 03- 846 Grants Management

Page 9 GAO- 03- 846 Grants Management

Page 10 GAO- 03- 846 Grants Management

Page 11 GAO- 03- 846 Grants Management

Page 12 GAO- 03- 846 Grants Management

Page 13 GAO- 03- 846 Grants Management

Page 14 GAO- 03- 846 Grants Management

Page 15 GAO- 03- 846 Grants Management

Page 16 GAO- 03- 846 Grants Management

Page 17 GAO- 03- 846 Grants Management

Page 18 GAO- 03- 846 Grants Management

Page 19 GAO- 03- 846 Grants Management

Page 20 GAO- 03- 846 Grants Management

Page 21 GAO- 03- 846 Grants Management

Page 22 GAO- 03- 846 Grants Management

Page 23 GAO- 03- 846 Grants Management

Page 24 GAO- 03- 846 Grants Management

Page 25 GAO- 03- 846 Grants Management

Page 26 GAO- 03- 846 Grants Management

Page 27 GAO- 03- 846 Grants Management

Page 28 GAO- 03- 846 Grants Management

Page 29 GAO- 03- 846 Grants Management

Page 30 GAO- 03- 846 Grants Management

Page 31 GAO- 03- 846 Grants Management

Page 32 GAO- 03- 846 Grants Management

Page 33 GAO- 03- 846 Grants Management

Page 34 GAO- 03- 846 Grants Management

Page 35 GAO- 03- 846 Grants Management

Page 36 GAO- 03- 846 Grants Management

Page 37 GAO- 03- 846 Grants Management

Page 38 GAO- 03- 846 Grants Management

Page 39 GAO- 03- 846 Grants Management

Page 40 GAO- 03- 846 Grants Management

Page 41 GAO- 03- 846 Grants Management

Page 42 GAO- 03- 846 Grants Management

Page 43 GAO- 03- 846 Grants Management

Appendix I

Appendix I Objectives, Scope, and Methodology

Page 44 GAO- 03- 846 Grants Management

Page 45 GAO- 03- 846 Grants Management

Appendix II

Page 46 GAO- 03- 846 Grants Management

Appendix III

Appendix III Results of Our Analysis of EPA*S 2002 Indepth Reviews

Page 47 GAO- 03- 846 Grants Management

Appendix III Results of Our Analysis of EPA*S 2002 Indepth Reviews

Page 48 GAO- 03- 846 Grants Management

Appendix III Results of Our Analysis of EPA*S 2002 Indepth Reviews

Page 49 GAO- 03- 846 Grants Management

Appendix III Results of Our Analysis of EPA*S 2002 Indepth Reviews

Page 50 GAO- 03- 846 Grants Management

Appendix III Results of Our Analysis of EPA*S 2002 Indepth Reviews

Page 51 GAO- 03- 846 Grants Management

Appendix III Results of Our Analysis of EPA*S 2002 Indepth Reviews

Page 52 GAO- 03- 846 Grants Management

Appendix III Results of Our Analysis of EPA*S 2002 Indepth Reviews

Page 53 GAO- 03- 846 Grants Management

Appendix III Results of Our Analysis of EPA*S 2002 Indepth Reviews

Page 54 GAO- 03- 846 Grants Management

Appendix III Results of Our Analysis of EPA*S 2002 Indepth Reviews

Page 55 GAO- 03- 846 Grants Management

Appendix III Results of Our Analysis of EPA*S 2002 Indepth Reviews

Page 56 GAO- 03- 846 Grants Management

Appendix III Results of Our Analysis of EPA*S 2002 Indepth Reviews

Page 57 GAO- 03- 846 Grants Management

Page 58 GAO- 03- 846 Grants Management

Appendix IV

Appendix IV Summary of EPA*s Grants Management Plan, 2003- 2008

Page 59 GAO- 03- 846 Grants Management

Appendix IV Summary of EPA*s Grants Management Plan, 2003- 2008

Page 60 GAO- 03- 846 Grants Management

Appendix IV Summary of EPA*s Grants Management Plan, 2003- 2008

Page 61 GAO- 03- 846 Grants Management

Appendix IV Summary of EPA*s Grants Management Plan, 2003- 2008

Page 62 GAO- 03- 846 Grants Management

Appendix IV Summary of EPA*s Grants Management Plan, 2003- 2008

Page 63 GAO- 03- 846 Grants Management

Appendix IV Summary of EPA*s Grants Management Plan, 2003- 2008

Page 64 GAO- 03- 846 Grants Management

Appendix IV Summary of EPA*s Grants Management Plan, 2003- 2008

Page 65 GAO- 03- 846 Grants Management

Appendix IV Summary of EPA*s Grants Management Plan, 2003- 2008

Page 66 GAO- 03- 846 Grants Management

Appendix IV Summary of EPA*s Grants Management Plan, 2003- 2008

Page 67 GAO- 03- 846 Grants Management

Page 68 GAO- 03- 846 Grants Management

Appendix V

Appendix V Comments from the Environmental Protection Agency Page 69 GAO-
03- 846 Grants Management

Page 70 GAO- 03- 846 Grants Management

Appendix VI

Page 71 GAO- 03- 846 Grants Management

Related GAO Products Page 72 GAO- 03- 846 Grants Management

GAO*s Mission The General Accounting Office, the audit, evaluation and
investigative arm of Congress, exists to support Congress in meeting its
constitutional responsibilities

and to help improve the performance and accountability of the federal
government for the American people. GAO examines the use of public funds;
evaluates federal programs and policies; and provides analyses,
recommendations, and other assistance to help Congress make informed
oversight, policy, and funding decisions. GAO*s commitment to good
government is reflected in its core values of accountability, integrity,
and reliability.

Obtaining Copies of GAO Reports and Testimony

The fastest and easiest way to obtain copies of GAO documents at no cost
is through the Internet. GAO*s Web site (www. gao. gov) contains abstracts
and fulltext files of current reports and testimony and an expanding
archive of older products. The Web site features a search engine to help
you locate documents using key words and phrases. You can print these
documents in their entirety, including charts and other graphics.

Each day, GAO issues a list of newly released reports, testimony, and
correspondence. GAO posts this list, known as *Today*s Reports,* on its
Web site daily. The list contains links to the full- text document files.
To have GAO e- mail this

list to you every afternoon, go to www. gao. gov and select *Subscribe to
e- mail alerts* under the *Order GAO Products* heading. Order by Mail or
Phone The first copy of each printed report is free. Additional copies are
$2 each. A check

or money order should be made out to the Superintendent of Documents. GAO
also accepts VISA and Mastercard. Orders for 100 or more copies mailed to
a single address are discounted 25 percent. Orders should be sent to:

U. S. General Accounting Office 441 G Street NW, Room LM Washington, D. C.
20548 To order by Phone: Voice: (202) 512- 6000 TDD: (202) 512- 2537 Fax:
(202) 512- 6061

To Report Fraud, Waste, and Abuse in Federal Programs

Contact: Web site: www. gao. gov/ fraudnet/ fraudnet. htm E- mail:
fraudnet@ gao. gov Automated answering system: (800) 424- 5454 or (202)
512- 7470

Public Affairs Jeff Nelligan, Managing Director, NelliganJ@ gao. gov (202)
512- 4800 U. S. General Accounting Office, 441 G Street NW, Room 7149
Washington, D. C. 20548

United States General Accounting Office Washington, D. C. 20548- 0001
Official Business Penalty for Private Use $300 Address Service Requested

Presorted Standard Postage & Fees Paid

GAO Permit No. GI00
*** End of document. ***