Juvenile Justice: Better Documentation of Discretionary Grant	 
Monitoring Is Needed (10-OCT-01, GAO-02-65).			 
								 
The Office of Juvenile Justice and Delinquency Prevention	 
(OJJDP)provides block grants and discretionary funding to help	 
states and communities prevent juvenile delinquency and improve  
their juvenile justice systems. OJJDP has specific program	 
monitoring and documentation requirements for its discretionary  
grants. These monitoring requirements include having the grant	 
manager make quarterly telephone calls, undertake on- and	 
off-site grant monitoring visits, and review interim and final	 
products. In a review of OJJDP's  most recent award of grants	 
active in fiscal years 1999 and 2000, GAO found that OJJDP's	 
grant monitoring activities were not consistently documented.	 
These findings are similar to those GAO reported in May 1996.	 
-------------------------Indexing Terms------------------------- 
REPORTNUM:   GAO-02-65						        
    ACCNO:   A02134						        
  TITLE:     Juvenile Justice: Better Documentation of Discretionary  
Grant Monitoring Is Needed					 
     DATE:   10/10/2001 
  SUBJECT:   Block grants					 
	     Discretionary grants				 
	     Grant monitoring					 
	     Juvenile delinquency				 
	     Records (documents)				 
	     Drug-Free Communities Support Program		 

******************************************************************
** This file contains an ASCII representation of the text of a  **
** GAO Testimony.                                               **
**                                                              **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced.  Tables are included, but    **
** may not resemble those in the printed version.               **
**                                                              **
** Please see the PDF (Portable Document Format) file, when     **
** available, for a complete electronic file of the printed     **
** document's contents.                                         **
**                                                              **
******************************************************************
GAO-02-65
     
The Honorable Bob Schaffer, House of Representatives

United States General Accounting Office

GAO

October 2001 JUVENILE JUSTICE Better Documentation of Discretionary Grant
Monitoring Is Needed

GAO- 02- 65

Page i GAO- 02- 65 Better Grant Monitoring Needed Letter 1

Results in Brief 2 Background 5 Scope and Methodology 5 Discretionary Grant
Monitoring Process 7 Grant Files Contained Limited Documentation That
Monitoring

Occurred 11 OJJDP Does Not Systematically Determine Staff Compliance With

Its Monitoring Requirements 18 Conclusions 19 Recommendations for Executive
Action 19 Agency Comments and Our Evaluation 20

Appendix I OJJDP?s Discretionary Grant Award Process 22 Program Plan Is to
Describe Goals and Priorities 25 Program Announcements and Application Kits
Are to Solicit

Applicants for Programs 25 Peer Review Panels Are to Help Determine Which
Grant

Applications Are Funded 25 Final Decision to Award Discretionary Grants 26

Appendix II Comments From the Department of Justice 28 GAO Comment 36

Appendix III GAO Contacts and Staff Acknowledgments 37 GAO Contacts 37
Acknowledgments 37

Figures

Figure 1: Key Steps in OJJDP?s Process for Planning and Announcing Grants 23
Figure 2: Overview of OJJDP Grant Application and Review

Process 24 Abbreviations OIG Office of the Inspector General OJJDP Office of
Juvenile Justice and Delinquency Prevention OJP Office of Justice Programs
Contents

Page 1 GAO- 02- 65 Better Grant Monitoring Needed

October 10, 2001 The Honorable Bob Schaffer House of Representatives

Dear Mr. Schaffer: The Office of Juvenile Justice and Delinquency Prevention
(OJJDP) was established to help states and communities prevent and control
juvenile delinquency and improve their juvenile justice systems. 1 OJJDP
provides funding to states, territories, localities, and private
organizations through block grants 2 and discretionary funding. 3 Between
September 1996 and September 2000, the number of active OJJDP discretionary
grants has more than tripled, from 240 to 790, and the related funding for
active grants almost doubled from $286 million to $555 million.

Monitoring these grant activities is a key management tool to help ensure
that funds are being properly spent. In May 1996, we testified that the
official grant files for discretionary grants generally contained plans for
agency monitoring, but little evidence that it occurred. 4

Your original request asked about various aspects of OJJDP?s grant
management. We will be responding to that request with two reports. This,
the first, addresses how well OJJDP monitors its grants to ensure that
grantees properly implement the programs that the grants support. The second
report will discuss OJJDP grantee reporting requirements, the

1 The Juvenile Justice and Delinquency Prevention Act of 1974, P. L. 93-
415, as amended, 42 U. S. C. 5601, et seq. It is one of five components of
the Department of Justice?s Office of Justice Programs (OJP); the other four
components are the National Institute of Justice, Bureau of Justice
Statistics, Bureau of Justice Assistance, and Office for Victims of Crime.

2 Block grants are awarded to the states to provide assistance to state and
local units of government for programs in accordance with legislative
requirements. 3 Discretionary grants are awarded to states, units of local
government, or private organizations at the discretion of the awarding
agency. Most discretionary awards are competitive in nature in that there
are limited funds available and a large number of potential recipients.

4 Testimony before the Subcommittee on Youth Violence, Committee on the
Judiciary, U. S. Senate, Juvenile Justice: Selected Issues Relating to
OJJDP?s Reauthorization

(GAO/ T- GGD- 96- 103, May 8, 1996).

United States General Accounting Office Washington, DC 20548

Page 2 GAO- 02- 65 Better Grant Monitoring Needed

number of juveniles OJJDP programs have directly served, and our analysis of
the program impact evaluations OJJDP has funded of its programs.

Specifically, we reviewed and are reporting on the (1) OJJDP grant
monitoring process, (2) extent to which OJJDP documented its discretionary
grant 5 monitoring activities, and (3) process it uses to oversee those
activities. In appendix I, we describe OJJDP?s award process for
discretionary grants.

To address these areas, we met with officials from OJJDP and OJP concerning
discretionary grant monitoring activities, including OJJDP?s activities to
oversee its monitoring. We reviewed OJP and OJJDP?s grant monitoring
requirements and selected and reviewed a representative sample of OJJDP
demonstration discretionary grants that were active in all of fiscal years
1999 and/ or 2000. We did this to determine whether OJJDP officials were
adhering to the agency?s discretionary grant monitoring requirements and how
the results of its monitoring efforts were documented in its grant files. We
also made a similar review of a representative sample of OJJDP training and
technical assistance grants. 6 To determine OJJDP?s oversight role of its
discretionary grant monitoring activities, we reviewed information such as
performance work plans. We also discussed with OJJDP officials the steps it
has taken to improve previously reported monitoring problems.

Our analysis did not address the quality of how OJJDP monitored its
grantees. For example, we reviewed documentation to determine whether OJJDP
made required site visits, but we could not review the quality of the site
visit contact.

OJJDP has specific program monitoring and documentation requirements for its
discretionary grants. These monitoring requirements include such activities
by the grant manager as making quarterly telephone calls, on and off- site
grant monitoring visits, and reviewing interim and final products. In
addition, all discretionary grantees are required to submit categorical

5 Includes cooperative agreements. 6 For both discretionary and training and
technical assistance grants, we express our confidence in the precision of
our sample results as a 95- percent confidence interval. Unless otherwise
noted, all percentage estimates have confidence intervals plus or minus 10
percentage points or less. Results in Brief

Page 3 GAO- 02- 65 Better Grant Monitoring Needed

assistance progress reports that summarize project activities and quarterly
financial status reports for review by grant managers. OJJDP also requires
certain documentation at the time of grant closeouts. At the time of our
review, OJJDP was reviewing its monitoring practices, including how to
improve its grant records.

Our review of OJJDP?s official grant files and grant managers? files for the
most recent award for a representative sample of grants active in all of
fiscal years 1999 and/ or 2000, showed that OJJDP was not consistently
documenting its grant monitoring activities. Our current observations are
similar to those we reported in May 1996 about the agency?s lack of
documentation of its monitoring activities.

 The grant managers? monitoring plans for the most recent awards for 79 of
89 demonstration grants and 34 of 45 training and technical assistance
grants had specific requirements for telephone contacts (e. g., monthly or
quarterly). There was no documentation showing that the requirement for
telephone contacts was met in 96 percent of the demonstration grants and in
any of the training and technical assistance grants.

 The grant managers? monitoring plans for the most recent awards for 66
demonstration grants and 20 training and technical assistance grants had
specific requirements for on- site visits. There was no documentation
showing that the requirement for site visits was met in 88 percent 7 of the
demonstration grants and 90 percent 8 of the training and technical
assistance grants.

 Progress and financial reports filed by the grantee are required for the
entire grant period. In 56 percent of the 89 demonstration grant files and
80 percent 9 of the 45 training and technical assistance grants files
reviewed, the progress reports did not cover the entire grant period. In 65
percent of the 89 demonstration grant files and 60 percent 10 of the 45
training and technical assistance grant files reviewed, the financial status
reports did not cover the entire grant period.

7 The estimated range is between 77 and 95 percent. 8 The estimated range is
between 68 and 99 percent. 9 The estimated range is between 69 and 88
percent. 10 The estimated range is between 48 and 71 percent.

Page 4 GAO- 02- 65 Better Grant Monitoring Needed

OJJDP does not systematically monitor grant managers? compliance with its
monitoring requirements or guidance, or assess the effectiveness of OJJDP
grant monitoring practices. According to OJJDP officials, one method of
overseeing compliance with the monitoring requirements is their review of
performance work plans. However, these plans referred, for example, to
monitoring grants to ensure that the projects meet their goals and
milestones, with no clear expectations regarding monitoring requirements,
such as site visits or telephone monitoring, or regarding documentation of
these contacts and reviews of grantees progress and financial reports.

We are making recommendations regarding OJJDP?s need to improve its
documentation of discretionary grant monitoring activities.

The Acting Assistant Attorney General provided us with written comments on a
draft of this report by letter dated September 19, 2001. In general, she
agreed with our findings and recommendations. However, she stated that we
had not adequately considered the fact that the associated cost of
monitoring has not been commensurate with OJJDP?s available budgetary
resources. She also commented that we had not adequately considered OJJDP?s
recent efforts in a major OJP- wide effort to improve grant monitoring
practices and establish uniform policies.

While we are encouraged by OJJDP?s stated dedication to improving its grant
monitoring practices, we believe the results of our review are sufficient to
conclude that OJJDP?s discretionary grant monitoring is not effective. We do
recognize in the report, the rapid growth in discretionary grants over the
past 5 years. However, we believe that monitoring is a critical management
control that allows OJJDP to know that federal dollars have been
appropriately spent. This conclusion is supported by the Department of
Justice?s Office of the Inspector General (OIG). In a December 2000, report
to the Congress, 11 the Inspector General cited grant management as one of
the 10 most serious management challenges facing Justice. The Inspector
General stated that Justice?s multi- billion dollar grant programs are a
high risk for fraud given the amount of money involved and the tens of
thousands of grantees. Among other things, the Inspector General said that
past OIG reviews have determined that many grantees did not submit the
required program monitoring and financial reports and that program
officials? on- site monitoring reviews did not consistently address all
grant conditions. The full text of the Acting

11 Management Challenges in the Department of Justice (Dec. 1, 2000).

Page 5 GAO- 02- 65 Better Grant Monitoring Needed

Assistant Attorney General?s comments and our evaluation of them are
presented in appendix II and elsewhere in this report, as appropriate.

OJJDP, within the U. S. Department of Justice, is the primary federal agency
responsible for the prevention and control of juvenile delinquency in the
United States. This includes preventing, treating and controlling youth gang
activity and violence through sponsored research, evaluation, and
demonstrating new approaches for communities across the country. The
initiatives are carried out by seven components within OJJDP: Research and
Program Development Division, Training and Technical Assistance Division,
Special Emphasis Division, State Relations and Assistance Division,
Information Dissemination Unit, Concentration of Federal Efforts Program,
and Child Protection Division. OJJDP annual appropriations have more than
tripled from $162 million in fiscal year 1996, with 71 authorized staff
positions, to $568 million in fiscal year 2000, with 87 authorized staff
positions.

In 1996, we reviewed the operations of OJJDP. We found that official grant
files for discretionary grants generally contained monitoring plans, but
little evidence that monitoring occurred. We reported that none of the grant
files had documentation of telephone contacts, site visits, or product
reviews. Also, no quarterly program and financial reports were included in
11 of the 78 files for which projects had been ongoing for at least 2
quarters at the time of our review, and one or more reports were missing
from another 61 files. In addition, only 6 of the 78 files had program and
financial reports for all quarters of work completed.

In commenting on our testimony, the OJJDP Deputy Administrator said that the
heavy workload of OJJDP staff may have resulted in a lack of monitoring
records and that, as a result of the our 1996 review, OJJDP would take the
steps necessary to improve records.

To address the first objective, to report on OJJDP?s grant monitoring
process, we reviewed applicable federal laws and regulations as well as OJP
and OJJDP policies and procedures. We also interviewed staff in OJJDP and
OJP?s Office of the Comptroller.

To accomplish the second objective, to report on OJJDP?s documentation of
its discretionary grant monitoring activities, we reviewed probability
samples of official grant files and grant managers? files using a data
collection instrument to record whether required monitoring activity
occurred- that is, that the progress reports, financial reports, and other
Background

Scope and Methodology

Page 6 GAO- 02- 65 Better Grant Monitoring Needed

required documents were included in the files. For each grant, we also
reviewed the most recent award covering 12 months to examine specific
monitoring requirements and activities, such as telephone calls and site
visits. Specifically, we reviewed a random sample of 89 of 545 OJJDP
demonstration discretionary grants, and 45 of 83 training and technical
assistance discretionary grants that were active in all of fiscal years 1999
and/ or 2000. The results of these samples are representative of the
populations from which they were drawn. We express our confidence in the
precision of our sample results as a 95- percent confidence interval. Unless
otherwise noted, all percentage estimates have confidence intervals plus or
minus 10 percentage points or less.

To determine the timeliness of grantee progress reports and financial status
reports, we reviewed all such reports for OJJDP demonstration and training
and technical assistance discretionary grants from the initiation of the
grant through the date of our review.

For our review, we provided a list of the selected grants to OJJDP and OJP?s
Comptroller. Because we did not always receive the grant files in the order
requested, we reviewed them out of order. When files were missing or lost,
we substituted replacement files. After our review was complete, some of the
missing files were located. We did not review the missing files but used the
substitute files that were a part of our random sample. Four grant files
were not found in time to be included in our review. 12 In addition, when
only one of the two files (the official grant file or the grant managers?
file) was available, we used the available file to complete the data
collection instrument. We have no reason to believe that the substitution of
files affected the validity of our sample because they were a part of our
random sample.

To determine whether OJJDP grant closeout procedures were implemented in
accordance with OJP policy, we reviewed grant files in our sample whose
project periods expired before August 31, 2000. For our review, we focused
on required closeout documentation, such as final progress or performance
reports, final financial status reports, grant

12 In commenting on a draft of this report, the Acting Assistant Attorney
General stated reasons for the four official grant files not being
accessible during our review. She said one file was not provided due to an
incorrect grant number on our list provided to the Comptroller?s Control
Desk when the official grant files were requested. However, the incorrect
grant number was contained in the list of grants that OJJDP provided to us.
One of the remaining grant files had been sent to storage. OJP staff had
checked out a third, and the last was missing. This last file is being
reconstructed according to the Acting Assistant Attorney General.

Page 7 GAO- 02- 65 Better Grant Monitoring Needed

checklist, and closeout notification and grant adjustment notice. Our
results should not be projected to the universe of cases in our sample.

To address the third audit objective, to report on the processes OJJDP uses
to oversee monitoring activities, we examined OJJDP?s monitoring and
oversight of its grant managers? compliance with applicable laws,
regulations, and procedures for monitoring. We obtained information from
OJJDP officials at headquarters and obtained data related to these
activities, such as performance work plans. To determine the requirements in
the performance work plans, OJJDP officials gave us a plan that they
considered to be representative.

Our work was done between March 2001 and September 2001 in accordance with
generally accepted government auditing standards.

After OJJDP awards discretionary grants, OJP policies require OJJDP to
monitor the grants and related activities and document the monitoring
results in the grant managers? program files and OJP?s Office of the
Comptroller official grant files. The monitoring is to ensure

 compliance with relevant statutes, regulations, policies, and guidelines;

 responsible oversight of awarded funds;

 implementation of required programs, goals, objectives, tasks, products,
time lines, and schedules;

 identification of issues and problems that may impede grant
implementation; and

 adjustments required by the grantee as approved by OJJDP. 13 One of the
OJJDP grant managers? main responsibilities is project monitoring. Each
grant manager prepares a monitoring plan as part of a grant manager?s
memorandum recommending initial or continuation funding. The level of
monitoring required is based upon the stated monitoring plan in the grant
manager?s memorandum. The memorandum includes

 an overview of the project;

 a detailed description of what type of activities the grantee plans on
implementing;

13 OJP, OJP Handbook: Policies and Procedures for the Administration of OJP
Grants

(OJP HB 4500.2C, Feb. 1992), p. 93. Discretionary Grant

Monitoring Process

Page 8 GAO- 02- 65 Better Grant Monitoring Needed

 a discussion of past monitoring activities and assessments, if the grant
is a multiyear grant that is awarded yearly;

 an identification of monitoring activities to be performed for the current
project period; and

 a discussion of the financial justification for the grant funds and of the
cost- effectiveness evaluation of the application.

Another part of the monitoring plan may list specific activities or
milestones to be monitored by the grant manager.

The 1991 OJJDP Monitoring Desk Book contains specific monitoring procedures
that grant managers used for monitoring OJJDP?s discretionary grants.
Monitoring activities documented included site visit reports and quarterly
monitoring telephone calls as well as other significant telephone calls.

Also, according to the OJP 1992 Handbook, all discretionary grantees are
required to submit categorical assistance progress reports that summarize
project activities and quarterly financial status reports for review by
grant managers. The OJP Handbook also stated that when all of the grant
funds are spent, all grant activities are completed, and all grantee and
grant managers? administrative requirements, including all paperwork and
review of grant files, are completed, the grant managers have to close out
the grant. (See app. I for a detailed discussion of the grant award
processes.)

The OJP Comptroller?s Control Desk is to maintain the official grant files
and is responsible for tracking the receipt of all grant documents. The
Control Desk receives the progress reports, files the original in the
official grant file, and forwards a copy to the cognizant program office.
The grant manager is responsible for the timely acceptance, review, and
analysis of progress reports.

The Comptroller?s Monitoring Division performs an in- house review of
various program and financial documents contained in the official grant
files. The Office of the Comptroller also performs risk- based on- site
financial reviews of grantee organizations to monitor administrative and
financial capability. The Comptroller applies risk- based criteria to a
universe of grants to develop a sample for each fiscal year monitoring plan.
The risk- based criteria include the dollar amount of the grant, new
grantees, new grant programs, and programs with known problems. The
Comptroller excludes from its sample any grantee organizations that were
subject to its other on- site financial monitoring or to audit by the Office
of the Inspector General during the last 2 years. According to OJP
officials,

Page 9 GAO- 02- 65 Better Grant Monitoring Needed

the purpose of the reviews is to provide as much of a comprehensive
financial review as possible without going on site. 14 Our review of the
results of the Comptroller?s in- house financial reviews of the grants in
our sample disclosed that they identified a number of problems similar to
our own observations. These included missing program and financial documents
and program and financial monitoring not always being documented in a timely
manner. 15

In commenting on a draft of this report, the Acting Assistant Attorney
General stated that there was no mention of delinquent documentation of
financial monitoring during our review, nor was there any evidence cited in
our draft report to support the contention that financial monitoring was not
documented in a timely manner. However, contrary to the Acting Assistant
Attorney General comment, our review found that there was no documentation
that financial reports were generally reviewed and that many of the reports
were not submitted in a timely manner. Specifically, we found one case where
no progress or financial reports had been submitted during the life of a
grant, and it was not discovered until almost a full year after the award
date (see p. 15).

In 1991, OJJDP issued a Monitoring Desk Book, which incorporated material
from OJP?s 1987 Handbook, including a formula for determining the monitoring
level requirements for the grant. According to OJJDP officials, this formula
is applied to determine how often a grantee will be visited and officially
contacted for monitoring purposes. This formula applies a weight to issues
such as program priority (i. e., how important is the grant within the
context of the OJJDP?s goals and objectives), size of grant, grantee?s need
for assistance, and anticipated difficulties in implementing the approach.
The grant manager assigns a weight or ?score?

to each of these criteria, which are added together for a total score. This
14 In October 2000, the Comptroller eliminated its review of progress
reports because Comptroller officials believed that it was a duplication of
efforts between the program offices and the Comptroller?s Office, and it
wanted to streamline closure of outstanding reviews.

15 The Monitoring Division made 939 in- house financial reviews in fiscal
year 1999, of which one OJJDP demonstration grant and five training and
technical assistance grants were included in our review. In fiscal year
2000, the division made 1,248 financial reviews, of which three OJJDP
demonstration grants and one training and technical assistance grant were
included in our review. As of August 2001, the financial reviews included
eight OJJDP demonstration grants and three training and technical assistance
grants that were included in our review. Outdated Monitoring

Policies and Procedures Were Being Used

Page 10 GAO- 02- 65 Better Grant Monitoring Needed

score assigns the grant to a category that defines the specific number of
on- site and off- site visits and the frequency of monitoring calls.

In February 1992, OJP revised its 1987 Handbook and eliminated the score
requirement to determine the monitoring level of a project. However, OJJDP
grant managers still use the instructions to determine the monitoring level
of a project. We found in our file reviews that 72 of the 89 demonstration
grant files and 14 of the 45 training and technical assistance grant files
had a score or level indicating monitoring requirements. Using the obsolete
monitoring requirement to establish grant manager?s grant monitoring
responsibilities may not result in the most efficient use of their time or
in the needed monitoring to ensure grantee compliance, as pointed out by the
Deputy Administrator in May 1996. He said that site visits did not occur
because they are expensive and time- consuming. He added that OJJDP might
need to revise some procedures, noting that a site visit once every 2 years
and some interim telephone monitoring may be more appropriate than annual
visits. Current OJP policy provides flexibility to determine monitoring
needs (e. g., site visits as appropriate) consistent with resource
availability as compared with a monitoring level that is associated with a
specific monitoring requirement that may not be achievable with existing
resources.

Since April 2001, OJJDP has been engaged in a review of its monitoring
practices. 16 According to OJJDP officials, this includes a thorough review
of its monitoring practices, including improvement of its grant records.
OJJDP indicated that a Working Group on Monitoring was created and charged
with reviewing its revised OJP Handbook: Policies and Procedures for the
Administration of OJP Grants to identify new and modified policies and
assess the need for training; to review OJJDP?s own policies and procedures
to ensure compliance with those in the revised OJP Handbook; and to
determine whether there is a continued need for the OJJDP Monitoring Desk
Book (1991), and if so, how it should be updated. In addition, OJJDP
recently embarked on an overhaul of its grant closeout practices to improve
its monitoring records. In spring 2000, it launched a grant closeout effort
jointly with the Office of the Comptroller that is intended to significantly
reduce the backlog of grants requiring closeout. As part of this activity,
grant files were reviewed, and relevant documentation was collected.

16 As of August 2001, OJJDP?s efforts were ongoing. OJJDP Current Activities

Page 11 GAO- 02- 65 Better Grant Monitoring Needed

OJJDP requires its grant managers to document their monitoring of
discretionary grants. This documentation is to include the development and
retention of a grant manager?s memorandum for each grant containing a plan
to follow for monitoring the project. In addition, progress reports
documenting progress achieved in relation to project milestones are required
from all grantees. Required documentation also is to include quarterly
financial reports from grantees. In addition, OJJDP requires certain
documentation at the time of grant closeouts.

While the grant files generally contained monitoring plans, 17 OJJDP grant
managers were not consistently documenting their monitoring efforts
according to the monitoring plans. The grant files generally did not contain
documentation showing required telephone contacts and site visits. 18
Further, progress and financial reports did not cover the entire grant
period over half of the time. For those grants in our sample that were
closed, most did not contain documentation showing compliance with closeout
procedures. We identified some of these same documentationrelated problems
in May 1996.

We compared the proposed monitoring activities in the grant manager?s
monitoring plan for 89 of 545 OJJDP demonstration discretionary grant files
and 45 of 83 OJJDP training and technical assistance discretionary grant
files to actual monitoring documentation. Our comparisons of the proposed to
actual documented monitoring revealed that OJJDP grant managers were not
consistently following the monitoring plan.

 Telephone Contacts. The grant managers? monitoring plans for the most
recent awards for 79 demonstration grants and 34 training and technical
assistance grants had specific requirements for telephone contacts (e. g.,
monthly or quarterly). There was no documentation showing that the
requirement for telephone contacts was met in 96 percent of the
demonstration grants and in any of the training and technical assistance
grants. Documentation was present that showed that some, but not all, of the
required telephone contacts had been made for 49 percent 19 of the

17 While required for every award, our review of each grant?s most recent
award showed that 1 of the 89 demonstration grant files and 3 of the 45
technical assistance grant files were missing the monitoring plan.

18 The absence of documentation in the grant file does not necessarily mean
that monitoring did not occur. 19 The estimated range is between 38 and 61
percent. Grant Files Contained

Limited Documentation That Monitoring Occurred

Page 12 GAO- 02- 65 Better Grant Monitoring Needed

demonstration grants and 41 percent 20 of the training and technical
assistance grants.

 Site Visits. The grant managers? monitoring plans for the most recent
awards for 66 demonstration grants and 20 training and technical assistance
grants had a specified number of on- site visits to be made. There was no
documentation showing that the requirement for site visits was met in 88
percent 21 of the demonstration grants and 90 percent 22 of the training and
technical assistance grants. Documentation was present that showed that
some, but not all, of the required on- site visits had been made for 6
percent of the demonstration grants and 10 percent 23 of the training and
technical assistance grants. When site visits are made to grantees, a
written site visit report is to be prepared. Of the awards that had any
evidence of site visits, such as memos or e- mails referring to the visit, 7
out of the 12 demonstration grants and 2 out of the 4 training and technical
assistance grants had any of the OJP- required, written site visit reports.

In commenting on a draft of this report, the Acting Assistant Attorney
General informed us that most grant managers prepare site visit reports
after an on- site review, as their travel vouchers are not forwarded for
payment unless the reports are attached. She noted that the grant managers
do not routinely forward copies of the reports to the official grant and
program files. She said that a copy of the reports is filed in the grant
manager?s travel folders that we did not review. However, according to the
OJP Handbook, the original site visit report is to be placed in the official
grant file. While the grant managers? travel folders may contain site visit
reports, the official grant files kept by the OJP Comptroller?s Control Desk
are the official record of the history and activity of the grant. As such,
we reviewed the official grant files as well as the grant managers? files
for required grant documents and not the travel voucher folders.

In another comment to a draft of this report, the Acting Assistant Attorney
General told us that monitoring plans that call for on- site visits by grant
managers are contingent upon staffing capacity and administrative resources
and not requirements. Our review indicated that, contrary to the

20 The estimated range is between 25 and 59 percent. 21 The estimated range
is between 77 and 95 percent. 22 The estimated range is between 68 and 99
percent. 23 The estimated range is between 1 and 32 percent.

Page 13 GAO- 02- 65 Better Grant Monitoring Needed

Acting Assistant Attorney General?s comment, certain specific program
monitoring requirements are included in the OJP Handbook, such as creating
monitoring plans and writing site visit reports within 10 days of a site
visit. To be useful, monitoring plans should be feasible. If they are not
feasible, then they will not be helpful guidance for monitoring. While it is
entirely reasonable to expect that some planned site visits will not take
place (for a variety of reasons), the proportion of site visits specified to
occur that were not documented seems to indicate that either they occurred
but were not documented or monitoring plans for site visits were infeasible.

 Progress and Financial Reports. According to the OJP Handbook, 24
recipients of discretionary grants are required to submit initial and then
semiannual categorical assistance progress reports documenting progress
achieved in relation to project milestones. Before December 31, 1995,
progress reports were due every quarter. Since March 1, 1996, progress
reports have been due every 6 months. This report must be submitted within
30 days after the end of the reporting period for the life of the award. A
final report, which provides a summary of progress toward achieving the
goals and objectives of the award, significant results, and any products
developed under the award, is due 90 days 25 after the end date of the
award.

We found that in 56 percent of the 89 demonstration grant files and 80
percent 26 of the 45 training and technical assistance grant files reviewed,
the progress reports did not cover the entire grant period. In addition, 7
percent of the demonstration grants files and 4 percent of the training and
technical assistance grants had no progress reports.

We also found that 56 percent of the progress reports submitted for
demonstration grants were filed on time; for the other 44 percent, the
number of days late ranged from 1 to 545, with 62 percent of the reports

24 In January 2001, OJP released a revised Grant Management Policies and
Procedures Manual to update and codify OJP?s current policies regarding its
business practices. According to OJP officials, this document superceded and
canceled OJP HB 4500. 2C,

Policies and Procedures for the Administration of OJP Grants, dated February
19, 1992. The revised policies and procedures were not applicable during the
period of our review.

25 The OJP Grant Management Policies and Procedures Manual dated January
2001, changed the final progress report due date from 90 days to 120 days.
The revised policies and procedures were not applicable during the period of
our review.

26 The estimated range is between 69 and 88 percent.

Page 14 GAO- 02- 65 Better Grant Monitoring Needed

for demonstration grants being late up to 30 days and 24 percent between 31
and 90 days. 27 We found that 42 percent of the training and technical
assistance grant reports were filed on time; for the other 58 percent, the
number of days late ranged from 1 to 297, with 76 percent being late up to
30 days and 15 percent between 31 and 90 days.

According to the OJP Handbook, all discretionary grantees are required to
submit quarterly financial status reports to OJP?s Office of the Comptroller
Control Desk, which maintains the official grant files. Copies are
disseminated to the appropriate financial analysts and cognizant grant
managers. These quarterly reports are required for all active grants, even
if there has been no financial activity during the reporting period.
According to OJP officials, reports are considered delinquent, for payment
purposes, if not received 45 days after the end of each calendar quarter.
The final financial status report is due 90 days 28 after the end of the
grant period or extension. Future awards and fund drawdowns may be withheld
if the progress and financial status reports are delinquent.

We found that in 65 percent of the 89 demonstration grant files and 60
percent 29 of the 45 training and technical assistance grant files reviewed,
the financial status reports did not cover the entire grant period. We also
found that 67 percent of the financial status reports submitted for
demonstration grants were filed on time; for the other 33 percent, the
number of days late ranged from 1 to 367, with 54 percent of the reports for
demonstration grants being late up to 30 days and 34 percent between 31 and
90 days. 30 We found that 70 percent of the training and technical

27 We reviewed all 321 demonstration grant progress reports and 252 training
and technical assistance grant progress reports in the files from the
initiation of the grant through the date of our review for grants that were
active in all of fiscal years 1999 and/ or 2000. We excluded 13 progress
reports for demonstration grants and 16 reports for training and technical
assistance grants because of data- related problems, such as missing
submission dates.

28 The OJP Grant Management Policies and Procedures Manual dated January
2001, changed the final financial status report due date from 90 days to 120
days. The revised policies and procedures were not applicable during the
period of our review.

29 The estimated range is between 48 and 71 percent. 30 We reviewed all 615
demonstration financial status reports and 603 training and technical
assistance financial status reports in the files from the initiation of the
grant through the date of our review for grants that were active in all of
fiscal years 1999 and/ or 2000. We excluded 18 financial status reports for
demonstration grants and 6 reports for training and technical assistance
grants because of data- related problems, such as missing submission dates.

Page 15 GAO- 02- 65 Better Grant Monitoring Needed

assistance grant reports were filed on time; for the other 30 percent, the
number of days late ranged from 1 to 353, with 72 percent being late up to
30 days, and 19 percent between 31 and 90 days.

Further, our review found no documentation in the files that OJJDP grant
managers regularly reviewed these reports, as required by OJP. Without such
reviews, OJJDP does not have assurances that the grants are being properly
administered. For example, one grant we reviewed had no documentation of
required semiannual categorical assistance progress reports or quarterly
financial reports in the file for the entire life of the award. The project
and budget periods ran from October 1, 1999, to September 30, 2000. A
$300,000 grant was awarded on March 7, 2000 (6 months after the start of the
budget period). The grantee was required to submit an initial and then
semiannual categorical assistance progress reports documenting progress
achieved in relation to project milestones, starting June 30, 2000, and
every 6 months thereafter. The grantee was also required to submit a
quarterly financial status report within 45 days after the quarter ended,
starting May 15, 2000, and every quarter thereafter.

Although no progress or financial reports had been submitted during the life
of the grant period, OJJDP did not discover this until almost a full year
from the award date. Nevertheless, OJJDP extended the grant period for an
additional year to allow the grantee to spend the funds for the project,
which it had not drawn down during the grant period. Missing progress
reports and quarterly financial reports increase the risk that OJJDP might
not be able to identify and rectify grantee?s program and financial problems
in a timely manner.

In commenting on a draft of this report, the Acting Assistant Attorney
General acknowledged this lack of documentation, but did not agree that it
meant that OJJDP had not been monitoring grants to ensure they were being
properly implemented. From our perspective, this lack of documentation
suggests to us that OJJDP does not know the extent to which grant managers
were carrying out their monitoring responsibilities to ensure that grants
were being properly implemented.

Despite the lack of evidence that grant managers were adhering to the
established monitoring plans, we did find that some grant monitoring was
occurring. For example, the files contained evidence that communication
occurred between grantees and the grant managers in the form of reports,
faxes, letters, and oral and e- mail communication on specific issues,
problems, or requests for information. However, much of the communication
between grantees and grant managers was not documented according to OJJDP
requirements.

Page 16 GAO- 02- 65 Better Grant Monitoring Needed

 Closeout Procedures. The OJP grant closeout process calls for an
accounting of programmatic accomplishments in relation to planned
activities; an inventory of all required financial, programmatic, and audit
reports; and an accounting of all federal funds. According to the OJP
Handbook, the grant manager is to complete an assessment report and other
closure requirements within 180 days after the end date of the award or any
approved extension. These requirements include a closeout checklist, overall
assessment of grant performance, final progress reports, final financial
status reports, and closeout notification and grant adjustment notice.

Our assessment of the closeout process did not include enough cases to
ensure that it is representative of all grant closeouts. 31 However, our
limited review showed that some grant files did not contain required
closeout materials.

At the time of our review, the following documents were missing from the
files:

 The required case checklists were not prepared for two of the eight
demonstration grants that expired and required closeout. Similarly, neither
of the two training and technical assistance grants had the required case
checklists.

 An overall assessment of grant performance was not prepared for two of the
eight demonstration grants. Neither of the two training and technical
assistance grants had this assessment.

 Final progress reports were missing from three of the eight demonstration
grant files. Neither of the two training and technical assistance grants had
final progress reports in the files.

 The final financial status report was missing from one of the eight
demonstration grant files. Neither of the two training and technical
assistance grants had a final financial status report in the files. And
there was no documentation that grant managers had reviewed any of the
reports or other official documents that were filed.

31 Because of the small number of closeout cases in our sample of active
grants in all of fiscal years 1999 and/ or 2000, we have no assurances that
the eight demonstration and two training assistance grants would be
representative of all closed out grants.

Page 17 GAO- 02- 65 Better Grant Monitoring Needed

 A closeout notification and grant adjustment notice to deobligate funds or
advise the grantee to retain records for 3 years was not prepared for three
of the eight expired demonstration grants. Similarly, neither of the two
training and technical assistance grants had a closeout notification and
grant adjustment notice.

In commenting on a draft of this report, the Acting Assistant Attorney
General noted the programmatic and fiscal closure process might not have
been completed for those grants until after our review. In our review of
closeout procedures, we waited at least the required 180 days before
reviewing the grant files to allow sufficient time for OJJDP to complete the
grant closeout process. However, the files we reviewed did not contain the
required closeout documents.

The Comptroller General?s internal control standards 32 require that all
transactions and other significant events be clearly documented and that the
documentation be readily available for examination. Appropriate
documentation is an internal control activity to help ensure that
management?s directives are carried out. Without such documentation, OJJDP
has no assurance that grants are meeting their goals and funds are being
used properly.

In commenting on a draft of this report, the Acting Assistant Attorney
General also said that almost 50 percent of the 89 Special Emphasis Division
grants that we reviewed were awarded in fiscal years 1998 and 1999 under the
Drug- Free Communities Support Program. Because of the time needed to
recruit, hire, and train new program staff, OJJDP handled 217 Drug- Free
Communities Support Program grants in those 2 years with less than adequate
(i. e., two) staff. According to the Acting Assistant Attorney General,
caseloads of this size make proper documentation a difficult goal to
achieve.

32 Standards for Internal Control in the Federal Government (GAO/ AIMD- 00-
21. 3. 1, Nov. 1999).

Page 18 GAO- 02- 65 Better Grant Monitoring Needed

OJJDP does not systematically monitor grant managers? compliance with its
monitoring requirements or guidance nor does it assess the effectiveness of
OJJDP grant monitoring practices. When we asked OJJDP officials about any
oversight of OJJDP grant managers to determine the adequacy of their
monitoring of OJJDP discretionary grants, they told us oversight takes
several forms. They pointed out that the OJP Handbook requires staff
performance to be carefully scrutinized against the performance work plans
(i. e., the critical performance elements and standards pertaining to
monitoring responsibilities). They said that OJJDP supervisory managers are
to address the following questions to determine how well each grant manager
with monitoring responsibilities is performing: (1) how informed is the
grant manager about his or her project; (2) what specific oversight and
review activities are being conducted; and (3) how are grantee performance
issues, if any, being resolved?

Our review of the performance work plan OJJDP considered to be
representative found that it tended to contain information that made it
difficult to precisely understand the nature of the monitoring to be
conducted. The performance work plan, for example, referred to monitoring
grants to ensure that projects meet their goals and milestones, with no
clear expectations regarding monitoring requirements, such as site visits or
telephone monitoring, and documentation of these contacts and reviews of
grantees? progress and financial reports.

In addition, according to OJJDP officials, they do not have an officewide
management information system that tracks monitoring activities specific to
each grant and links these to grant monitoring staff. 33 However, OJJDP
officials said they take several actions to ensure that staff are adequately
performing their monitoring responsibilities. They said the grant manager?s
memorandum is carefully scrutinized to determine that the proper information
regarding grant goals, objectives, tasks, program activities, and products
are included. Division directors are to review requests for site visits on a
quarterly or semiannual basis as part of their development of travel
budgets. Site visit reports are reviewed at the time or within a reasonable
period of submission of travel reimbursement requests. Finally, division
directors are routinely consulted regarding corrective action on grants.

33 OJP?s Office of the Comptroller has developed an automated tracking
system that flags missing and late progress and financial reports. This
information is available to OJJDP grant managers. However, OJJDP does not
have a system for tracking telephone contacts, site visits, review of
progress and financial reports, or other reporting requirements. OJJDP Does
Not

Systematically Determine Staff Compliance With Its Monitoring Requirements

Page 19 GAO- 02- 65 Better Grant Monitoring Needed

In our May 1996 work, we reported that OJJDP?s files for discretionary
grants generally contained monitoring plans but little documentation that
monitoring occurred. OJJDP said that a heavy workload might have resulted in
a lack of monitoring records, but that it would take the steps necessary to
improve records. OJJDP has made limited progress in addressing the
monitoring problems we identified in 1996.

Our review showed that monitoring activities continue to be insufficiently
documented, and because of this, neither OJJDP nor we can determine the
level of monitoring being performed by grant managers as required by OJP,
OJJDP, and the Comptroller General?s internal control standards.

OJJDP also cannot rely on supervisory oversight of its grant managers to
ensure that monitoring is being performed as required. The use of staff
performance work plans seems problematic as a means of reviewing monitoring
activities because the requirements are written in such general terms. Grant
managers? performance work plans do not include clear expectations and
accountability for monitoring called for in grant managers? monitoring plans
as well as follow- up requirements to notify and assist grantees who
consistently submit late or no progress or financial reports. Further, the
approval of grant managers? memorandums or preapproval of site visits does
not ensure that required monitoring is taking place, and none of these
oversight activities is designed to ensure that proper documentation of
monitoring activity is taking place.

Current activities to review monitoring policies and practices and to
complete documentation for closed grants could result in positive changes.
However, these activities are too new to be evaluated in relation to their
potential for improving grant monitoring or its documentation.

The current review of monitoring policies and practices affords an
opportunity for OJJDP to assess monitoring in light of our recent findings.
To facilitate and improve the management of program grant monitoring, we
recommend that the Attorney General direct OJJDP, as part of its ongoing
review of monitoring policies and practices, to determine whether the
monitoring documentation problems that we identified were an indication of
grant monitoring requirements not being met or of a failure to document
activities in the official grant files that did, in fact, take place.

If monitoring requirements are not being met, we recommend that the Attorney
General direct OJJDP to determine why this is so and to consider those
reasons as it develops solutions for improving compliance with the
requirements. Conclusions

Recommendations for Executive Action

Page 20 GAO- 02- 65 Better Grant Monitoring Needed

If it is determined that required monitoring is taking place but is not
being documented, we recommend that the Attorney General direct OJJDP to
develop and enforce clear expectations regarding monitoring requirements.
This policy should, among other things, require supervisory review to ensure
that monitoring activities are being carried out and documented as
prescribed. Further, grant managers? performance work plans should include
clear expectations and accountability for monitoring called for in grant
managers? monitoring plans as well as follow- up requirements to notify and
assist grantees who consistently submit late or no progress or financial
reports.

We provided a copy of this report to the Attorney General for review and
comment. In a September 19, 2001, letter, the Acting Assistant Attorney
General commented on a draft of this report. Her comments are summarized
below and are presented in their entirety in appendix II.

The Acting Assistant Attorney General said that overall, the report provides
useful information in highlighting management and monitoring activities in
need of improvement. Senior staff within OJJDP have been directed to address
each of the issues identified in the draft report to focus on adjusting the
nature and types of monitoring activities pursuant to these types of grants,
thus ensuring that every active grant receives a programmatic assessment.
OJJDP will also be considering ways to improve and streamline grant
management monitoring activities to ensure that program monitoring efforts
are documented and useful, including the development of a management
information system and the use of private contractors to assist in some of
these review functions.

The Acting Assistant Attorney General also said that the draft report could
have gone further in acknowledging the impact of OJJDP?s enormous growth of
the past 5 years on its ability to monitor recipients. We recognized that
the number of active OJJDP discretionary grants has more than tripled over
the past 5 years. However, it is OJJDP?s responsibility to initiate action,
such as to request additional funding to meet its monitoring
responsibilities or revise its monitoring efforts to comport with its
available resources. In commenting on our May 1996 testimony, the OJJDP
Deputy Administrator recognized that OJJDP?s heavy workload might have
resulted in a lack of monitoring records, but that he would take steps
necessary to improve records. However, our current review disclosed that
similar monitoring problems persist.

Further, the Acting Assistant Attorney General said that we did not
acknowledge OJJDP?s efforts in a major OJP- wide effort to improve grant
Agency Comments

and Our Evaluation

Page 21 GAO- 02- 65 Better Grant Monitoring Needed

monitoring practices and establish uniform policies. We recognized some of
the efforts to review its monitoring practices, such as revising the OJP
Handbook: Policies and Procedures for the Administration of OJP Grants and
its recent embarking on an overhaul of its grant closeout practices to
improve its monitoring records. Such efforts, we believe, should help
address the monitoring problems we identified.

The Acting Assistant Attorney General concurred with our recommendation that
grant managers? performance work plans include clear expectations and
accountability for monitoring. She said performance work plans put into
effect after February 2001 for grant managers in OJJDP?s Special Emphasis
Division more clearly reflect monitoring responsibilities and provide for
better supervisory controls. Further, she noted that OJJDP?s Special
Emphasis Division has put into place a new process designating one staff
member with responsibility for file ?audits? to help ensure that files are
in order and that documentation of monitoring is in place.

As agreed with your office, unless you publicly announce the contents of
this report earlier, we plan no further distribution of it until 30 days
from the date of this letter. At that time, we will send copies to the
Chairmen and Ranking Members of the Senate Judiciary Committee; Senate
Subcommittee on Youth Violence; House Committee on Education and the
Workforce; House Subcommittee on Early Childhood, Youth and Families;
Attorney General; and Director, Office of Management and Budget. We will
also make copies available to others upon request.

If you or your staff have any questions about this report, please contact me
or James M. Blume at (202) 512- 8777. Key contributors to this report are
acknowledged in appendix III.

Sincerely yours, Laurie E. Ekstrand Director, Justice Issues

Appendix I: OJJDP?s Discretionary Grant Award Process

Page 22 GAO- 02- 65 Better Grant Monitoring Needed

Each year, OJJDP publishes a proposed program plan seeking public comment
about proposed discretionary funding opportunities for activities covered
under parts C and D (all national programs and gang- related activities) of
the Juvenile Justice and Delinquency Prevention Act. The proposed plan is
published in the Federal Register. Program announcements and solicitations
for discretionary funding opportunities are published as they become
available. The announcements and solicitations provide details about
specific funding opportunities, including eligibility requirements and
deadlines. These include funding opportunities for tribal youth and drug-
free communities as well as funding opportunities associated with parts C
and D. Once discretionary grant applications are submitted, OJJDP said that
its contractor 1 conducts an initial review to eliminate any applications
that do not comply with the specified grant instructions. After ineligible
grant applications are eliminated, OJJDP has a peer review panel, which
consists of three or more outside experts who evaluate and rank OJJDP?s
grant applications. After the peer review panel has ranked the applications,
OJJDP?s grant managers 2 do an in- depth review of applications and make
formal recommendations regarding the funding of individual grants to the
OJJDP Administrator.

As shown in figure 1, the key parts of OJJDP?s process for planning and
announcing discretionary grants are issuance of a program plan, issuance of
discretionary program announcements, and distribution of application kits.

1 OJJDP?s current contractor is the Juvenile Justice Research Center at
Aspen Systems. 2 OJJDP?s grant managers are OJJDP officials with
responsibility for overseeing and monitoring grants. Appendix I: OJJDP?s
Discretionary Grant

Award Process

Appendix I: OJJDP?s Discretionary Grant Award Process

Page 23 GAO- 02- 65 Better Grant Monitoring Needed

Figure 1: Key Steps in OJJDP?s Process for Planning and Announcing Grants

Source: GAO analysis of OJJDP documents and interviews with OJJDP staff.

As shown in figure 2, according to OJJDP officials, the key parts of the
grant application and review process include initial screening of an
application by the OJJDP contractor, peer review of applications, and the
final selection of applications for funding by the OJJDP Administrator or
Assistant Attorney General.

Step 1 Step 2 OJJDP program plan

Development Revision

OJJDP program announcement

Development

Federal Register

publishes proposed program plan

Federal Register

publishes program announcement

notice Contractor mails out

application kits for discretionary

grants Applicant applies

for discretionary Final program

plan issued Public has 45- day

comment period grant

Appendix I: OJJDP?s Discretionary Grant Award Process

Page 24 GAO- 02- 65 Better Grant Monitoring Needed

Figure 2: Overview of OJJDP Grant Application and Review Process

a Some program managers will do initial screening only for specific
programs. b Quality control checks are done by program managers before the
peer review.

Source: GAO analysis of OJJDP documents and interviews with OJJDP staff.
Contractor

does initial screening of grant

applications a Peer review panel

evaluates application and makes funding recommendations b Does application

meet requirements?

OJJDP Administrator makes tentative selection of application for funding

OJJDP Administrator approves grant awards for research, evaluation,
statistics

and other grants, as delegated by

the Assistant Attorney General

OJP Comptroller conducts financial

review Applications are no

longer processed Juvenile Justice

Research Center at Aspen Sytems

receives the applications

Program managers review recommendations of peer review panelists

and make formal funding recommendations

to OJJDP Administrator

Assistant Attorney General, OJP, approves grants

in accordance with the guidance outlined in the current

appropriations law; e. g., training and technical assistance

and demonstration Yes

No

Appendix I: OJJDP?s Discretionary Grant Award Process

Page 25 GAO- 02- 65 Better Grant Monitoring Needed

OJJDP?s proposed program plan describes OJJDP?s goals and priorities and the
particular program activities it plans to carry out during the fiscal year
under parts C and D of Title II of the OJJDP Act. OJJDP publicly announces
its proposed program plan in the Federal Register. After a 45day comment
period, OJJDP is to publish a final plan 3 that is to take into
consideration comments received during the comment period.

According to an official, the OJJDP program plan is generally drafted by the
Administrator and senior managers and based on (1) congressional and
departmental priorities, (2) their knowledge and expertise, and (3) input
during the course of the year from a variety of sources. These sources
include program reviews by OJJDP staff and comments from practitioners in
the field, officials from Justice and OJP bureaus and offices, and other
federal agencies.

Following publication of the final program plan, OJJDP is to publish in the

Federal Register a notice of discretionary grant programs and to announce
the availability of an application kit for grants. The notice is to include
the availability of funds, criteria for selection of grant applicants,
procedures applicable to the submission and review of application for
submission and review of applications for assistance, and information on how
to obtain an application kit.

Upon receipt of discretionary grant applications, the OJJDP contractor does
an initial review of each application, using an application review
checklist. The objective of this initial review is to determine whether
applications are complete and eligible for federal funding.

OJJDP contracts out the administrative arrangements (e. g., arranging panels
and notifying reviewers) for the peer review of its discretionary grant
applications. The contractor maintains a list of qualified consultants from
which peer reviewers were selected by OJJDP. An OJJDP official estimated
that there were 73 peer review panels in fiscal year 2000 for its 22 grant
programs. The grant managers identify the particular expertise they want the
peer reviewer to have. The contractor generates a list of potential
reviewers with their resumes, background information, and other helpful
information. The grant managers recommend peer reviewers on

3 The plan is to be published before December 31 each fiscal year. Program
Plan Is to

Describe Goals and Priorities

Program Announcements and Application Kits Are to Solicit Applicants for
Programs

Peer Review Panels Are to Help Determine Which Grant Applications Are Funded

Appendix I: OJJDP?s Discretionary Grant Award Process

Page 26 GAO- 02- 65 Better Grant Monitoring Needed

the basis of the contractor?s list and give the list to the Deputy
Administrator for approval. 4 After the Deputy Administrator approves the
list, it is sent to the Administrator. Once the Administrator selects the
peer reviewers, the contractor makes arrangements with the individuals
selected and manages the peer review process.

According to an OJJDP official, consultants performing the peer reviews are
reimbursed at a flat rate of $150 a day. OJJDP guidance requires that the
grant managers use the following criteria to help achieve balance on the
peer review panel.

 Each reviewer should have expertise in or complementary to the subject
area under review.

 Where possible, the peer review panel should be composed of a mix of
researchers, practitioners, and academicians.

 Special attention should be paid to obtaining qualified representation
from women and minority groups.

OJJDP?s peer review process is advisory and is to supplement and assist
OJJDP?s consideration of applications. However, an OJJDP official noted
that, in practice, results of the peer review panels? consideration of
application were almost always accepted.

After completion of the peer review panels, OJJDP?s grant managers do an in-
depth review of the peer review panels? ranked list. The OJJDP grant
managers make recommendations concerning applications to receive funding in
a memorandum to the Administrator, and the Administrator is to make
tentative grant selections. The OJJDP Administrator makes final awards for
research and evaluation grants, while final awards for all training and
technical assistance and demonstration grants are made by the Assistant
Attorney General. After the applications are approved, OJP?s Office of the
Comptroller does a financial review of these applications to determine
whether the applicant has the necessary resources and integrity

4 The Juvenile Justice and Delinquency Prevention Act of 1974, as amended,
requires OJJDP to have formal peer review process for discretionary grant
awards (National Programs). 42 U. S. C. 5601, 5665a (d). However, OJJDP
chooses to require all the competitive discretionary grant applications to
be involved in its peer review. Noncompetitive grants, earmarks, and
uniquely qualified grants do not go through a formal peer review process.
OJJDP usually has a maximum of three grants that are uniquely qualified,
which are reviewed and approved by the Administrator. Final Decision to

Award Discretionary Grants

Appendix I: OJJDP?s Discretionary Grant Award Process

Page 27 GAO- 02- 65 Better Grant Monitoring Needed

to account for and administer federal funds properly and whether budget and
cost data in the application were allowable, effective, and reasonable.

Appendix II: Comments From the Department of Justice

Page 28 GAO- 02- 65 Better Grant Monitoring Needed

Appendix II: Comments From the Department of Justice

Note: GAO comments supplementing those in the report text appear at the end
of this appendix.

Appendix II: Comments From the Department of Justice

Page 29 GAO- 02- 65 Better Grant Monitoring Needed

See footnote 12. See comment 1.

Appendix II: Comments From the Department of Justice

Page 30 GAO- 02- 65 Better Grant Monitoring Needed

Now last paragraph. Now on p. 8.

Appendix II: Comments From the Department of Justice

Page 31 GAO- 02- 65 Better Grant Monitoring Needed

Now on p. 9, footnote 14. Now on p. 9.

Appendix II: Comments From the Department of Justice

Page 32 GAO- 02- 65 Better Grant Monitoring Needed

Now on p. 14. See pp. 12- 13.

See p. 17. Now on p. 11.

Appendix II: Comments From the Department of Justice

Page 33 GAO- 02- 65 Better Grant Monitoring Needed

Now on p. 16, 1 st bullet. Now on p. 16, 4 th bullet.

Now on p. 15. See p. 14, footnote 28.

Appendix II: Comments From the Department of Justice

Page 34 GAO- 02- 65 Better Grant Monitoring Needed

Now on p. 20. See p. 12.

Now on pp. 19- 20. Now on p. 17.

Appendix II: Comments From the Department of Justice

Page 35 GAO- 02- 65 Better Grant Monitoring Needed

Appendix II: Comments From the Department of Justice

Page 36 GAO- 02- 65 Better Grant Monitoring Needed

The following are GAO?s comments on the Department of Justice?s September
19, 2001, letter.

1. According to the OJP Handbook: Policies and Procedures for the
Administration of OJP Grants (Feb. 1992), official grant files kept by the
Office of the Comptroller Control Desk are to contain documents relating to
each grantee, including progress and financial reports and site visit
reports. In addition, for documentation to be readily available for
examination, as required by the Comptroller General?s internal control
standards, keeping them in the official grant file seems appropriate. GAO
Comment

Page 37 GAO- 02- 65 Better Grant Monitoring Needed

Laurie E. Ekstrand (202) 512- 8777 James M. Blume (202) 512- 8777

In addition to those named above, R. Rochelle Burns, Jill Roth, Michele
Fejfar, Sidney H. Schwartz, Michele J. Tong, Jerome T. Sandau, and Ann H.
Finley made key contributions to this report. Appendix III: GAO Contacts and
Staff

Acknowledgments GAO Contacts Acknowledgments

(440026)

The General Accounting Office, the investigative arm of Congress, exists to
support Congress in meeting its constitutional responsibilities and to help
improve the performance and accountability of the federal government for the
American people. GAO examines the use of public funds; evaluates federal
programs and policies; and provides analyses, recommendations, and other
assistance to help Congress make informed oversight, policy, and funding
decisions. GAO?s commitment to good government is reflected in its core
values of accountability, integrity, and reliability.

The fastest and easiest way to obtain copies of GAO documents is through the
Internet. GAO?s Web site (www. gao. gov) contains abstracts and full- text
files of current reports and testimony and an expanding archive of older
products. The Web site features a search engine to help you locate documents
using key words and phrases. You can print these documents in their
entirety, including charts and other graphics.

Each day, GAO issues a list of newly released reports, testimony, and
correspondence. GAO posts this list, known as ?Today?s Reports,? on its Web
site daily. The list contains links to the full- text document files. To
have GAO E- mail this list to you every afternoon, go to our home page and
complete the easy- to- use electronic order form found under ?To Order GAO
Products.?

The first copy of each printed report is free. Additional copies are $2
each. A check or money order should be made out to the Superintendent of
Documents. GAO also accepts VISA and Mastercard. Orders for 100 or more
copies mailed to a single address are discounted 25 percent. Orders should
be sent to:

U. S. General Accounting Office P. O. Box 37050 Washington, D. C. 20013

To order by Phone: Voice: (202) 512- 6000 TDD: (301) 413- 0006 Fax: (202)
258- 4066

GAO Building Room 1100, 700 4th Street, NW (corner of 4th and G Streets, NW)
Washington, D. C. 20013

Contact: Web site: www. gao. gov/ fraudnet/ fraudnet. htm, E- mail:
fraudnet@ gao. gov, or 1- 800- 424- 5454 (automated answering system).

Jeff Nelligan, Managing Director, NelliganJ@ gao. gov (202) 512- 4800 U. S.
General Accounting Office, 441 G. Street NW, Room 7149, Washington, D. C.
20548 GAO?s Mission

Obtaining Copies of GAO Reports and Testimony

Order by Mail or Phone Visit GAO?s Document Distribution Center

To Report Fraud, Waste, and Abuse in Federal Programs

Public Affairs
*** End of document. ***