Grants Management: Despite Efforts to Improve Weed and Seed
Program Management, Challenges Remain (24-MAR-04, GAO-04-245).
The Weed and Seed program, within the Department of Justice's
Office of Justice Programs (OJP), aims to prevent and reduce
violent crime in targeted neighborhoods, but it cannot optimize
its effectiveness without sound management practices. In 1999,
GAO made four recommendations to the Executive Office for Weed
and Seed (EOWS) to improve the program's management, including
(1) developing adequate internal controls to fully document
decisions, (2) improving program monitoring, (3) developing
criteria for determining when sites have become self-sustaining
and when to reduce or withdraw program funding, and (4)
developing additional performance measures. GAO did this study to
assess progress in implementing these recommendations.
-------------------------Indexing Terms-------------------------
REPORTNUM: GAO-04-245
ACCNO: A09559
TITLE: Grants Management: Despite Efforts to Improve Weed and
Seed Program Management, Challenges Remain
DATE: 03/24/2004
SUBJECT: Community development programs
Crime prevention
Internal controls
Performance measures
Program evaluation
Program management
Records (documents)
Reporting requirements
Evaluation criteria
Program graduation
DOJ Operation Weed and Seed Program
******************************************************************
** This file contains an ASCII representation of the text of a **
** GAO Product. **
** **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced. Tables are included, but **
** may not resemble those in the printed version. **
** **
** Please see the PDF (Portable Document Format) file, when **
** available, for a complete electronic file of the printed **
** document's contents. **
** **
******************************************************************
GAO-04-245
United States General Accounting Office
GAO Report to the Chairman, Committee on the Judiciary, House of Representatives
March 2004
GRANTS MANAGEMENT
Despite Efforts to Improve Weed and Seed Program Management, Challenges Remain
GAO-04-245
Highlights of GAO-04-245, a report to the Chairman, Committee on the
Judiciary, House of Representatives
The Weed and Seed program, within the Department of Justice's Office of
Justice Programs (OJP), aims to prevent and reduce violent crime in
targeted neighborhoods, but it cannot optimize its effectiveness without
sound management practices. In 1999, GAO made four recommendations to the
Executive Office for Weed and Seed (EOWS) to improve the program's
management, including (1) developing adequate internal controls to fully
document decisions, (2) improving program monitoring, (3) developing
criteria for determining when sites have become self-sustaining and when
to reduce or withdraw program funding, and (4) developing additional
performance measures. GAO did this study to assess progress in
implementing these recommendations.
GAO recommends that the Attorney General require the Assistant Attorney
General for OJP to ensure that the Executive Office for Weed and Seed
fully implement the intent of GAO's previous recommendations.
In commenting on a draft of this report, the Department of Justice agreed
to strengthen controls on maintaining documentation and take further steps
to define and apply criteria for self-sustainability. Justice believes
that the studies currently under way will help develop outcome measures.
www.gao.gov/cgi-bin/getrpt?GAO-04-245.
To view the full product, including the scope and methodology, click on
the link above. For more information, contact Laurie Ekstrand at (202)
512-8777 or [email protected].
March 2004
GRANTS MANAGEMENT
Despite Efforts to Improve Weed and Seed Program Management, Challenges Remain
Despite some progress toward addressing GAO's recommendations aimed at
improving program management, GAO's review shows that EOWS has not fully
implemented the management improvement recommendations GAO made in 1999.
First, although EOWS has revised its internal controls to require that
significant qualification and funding decisions be documented and readily
available in the central grant files for review, EOWS has not always
ensured that its policies and procedures were followed, for the grant
files GAO reviewed. Second, EOWS reported taking a number of actions
intended to improve program monitoring, such as mandating the timely
submission of progress reports and adequate recording of site visits as
GAO recommended. Nonetheless, GAO found that while EOWS was able to
provide such documentation before its review ended, documentation was not
available in some of the central grant files GAO reviewed. Thus, the
documentation was not readily available for external reviewers, as
required by OJP policies and GAO's internal control standards. Third, GAO
found that EOWS still lacks fully developed criteria to determine when
sites become self-sustaining and when to reduce or withdraw Weed and Seed
funds because of the level of sustainability, even though sustainability
is a central goal of the program. At the time of GAO's review, no site's
funding had been reduced or withdrawn because of sustainability during the
13 years of the program's existence. Fourth, EOWS has not developed
outcome performance measures that can be used to adequately track progress
toward program outcomes of the Weed and Seed program. While EOWS has
initiated studies on how to develop performance measures, at the time of
GAO's review, none of these studies had been completed. Without
requirements to monitor improvements and assign accountability, progress
will be difficult to achieve.
A sign displayed in front of a school in a Weed and Seed site.
Contents
Letter
Results in Brief
Background
Internal Controls Have Been Developed, but Challenges Remain
EOWS Has Taken Steps to Improve Program-Monitoring
Documentation, but Challenges Remain
Despite Some Steps, Challenges Remain in Developing Criteria for
Self-Sustainability
Performance Measures Generally Did Not Track Program
Outcomes
Conclusions
Recommendations for Executive Action
Agency Comments and Our Evaluation
1
2 3 7
9
11
12 16 17 17
Appendix I Scope and Methodology
Appendix II GAO Site Visit Summary
Appendix III Proposed Legislation Could Affect Self-Sustainability 29
Appendix IV Executive Office for Weed and Seed Selected Activities
Appendix V Activity Data Collected by EOWS in Fiscal Year 2003 34
Appendix VI Description of EOWS's Ongoing and Completed
Studies 36
Appendix VII Comments from the Department of Justice 37
Appendix VIII GAO Contacts and Staff Acknowledgments 42
GAO Contacts 42 Staff Acknowledgments 42
Related GAO Products
Tables
Table 1: EOWS Funding History for Fiscal Years 1999-2003 4 Table 2: Sites'
Weed and Seed Funding History-Fiscal Years 19992003 24 Table 3: Examples
of Activities Funded by Weed and Seed in Pittsburgh, Charleston, and North
Charleston 27 Table 4: Examples of Leveraging Efforts at Weed and Seed
Sites in Pittsburgh, Charleston, and North Charleston 28
Abbreviations
AAG Assistant Attorney General
COPS Community Oriented Policing Services
DFCSP Drug-Free Communities Support Program
EOWS Executive Office for Weed and Seed
FBI Federal Bureau of Investigation
FY fiscal year
GMS grant management system
GPRA Government Performance and Results Act
HUD Housing and Urban Development
JRSA Justice Research and Statistics Association
OJP Office of Justice Programs
OMB Office of Management and Budget
OR official recognition
This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed in
its entirety without further permission from GAO. However, because this
work may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this material
separately.
United States General Accounting Office Washington, DC 20548
March 24, 2004
The Honorable F. James Sensenbrenner, Jr.
Chairman
Committee on the Judiciary
House of Representatives
Dear Mr. Chairman:
The Department of Justice's (Justice) Weed and Seed program proposes to
"weed" out crime from targeted neighborhoods and "seed" them with a
variety of programs and resources to prevent crime from recurring. Weed
and Seed sites have sponsored activities such as police bike patrols,
community cleanups, youth and recreational activities, and computer
training. According to Justice, a central tenet of the Weed and Seed
program is for local Weed and Seed sites to develop partnerships with
other federal, state, and local governments and private sector agencies to
leverage federal Weed and Seed grant funds with additional resources
from these partners to promote weeding and seeding activities. These
additional resources are intended to help the sites achieve the goal of
becoming self-sustaining after Weed and Seed funding ends and to sustain
crime reduction and community revitalization activities to ensure stable
communities.
This report responds to your request to assess the Executive Office for
Weed and Seed's (EOWS) efforts to implement the management
improvement recommendations we made in 1999.1 It is not intended to
evaluate the overall management or results of the program. In our
previous report on the Weed and Seed program, we recommended that
EOWS improve its management of the Weed and Seed program.
Specifically, we recommended that EOWS (1) develop adequate internal
controls to ensure that the basis and the rationale for new and existing
site
qualification and funding decisions are always fully documented,
(2) improve program monitoring to ensure that sites meet the grant
requirement of submitting progress reports and that EOWS site visits are
documented, (3) develop criteria for determining when sites are self
1U.S. General Accounting Office, Federal Grants: More Can Be Done to
Improve Weed and Seed Program Management, GAO/GGD-99-110 (Washington,
D.C.: July 1999).
sustaining and when to reduce or withdraw program funding, and (4) develop
additional performance measures that track program outcomes. This report
summarizes our assessment of EOWS's efforts to address our
recommendations.
To obtain information on EOWS's efforts, we reviewed relevant documents,
including EOWS's policies and procedures, monitoring documentation, and
agency staffing and budget data. We reviewed GAO, Congressional Research
Service, and Justice's Office of Inspector General reports. In addition,
we interviewed officials from EOWS, the Office of Justice Programs, other
entities in the Justice Department, and researchers in performance
measurement and evaluation at the Justice Research and Statistics
Association and the Urban Institute in Washington, D.C. We visited and
interviewed program staff at three Weed and Seed sites. We reviewed 30
randomly selected Weed and Seed grant files to identify the actions taken
by EOWS for those grants to document its qualification and funding
decisions and the steps EOWS took to ensure that grant documentation
requirements were met. Since the files we reviewed were not representative
of all EOWS grant files, we cannot project the results to the larger
population. However, the information helps to identify the level of
implementation of our recommendations for those grants. Additional
information about the report's scope and methodology is presented in
appendix I. We conducted this engagement in accordance with generally
accepted government auditing standards.
Despite some progress toward addressing our recommendations aimed at
improving program management, our review shows that EOWS has not fully
implemented any of the management improvement recommendations we made in
1999. First, EOWS has revised its internal controls to require that
significant qualification and funding decisions be documented and readily
available in two types of centralized files: official recognition and
official grant files. However, EOWS has not always ensured that the
documentation was readily available and that its policies and procedures
for internal controls were followed for the official recognition and
official grant files we reviewed. Second, EOWS reported to us that it had
taken a number of actions intended to improve program monitoring, such as
mandating the timely submission of progress reports and adequate recording
of site visits, as we recommended. Nonetheless, we found that while EOWS
was able to provide such documentation before our review ended,
documentation was not available in some of the central grant files we
reviewed. Thus, the documentation was not readily available for EOWS
management or external reviewers, as required by the Office of Justice
Programs (OJP) policies and GAO's internal control standards. Third, we
Results in Brief
found that EOWS still lacks fully developed criteria we recommended that
they develop to determine when sites have become self-sustaining and when
to reduce or withdraw Weed and Seed funds because of the level of
sustainability, even though sustainability is a central goal of the
program. At the time of our review, no site's funding had been reduced or
withdrawn because of sustainability during the 13 years of the program's
existence. Fourth, EOWS began developing additional performance measures
to better assess how well sites are meeting program objectives, as we
recommended in 1999. However, our work showed that although EOWS collected
data on a variety of activities taking place at Weed and Seed sites, they
generally did not measure the extent to which grantees were weeding crime
from neighborhoods and preventing it from recurring. While EOWS has
initiated studies on how to develop additional performance measures, at
the time of our review, none of these studies had been completed.
To further improve program management, we make a recommendation to the
Attorney General for four actions to help ensure full implementation of
the recommendations we made in our 1999 report. In commenting on a draft
of this report, Justice partially agreed with our recommendations and
mentioned actions being taken or planned in response.
The Weed and Seed program is a Justice discretionary grant program within
OJP.2 The Assistant Attorney General for OJP is responsible for the
overall management and oversight of offices within OJP, including EOWS.
The Assistant Attorney General for OJP sets policies, promotes
coordination among OJP bureaus and offices, and ensures that EOWS follows
its policies and procedures. EOWS provides funding to grantees to help
prevent and control crime and improve the quality of life in targeted
high-crime neighborhoods across the country.3 It is a joint federal,
state, and local program for coordinated law enforcement and neighborhood
reinvestment. Federal program funding is to support Weed and Seed sites
and to provide training and technical assistance.4
2Discretionary grants are awarded to eligible grantees, most often on a
competitive basis.
3A grantee is an entity that receives funding from EOWS to implement the
Weed and Seed program. The grantee distributes the funds to sites and is
responsible for ensuring that sites comply with the terms of the grant. A
grantee may have more than one site.
4A site is a geographically defined area ranging in size from several
neighborhood blocks to several square miles. With the input of the local
U.S. Attorney's Office and the site's steering committee, each site
develops and implements its own Weed and Seed program.
Background
The Weed and Seed program has grown since it began in fiscal year 1991
with three pilot sites in Kansas City, Missouri; Omaha, Nebraska; and
Trenton, New Jersey, and a relatively small investment of federal money.
From fiscal year 1999 through 2003, the number of Weed and Seed sites
increased from 163 to 221, while the total annual program budget generally
increased from about $45 million to $70 million. See table 1 for fiscal
years 1999-2003 data on the Weed and Seed program, including EOWS's
funding history. In fiscal year 2003, with a budget of about $70 million,
EOWS awarded grants to 221 Weed and Seed sites.
Table 1: EOWS Funding History for Fiscal Years 1999-2003
Prior year Funds from
Unobligated recoveries Funds from the Housing
and
balance other the asset and Urban
carried
forward Total
from reimbursable EOWS forfeiture funding
Development Number of
Fiscal prior year adjustments appropriation funda (HUD)b available
year funded sites
1999 $4,122,668 $714,466 $33,500,000 $6,500,000 $0 $44,837,134
2000 $4,129,852 $2,066,682 $33,500,000 $6,500,000 $0 $46,196,533c
2001 $4,955,423 $1,269,708 $33,925,200 $15,500,000 $0 $55,650,331
2002 $584,278 $2,359,292 $58,918,000 $0 $10,000,000 $71,861,570
2003 $4,102,334 $2,713,913 $58,542,000 $0 $4,935,000 $70,293,247
Source: OJP's Office of Budget and Management Services data. GAO did not
verify the data.
Note: For EOWS funding history prior to 1999, see GAO/GGD-99-110.
aThe proceeds from the asset forfeitures fund were used for federal and
state law enforcement purposes. EOWS received these funds from Justice,
under a reimbursable agreement, for the payment of various costs incurred
by state and local law enforcement officers that participated in joint
federal law enforcement operations with federal agencies.
bHUD made $10 million in additional funding available, under a
reimbursable agreement, to help Weed and Seed sites reduce drug-related
crimes in public housing.
cDoes not add because of rounding.
EOWS is responsible for the national management and administration of the
Weed and Seed program, including developing policy and providing guidance
and oversight. EOWS currently administers the Weed and Seed program with a
staff of 2 management officials, 10 grant monitors, 3 support staff, 3
detailees,5 and 6 contractors.
5Two detailees are from the Federal Bureau of Investigation (FBI) and one
is from the Department of the Interior. The FBI detailees work part-time
at EOWS.
Before a community can apply for Weed and Seed grant funding and become an
eligible site, it must first apply for and gain official recognition from
EOWS. In order to obtain official recognition, a potential Weed and Seed
site must show in its application that it has a strategy for weeding and
preventing crime. Once the application has been received, EOWS creates and
maintains the official recognition files that are intended to include
documentation such as the rationale for decisions to grant or deny
official recognition to an applicant. Official recognition requires the
U.S. Attorney in the area where the Weed and Seed site is to be located to
organize a local steering committee made up of various federal, state, and
local representatives, including residents, to be responsible for local
administration of the program.6 For official recognition, a site is also
required to develop a management plan, engage residents and other partners
in its activities, and develop a comprehensive program to weed out crime
and gang activity and seed the area with social services, economic
services, and economic revitalization.
Weed and Seed program guidance requires that its sites show plans for
addressing four required elements: (1) law enforcement; (2) community
policing; (3) crime prevention and intervention, and substance abuse
prevention, intervention, and treatment; and (4) neighborhood restoration.
According to EOWS, law enforcement should attempt to eliminate the most
violent offenders by coordinating and integrating the efforts of federal,
state, and local law enforcement agencies in targeted high-crime
neighborhoods. The objective of community policing is to raise the level
of citizen and community involvement in crime prevention and intervention
activities. Crime and substance abuse prevention, intervention, and
treatment should include youth services, school programs, community and
social programs, and support groups. Finally, neighborhood restoration
should focus on assistance to distressed neighborhoods through economic
and housing development. Weed and Seed sites fund a variety of law
enforcement and community activities. For example, law enforcementfunded
activities range from participation in a multijurisdictional task force to
conducting bike patrols in the community. See appendix II for additional
information on Weed and Seed activities at the sites GAO visited.
6There are 93 U.S. Attorneys throughout the United States, Puerto Rico,
the Virgin Islands, Guam, and the Northern Mariana Islands. Each U.S.
Attorney is the chief federal law enforcement officer of the United States
within his or her particular jurisdiction. The U.S. Attorneys serve as the
nation's principal litigators under the direction of the Attorney General.
Applicants that are officially recognized are eligible to apply for
funding for up to 5 years. OJP's Office of the Comptroller creates and
maintains official grant files for sites that are awarded funding by EOWS.
Applicants are rated based on the strategy they developed in response to
the four required elements, as stated above.
In our 1999 report, we made four recommendations to help EOWS improve
program monitoring and management. The first was to develop adequate
internal controls to fully document significant qualification and funding
decisions. We found that EOWS lacked internal controls requiring
significant program management decisions be documented. Therefore, EOWS
was not able to ensure that it was making the best decisions about
allocating available funds. Our second recommendation was to improve
program monitoring to ensure that sites met the grant requirement of
submitting progress reports and that EOWS site visits were documented. We
found that EOWS did not always ensure that local sites submitted progress
reports and that grant monitors documented the results of site visits.
Such documentation would help EOWS management and grant monitors determine
how sites are meeting program objectives and how well sites are complying
with grant requirements, and assist them in making future grant
qualification decisions. Our third recommendation was to develop criteria
for determining when sites have become selfsustaining and when to reduce
or withdraw program funding. We found that although self-sustainability is
central to the program, no site's funding had been reduced or withdrawn as
a result of its efforts to become selfsustaining in the 9 years of the
program's existence. Establishing such criteria is important because the
Weed Seed program was founded on the premise that federal funding would
continue for a finite period, after which a Weed and Seed site would be
self-sustaining. Our fourth recommendation was to develop additional
performance measures to track program outcomes. We found that EOWS's
performance indicators generally tracked activities rather than program
results; therefore, EOWS was not able to measure the success of the
program.
EOWS generally agreed with three of the four recommendations presented in
the report and discussed future actions it planned to take. EOWS officials
disagreed with our recommendation on self-sustainability. They stated that
developing criteria to ascertain self-sustainment is redundant since EOWS
adopted a 5-year rule under which it could discontinue awarding funding to
qualifying sites unless the sites expanded to an additional neighborhood
site.
Internal Controls Have Been Developed, but Challenges Remain
Although EOWS has developed internal controls intended to require that
significant qualification and funding decisions be documented and readily
available for review as we recommended in 1999, these policies and
procedures are generally not being followed in the files we reviewed. In
response to our 1999 recommendation on internal controls, EOWS established
policies and procedures intended to ensure that significant qualification
decisions were documented. In addition, OJP requires that qualification
and funding decisions are to be documented in official recognition and
official grant files. Our review of 20 official recognition and 10
official grant files showed that some of EOWS's official recognition and
official grant files were missing full documentation regarding the
qualification and funding decisions. 7 However, before our file review
ended, EOWS officials produced further documentation, which they
acknowledged was not in the official recognition and official grant files
as required, but rather in the personal working files of grant monitors
and thus not readily available to EOWS management and external reviewers.
This lack of ready availability is not in keeping with EOWS's and OJP's
policies and procedures, or with the Comptroller General's standards for
internal controls.8 Without having official recognition and official grant
files complete and readily available, it may delay and complicate EOWS
officials' oversight of the documentation of qualification and funding
decisions.
Policies and Procedures Have Been Developed to Document Significant
Decisions
Both EOWS and OJP policies and procedures have been developed to help
ensure the documentation of significant decisions. In 2000, EOWS developed
a policies and procedures guide in response to the recommendation we made
in our 1999 report intended to ensure that significant qualification and
funding decisions for new and existing sites9 were always fully
documented, and further revised the guide in 2003. This guide requires
that all documentation pertaining to official recognition decisions be
kept in the official recognition files. EOWS is also required to follow
the policies in the OJP Grants Management Policies and Procedures Manual,
which requires that EOWS fully document program management decisions in
both official recognition files and official grant
7When applicants apply for funding, EOWS creates and maintains official
recognition files. If applicants receive funding, OJP creates and
maintains official grant files.
8U.S. General Accounting Office, Standards for Internal Control in the
Federal Government, GAO/AIMD-00-21.3.1 (Washington, D.C.: November 1999).
9Throughout the rest of this report, we refer to both new and existing
sites as sites.
files. The OJP manual specifically requires that the official recognition
files and official grant files contain key documents such as ratings
information, rejection letters, and applications.
Official Recognition and Official Grant Files Were Incomplete
Despite EOWS's and OJP's efforts to require the documentation of
qualification and funding decisions in official recognition and official
grant files, some of EOWS's official recognition and official grant files
we reviewed were incomplete at the time of our file review. In reviewing
the official recognition files, we randomly selected 10 files to review
from the 94 files submitted for official recognition in fiscal year 2002.
Additional information about how we selected the files is in appendix I.
We found sufficient documentation in 7 of the 10 files to determine the
basis and rationale for decisions to award official recognition. However,
in the remaining 3 files we were unable to determine the basis and
rationale for such decisions because documentation was insufficient. For
example, key documents such as the ratings information that EOWS grant
monitors use to record their assessment of the official recognition
applications did not contain the basis and rationale for the award
decision.10 Additionally, OJP's grant policies and procedures require
EOWS's grant monitors to prepare a rejection letter, informing applicants
of reasons for rejection when funding is denied and place a copy of the
rejection letter in the official recognition file. We reviewed all 10
official recognition files for the applicants that were eligible to apply
for funding in fiscal year 2002 but were rejected. We found a rejection
letter was missing in 7 of the 10 files.
We also reviewed documents in EOWS's official grant files to assess EOWS's
efforts to comply with EOWS and OJP policies for fully documenting funding
decisions. We randomly selected 10 official grant files to review from the
31 sites that were funded in fiscal year 2002. We found that none of the
10 official grant files fully documented funding decisions for Weed and
Seed sites. For example, the application, a basic component of the
official grant file, was missing in 8 of the 10 files.
Challenges Persist in Several directives require that grant documentation
be readily available. Making Documentation OJP's policies and procedures
manual requires that documentation be Readily Available kept readily
available in the official recognition and official grant files, so
that OJP and EOWS management can identify and resolve any problems or
10EOWS uses the information in the documents as the basis to award
official recognition.
deficiencies in grantees' compliance with relevant policies and
procedures. OJP's policies and procedures manual requires that all
pertinent information that should be in the official recognition and the
official grant files be kept in a centralized location to facilitate
reviewing for completeness. In addition, the Comptroller General's
guidance on internal controls in the federal government, Standards for
Internal Control in the Federal Government, requires that the
documentation for all transactions and significant events be readily
available for examination. The Assistant Attorney General for OJP is
responsible for holding EOWS accountable for following its policies and
procedures. In reference to the ratings information and rejection letters
missing from the official recognition files, EOWS produced the documents
after our initial review, informing us that the documents were kept in the
personal files of the grant monitors--rather than the official recognition
files--a practice not in keeping with EOWS's and OJP's policies and
procedures. In reference to the missing applications, EOWS officials also
produced them after our initial review, but did not specify where they
found them. Not having complete and readily available official recognition
and official grant files may delay and make it more difficult for OJP and
EOWS officials to perform management functions such as overseeing and
assessing the documentation and oversight of qualification and funding
decisions. While we eventually were able to obtain documentation showing
the basis and rationale for EOWS decision making, the incomplete official
recognition and official grant files made the review difficult and
time-consuming.
In responding to a draft of this report, Justice stated that electronic
documents maintained in its grant management system (GMS) are considered a
part of the official recognition and official grant files and are readily
available to EOWS, and that they should have been readily available to,
and easily accessible by, GAO.
In response to our 1999 recommendation, EOWS has taken steps to improve
program monitoring, as with the documentation of qualification and funding
decisions, but challenges remain in making the monitoring documentation
readily available. While OJP requires progress reports and site visit
reports to be included in the official grant files, our review in 2003 of
10 official grant files showed that some of EOWS's files were missing full
documentation. However, before our review ended, EOWS provided further
documentation that fully documented progress reports and site visits,
which it acknowledges was not in the official grant files, and thus not
readily available. This lack of readily available documentation is not in
keeping with OJP's policies and procedures. Not having complete and
readily available official grant files may delay and complicate EOWS
EOWS Has Taken Steps to Improve Program-Monitoring Documentation, but
Challenges Remain
officials' and external reviewers' assessment of whether EOWS's monitoring
requirements are being followed.
Official Grant Files Were Incomplete
Despite EOWS's efforts, some of the official grant files we reviewed were
incomplete at the time of our file review. We randomly selected 10 files
from the 31 sites that were first funded in fiscal year 200211 to
determine whether the sites had submitted the required progress reports
from January 1, 2003, through June 30, 2003.12 Of the 10 files we
reviewed, 6 did not contain any of the required progress reports for the
period. The remaining 4 files included the required progress reports.
OJP's policies and procedures require EOWS to ensure that progress reports
are included in the official grant files. However, when we asked EOWS
about the missing progress reports, EOWS provided us with the requested
documents. We asked, but EOWS did not specify where it found the missing
documentation.
In addition, EOWS's grant monitors are required to conduct site visits a
minimum of every 18 months and document their visits. This documentation
is meant to convey to EOWS management officials how well sites are
complying with grant requirements and is to be used by EOWS in making
funding decisions. Grant monitors are to prepare a report of the visits
and forward a copy to the Office of the Comptroller for inclusion in the
official grant file. We reviewed the same 10 grant files discussed above
to determine whether site visits were fully documented. We found that a
site visit report was not yet due for 7 of the 10 files. In the remaining
3 files where a site visit was due, none of the 3 files contained
documentation that the site visit had been conducted. We requested to
speak to the responsible grant monitors to determine whether site visits
had been conducted. EOWS officials told us that the monitors were no
longer involved with the program but provided additional documentation for
the 3 files we reviewed. The documentation showed that of the 3 sites,
only 1 visit had been conducted; the other 2 were scheduled but had been
delayed. While we only reviewed 3 files where a site visit was due, the
lack
11The remaining 187 of the 218 sites received funding in fiscal year 2002,
but were initially funded prior to fiscal year 2002.
12Progress reports describe Weed and Seed activities and the
accomplishment of objectives in a site's funding application. Progress
reports help EOWS officials determine how sites are meeting program
objectives and assist them in making future grant decisions. Progress
reports are due 30 days after June 30 and December 31, respectively.
of documentation in all 3 files, as well as the fact that only 1 of 3 site
visits had been conducted, added to our concerns about the completeness of
official grant files for program oversight.
Challenges Persist in Making Monitoring Documentation Readily Available
Despite Some Steps, Challenges Remain in Developing Criteria for
Self-Sustainability
As we have previously mentioned, OJP's policies and procedures manual and
the Comptroller General's guidance on internal controls require that
documentation should be kept readily available for examination. Subsequent
to our finding missing progress reports and site visit reports, EOWS
provided us with additional documentation that demonstrated that the
progress reports and one of the site visit reports had been completed.
When asked about where they found the missing progress reports and site
visit reports, EOWS officials did not specify where they found the missing
documentation.
EOWS has established a rule and set some activities for sites to complete
to encourage them to become self-sustaining. However, it has still not
fully developed criteria to determine when sites have become
self-sustaining and when to reduce or withdraw Weed and Seed grant funds,
as we recommended in 1999. To become self-sustaining, sites must leverage
additional resources from sources other than EOWS to sustain the Weed and
Seed program without EOWS funds. We found that EOWS has not reduced or
withdrawn funds from any Weed and Seed sites for reasons related to
becoming self-sustaining. This is important because without overall
program funding increases, new sites cannot be funded unless funding can
be reduced or discontinued from sites that have achieved
selfsustainability.
EOWS Encourages Self-Sustainability, yet Needs Criteria to Make Funding
Decisions
In 1999, EOWS established a rule to encourage self-sustainability that was
generally intended to limit Weed and Seed funding to a site to 5 calendar
years and require grantees to shift the majority of EOWS funds to a
different site after 5 years.13 According to EOWS officials, EOWS also
encouraged sites to become self-sustaining by establishing activities to
be completed during each year of the 5-year grant.14 Program monitors are
to assess sites' completion of these activities by recording results on a
checklist.
13Grantees may have more than one site.
14EOWS refers to these activities as benchmarks. For a list of EOWS's
benchmarks, see appendix IV.
We found that while EOWS undertook efforts to encourage
selfsustainability, it has not fully developed criteria to make funding
decisions based on whether sites had achieved self-sustainability. In
fiscal year 2002, 73 sites, which had completed 5 years of EOWS funding,
reapplied for funding. EOWS approved an additional round of 5-year funding
to 67 of these sites. EOWS denied funding to 6 sites, but not for reasons
related to self-sustainability. At the time of our review, no site's
funding had been reduced or withdrawn because the site had achieved
self-sustainability, even though EOWS has funded some sites since the
early 1990s. In relation to the activities EOWS set to encourage sites to
achieve self-sustainability, because these activities were established in
2003, we could not assess the sites' progress toward completing them. As
we reported in 1999, without criteria, EOWS does not have a basis for
determining when sites are selfsustaining and when to reduce or withdraw
Weed and Seed funds.
EOWS officials told us that OJP is currently developing criteria for
selfsustainability for EOWS and other OJP programs and that a report
detailing the criteria may be completed in 2004. According to Justice,
EOWS developed a new criterion for self-sustainability in the FY2003
Competitive Application Kit regarding whether or not a full-time
coordinator is funded by a reallocation of resources other than the Weed
and Seed grant.
For information about proposed legislation that could affect
selfsustainability for Weed and Seed sites, similar to other Justice
programs, see appendix III.
Performance In 1999, we reported15 that while EOWS had developed various
performance measures in an attempt to respond to the Government Measures
Generally Performance and Results Act (GPRA) of 1993,16 EOWS's measures
Did Not Track generally did not track program outcomes.17 GPRA seeks to
shift the focus
of federal management and decision making away from activities Program
Outcomes performed to outcomes, or the results of activities undertaken.
Since our
15See GAO/GGD-99-110.
16P.L. 103-62.
17Performance measures translate program goals into concrete, observable
conditions that determine what data to collect to learn whether progress
has been made toward achieving program goals. Such measures are meant to
cover the key aspects of performance that will enable programs to assess
accomplishments, make decisions, realign processes, and assign
accountability.
last report, EOWS has continued to refine and develop its performance
measures, but we found that EOWS still generally collects site activity
data rather than measuring outcomes. EOWS also generally does not use
intermediate measures, which represent conditions believed to precede or
contribute to achieving the ultimate outcomes that may be considered
constructive steps toward measuring outcomes. While assessing success
using outcomes and/or intermediate measures can be difficult, doing so is
important to program management and to policy makers for understanding
whether the program is achieving the intended results, and for identifying
opportunities for improvement.
EOWS Generally Collects Activity Data Rather Than Outcome Performance
Measures
Our review showed that EOWS generally collects activity data, but has not
developed outcome or intermediate measures that enable EOWS management to
track the success of the Weed and Seed program in meeting its goals.18 As
established in prior work by GAO, outcome measures help officials track
the success of their programs in meeting program goals.19 EOWS mostly
collects data about site activities, such as whether sites have foot
patrols, safe havens, and provide job training. An intermediate measure,
for example in relation to job training programs, might be the extent to
which those who attend job training obtain employment. This intermediate
measure rests on the assumption that individuals who are employed are less
likely to commit crimes. See appendix V for activity data collected by
EOWS.
EOWS officials told us that to measure its success in reducing violent
crime, they collect data on the number of homicides and consider homicides
the significant indicator for measuring performance outcomes. However,
using the number of homicides as an outcome measure indicative of program
success is problematic for several reasons. First, because homicides are
relatively rare, even in high crime areas, homicide trends may be too
unstable (fluctuate too much from year to year) to assess the success of
the Weed and Seed program. Second, outcome measures, such as homicides,
can present some methodological challenges
18EOWS officials told us that the Weed and Seed program has three main
goals: (1) reduce violent crime, (2) reduce drug crime, and (3) coordinate
Weed and Seed funds with other resources. Such goals are related to the
overall intent of the program to weed out crime from targeted
neighborhoods and seed them with a variety of programs to prevent crime
from recurring.
19U.S. General Accounting Office, Managing for Results: Analytic
Challenges in Measuring Performance, GAO/HEHS/GGD-97-138 (Washington,
D.C.: May 30, 1997).
because it is difficult to draw a direct causal link between the homicides
and a program's work. Economic trends and other law enforcement
initiatives could also be responsible for the observed outcomes. If
homicides are used as a performance measure, any analysis should attempt
to control for other factors influencing the outcomes. Third, given the
broad nature of this program goal --to reduce violent crime, measuring
outcomes in relation to only one type of violent crime also seems
problematic because it is too narrowly focused. According to EOWS
officials, homicide data was selected because of its reliability, and
because gang-related homicides are a significant indicator of the success
of a Weed and Seed strategy.
In February 2004,the Office of Management and Budget (OMB) issued a
detailed assessment of the Weed and Seed program.20 According to OMB, Weed
and Seed has not demonstrated results. OMB's report states that Weed and
Seed lacks clear targets or timelines for reducing violent crime.
Additionally, the report states that Weed and Seed has difficulty
collecting data and developing aggregate measures for assessing program
performance.
GAO Has Previously Reported on How Other Federal Programs Have Developed
Effective Performance Measures, Despite Challenges
Although the Weed and Seed program faces many challenges in developing
outcome measures for its various activities, we have previously reported
that other federal programs have developed effective performance measures
even under difficult circumstances.21 These reports identified a variety
of strategies that other federal programs have used to develop performance
measures. For example, we reported that some federal programs utilized a
mix of outcome and intermediate measures.22 This combination of measures
allowed them to minimize the risk of not showing outcomes because of their
limited control over external factors such as economic trends, which may
prevent programs from achieving intended outcomes. Intermediate measures
are also helpful to show progress when it is expected to take many years
before the desired outcome is likely to be achieved. In such instances,
progress toward program outcomes may be demonstrated through intermediate
outcomes.
20OMB periodically assesses goals and results of federal programs, such as
Weed and Seed, and reports on its findings. The purpose is to tie
performance with the budget process.
21U.S. General Accounting Office, Managing for Results: Measuring Program
Results That Are Under Limited Federal Control,GAO/GGD-99-16 (Washington,
D.C.: December 11, 1998) and GAO/HEHS/GGD-97-138.
22See GAO/GGD-99-16.
We also reported that where measuring outcomes is, after careful
consideration, deemed infeasible, intermediate measures can be used to
track progress toward outcomes for programs such as Weed and Seed.23
In addition, we have reported other ways federal agencies have used
intermediate outcomes.24 For example, the National Highway Traffic Safety
Administration uses the rate of safety belt use as an intermediate measure
of its goal to reduce motor vehicle crashes and the resulting fatalities
and injuries. As we reported, an increase in the use of safety belts is
considered an intermediate outcome--not an end outcome--because it is
desirable not in itself but because it has been demonstrated to contribute
to the ultimate goal--reducing highway-related fatalities and injuries.25
Additional Performance Measures Are Being Developed for EOWS
The Justice Research and Statistics Association and the Urban Institute
are currently developing additional outcome performance measures under
contracts awarded by EOWS in 2002 and 2003. According to EOWS officials,
these research organizations have the expertise needed to develop outcome
performance measures for the Weed and Seed program. We interviewed
researchers from these organizations and spoke to EOWS officials about
their research plans. However, none of the studies on developing
additional performance measures had been completed at the time of our
review, and it is too soon to tell whether the studies will produce
outcome measures needed to adequately assess the Weed and Seed program.26
The Justice Research Statistics Association and other researchers were
conducting studies on crime data and evaluation during our review; one of
those studies was submitted to EOWS in November 2003. See appendix VI for
a description of EOWS's ongoing and completed studies.
23U.S General Accounting Office, Law Enforcement: Better Performance
Measures Needed to Assess Results of Justice's Office of Science and
Technology, GAO-04-198 (Washington, D.C.: November 2003).
24See GAO/GGD-99-16.
25 It is commonly accepted that safety belt use reduces fatality rates and
the severity of injuries. See U.S. General Accounting Office, Highway
Safety: Safety Belts Use Laws Save Lives and Reduce Costs to Society,
GAO/RCED-92-106 (Washington, D.C.: May 1992)
26In November 2003 the Justice Research and Statistics Association
completed a study comparing homicide trends in sites to their host
jurisdictions. However, this study relied on homicide rates collected by
EOWS, rather than developing additional performance measures.
Conclusions
We believe that 5 years has been ample time for OJP and EOWS to fully
implement the recommendations we made in our 1999 report. EOWS has
developed and partially implemented policies requiring the completion of
documentation to support major decisions, such as qualification and
funding decisions, and recorded monitoring information. But by failing to
ensure that these documents are appropriately maintained in official
recognition and grant files, the documents are not readily available to
meet their purpose. That is, the intent of this type of internal control
is to ensure that both management and external reviewers such as auditors
can adequately perform their responsibilities, for example, reviewing work
and making management decisions based on complete and accurate
information. While EOWS was eventually able to produce the documentation
we requested, failure to appropriately file the documentation made the
process of using it very inefficient and, in effect, may have defeated the
purpose of having it completed.
While EOWS may have moved forward in addressing its self-sustainability
goals by recently developing an activity checklist for grantees to use to
document actions that are intended to achieve self-sustainability, the
activities do not constitute criteria for determining when sites should be
considered self-sustaining and consequently have federal funds reduced or
discontinued. One of the benefits of such criteria is to enable EOWS to
determine when current projects are likely to be able to self-sustain so
that available funding can be used to help reduce crime and achieve other
benefits in other deserving communities. Because no sites have had their
funding withdrawn because they were deemed self-sustaining during the
13-year life of this program, EOWS may be foregoing the opportunity to use
the funds in another location where the need for federal funding is
greater or EOWS may need less overall funding. Without criteria to
determine when federal funds are no longer needed, EOWS also runs the risk
of providing funds beyond what is needed to sustain some sites.
EOWS also needs performance measures that focus on program outcomes,
and/or achievement of intermediate goals, so that it and those that
provide oversight, such as the Congress and OMB, will be able to
adequately assess the extent to which the program is achieving its goals.
Management depends on this type of assessment to make the strategic and
operations decisions needed to achieve the program's missions and goals.
Congress and oversight agencies need this type of assessment so they can
make funding decisions and help ensure that EOWS is in the best position
and has the best tools to accomplish its mission. While EOWS recognizes
the need for outcome performance measures and has funded studies to help
Recommendations for Executive Action
Agency Comments and Our Evaluation
develop them, it is unclear at this time whether these studies will be
able to suggest the needed measures.
We recommend that the Attorney General of the United States require the
Assistant Attorney General for OJP to ensure that the Executive Office for
Weed and Seed fully implement the intent of our previous recommendations
by taking the following four steps:
o maintain the documentation of the basis and rationale for
qualification and funding decisions in appropriate grant files;
o retain progress reports and site visit reports in official grant files;
o clearly define criteria to assess when sites are self-sustaining and
apply the criteria to sites when making further funding decisions; and
o develop outcome performance measures--or, where measuring outcome is,
after careful consideration, deemed infeasible, intermediate
measures--that can be used to adequately track progress toward program
outcomes of the Weed and Seed program.
We provided a draft of this report to the Attorney General of the United
States for review and comment. In a March 11, 2004, letter, the Assistant
Attorney General (AAG) for OJP commented on the draft. Her written
comments are summarized below and presented in their entirety in appendix
VII. Based on Justice's comments, we modified the text of our report where
appropriate and also incorporated Justice's technical comments into this
report where appropriate.
In the AAG's response, the Justice Department generally agreed with our
recommendations on maintaining the documentation of the basis and
rationale for qualification and funding decisions in appropriate grant
files and retaining progress reports and site visit reports in official
grant files. Justice agreed in part with our recommendation on
self-sustainability criteria and fully agreed with our recommendation on
outcome performance measures, and offered additional information to show
it had moved forward in responding to our past recommendation.
Justice agreed that controls should be strengthened to ensure that the
basis and rationale for qualification and funding decisions are documented
in the centrally maintained official recognition (OR) files. In Justice's
comments, the AAG explained that electronic documents maintained in its
grant management system (GMS) are considered a part of the official
recognition file and are readily available to EOWS, and that they should
have been, but were not, readily available to, and easily accessible by,
GAO. As of the current fiscal year, OJP will not only post all
solicitations
and receive all grant applications via GMS, but also receive and maintain
all grant progress reports on GMS. In this way, there will be no
opportunity to misplace official grant documents. Justice stated that
OJP's Office of the Comptroller will implement procedures to ensure that
external reviewers have ready access to GMS information.
Justice agreed with our second recommendation on retaining progress
reports and site visit reports in official grant files. Justice noted that
EOWS recently implemented a progress reporting module in GMS that now
enables recipients of grants awarded through GMS to submit semi annual
progress reports electronically. OJP is developing a monitoring module
that will enable program monitors to record and report on site visits
directly in GMS. In the interim, EOWS and the Office of the Comptroller
will take steps to strengthen controls to ensure that copies of completed
site visit reports are included in the official grant file.
Justice agreed in part with our third recommendation on
self-sustainability and noted that EOWS will further define the criteria
to assess when sites are self-sustaining and apply the criteria when
making funding decisions. Further, Justice said it is promoting a
"graduation" process to bring sites to sustainability and cease providing
Weed and Seed funding to those sites, thus making funding available to
assist newer developing sites. However, Justice does not agree that EOWS
has never set criteria for selfsustainability. Justice said that EOWS set
a criterion for self-sustainability in the FY2003 Weed and Seed
Competitive Application Kit, which asked if the application provided for a
full-time coordinator funded by reallocation of existing resources other
than the Weed and Seed grant. According to Justice, applications were
ranked against this criterion. According to EOWS officials, in March 2004,
EOWS will announce an additional criterion specific to measuring
self-sustainability. The new criterion will require grant applicants to
identify other funding sources at a level five times the EOWS
contribution. This criterion will be considered when making funding
decisions.
We applaud EOWS's commitment to further develop criteria to assess when
sites are self-sustaining and apply the criteria to sites when making
further funding decisions. However, Justice's comments did not provide
specific information about how the Application Kit criterion has been used
to determine self-sustainability. We added Justice's belief that this
criterion relates to self-sustainability to the text, but also note that
over the 13-year history of the program, funding has never been reduced or
withdrawn from a site because a site was deemed to be self-sustaining.
Justice agrees that developing outcome performance measures is important
and states that EOWS has been working to develop measures that track
progress toward program outcomes in the Weed and Seed Program. In a letter
commenting on this draft report, Justice provided a summary of results
from JRSA's study to support EOWS's use of the change in homicides as a
measure of program success. According to Justice's summary, this study
concluded that measuring reductions of homicides is an adequate
performance measure of the Weed and Seed program because of its
reliability and explicit selection as a goal by over 30 percent of the
Weed and Seed sites participating in the Crime Pattern Study. In addition,
Justice includes a 5 percent reduction in homicides in Weed and Seed sites
as part of its Strategic Plan. With its comments, Justice included
summaries of two recent research reports. One of these, Analysis of
Homicide in Weed and Seed Sites, showed positive results in homicide
trends in Weed and Seed sites when compared to host jurisdictions. Another
report, Crime Pattern Analysis (Three-Top Crime Study), showed that after
the third year of Weed and Seed program implementation, sites encountered
a significant decrease in reported crimes.
Because the Crime Pattern Analysis study was not completed at the time of
our review, we could not fully assess its methodological rigor and whether
it would result in effective outcome performance measures. Although we
believe that EOWS may be moving in the right direction regarding
developing additional performance measures and we mention the studies in
the report text, we also believe that homicides have shortcomings as
described in the text when used as the only measure of the program's
performance. In addition, our review of the Analysis of Homicide in Local
Weed and Seed Sites study generated several concerns. First, the study
used changes in the raw volume of homicides, instead of changes in
homicide rates, as its indicator of success or failure of program sites.
This method does not account for changes in the population as would be
accounted for were a homicide rate measured. Second, the study used the
larger host jurisdictions for specific Weed and Seed sites as the
comparison locations to which the Weed and Seed sites are compared. In
doing so, Weed and Seed sites' homicide data are included with host
jurisdictions' homicide data, thus making the differences between them
extremely difficult to interpret. It is also worth noting that this study
evaluates Weed and Seed sites only on the basis of decreases in homicides,
while Weed and Seed was intended to decrease crime more generally.
We are sending copies of this report to the Attorney General, appropriate
congressional committees, and other interested parties. We will also make
copies available to others upon request. In addition, the report will be
available at no charge on GAO's Web site at http://www.gao.gov. If you or
your staff has any questions on this report, please call Weldon McPhail on
(202) 512-8644 or me on (202) 512-8777.
Sincerely yours,
Laurie E. Ekstrand, Director Homeland Security and Justice Issues
Appendix I: Scope and Methodology
To determine what actions the Executive Office for Weed and Seed (EOWS)
has taken to develop an adequate internal control to ensure full
documentation of the basis and rationale for qualifying new and existing
sites for funding, we reviewed (1) the criteria used to determine which
new and existing sites should be qualified for funding, and (2) EOWS's
policies and guidance. To gather this information, we interviewed
officials from Justice and EOWS and reviewed pertinent documents,
including EOWS policies and procedures, official recognition and grant
applications, management oversight and monitoring documentation, and
budget reports. In addition, we randomly selected 10 of 94 fiscal year
2002 official recognition files submitted to EOWS for review. These 10
files included 5 files from new applicants and 5 files from existing sites
that applied in 2002 and were funded in fiscal year 2002. We reviewed 10
of the 14 official recognition files that EOWS decided not to fund in
fiscal year 2002. We also reviewed 10 randomly selected official grant
files from the 31 new and existing sites that were approved for funding in
fiscal year 2002. Because of limited resources, we did not attempt to
review all the files. While the small sample size prevents us from making
reliable generalizations, the 30 files we reviewed represented various
types of files that EOWS maintains. We selected the files from a list that
was provided to us by EOWS. To ensure the list of files was sufficient for
the purpose of our review, we spoke to knowledgeable EOWS officials about
the completeness and accuracy of the list.
To assess what steps EOWS has taken to improve program monitoring to
ensure that sites meet the grant requirement of submitting progress
reports, and that EOWS site visits are documented, we reviewed EOWS
program grant guidance, including training offered, the EOWS monitoring
guidance used by grant monitors when conducting site visits, and the grant
files for the Weed and Seed sites that we visited: Pittsburgh,
Pennsylvania; Charleston, South Carolina; and North Charleston, South
Carolina. We selected these sites from the 221 sites funded by EOWS in
fiscal year 2003. Based on our discussions with EOWS officials about the
sites, we selected three sites, which although not representative of all
sites, had received Weed and Seed grant funds since the early 1990s, which
enabled us to learn about their efforts to implement the Weed and Seed
strategy over time. We also reviewed selected monitoring visit reports
prepared by grant monitors for these sites and biannual progress reports
submitted in fiscal years 2002 and 2003. We interviewed EOWS officials and
grant coordinators at these 3 sites regarding procedures used for
monitoring Weed and Seed sites. We also reviewed the same 10 randomly
selected official grant files discussed above from the 31 new and existing
sites that were funded in fiscal year 2002 to determine whether the
progress reports
Appendix I: Scope and Methodology
and site visit reports were in the official grant files and readily
available. The sites were funded under EOWS's competitive application and
did not include sites funded under EOWS's continuation application.1 While
the 10 files we reviewed represented various types of files that EOWS
maintains, the small sample size prevents us from making reliable
generalizations about all official grant files. To ensure the list of
files was sufficient for the purpose of our review, we spoke to a
knowledgeable official about the completeness and accuracy of the list.
To assess what criteria, if any, EOWS has developed when sites are
selfsustaining and when to reduce or withdraw program funding, we asked
EOWS officials for any relevant information. Following repeated requests,
EOWS officials did not provide any documentation on criteria that they had
developed. They did, however, provide information on the five-year rule
for self-sustainability and the sites' activities, which we reviewed.
To determine what additional performance measures EOWS has developed that
track program outcomes, we interviewed officials from EOWS and the sites
we visited. We reviewed pertinent documents, including EOWS policies and
procedures, grant applications, and data collected pursuant to the
Government Performance and Results Act of 1993 (GPRA). In relation to the
studies EOWS commissioned to develop program-wide performance measures, we
interviewed the authors of the studies under way to discuss their research
plans. At the time of our review, none of the studies on developing
additional performance measures had been completed.
1Sites funded under EOWS's competitive application can be either the first
Weed and Seed site in the jurisdiction that received official recognition
or a new geographical area (that is, not contiguous with an existing or
former target area) in a jurisdiction with an existing (or continuation)
or former Weed and Seed site. Sites funded under EOWS's continuation
application are in their second, third, fourth, or fifth year of funding
and implementing the Weed and Seed program.
Appendix I: Scope and Methodology
We conducted our audit work between June and December 2003 in accordance
with generally accepted government auditing standards.
Appendix II: GAO Site Visit Summary
Table 2: Sites' Weed and Seed Funding History-Fiscal Years 1999-2003
Source: GAO site visits and EOWS data.
aPittsburgh's Weed and Seed funds for fiscal year 2002 were amended into
the 2001 grant award.
bPittsburgh's Weed and Seed funds for fiscal year 2003 were amended into
the 2001 grant award.
Pittsburgh, Pa. Background
Pittsburgh established its first Weed and Seed site, the Hill District, in
1992. In 1995, Hazelwood became the second Pittsburgh site. In 1997 and
1999, East Liberty and Homewood became the third and fourth Weed and Seed
sites in Pittsburgh. Pittsburgh currently receives funds for three sites:
East Liberty, Hazelwood, and Homewood. The sites are located between the
Allegheny and Monongahela rivers on the eastern side of the city. The
population of the target area is 10,897 in East Liberty, 15,849 in
Hazelwood, and 9,283 in Homewood. In fiscal year 2003, the Pittsburgh
sites in total received $435,000 in Weed and Seed funding. See table 2 for
the Weed and Seed funding history for Pittsburgh's three sites.
Activities
Pittsburgh's weeding activities include funding a drug violence initiative
headed by the Federal Bureau of Investigation (FBI) and a communitybased
crime prevention activity coordinated with the police. Pittsburgh's
seeding activities include funding a drug education youth camp that
provides children with leadership training and fitness tests, and a
downtown revitalization program. See table 3 for examples of the types of
activities funded by the Pittsburgh Weed and Seed program.
Appendix II: GAO Site Visit Summary
Charleston, S.C.
Leveraging Efforts
An important goal of the Weed and Seed program is for sites to leverage
resources from sources other than EOWS in order to become selfsustaining.
During our site visit and through documentation obtained from Pittsburgh
Weed and Seed program staff, we identified several partnerships
established by the Pittsburgh Weed and Seed program to leverage resources.
These cooperative arrangements involved partners such as the University of
Pittsburgh Medical Center and the Pittsburgh Harlequin Rugby Association.
Table 4 provides specific examples of leveraging efforts that we
identified.1 Pittsburgh Weed and Seed program staff estimate that they
coordinated and leveraged approximately $1.1 million in 2002.
Background
Charleston became a Weed and Seed site in the spring of 1992. The
Charleston Weed and Seed target area includes approximately 8 square miles
on the Charleston Peninsula and in the city of Charleston. The entire Weed
and Seed site is located in Charleston's Renewal Community. The population
of the target area, encompassing 13 neighborhoods, is roughly 17,000. In
fiscal year 2003, the site received $275,000 in Weed and Seed funding. See
table 2 for Charleston's Weed and Seed funding history.
Activities
Charleston's weeding activities include supporting a youth court with a
Weed and Seed juvenile investigator and funding community policing foot
patrols. Charleston's seeding activities include supporting a police-youth
after-school boxing program, and boarding up abandoned and vacant
buildings to improve the appearance of the area. See table 3 for examples
of the types of activities funded by the Charleston Weed and Seed program.
Leveraging Efforts
An important goal of the Weed and Seed program is for sites to leverage
resources from sources other than EOWS in order to become selfsustaining.
During our site visit and through documentation obtained from Charleston,
we identified several partnerships established by the
1GAO did not verify the funds leveraged by Weed and Seed sites.
Appendix II: GAO Site Visit Summary
North Charleston, S.C.
Charleston Weed and Seed program to leverage additional resources. These
cooperative arrangements involved partners such as the FBI and the College
of Charleston's School of Education. Table 4 provides specific examples of
leveraging efforts that we identified.2 Charleston Weed and Seed program
staff estimate that they were able to coordinate and leverage
approximately $1 million in 2002.
Background
North Charleston was originally funded as a Weed and Seed site in August
1993. The city of North Charleston is located about 7 miles north of the
city of Charleston. Today, North Charleston has four target neighborhoods:
Union Heights, Chicora/Cherokee, Accabee, and Liberty Hill. The population
of the target area is 13,606. In fiscal year 2003, the site received
$225,000 in Weed and Seed funding. See table 2 for North Charleston's Weed
and Seed funding history.
Activities
North Charleston's weeding activities include participating in Operation
Cease Fire with the U.S. Attorney's Office, to reduce gun crime in target
neighborhoods, and partnering with local law enforcement to improve the
bike patrol program. North Charleston's seeding activities include
developing the Drug Education for Youth program and supporting an annual
neighborhood cleanup. See table 3 for examples of the types of activities
funded by the North Charleston Weed and Seed program.
Leveraging Efforts
An important goal of the Weed and Seed program is for sites to leverage
resources from sources other than EOWS in order to become selfsustaining.
During our site visit and through documentation obtained from North
Charleston, we identified several partnerships established by the North
Charleston Weed and Seed program to leverage resources. These cooperative
arrangements involved partners such as the United States Air Force and the
Medical University of South Carolina. Table 4 provides specific examples
of leveraging efforts that we identified.3 North
Charleston Weed and Seed program staff estimate that they coordinated and
leveraged approximately $3.3 million in 2002.
2GAO did not verify the funds leveraged by Weed and Seed sites. 3GAO did
not verify funds leveraged by Weed and Seed sites.
Appendix II: GAO Site Visit Summary
Table 3: Examples of Activities Funded by Weed and Seed in Pittsburgh,
Charleston, and North Charleston
Program
Site component Activity Weed and Seed Description
partner
Law enforcement Drug violence
Safe Streets U.S. Attorney's initiative headed by
Pittsburgh, Office, the FBI.
Pa. program Federal Bureau of
Investigation
(FBI)
Community Bike Community Community-based crime
safety residents and prevention activity
policing classes police officers coordinated with the
and police.
registration
About 60 children
Prevention, Drug Boy Scouts of participated in
Education America, leadership training
intervention, and U.S. Air Force and fitness tests.
For Youth
treatment (DEFY)
Camp
Hazelwood A community-wide effort
Neighborhood Initiative, Inc., to reestablish the
Improved local
and the Second business district,
restoration housing Avenue revitalize a 200-acre
area, reclaim
Main Street historic buildings, and
Program build new housing.
Law enforcement First-time juvenile
Charleston, Youth Court South Carolina Bar offenders are tried by
their peers
S.C. Association, Young and given an opportunity
to improve their behavior.
Lawyers Division,
Charleston
Enterprise
Community
Community Foot patrols Charleston Weed Weed and Seed officers
and patrol the community on
policing Seed Patrol foot.
Officers,
Charleston Police
Department
Police Participants include
Athletic Police Athletic junior and senior
Prevention, League boxers and
intervention, and League Boxing coaches.
treatment
Neighborhood Boarding up Charleston Police The city boarded up abandoned
and vacant
restoration abandoned and Department, Maintenance buildings in Weed and
Seed neighborhoods. vacant buildings Division
U.S. Attorney's Office, A gun
North Law enforcement Cease Fire control program that began in
January 2003
Charleston, North Charleston Police to reduce
program gun crime in target areas.
S.C. Department
North Charleston Police Two
officers on bike patrol are
Community Bike patrol dedicated
policing Department exclusively to the Weed
and Seed target area.
Charleston Air Force Base Weed and
Prevention, DEFY program Seed staff was developing a DEFY
intervention, and program for participants from the
Weed and Seed
treatment target area.
Neighborhood residents An annual
neighborhood cleanup was held in
Neighborhood Neighborhood April
restoration cleanup 2003.
Source: GAO site visits and EOWS.
Appendix II: GAO Site Visit Summary
Table 4: Examples of Leveraging Efforts at Weed and Seed Sites in Pittsburgh,
Charleston, and North Charleston
Type of Name of partner
partnership or
Weed and Seed or cooperative cooperative
site arrangement arrangement Description
The FBI leads the Safe
Federal FBI Streets program in
Pittsburgh, Pa. government Pittsburgh to
reduce street gang and
drug related violence.
State Police officers
are assisting the U.S.
State government State police Attorney's Office
in implementing
Justice's Project Safe
Neighborhoods (PSN)
initiative to reduce gun
violence.
Local government, Allegheny County Health In March and April 2000,
Pittsburgh began a partnership with
university Department (ACHD) and ACHD, UPMC, and others to conduct medical
and dental University of Pittsburgh screening at community events. The
first major effort of the Medical Center (UPMC) partnership was a lead
abatement project, which identified and cleaned contaminated dwellings.
The Pittsburgh Urban Redevelopment Authority and the Housing Authority
also participated in this effort.
Pittsburgh The partnership with
Local community Harlequins the Harlequins began in
March 2001.
The Harlequins
Rugby Association established two teams
of 20 children to play
touch rugby while also
incorporating training
in teamwork,
partnership, and
discipline into the
sport.
The FBI leads the
Charleston, S.C. Federal FBI Charleston Safe Streets
government Task Force to
reduce drug-related
activities.
Police officers train
Local Charleston Police youth in law
government enforcement procedures
and tactics as well as
Department physical training. The
goal of this
program is to instill
responsibility in
participants and to
prepare them for
possible careers in law
enforcement.
Graduate interns tutor
College of students from Safe
Local college Charleston's Havens.
School of
Education
The DEFY program will
North Federal U.S. Air Force be conducted
Charleston, government year-round, through a
S.C. partnership with the
Air Force.
North Charleston Police officers
Local Police instruct youth on crime
government prevention during
Department school breaks at Safe
Havens.
Through this
partnership,
Local Medical information was
university University of distributed to
South Carolina Hispanic families in
Crime the Weed and Seed
target area about a
program designed to
Victims Center provide medical
services to Hispanic
families.
Source: GAO site visits and EOWS.
Note: Grantees also receive funds from other federal programs and
non-federal funds.
Appendix III: Proposed Legislation Could Affect Self-Sustainability
Proposed legislation could affect self-sustainability for Weed and Seed
sites by mandating that EOWS impose more rigorous requirements on its
grantees, similar to some other Justice programs. Unlike some other
Justice discretionary grant programs, the law establishing the Weed and
Seed program currently does not have a matching funding requirement,1 nor
does it limit the maximum number of years a grantee can receive funding.2
H.R. 3036, a bill to reauthorize the Department of Justice for fiscal
years 2004 through 2006, was introduced on September 9, 2003. This bill
includes a provision that would limit the federal contribution to 75
percent of the total Weed and Seed program costs. The bill would also
impose limitations on the duration of grants funded under the Weed and
Seed program, limiting to 10 the total number of fiscal years a grantee
may receive grants. Additionally, the bill requires that in order for a
grantee to be eligible for a grant, it must agree to formulate a timely
and effective plan to independently sustain the Weed and Seed program when
federal funding ends.
Currently there are other federal grant programs that require a funding
match and a limit on the duration of grant awards.3 For example, the
Drug-Free Communities Support Program (DFCSP) and Community Oriented
Policing Services (COPS) both have matching requirements and clearly
specify the number of years a grantee may be funded. DFCSP is a
discretionary grant program with a budget of about $60 million in fiscal
year 2003. DFCSP requires a dollar-for-dollar match, limits grantees to no
more than 5 years of funding, and reduces awards by 25 percent in the
fourth and fifth years. The COPS program, with a budget of about $929
million in fiscal year 2003, generally requires grantees to contribute
1A matching requirement requires grant recipients to contribute their own
funds to obtain federal grant funds. The Weed and Seed program is
authorized and funded by the Department of Justice annual appropriation
acts. These acts have not included either a requirement for matching funds
or a limitation on the maximum number of years a grantee may receive
funding. See for example, Departments of Commerce, Justice, and State, the
Judiciary, and Related Agencies Appropriations Act, 1998, P.L. 105-119,
111 Stat. 2440, 2454.
2As discussed earlier, beginning in 1999, EOWS imposed a 5-year site
expansion rule, under which a site is awarded funding for 5 years and may
receive an additional 5 years of funding if the majority of funds are
shifted to a new site. This is not a requirement imposed by the law.
3U.S. General Accounting Office, Federal Grants: Design Improvements Could
Help Federal Resources Go Further, GAO/AIMD-97-7 (Washington, D.C.:
December 1996).
Appendix III: Proposed Legislation Could Affect Self-Sustainability
25 percent of the costs of the program. The federal share of a COPS grant
that covers more than 1 year must decline year to year.4
4Grants for hiring and rehiring may be renewed for up to 5 years. Grants
for other purposes may not cover more than 3 years.
Appendix IV: Executive Office for Weed and Seed Selected Activities
Year 1 Year 2 Year 3 Year 4 Year 5
The local U.S. The steering committee continually
Attorney's Office educates, guides, and reevaluates
establishes a steering site management.
committee, which
establishes clear
roles and
responsibilities for the
site's organizational
and management
a
structure.
The steering Steering committee meets at least Steering committee -> ->
committee meets monthly. and organizational
monthly and develops structure continue to
a job description for function. Steering
the site coordinator. committee meets at
least quarterly.
The grantee and the The grantee and/or local U.S. ->
site coordinator Attorney's Office evaluates the site
establish procedures coordinator's job performance,
for tracking grant with input from the steering
funds and committee. b
expenditures.
The site coordinator and the steering committee work together to ensure
timely expenditure of grant funds.
Based on the time The site's strategy to achieve The site continues to
line, goals, and measurable outcomes in the four implement strategy
objectives required program areas (especially for achieving goals
established by the law enforcement/community and objectives in all
site's official policing) is on track. The site four required program
recognition,c the site continues to collect data related to elements.
begins implementing goals and objectives.
strategies and
developing baseline
data, addressing the
four required program
elements. d
-> ->
-> The site makes plans to continue community development efforts and
strategic planning through various funding sources.
The site begins The site The site The site The site's
to continues continues continues community
program
publicity and program program leaders are
publicize the outreach to publicity and publicity working to
recruit and
program, to neighborhood outreach outreach sustain efforts
recruit leaders and efforts to efforts to and
neighborhood mobilize recruit mobilize continue
community. neighborhood community community
leaders, and to leaders and to and recruit mobilization.
new
mobilize the mobilize the community
leaders.
community. community
while
identifying
and
building
sustainable
leadership
among
community
members.
Appendix IV: Executive Office for Weed and Seed Selected Activities
Year 1 Year 2 Year 3 Year 4 Year 5
The site The site The Weed and Community The Weed and
begins continues to Seed leaders Seed
coordinate its
coordinating efforts with effort begins have community is
its local to be established included in
initiatives,
efforts with particularly working city/community
related programs with included in relationships planning.
local
local parks and city/community with local
initiatives, recreation plans. officials.
departments.
particularly
law
enforcement
resources and
activities.
The steering committee develops The special emphasis and implements the
special program is fully emphasis program based on operational. community
needs.
The site submits -> -> -> ->
timely grant
expenditure reports to
Executive Office for
Weed and Seed and
Office of Justice
Programs.
The site submits -> -> -> The site submits a final
timely progress progress report.
reports, providing
program outcome
information related to
the achievement of
the goals and
objectives stated in
the official recognition
and grant
applications.
The site communicates regularly -> ->
with all partners, including U.S.
Attorney's Office, Executive Office
for Weed and Seed, and
community members, using
newsletters, faxes, neighborhood
visits, e-mail, community meetings,
and so forth.
The steering -> -> The steering The steering committee
committee and the committee and site determines if there is
a
site coordinator coordinator review need to continue the
evaluate site previous years' grant program in other
progress to determine applications to neighborhoods based on
site's technical determine plans for the program.
assistance needs, if final year of official
any. recognition.
Appendix IV: Executive Office for Weed and Seed Selected Activities
Year 1 Year 2 Year 3 Year 4 Year 5
The site develops -> -> -> ->
operating plans for
next year.
The site begins The site establishes a Subcommittee
developing strategies subcommittee to look continues to search for
for long-term self- for additional funding additional resources to
sustainment. -> or in-kind resources sustain the program. ->
to sustain the
program.
The site begins to develop plans The site identifies an The site undergoes
By the end of the year,
for evaluating its programs. evaluation tool and program evaluation to
the site releases its full begins developing an assess overall evaluation
report, evaluation based on effectiveness of official assessing the
overall Executive Office for recognition strategy program Weed and Seed
site implementation. accomplishments and its evaluation literature.
effectiveness.
Source: EOWS data.
Note: Arrow indicates action from the previous year continues to the
current year.
aA site is a geographically defined area ranging in size from several
neighborhood blocks to several square miles. With the input of the U.S.
Attorney's Office and the steering committee, each site develops and
implements its own Weed and Seed strategy.
bA grantee is an entity that receives funding from EOWS to implement the
Weed and Seed program. The grantee distributes the funds to sites and is
responsible for ensuring that sites comply with the terms of the grant. A
grantee may have more than one site.
cOfficial recognition is a designation given to a grantee by EOWS that
signifies that EOWS has approved the grantee's Weed and Seed strategy in a
specific site. A grantee must receive official recognition from EOWS
before a grantee can apply for funding. Official recognition requires,
among other things, that the site develop a management plan and a
comprehensive strategy for implementing the Weed and Seed strategy.
dThe four required program elements that sites are required to address are
(1) law enforcement; (2) community policing; (3) crime and substance
prevention, intervention, and treatment; and (4) neighborhood restoration.
Appendix V: Activity Data Collected by EOWS in Fiscal Year 2003
Activity data collected by EOWS
1. Does site include a multijurisdictional task force
2. Does site participate in Project Safe Neighborhoods
3. Does site have foot patrols
4. Does site have bike patrols
5. Does site have substations
6. Does site have crime watch
7. Do police participate in community meetings
8. Number of homicides for 3 preceding calendar years for Weed
and Seed site
9. Number of homicides for 3 preceding calendar years for entire
jurisdiction
a
10. Number of total drug arrests
11. Number of heroin drug arrests
12. Number of cocaine (not including crack) drug arrests
13. Number of crack cocaine drug arrests
14. Number of marijuana drug arrests
15. Number of methamphetamines only drug arrests
16. Number of safe haven facilities
17. Number of safe havens receiving EOWS funding
18. Does site provide academic courses and tutoring
19. Does site provide mentoring
20. Does site provide prevention education
21. Does site provide dispute resolution and mediation
22. Does site provide recreation and athletics
23. Does site provide job training
24. Does site provide job placement
25. Does site provide antidrug education
26. Does site provide community police co-located in safe
haven
27. Does site provide safe corridors (school escorts for
children)
28. Does site provide summer day camp
29. Does site provide youth leadership training
30. Does site provide boys and girls club programs
31. Does site provide scouting programs
32. Does site provide military cadet training
33. Does site provide antigang education and training
Appendix V: Activity Data Collected by EOWS in Fiscal Year 2003
Activity data collected by EOWS
34. Does site provide Communities in Schools programs
35. Does site provide performance or applied arts programs
36. Does site provide victim assistance programs
37. Does site provide community projects, such as cleanups
38. Does site provide general health-screening services
39. Does site provide lead-poisoning-screening service
40. Number of persons receiving safe haven services
41. Number of community development corporations within
site area
42. Is site constructing and renovating housing
developments
43. Is site constructing and renovating commercial
developments
44. Is site constructing and renovating business and
community partnerships
45. Number of community cleanups done using EOWS
support
46. Do Weed and Seed activities relate to or involve
Brownfields Assessment Demonstration Pilots
47. Do Weed and Seed activities relate to or involve
Brownfields Cleanup and Revolving Load Fund Pilots
48. Do Weed and Seed activities relate to or involve
Brownfields Showcase Communities
49. Do Weed and Seed activities relate to or involve
Brownfields Tax Incentive
50. Source of non-EOWS fundingb
51. Amount of non-EOWS grant funding
52. Source and type of in-kind contributions
Source: EOWS data.
aIn addition to the number of drug arrests, EOWS also requests that each
site report the number of sale, manufacturing, and possession drug arrests
for each type of drug.
bNon-EOWS grant funding includes funding from other federal, state, local,
and private sources.
Appendix VI: Description of EOWS's Ongoing and Completed Studies
Title of study and When results are research partner Purpose of study
expected
A Comparison of Compare homicides in Weed and Seed sites with homicides in
the jurisdictions that Report was expected
Homicide Trends in sites are located in, to determine whether homicides
will decline faster in Weed and in January 2004 but
Local Weed and Seed Seed sites than in the remainder of the jurisdiction.
was issued in
Sites Relative to Their November 2003
Host Jurisdictions,
1996 to 2001
Justice Research and
Statistics Association
(JRSA)
Performance Indicator Develop additional performance indicators that will
be used by EOWS and Weed and March 2004 Study Seed sites to evaluate
program performance with respect to crime control.
Urban Institute
Weed and Seed Crime Summarize the types of crime selected as "target"
crimes by Weed and Seed sites March 2004 Pattern Data Collection (sites
choose three crimes) to determine crime reduction strategies. Assess the
degree of success of Weed and Seed sites by analyzing the percentage of
reduction Justice Research and in crime compared with the percentage in
the rest of the jurisdiction that they are
Statistics Association located in.
Meta-Analysis Summarize the completed evaluations of Mid 2004
individual Weed and Seed sites. The
analysis will include more than 80
Evaluation studies. JRSA's preliminary analysis
(March 3,
2003) includes 36 evaluations. Of the 36
evaluations, JRSA determined that 5 were
Justice Research and process, 19 were impact, and 9 were both
process and impact evaluations. The
Statistics Association remaining 3 evaluations were not
identified as process or impact.
Note: A process evaluation addresses
whether the program is working as
intended.
An impact evaluation isolates the effects
of a particular program or factor from all
other potential contributing factors that
could also effect change.
Local Pilot Evaluations Conduct local evaluations of Weed and Seed sites.
In fiscal year 2002, EOWS funded Mid 2004 13 local evaluations. Each site
contracts with its own researcher.
Local researchers
Source: EOWS and research organizations.
Appendix VII: Comments from the Department of Justice
Appendix VII: Comments from the Department of Justice
Appendix VII: Comments from the Department of Justice
Appendix VII: Comments from the Department of Justice
Appendix VII: Comments from the Department of Justice
Appendix VIII: GAO Contacts and Staff Acknowledgments
GAO Contacts
Staff Acknowledgments
Laurie Ekstrand (202) 512-2758 Weldon McPhail (202) 512-8644
In addition to those named above, the following individuals made key
contributions to this report: Lisa G. Shibata, Kirstin B. L. Nelson, Cady
L. Summers, David Alexander, Sidney Schwartz, Kevin L. Jackson, Elizabeth
Curda, Tom James, Denise Fantone, Jan Montgomery, Ann H. Finley, Katherine
M. Davis, and Leo Barbour.
Related GAO Products
Law Enforcement: Better Performance Measures Needed to Assess Results of
Justice's Office of Science and Technology, GAO-04-198. Washington, D.C.:
November 14, 2003.
Standards for Internal Control in the Federal Government.
GAO/AIMD-00-21.3.1. Washington, D.C.: November 1999.
Federal Grants: More Can Be Done to Improve Weed and Seed Program
Management. GAO/GGD-99-110. Washington, D.C.: July 16, 1999.
Small Business Administration: Enhancements Needed for Loan Monitoring
System Benchmark Study. GAO/AIMD-99-165. Washington, D.C.: May 14, 1999.
Managing for Results: Measuring Program Results That Are Under Limited
Federal Control. GAO/GGD-99-16. Washington, D.C.: December 11, 1998.
Managing for Results: Analytic Challenges in Measuring Performance.
GAO/HEHS/GGD-97-138. Washington, D.C.: May 30, 1997.
Federal Grants: Design Improvements Could Help Federal Resources Go
Further. GAO/AIMD-97-7. Washington, D.C.: December 18, 1996.
Highway Safety: Safety Belts Use Laws Save Lives and Reduce Costs to
Society. GAO/RCED-92-106. Washington, D.C.: May 1992.
GAO's Mission
Obtaining Copies of GAO Reports and Testimony
The General Accounting Office, the audit, evaluation and investigative arm
of Congress, exists to support Congress in meeting its constitutional
responsibilities and to help improve the performance and accountability of
the federal government for the American people. GAO examines the use of
public funds; evaluates federal programs and policies; and provides
analyses, recommendations, and other assistance to help Congress make
informed oversight, policy, and funding decisions. GAO's commitment to
good government is reflected in its core values of accountability,
integrity, and reliability.
The fastest and easiest way to obtain copies of GAO documents at no cost
is through the Internet. GAO's Web site (www.gao.gov) contains abstracts
and fulltext files of current reports and testimony and an expanding
archive of older products. The Web site features a search engine to help
you locate documents using key words and phrases. You can print these
documents in their entirety, including charts and other graphics.
Each day, GAO issues a list of newly released reports, testimony, and
correspondence. GAO posts this list, known as "Today's Reports," on its
Web site daily. The list contains links to the full-text document files.
To have GAO e-mail this list to you every afternoon, go to www.gao.gov and
select "Subscribe to e-mail alerts" under the "Order GAO Products"
heading.
Order by Mail or Phone The first copy of each printed report is free.
Additional copies are $2 each. A check or money order should be made out
to the Superintendent of Documents. GAO also accepts VISA and Mastercard.
Orders for 100 or more copies mailed to a single address are discounted 25
percent. Orders should be sent to:
U.S. General Accounting Office 441 G Street NW, Room LM Washington, D.C.
20548
To order by Phone: Voice: (202) 512-6000 TDD: (202) 512-2537 Fax: (202)
512-6061
To Report Fraud, Contact: Web site: www.gao.gov/fraudnet/fraudnet.htm
Waste, and Abuse in E-mail: [email protected]
Federal Programs Automated answering system: (800) 424-5454 or (202)
512-7470
Jeff Nelligan, Managing Director, [email protected] (202) 512-4800
Public Affairs U.S. General Accounting Office, 441 G Street NW, Room 7149
Washington, D.C. 20548
*** End of document. ***