    Grants Management: Despite Efforts to Improve Weed and Seed
    Program Management, Challenges Remain (24-MAR-04, GAO-04-245).

    The Weed and Seed program, within the Department of Justice's
    Office of Justice Programs (OJP), aims to prevent and reduce
    violent crime in targeted neighborhoods, but it cannot optimize
    its effectiveness without sound management practices. In 1999,
    GAO made four recommendations to the Executive Office for Weed
    and Seed (EOWS) to improve the program's management, including
    (1) developing adequate internal controls to fully document
    decisions, (2) improving program monitoring, (3) developing
    criteria for determining when sites have become self-sustaining
    and when to reduce or withdraw program funding, and (4)
    developing additional performance measures. GAO did this study to
    assess progress in implementing these recommendations.
    -------------------------Indexing Terms-------------------------
    REPORTNUM:   GAO-04-245
        ACCNO:   A09559
      TITLE:     Grants Management: Despite Efforts to Improve Weed and
    Seed Program Management, Challenges Remain
         DATE:   03/24/2004
      SUBJECT:   Community development programs
    	     Crime prevention
    	     Internal controls
    	     Performance measures
    	     Program evaluation
    	     Program management
    	     Records (documents)
    	     Reporting requirements
    	     Evaluation criteria
    	     Program graduation
    	     DOJ Operation Weed and Seed Program

    ******************************************************************
    ** This file contains an ASCII representation of the text of a  **
    ** GAO Product.                                                 **
    **                                                              **
    ** No attempt has been made to display graphic images, although **
    ** figure captions are reproduced.  Tables are included, but    **
    ** may not resemble those in the printed version.               **
    **                                                              **
    ** Please see the PDF (Portable Document Format) file, when     **
    ** available, for a complete electronic file of the printed     **
    ** document's contents.                                         **
    **                                                              **
    ******************************************************************
    GAO-04-245

    United States General Accounting Office

    GAO	Report to the Chairman, Committee on the Judiciary, House of Representatives

    March 2004

    GRANTS MANAGEMENT

     Despite Efforts to Improve Weed and Seed Program Management, Challenges Remain

    GAO-04-245

    Highlights of GAO-04-245, a report to the Chairman, Committee on the
    Judiciary, House of Representatives

    The Weed and Seed program, within the Department of Justice's Office of
    Justice Programs (OJP), aims to prevent and reduce violent crime in
    targeted neighborhoods, but it cannot optimize its effectiveness without
    sound management practices. In 1999, GAO made four recommendations to the
    Executive Office for Weed and Seed (EOWS) to improve the program's
    management, including (1) developing adequate internal controls to fully
    document decisions, (2) improving program monitoring, (3) developing
    criteria for determining when sites have become self-sustaining and when
    to reduce or withdraw program funding, and (4) developing additional
    performance measures. GAO did this study to assess progress in
    implementing these recommendations.

    GAO recommends that the Attorney General require the Assistant Attorney
    General for OJP to ensure that the Executive Office for Weed and Seed
    fully implement the intent of GAO's previous recommendations.

    In commenting on a draft of this report, the Department of Justice agreed
    to strengthen controls on maintaining documentation and take further steps
    to define and apply criteria for self-sustainability. Justice believes
    that the studies currently under way will help develop outcome measures.

    www.gao.gov/cgi-bin/getrpt?GAO-04-245.

    To view the full product, including the scope and methodology, click on
    the link above. For more information, contact Laurie Ekstrand at (202)
    512-8777 or Ekstrandl@gao.gov.

    March 2004

    GRANTS MANAGEMENT

    Despite Efforts to Improve Weed and Seed Program Management, Challenges Remain

    Despite some progress toward addressing GAO's recommendations aimed at
    improving program management, GAO's review shows that EOWS has not fully
    implemented the management improvement recommendations GAO made in 1999.
    First, although EOWS has revised its internal controls to require that
    significant qualification and funding decisions be documented and readily
    available in the central grant files for review, EOWS has not always
    ensured that its policies and procedures were followed, for the grant
    files GAO reviewed. Second, EOWS reported taking a number of actions
    intended to improve program monitoring, such as mandating the timely
    submission of progress reports and adequate recording of site visits as
    GAO recommended. Nonetheless, GAO found that while EOWS was able to
    provide such documentation before its review ended, documentation was not
    available in some of the central grant files GAO reviewed. Thus, the
    documentation was not readily available for external reviewers, as
    required by OJP policies and GAO's internal control standards. Third, GAO
    found that EOWS still lacks fully developed criteria to determine when
    sites become self-sustaining and when to reduce or withdraw Weed and Seed
    funds because of the level of sustainability, even though sustainability
    is a central goal of the program. At the time of GAO's review, no site's
    funding had been reduced or withdrawn because of sustainability during the
    13 years of the program's existence. Fourth, EOWS has not developed
    outcome performance measures that can be used to adequately track progress
    toward program outcomes of the Weed and Seed program. While EOWS has
    initiated studies on how to develop performance measures, at the time of
    GAO's review, none of these studies had been completed. Without
    requirements to monitor improvements and assign accountability, progress
    will be difficult to achieve.

    A sign displayed in front of a school in a Weed and Seed site.

    Contents

      Letter

    Results in Brief
    Background
    Internal Controls Have Been Developed, but Challenges Remain
    EOWS Has Taken Steps to Improve Program-Monitoring

    Documentation, but Challenges Remain
    Despite Some Steps, Challenges Remain in Developing Criteria for
    Self-Sustainability
    Performance Measures Generally Did Not Track Program

    Outcomes
    Conclusions
    Recommendations for Executive Action
    Agency Comments and Our Evaluation

    1

    2 3 7

    9

    11

    12 16 17 17

    Appendix I Scope and Methodology

    Appendix II GAO Site Visit Summary

    Appendix III Proposed Legislation Could Affect Self-Sustainability 29

    Appendix IV	Executive Office for Weed and Seed Selected Activities

    Appendix V Activity Data Collected by EOWS in Fiscal Year 2003 34

    Appendix VI	Description of EOWS's Ongoing and Completed
    Studies 36

      Appendix VII Comments from the Department of Justice 37

    Appendix VIII GAO Contacts and Staff Acknowledgments 42

    GAO Contacts 42 Staff Acknowledgments 42

    Related GAO Products

      Tables

    Table 1: EOWS Funding History for Fiscal Years 1999-2003 4 Table 2: Sites'
    Weed and Seed Funding History-Fiscal Years 19992003 24 Table 3: Examples
    of Activities Funded by Weed and Seed in Pittsburgh, Charleston, and North
    Charleston 27 Table 4: Examples of Leveraging Efforts at Weed and Seed
    Sites in Pittsburgh, Charleston, and North Charleston 28

    Abbreviations

    AAG Assistant Attorney General
    COPS Community Oriented Policing Services
    DFCSP Drug-Free Communities Support Program
    EOWS Executive Office for Weed and Seed
    FBI Federal Bureau of Investigation
    FY fiscal year
    GMS grant management system
    GPRA Government Performance and Results Act
    HUD Housing and Urban Development
    JRSA Justice Research and Statistics Association
    OJP Office of Justice Programs
    OMB Office of Management and Budget
    OR official recognition

    This is a work of the U.S. government and is not subject to copyright
    protection in the United States. It may be reproduced and distributed in
    its entirety without further permission from GAO. However, because this
    work may contain copyrighted images or other material, permission from the
    copyright holder may be necessary if you wish to reproduce this material
    separately.

    United States General Accounting Office Washington, DC 20548

    March 24, 2004

    The Honorable F. James Sensenbrenner, Jr.
    Chairman
    Committee on the Judiciary
    House of Representatives

    Dear Mr. Chairman:

    The Department of Justice's (Justice) Weed and Seed program proposes to
    "weed" out crime from targeted neighborhoods and "seed" them with a
    variety of programs and resources to prevent crime from recurring. Weed
    and Seed sites have sponsored activities such as police bike patrols,
    community cleanups, youth and recreational activities, and computer
    training. According to Justice, a central tenet of the Weed and Seed
    program is for local Weed and Seed sites to develop partnerships with
    other federal, state, and local governments and private sector agencies to
    leverage federal Weed and Seed grant funds with additional resources
    from these partners to promote weeding and seeding activities. These
    additional resources are intended to help the sites achieve the goal of
    becoming self-sustaining after Weed and Seed funding ends and to sustain
    crime reduction and community revitalization activities to ensure stable
    communities.

    This report responds to your request to assess the Executive Office for
    Weed and Seed's (EOWS) efforts to implement the management
    improvement recommendations we made in 1999.1 It is not intended to
    evaluate the overall management or results of the program. In our
    previous report on the Weed and Seed program, we recommended that
    EOWS improve its management of the Weed and Seed program.
    Specifically, we recommended that EOWS (1) develop adequate internal
    controls to ensure that the basis and the rationale for new and existing
    site
    qualification and funding decisions are always fully documented,
    (2) improve program monitoring to ensure that sites meet the grant
    requirement of submitting progress reports and that EOWS site visits are
    documented, (3) develop criteria for determining when sites are self

    1U.S. General Accounting Office, Federal Grants: More Can Be Done to
    Improve Weed and Seed Program Management, GAO/GGD-99-110 (Washington,
    D.C.: July 1999).

    sustaining and when to reduce or withdraw program funding, and (4) develop
    additional performance measures that track program outcomes. This report
    summarizes our assessment of EOWS's efforts to address our
    recommendations.

    To obtain information on EOWS's efforts, we reviewed relevant documents,
    including EOWS's policies and procedures, monitoring documentation, and
    agency staffing and budget data. We reviewed GAO, Congressional Research
    Service, and Justice's Office of Inspector General reports. In addition,
    we interviewed officials from EOWS, the Office of Justice Programs, other
    entities in the Justice Department, and researchers in performance
    measurement and evaluation at the Justice Research and Statistics
    Association and the Urban Institute in Washington, D.C. We visited and
    interviewed program staff at three Weed and Seed sites. We reviewed 30
    randomly selected Weed and Seed grant files to identify the actions taken
    by EOWS for those grants to document its qualification and funding
    decisions and the steps EOWS took to ensure that grant documentation
    requirements were met. Since the files we reviewed were not representative
    of all EOWS grant files, we cannot project the results to the larger
    population. However, the information helps to identify the level of
    implementation of our recommendations for those grants. Additional
    information about the report's scope and methodology is presented in
    appendix I. We conducted this engagement in accordance with generally
    accepted government auditing standards.

    Despite some progress toward addressing our recommendations aimed at
    improving program management, our review shows that EOWS has not fully
    implemented any of the management improvement recommendations we made in
    1999. First, EOWS has revised its internal controls to require that
    significant qualification and funding decisions be documented and readily
    available in two types of centralized files: official recognition and
    official grant files. However, EOWS has not always ensured that the
    documentation was readily available and that its policies and procedures
    for internal controls were followed for the official recognition and
    official grant files we reviewed. Second, EOWS reported to us that it had
    taken a number of actions intended to improve program monitoring, such as
    mandating the timely submission of progress reports and adequate recording
    of site visits, as we recommended. Nonetheless, we found that while EOWS
    was able to provide such documentation before our review ended,
    documentation was not available in some of the central grant files we
    reviewed. Thus, the documentation was not readily available for EOWS
    management or external reviewers, as required by the Office of Justice
    Programs (OJP) policies and GAO's internal control standards. Third, we

      Results in Brief

    found that EOWS still lacks fully developed criteria we recommended that
    they develop to determine when sites have become self-sustaining and when
    to reduce or withdraw Weed and Seed funds because of the level of
    sustainability, even though sustainability is a central goal of the
    program. At the time of our review, no site's funding had been reduced or
    withdrawn because of sustainability during the 13 years of the program's
    existence. Fourth, EOWS began developing additional performance measures
    to better assess how well sites are meeting program objectives, as we
    recommended in 1999. However, our work showed that although EOWS collected
    data on a variety of activities taking place at Weed and Seed sites, they
    generally did not measure the extent to which grantees were weeding crime
    from neighborhoods and preventing it from recurring. While EOWS has
    initiated studies on how to develop additional performance measures, at
    the time of our review, none of these studies had been completed.

    To further improve program management, we make a recommendation to the
    Attorney General for four actions to help ensure full implementation of
    the recommendations we made in our 1999 report. In commenting on a draft
    of this report, Justice partially agreed with our recommendations and
    mentioned actions being taken or planned in response.

    The Weed and Seed program is a Justice discretionary grant program within
    OJP.2 The Assistant Attorney General for OJP is responsible for the
    overall management and oversight of offices within OJP, including EOWS.
    The Assistant Attorney General for OJP sets policies, promotes
    coordination among OJP bureaus and offices, and ensures that EOWS follows
    its policies and procedures. EOWS provides funding to grantees to help
    prevent and control crime and improve the quality of life in targeted
    high-crime neighborhoods across the country.3 It is a joint federal,
    state, and local program for coordinated law enforcement and neighborhood
    reinvestment. Federal program funding is to support Weed and Seed sites
    and to provide training and technical assistance.4

    2Discretionary grants are awarded to eligible grantees, most often on a
    competitive basis.

    3A grantee is an entity that receives funding from EOWS to implement the
    Weed and Seed program. The grantee distributes the funds to sites and is
    responsible for ensuring that sites comply with the terms of the grant. A
    grantee may have more than one site.

    4A site is a geographically defined area ranging in size from several
    neighborhood blocks to several square miles. With the input of the local
    U.S. Attorney's Office and the site's steering committee, each site
    develops and implements its own Weed and Seed program.

      Background

    The Weed and Seed program has grown since it began in fiscal year 1991
    with three pilot sites in Kansas City, Missouri; Omaha, Nebraska; and
    Trenton, New Jersey, and a relatively small investment of federal money.
    From fiscal year 1999 through 2003, the number of Weed and Seed sites
    increased from 163 to 221, while the total annual program budget generally
    increased from about $45 million to $70 million. See table 1 for fiscal
    years 1999-2003 data on the Weed and Seed program, including EOWS's
    funding history. In fiscal year 2003, with a budget of about $70 million,
    EOWS awarded grants to 221 Weed and Seed sites.

                Table 1: EOWS Funding History for Fiscal Years 1999-2003

                         Prior year                            Funds from
           Unobligated recoveries                 Funds from  the Housing
                       and
           balance            other                 the asset   and Urban
           carried
               forward                                                           Total
                  from reimbursable          EOWS  forfeiture                  funding
                                                              Development    Number of
    Fiscal  prior year  adjustments appropriation       funda      (HUD)b    available
     year                                                                 funded sites
     1999   $4,122,668     $714,466   $33,500,000 $6,500,000           $0 $44,837,134
     2000   $4,129,852   $2,066,682   $33,500,000 $6,500,000           $0 $46,196,533c
     2001   $4,955,423   $1,269,708   $33,925,200 $15,500,000          $0 $55,650,331
     2002     $584,278   $2,359,292   $58,918,000          $0 $10,000,000 $71,861,570
     2003   $4,102,334   $2,713,913   $58,542,000          $0  $4,935,000 $70,293,247

    Source: OJP's Office of Budget and Management Services data. GAO did not
    verify the data.

    Note: For EOWS funding history prior to 1999, see GAO/GGD-99-110.

    aThe proceeds from the asset forfeitures fund were used for federal and
    state law enforcement purposes. EOWS received these funds from Justice,
    under a reimbursable agreement, for the payment of various costs incurred
    by state and local law enforcement officers that participated in joint
    federal law enforcement operations with federal agencies.

    bHUD made $10 million in additional funding available, under a
    reimbursable agreement, to help Weed and Seed sites reduce drug-related
    crimes in public housing.

    cDoes not add because of rounding.

    EOWS is responsible for the national management and administration of the
    Weed and Seed program, including developing policy and providing guidance
    and oversight. EOWS currently administers the Weed and Seed program with a
    staff of 2 management officials, 10 grant monitors, 3 support staff, 3
    detailees,5 and 6 contractors.

    5Two detailees are from the Federal Bureau of Investigation (FBI) and one
    is from the Department of the Interior. The FBI detailees work part-time
    at EOWS.

    Before a community can apply for Weed and Seed grant funding and become an
    eligible site, it must first apply for and gain official recognition from
    EOWS. In order to obtain official recognition, a potential Weed and Seed
    site must show in its application that it has a strategy for weeding and
    preventing crime. Once the application has been received, EOWS creates and
    maintains the official recognition files that are intended to include
    documentation such as the rationale for decisions to grant or deny
    official recognition to an applicant. Official recognition requires the
    U.S. Attorney in the area where the Weed and Seed site is to be located to
    organize a local steering committee made up of various federal, state, and
    local representatives, including residents, to be responsible for local
    administration of the program.6 For official recognition, a site is also
    required to develop a management plan, engage residents and other partners
    in its activities, and develop a comprehensive program to weed out crime
    and gang activity and seed the area with social services, economic
    services, and economic revitalization.

    Weed and Seed program guidance requires that its sites show plans for
    addressing four required elements: (1) law enforcement; (2) community
    policing; (3) crime prevention and intervention, and substance abuse
    prevention, intervention, and treatment; and (4) neighborhood restoration.
    According to EOWS, law enforcement should attempt to eliminate the most
    violent offenders by coordinating and integrating the efforts of federal,
    state, and local law enforcement agencies in targeted high-crime
    neighborhoods. The objective of community policing is to raise the level
    of citizen and community involvement in crime prevention and intervention
    activities. Crime and substance abuse prevention, intervention, and
    treatment should include youth services, school programs, community and
    social programs, and support groups. Finally, neighborhood restoration
    should focus on assistance to distressed neighborhoods through economic
    and housing development. Weed and Seed sites fund a variety of law
    enforcement and community activities. For example, law enforcementfunded
    activities range from participation in a multijurisdictional task force to
    conducting bike patrols in the community. See appendix II for additional
    information on Weed and Seed activities at the sites GAO visited.

    6There are 93 U.S. Attorneys throughout the United States, Puerto Rico,
    the Virgin Islands, Guam, and the Northern Mariana Islands. Each U.S.
    Attorney is the chief federal law enforcement officer of the United States
    within his or her particular jurisdiction. The U.S. Attorneys serve as the
    nation's principal litigators under the direction of the Attorney General.

    Applicants that are officially recognized are eligible to apply for
    funding for up to 5 years. OJP's Office of the Comptroller creates and
    maintains official grant files for sites that are awarded funding by EOWS.
    Applicants are rated based on the strategy they developed in response to
    the four required elements, as stated above.

    In our 1999 report, we made four recommendations to help EOWS improve
    program monitoring and management. The first was to develop adequate
    internal controls to fully document significant qualification and funding
    decisions. We found that EOWS lacked internal controls requiring
    significant program management decisions be documented. Therefore, EOWS
    was not able to ensure that it was making the best decisions about
    allocating available funds. Our second recommendation was to improve
    program monitoring to ensure that sites met the grant requirement of
    submitting progress reports and that EOWS site visits were documented. We
    found that EOWS did not always ensure that local sites submitted progress
    reports and that grant monitors documented the results of site visits.
    Such documentation would help EOWS management and grant monitors determine
    how sites are meeting program objectives and how well sites are complying
    with grant requirements, and assist them in making future grant
    qualification decisions. Our third recommendation was to develop criteria
    for determining when sites have become selfsustaining and when to reduce
    or withdraw program funding. We found that although self-sustainability is
    central to the program, no site's funding had been reduced or withdrawn as
    a result of its efforts to become selfsustaining in the 9 years of the
    program's existence. Establishing such criteria is important because the
    Weed Seed program was founded on the premise that federal funding would
    continue for a finite period, after which a Weed and Seed site would be
    self-sustaining. Our fourth recommendation was to develop additional
    performance measures to track program outcomes. We found that EOWS's
    performance indicators generally tracked activities rather than program
    results; therefore, EOWS was not able to measure the success of the
    program.

    EOWS generally agreed with three of the four recommendations presented in
    the report and discussed future actions it planned to take. EOWS officials
    disagreed with our recommendation on self-sustainability. They stated that
    developing criteria to ascertain self-sustainment is redundant since EOWS
    adopted a 5-year rule under which it could discontinue awarding funding to
    qualifying sites unless the sites expanded to an additional neighborhood
    site.

      Internal Controls Have Been Developed, but Challenges Remain

    Although EOWS has developed internal controls intended to require that
    significant qualification and funding decisions be documented and readily
    available for review as we recommended in 1999, these policies and
    procedures are generally not being followed in the files we reviewed. In
    response to our 1999 recommendation on internal controls, EOWS established
    policies and procedures intended to ensure that significant qualification
    decisions were documented. In addition, OJP requires that qualification
    and funding decisions are to be documented in official recognition and
    official grant files. Our review of 20 official recognition and 10
    official grant files showed that some of EOWS's official recognition and
    official grant files were missing full documentation regarding the
    qualification and funding decisions. 7 However, before our file review
    ended, EOWS officials produced further documentation, which they
    acknowledged was not in the official recognition and official grant files
    as required, but rather in the personal working files of grant monitors
    and thus not readily available to EOWS management and external reviewers.
    This lack of ready availability is not in keeping with EOWS's and OJP's
    policies and procedures, or with the Comptroller General's standards for
    internal controls.8 Without having official recognition and official grant
    files complete and readily available, it may delay and complicate EOWS
    officials' oversight of the documentation of qualification and funding
    decisions.

        Policies and Procedures Have Been Developed to Document Significant
        Decisions

    Both EOWS and OJP policies and procedures have been developed to help
    ensure the documentation of significant decisions. In 2000, EOWS developed
    a policies and procedures guide in response to the recommendation we made
    in our 1999 report intended to ensure that significant qualification and
    funding decisions for new and existing sites9 were always fully
    documented, and further revised the guide in 2003. This guide requires
    that all documentation pertaining to official recognition decisions be
    kept in the official recognition files. EOWS is also required to follow
    the policies in the OJP Grants Management Policies and Procedures Manual,
    which requires that EOWS fully document program management decisions in
    both official recognition files and official grant

    7When applicants apply for funding, EOWS creates and maintains official
    recognition files. If applicants receive funding, OJP creates and
    maintains official grant files.

    8U.S. General Accounting Office, Standards for Internal Control in the
    Federal Government, GAO/AIMD-00-21.3.1 (Washington, D.C.: November 1999).

    9Throughout the rest of this report, we refer to both new and existing
    sites as sites.

    files. The OJP manual specifically requires that the official recognition
    files and official grant files contain key documents such as ratings
    information, rejection letters, and applications.

        Official Recognition and Official Grant Files Were Incomplete

    Despite EOWS's and OJP's efforts to require the documentation of
    qualification and funding decisions in official recognition and official
    grant files, some of EOWS's official recognition and official grant files
    we reviewed were incomplete at the time of our file review. In reviewing
    the official recognition files, we randomly selected 10 files to review
    from the 94 files submitted for official recognition in fiscal year 2002.
    Additional information about how we selected the files is in appendix I.
    We found sufficient documentation in 7 of the 10 files to determine the
    basis and rationale for decisions to award official recognition. However,
    in the remaining 3 files we were unable to determine the basis and
    rationale for such decisions because documentation was insufficient. For
    example, key documents such as the ratings information that EOWS grant
    monitors use to record their assessment of the official recognition
    applications did not contain the basis and rationale for the award
    decision.10 Additionally, OJP's grant policies and procedures require
    EOWS's grant monitors to prepare a rejection letter, informing applicants
    of reasons for rejection when funding is denied and place a copy of the
    rejection letter in the official recognition file. We reviewed all 10
    official recognition files for the applicants that were eligible to apply
    for funding in fiscal year 2002 but were rejected. We found a rejection
    letter was missing in 7 of the 10 files.

    We also reviewed documents in EOWS's official grant files to assess EOWS's
    efforts to comply with EOWS and OJP policies for fully documenting funding
    decisions. We randomly selected 10 official grant files to review from the
    31 sites that were funded in fiscal year 2002. We found that none of the
    10 official grant files fully documented funding decisions for Weed and
    Seed sites. For example, the application, a basic component of the
    official grant file, was missing in 8 of the 10 files.

    Challenges Persist in Several directives require that grant documentation
    be readily available. Making Documentation OJP's policies and procedures
    manual requires that documentation be Readily Available kept readily
    available in the official recognition and official grant files, so

    that OJP and EOWS management can identify and resolve any problems or

    10EOWS uses the information in the documents as the basis to award
    official recognition.

    deficiencies in grantees' compliance with relevant policies and
    procedures. OJP's policies and procedures manual requires that all
    pertinent information that should be in the official recognition and the
    official grant files be kept in a centralized location to facilitate
    reviewing for completeness. In addition, the Comptroller General's
    guidance on internal controls in the federal government, Standards for
    Internal Control in the Federal Government, requires that the
    documentation for all transactions and significant events be readily
    available for examination. The Assistant Attorney General for OJP is
    responsible for holding EOWS accountable for following its policies and
    procedures. In reference to the ratings information and rejection letters
    missing from the official recognition files, EOWS produced the documents
    after our initial review, informing us that the documents were kept in the
    personal files of the grant monitors--rather than the official recognition
    files--a practice not in keeping with EOWS's and OJP's policies and
    procedures. In reference to the missing applications, EOWS officials also
    produced them after our initial review, but did not specify where they
    found them. Not having complete and readily available official recognition
    and official grant files may delay and make it more difficult for OJP and
    EOWS officials to perform management functions such as overseeing and
    assessing the documentation and oversight of qualification and funding
    decisions. While we eventually were able to obtain documentation showing
    the basis and rationale for EOWS decision making, the incomplete official
    recognition and official grant files made the review difficult and
    time-consuming.

    In responding to a draft of this report, Justice stated that electronic
    documents maintained in its grant management system (GMS) are considered a
    part of the official recognition and official grant files and are readily
    available to EOWS, and that they should have been readily available to,
    and easily accessible by, GAO.

    In response to our 1999 recommendation, EOWS has taken steps to improve
    program monitoring, as with the documentation of qualification and funding
    decisions, but challenges remain in making the monitoring documentation
    readily available. While OJP requires progress reports and site visit
    reports to be included in the official grant files, our review in 2003 of
    10 official grant files showed that some of EOWS's files were missing full
    documentation. However, before our review ended, EOWS provided further
    documentation that fully documented progress reports and site visits,
    which it acknowledges was not in the official grant files, and thus not
    readily available. This lack of readily available documentation is not in
    keeping with OJP's policies and procedures. Not having complete and
    readily available official grant files may delay and complicate EOWS

      EOWS Has Taken Steps to Improve Program-Monitoring Documentation, but
      Challenges Remain

    officials' and external reviewers' assessment of whether EOWS's monitoring
    requirements are being followed.

        Official Grant Files Were Incomplete

    Despite EOWS's efforts, some of the official grant files we reviewed were
    incomplete at the time of our file review. We randomly selected 10 files
    from the 31 sites that were first funded in fiscal year 200211 to
    determine whether the sites had submitted the required progress reports
    from January 1, 2003, through June 30, 2003.12 Of the 10 files we
    reviewed, 6 did not contain any of the required progress reports for the
    period. The remaining 4 files included the required progress reports.
    OJP's policies and procedures require EOWS to ensure that progress reports
    are included in the official grant files. However, when we asked EOWS
    about the missing progress reports, EOWS provided us with the requested
    documents. We asked, but EOWS did not specify where it found the missing
    documentation.

    In addition, EOWS's grant monitors are required to conduct site visits a
    minimum of every 18 months and document their visits. This documentation
    is meant to convey to EOWS management officials how well sites are
    complying with grant requirements and is to be used by EOWS in making
    funding decisions. Grant monitors are to prepare a report of the visits
    and forward a copy to the Office of the Comptroller for inclusion in the
    official grant file. We reviewed the same 10 grant files discussed above
    to determine whether site visits were fully documented. We found that a
    site visit report was not yet due for 7 of the 10 files. In the remaining
    3 files where a site visit was due, none of the 3 files contained
    documentation that the site visit had been conducted. We requested to
    speak to the responsible grant monitors to determine whether site visits
    had been conducted. EOWS officials told us that the monitors were no
    longer involved with the program but provided additional documentation for
    the 3 files we reviewed. The documentation showed that of the 3 sites,
    only 1 visit had been conducted; the other 2 were scheduled but had been
    delayed. While we only reviewed 3 files where a site visit was due, the
    lack

    11The remaining 187 of the 218 sites received funding in fiscal year 2002,
    but were initially funded prior to fiscal year 2002.

    12Progress reports describe Weed and Seed activities and the
    accomplishment of objectives in a site's funding application. Progress
    reports help EOWS officials determine how sites are meeting program
    objectives and assist them in making future grant decisions. Progress
    reports are due 30 days after June 30 and December 31, respectively.

    of documentation in all 3 files, as well as the fact that only 1 of 3 site
    visits had been conducted, added to our concerns about the completeness of
    official grant files for program oversight.

        Challenges Persist in Making Monitoring Documentation Readily Available

      Despite Some Steps, Challenges Remain in Developing Criteria for
      Self-Sustainability

    As we have previously mentioned, OJP's policies and procedures manual and
    the Comptroller General's guidance on internal controls require that
    documentation should be kept readily available for examination. Subsequent
    to our finding missing progress reports and site visit reports, EOWS
    provided us with additional documentation that demonstrated that the
    progress reports and one of the site visit reports had been completed.
    When asked about where they found the missing progress reports and site
    visit reports, EOWS officials did not specify where they found the missing
    documentation.

    EOWS has established a rule and set some activities for sites to complete
    to encourage them to become self-sustaining. However, it has still not
    fully developed criteria to determine when sites have become
    self-sustaining and when to reduce or withdraw Weed and Seed grant funds,
    as we recommended in 1999. To become self-sustaining, sites must leverage
    additional resources from sources other than EOWS to sustain the Weed and
    Seed program without EOWS funds. We found that EOWS has not reduced or
    withdrawn funds from any Weed and Seed sites for reasons related to
    becoming self-sustaining. This is important because without overall
    program funding increases, new sites cannot be funded unless funding can
    be reduced or discontinued from sites that have achieved
    selfsustainability.

        EOWS Encourages Self-Sustainability, yet Needs Criteria to Make Funding
        Decisions

    In 1999, EOWS established a rule to encourage self-sustainability that was
    generally intended to limit Weed and Seed funding to a site to 5 calendar
    years and require grantees to shift the majority of EOWS funds to a
    different site after 5 years.13 According to EOWS officials, EOWS also
    encouraged sites to become self-sustaining by establishing activities to
    be completed during each year of the 5-year grant.14 Program monitors are
    to assess sites' completion of these activities by recording results on a
    checklist.

    13Grantees may have more than one site.

    14EOWS refers to these activities as benchmarks. For a list of EOWS's
    benchmarks, see appendix IV.

    We found that while EOWS undertook efforts to encourage
    selfsustainability, it has not fully developed criteria to make funding
    decisions based on whether sites had achieved self-sustainability. In
    fiscal year 2002, 73 sites, which had completed 5 years of EOWS funding,
    reapplied for funding. EOWS approved an additional round of 5-year funding
    to 67 of these sites. EOWS denied funding to 6 sites, but not for reasons
    related to self-sustainability. At the time of our review, no site's
    funding had been reduced or withdrawn because the site had achieved
    self-sustainability, even though EOWS has funded some sites since the
    early 1990s. In relation to the activities EOWS set to encourage sites to
    achieve self-sustainability, because these activities were established in
    2003, we could not assess the sites' progress toward completing them. As
    we reported in 1999, without criteria, EOWS does not have a basis for
    determining when sites are selfsustaining and when to reduce or withdraw
    Weed and Seed funds.

    EOWS officials told us that OJP is currently developing criteria for
    selfsustainability for EOWS and other OJP programs and that a report
    detailing the criteria may be completed in 2004. According to Justice,
    EOWS developed a new criterion for self-sustainability in the FY2003
    Competitive Application Kit regarding whether or not a full-time
    coordinator is funded by a reallocation of resources other than the Weed
    and Seed grant.

    For information about proposed legislation that could affect
    selfsustainability for Weed and Seed sites, similar to other Justice
    programs, see appendix III.

    Performance In 1999, we reported15 that while EOWS had developed various

    performance measures in an attempt to respond to the Government Measures
    Generally Performance and Results Act (GPRA) of 1993,16 EOWS's measures
    Did Not Track generally did not track program outcomes.17 GPRA seeks to
    shift the focus

    of federal management and decision making away from activities Program
    Outcomes performed to outcomes, or the results of activities undertaken.
    Since our

    15See GAO/GGD-99-110.

    16P.L. 103-62.

    17Performance measures translate program goals into concrete, observable
    conditions that determine what data to collect to learn whether progress
    has been made toward achieving program goals. Such measures are meant to
    cover the key aspects of performance that will enable programs to assess
    accomplishments, make decisions, realign processes, and assign
    accountability.

    last report, EOWS has continued to refine and develop its performance
    measures, but we found that EOWS still generally collects site activity
    data rather than measuring outcomes. EOWS also generally does not use
    intermediate measures, which represent conditions believed to precede or
    contribute to achieving the ultimate outcomes that may be considered
    constructive steps toward measuring outcomes. While assessing success
    using outcomes and/or intermediate measures can be difficult, doing so is
    important to program management and to policy makers for understanding
    whether the program is achieving the intended results, and for identifying
    opportunities for improvement.

        EOWS Generally Collects Activity Data Rather Than Outcome Performance
        Measures

    Our review showed that EOWS generally collects activity data, but has not
    developed outcome or intermediate measures that enable EOWS management to
    track the success of the Weed and Seed program in meeting its goals.18 As
    established in prior work by GAO, outcome measures help officials track
    the success of their programs in meeting program goals.19 EOWS mostly
    collects data about site activities, such as whether sites have foot
    patrols, safe havens, and provide job training. An intermediate measure,
    for example in relation to job training programs, might be the extent to
    which those who attend job training obtain employment. This intermediate
    measure rests on the assumption that individuals who are employed are less
    likely to commit crimes. See appendix V for activity data collected by
    EOWS.

    EOWS officials told us that to measure its success in reducing violent
    crime, they collect data on the number of homicides and consider homicides
    the significant indicator for measuring performance outcomes. However,
    using the number of homicides as an outcome measure indicative of program
    success is problematic for several reasons. First, because homicides are
    relatively rare, even in high crime areas, homicide trends may be too
    unstable (fluctuate too much from year to year) to assess the success of
    the Weed and Seed program. Second, outcome measures, such as homicides,
    can present some methodological challenges

    18EOWS officials told us that the Weed and Seed program has three main
    goals: (1) reduce violent crime, (2) reduce drug crime, and (3) coordinate
    Weed and Seed funds with other resources. Such goals are related to the
    overall intent of the program to weed out crime from targeted
    neighborhoods and seed them with a variety of programs to prevent crime
    from recurring.

    19U.S. General Accounting Office, Managing for Results: Analytic
    Challenges in Measuring Performance, GAO/HEHS/GGD-97-138 (Washington,
    D.C.: May 30, 1997).

    because it is difficult to draw a direct causal link between the homicides
    and a program's work. Economic trends and other law enforcement
    initiatives could also be responsible for the observed outcomes. If
    homicides are used as a performance measure, any analysis should attempt
    to control for other factors influencing the outcomes. Third, given the
    broad nature of this program goal --to reduce violent crime, measuring
    outcomes in relation to only one type of violent crime also seems
    problematic because it is too narrowly focused. According to EOWS
    officials, homicide data was selected because of its reliability, and
    because gang-related homicides are a significant indicator of the success
    of a Weed and Seed strategy.

    In February 2004,the Office of Management and Budget (OMB) issued a
    detailed assessment of the Weed and Seed program.20 According to OMB, Weed
    and Seed has not demonstrated results. OMB's report states that Weed and
    Seed lacks clear targets or timelines for reducing violent crime.
    Additionally, the report states that Weed and Seed has difficulty
    collecting data and developing aggregate measures for assessing program
    performance.

        GAO Has Previously Reported on How Other Federal Programs Have Developed
        Effective Performance Measures, Despite Challenges

    Although the Weed and Seed program faces many challenges in developing
    outcome measures for its various activities, we have previously reported
    that other federal programs have developed effective performance measures
    even under difficult circumstances.21 These reports identified a variety
    of strategies that other federal programs have used to develop performance
    measures. For example, we reported that some federal programs utilized a
    mix of outcome and intermediate measures.22 This combination of measures
    allowed them to minimize the risk of not showing outcomes because of their
    limited control over external factors such as economic trends, which may
    prevent programs from achieving intended outcomes. Intermediate measures
    are also helpful to show progress when it is expected to take many years
    before the desired outcome is likely to be achieved. In such instances,
    progress toward program outcomes may be demonstrated through intermediate
    outcomes.

    20OMB periodically assesses goals and results of federal programs, such as
    Weed and Seed, and reports on its findings. The purpose is to tie
    performance with the budget process.

    21U.S. General Accounting Office, Managing for Results: Measuring Program
    Results That Are Under Limited Federal Control,GAO/GGD-99-16 (Washington,
    D.C.: December 11, 1998) and GAO/HEHS/GGD-97-138.

    22See GAO/GGD-99-16.

    We also reported that where measuring outcomes is, after careful
    consideration, deemed infeasible, intermediate measures can be used to
    track progress toward outcomes for programs such as Weed and Seed.23

    In addition, we have reported other ways federal agencies have used
    intermediate outcomes.24 For example, the National Highway Traffic Safety
    Administration uses the rate of safety belt use as an intermediate measure
    of its goal to reduce motor vehicle crashes and the resulting fatalities
    and injuries. As we reported, an increase in the use of safety belts is
    considered an intermediate outcome--not an end outcome--because it is
    desirable not in itself but because it has been demonstrated to contribute
    to the ultimate goal--reducing highway-related fatalities and injuries.25

        Additional Performance Measures Are Being Developed for EOWS

    The Justice Research and Statistics Association and the Urban Institute
    are currently developing additional outcome performance measures under
    contracts awarded by EOWS in 2002 and 2003. According to EOWS officials,
    these research organizations have the expertise needed to develop outcome
    performance measures for the Weed and Seed program. We interviewed
    researchers from these organizations and spoke to EOWS officials about
    their research plans. However, none of the studies on developing
    additional performance measures had been completed at the time of our
    review, and it is too soon to tell whether the studies will produce
    outcome measures needed to adequately assess the Weed and Seed program.26
    The Justice Research Statistics Association and other researchers were
    conducting studies on crime data and evaluation during our review; one of
    those studies was submitted to EOWS in November 2003. See appendix VI for
    a description of EOWS's ongoing and completed studies.

    23U.S General Accounting Office, Law Enforcement: Better Performance
    Measures Needed to Assess Results of Justice's Office of Science and
    Technology, GAO-04-198 (Washington, D.C.: November 2003).

    24See GAO/GGD-99-16.

    25 It is commonly accepted that safety belt use reduces fatality rates and
    the severity of injuries. See U.S. General Accounting Office, Highway
    Safety: Safety Belts Use Laws Save Lives and Reduce Costs to Society,
    GAO/RCED-92-106 (Washington, D.C.: May 1992)

    26In November 2003 the Justice Research and Statistics Association
    completed a study comparing homicide trends in sites to their host
    jurisdictions. However, this study relied on homicide rates collected by
    EOWS, rather than developing additional performance measures.

    Conclusions

    We believe that 5 years has been ample time for OJP and EOWS to fully
    implement the recommendations we made in our 1999 report. EOWS has
    developed and partially implemented policies requiring the completion of
    documentation to support major decisions, such as qualification and
    funding decisions, and recorded monitoring information. But by failing to
    ensure that these documents are appropriately maintained in official
    recognition and grant files, the documents are not readily available to
    meet their purpose. That is, the intent of this type of internal control
    is to ensure that both management and external reviewers such as auditors
    can adequately perform their responsibilities, for example, reviewing work
    and making management decisions based on complete and accurate
    information. While EOWS was eventually able to produce the documentation
    we requested, failure to appropriately file the documentation made the
    process of using it very inefficient and, in effect, may have defeated the
    purpose of having it completed.

    While EOWS may have moved forward in addressing its self-sustainability
    goals by recently developing an activity checklist for grantees to use to
    document actions that are intended to achieve self-sustainability, the
    activities do not constitute criteria for determining when sites should be
    considered self-sustaining and consequently have federal funds reduced or
    discontinued. One of the benefits of such criteria is to enable EOWS to
    determine when current projects are likely to be able to self-sustain so
    that available funding can be used to help reduce crime and achieve other
    benefits in other deserving communities. Because no sites have had their
    funding withdrawn because they were deemed self-sustaining during the
    13-year life of this program, EOWS may be foregoing the opportunity to use
    the funds in another location where the need for federal funding is
    greater or EOWS may need less overall funding. Without criteria to
    determine when federal funds are no longer needed, EOWS also runs the risk
    of providing funds beyond what is needed to sustain some sites.

    EOWS also needs performance measures that focus on program outcomes,
    and/or achievement of intermediate goals, so that it and those that
    provide oversight, such as the Congress and OMB, will be able to
    adequately assess the extent to which the program is achieving its goals.
    Management depends on this type of assessment to make the strategic and
    operations decisions needed to achieve the program's missions and goals.
    Congress and oversight agencies need this type of assessment so they can
    make funding decisions and help ensure that EOWS is in the best position
    and has the best tools to accomplish its mission. While EOWS recognizes
    the need for outcome performance measures and has funded studies to help

      Recommendations for Executive Action

    Agency Comments and Our Evaluation

    develop them, it is unclear at this time whether these studies will be
    able to suggest the needed measures.

    We recommend that the Attorney General of the United States require the
    Assistant Attorney General for OJP to ensure that the Executive Office for
    Weed and Seed fully implement the intent of our previous recommendations
    by taking the following four steps:

    o  	maintain the documentation of the basis and rationale for
    qualification and funding decisions in appropriate grant files;

    o  retain progress reports and site visit reports in official grant files;

    o  	clearly define criteria to assess when sites are self-sustaining and
    apply the criteria to sites when making further funding decisions; and

    o  	develop outcome performance measures--or, where measuring outcome is,
    after careful consideration, deemed infeasible, intermediate
    measures--that can be used to adequately track progress toward program
    outcomes of the Weed and Seed program.

    We provided a draft of this report to the Attorney General of the United
    States for review and comment. In a March 11, 2004, letter, the Assistant
    Attorney General (AAG) for OJP commented on the draft. Her written
    comments are summarized below and presented in their entirety in appendix
    VII. Based on Justice's comments, we modified the text of our report where
    appropriate and also incorporated Justice's technical comments into this
    report where appropriate.

    In the AAG's response, the Justice Department generally agreed with our
    recommendations on maintaining the documentation of the basis and
    rationale for qualification and funding decisions in appropriate grant
    files and retaining progress reports and site visit reports in official
    grant files. Justice agreed in part with our recommendation on
    self-sustainability criteria and fully agreed with our recommendation on
    outcome performance measures, and offered additional information to show
    it had moved forward in responding to our past recommendation.

    Justice agreed that controls should be strengthened to ensure that the
    basis and rationale for qualification and funding decisions are documented
    in the centrally maintained official recognition (OR) files. In Justice's
    comments, the AAG explained that electronic documents maintained in its
    grant management system (GMS) are considered a part of the official
    recognition file and are readily available to EOWS, and that they should
    have been, but were not, readily available to, and easily accessible by,
    GAO. As of the current fiscal year, OJP will not only post all
    solicitations

    and receive all grant applications via GMS, but also receive and maintain
    all grant progress reports on GMS. In this way, there will be no
    opportunity to misplace official grant documents. Justice stated that
    OJP's Office of the Comptroller will implement procedures to ensure that
    external reviewers have ready access to GMS information.

    Justice agreed with our second recommendation on retaining progress
    reports and site visit reports in official grant files. Justice noted that
    EOWS recently implemented a progress reporting module in GMS that now
    enables recipients of grants awarded through GMS to submit semi annual
    progress reports electronically. OJP is developing a monitoring module
    that will enable program monitors to record and report on site visits
    directly in GMS. In the interim, EOWS and the Office of the Comptroller
    will take steps to strengthen controls to ensure that copies of completed
    site visit reports are included in the official grant file.

    Justice agreed in part with our third recommendation on
    self-sustainability and noted that EOWS will further define the criteria
    to assess when sites are self-sustaining and apply the criteria when
    making funding decisions. Further, Justice said it is promoting a
    "graduation" process to bring sites to sustainability and cease providing
    Weed and Seed funding to those sites, thus making funding available to
    assist newer developing sites. However, Justice does not agree that EOWS
    has never set criteria for selfsustainability. Justice said that EOWS set
    a criterion for self-sustainability in the FY2003 Weed and Seed
    Competitive Application Kit, which asked if the application provided for a
    full-time coordinator funded by reallocation of existing resources other
    than the Weed and Seed grant. According to Justice, applications were
    ranked against this criterion. According to EOWS officials, in March 2004,
    EOWS will announce an additional criterion specific to measuring
    self-sustainability. The new criterion will require grant applicants to
    identify other funding sources at a level five times the EOWS
    contribution. This criterion will be considered when making funding
    decisions.

    We applaud EOWS's commitment to further develop criteria to assess when
    sites are self-sustaining and apply the criteria to sites when making
    further funding decisions. However, Justice's comments did not provide
    specific information about how the Application Kit criterion has been used
    to determine self-sustainability. We added Justice's belief that this
    criterion relates to self-sustainability to the text, but also note that
    over the 13-year history of the program, funding has never been reduced or
    withdrawn from a site because a site was deemed to be self-sustaining.

    Justice agrees that developing outcome performance measures is important
    and states that EOWS has been working to develop measures that track
    progress toward program outcomes in the Weed and Seed Program. In a letter
    commenting on this draft report, Justice provided a summary of results
    from JRSA's study to support EOWS's use of the change in homicides as a
    measure of program success. According to Justice's summary, this study
    concluded that measuring reductions of homicides is an adequate
    performance measure of the Weed and Seed program because of its
    reliability and explicit selection as a goal by over 30 percent of the
    Weed and Seed sites participating in the Crime Pattern Study. In addition,
    Justice includes a 5 percent reduction in homicides in Weed and Seed sites
    as part of its Strategic Plan. With its comments, Justice included
    summaries of two recent research reports. One of these, Analysis of
    Homicide in Weed and Seed Sites, showed positive results in homicide
    trends in Weed and Seed sites when compared to host jurisdictions. Another
    report, Crime Pattern Analysis (Three-Top Crime Study), showed that after
    the third year of Weed and Seed program implementation, sites encountered
    a significant decrease in reported crimes.

    Because the Crime Pattern Analysis study was not completed at the time of
    our review, we could not fully assess its methodological rigor and whether
    it would result in effective outcome performance measures. Although we
    believe that EOWS may be moving in the right direction regarding
    developing additional performance measures and we mention the studies in
    the report text, we also believe that homicides have shortcomings as
    described in the text when used as the only measure of the program's
    performance. In addition, our review of the Analysis of Homicide in Local
    Weed and Seed Sites study generated several concerns. First, the study
    used changes in the raw volume of homicides, instead of changes in
    homicide rates, as its indicator of success or failure of program sites.
    This method does not account for changes in the population as would be
    accounted for were a homicide rate measured. Second, the study used the
    larger host jurisdictions for specific Weed and Seed sites as the
    comparison locations to which the Weed and Seed sites are compared. In
    doing so, Weed and Seed sites' homicide data are included with host
    jurisdictions' homicide data, thus making the differences between them
    extremely difficult to interpret. It is also worth noting that this study
    evaluates Weed and Seed sites only on the basis of decreases in homicides,
    while Weed and Seed was intended to decrease crime more generally.

    We are sending copies of this report to the Attorney General, appropriate
    congressional committees, and other interested parties. We will also make
    copies available to others upon request. In addition, the report will be

    available at no charge on GAO's Web site at http://www.gao.gov. If you or
    your staff has any questions on this report, please call Weldon McPhail on
    (202) 512-8644 or me on (202) 512-8777.

    Sincerely yours,

    Laurie E. Ekstrand, Director Homeland Security and Justice Issues

                           Appendix I: Scope and Methodology

    To determine what actions the Executive Office for Weed and Seed (EOWS)
    has taken to develop an adequate internal control to ensure full
    documentation of the basis and rationale for qualifying new and existing
    sites for funding, we reviewed (1) the criteria used to determine which
    new and existing sites should be qualified for funding, and (2) EOWS's
    policies and guidance. To gather this information, we interviewed
    officials from Justice and EOWS and reviewed pertinent documents,
    including EOWS policies and procedures, official recognition and grant
    applications, management oversight and monitoring documentation, and
    budget reports. In addition, we randomly selected 10 of 94 fiscal year
    2002 official recognition files submitted to EOWS for review. These 10
    files included 5 files from new applicants and 5 files from existing sites
    that applied in 2002 and were funded in fiscal year 2002. We reviewed 10
    of the 14 official recognition files that EOWS decided not to fund in
    fiscal year 2002. We also reviewed 10 randomly selected official grant
    files from the 31 new and existing sites that were approved for funding in
    fiscal year 2002. Because of limited resources, we did not attempt to
    review all the files. While the small sample size prevents us from making
    reliable generalizations, the 30 files we reviewed represented various
    types of files that EOWS maintains. We selected the files from a list that
    was provided to us by EOWS. To ensure the list of files was sufficient for
    the purpose of our review, we spoke to knowledgeable EOWS officials about
    the completeness and accuracy of the list.

    To assess what steps EOWS has taken to improve program monitoring to
    ensure that sites meet the grant requirement of submitting progress
    reports, and that EOWS site visits are documented, we reviewed EOWS
    program grant guidance, including training offered, the EOWS monitoring
    guidance used by grant monitors when conducting site visits, and the grant
    files for the Weed and Seed sites that we visited: Pittsburgh,
    Pennsylvania; Charleston, South Carolina; and North Charleston, South
    Carolina. We selected these sites from the 221 sites funded by EOWS in
    fiscal year 2003. Based on our discussions with EOWS officials about the
    sites, we selected three sites, which although not representative of all
    sites, had received Weed and Seed grant funds since the early 1990s, which
    enabled us to learn about their efforts to implement the Weed and Seed
    strategy over time. We also reviewed selected monitoring visit reports
    prepared by grant monitors for these sites and biannual progress reports
    submitted in fiscal years 2002 and 2003. We interviewed EOWS officials and
    grant coordinators at these 3 sites regarding procedures used for
    monitoring Weed and Seed sites. We also reviewed the same 10 randomly
    selected official grant files discussed above from the 31 new and existing
    sites that were funded in fiscal year 2002 to determine whether the
    progress reports

    Appendix I: Scope and Methodology

    and site visit reports were in the official grant files and readily
    available. The sites were funded under EOWS's competitive application and
    did not include sites funded under EOWS's continuation application.1 While
    the 10 files we reviewed represented various types of files that EOWS
    maintains, the small sample size prevents us from making reliable
    generalizations about all official grant files. To ensure the list of
    files was sufficient for the purpose of our review, we spoke to a
    knowledgeable official about the completeness and accuracy of the list.

    To assess what criteria, if any, EOWS has developed when sites are
    selfsustaining and when to reduce or withdraw program funding, we asked
    EOWS officials for any relevant information. Following repeated requests,
    EOWS officials did not provide any documentation on criteria that they had
    developed. They did, however, provide information on the five-year rule
    for self-sustainability and the sites' activities, which we reviewed.

    To determine what additional performance measures EOWS has developed that
    track program outcomes, we interviewed officials from EOWS and the sites
    we visited. We reviewed pertinent documents, including EOWS policies and
    procedures, grant applications, and data collected pursuant to the
    Government Performance and Results Act of 1993 (GPRA). In relation to the
    studies EOWS commissioned to develop program-wide performance measures, we
    interviewed the authors of the studies under way to discuss their research
    plans. At the time of our review, none of the studies on developing
    additional performance measures had been completed.

    1Sites funded under EOWS's competitive application can be either the first
    Weed and Seed site in the jurisdiction that received official recognition
    or a new geographical area (that is, not contiguous with an existing or
    former target area) in a jurisdiction with an existing (or continuation)
    or former Weed and Seed site. Sites funded under EOWS's continuation
    application are in their second, third, fourth, or fifth year of funding
    and implementing the Weed and Seed program.

    Appendix I: Scope and Methodology

    We conducted our audit work between June and December 2003 in accordance
    with generally accepted government auditing standards.

                          Appendix II: GAO Site Visit Summary

    Table 2: Sites' Weed and Seed Funding History-Fiscal Years 1999-2003

    Source: GAO site visits and EOWS data.

    aPittsburgh's Weed and Seed funds for fiscal year 2002 were amended into
    the 2001 grant award.

    bPittsburgh's Weed and Seed funds for fiscal year 2003 were amended into
    the 2001 grant award.

    Pittsburgh, Pa. Background

    Pittsburgh established its first Weed and Seed site, the Hill District, in
    1992. In 1995, Hazelwood became the second Pittsburgh site. In 1997 and
    1999, East Liberty and Homewood became the third and fourth Weed and Seed
    sites in Pittsburgh. Pittsburgh currently receives funds for three sites:
    East Liberty, Hazelwood, and Homewood. The sites are located between the
    Allegheny and Monongahela rivers on the eastern side of the city. The
    population of the target area is 10,897 in East Liberty, 15,849 in
    Hazelwood, and 9,283 in Homewood. In fiscal year 2003, the Pittsburgh
    sites in total received $435,000 in Weed and Seed funding. See table 2 for
    the Weed and Seed funding history for Pittsburgh's three sites.

    Activities

    Pittsburgh's weeding activities include funding a drug violence initiative
    headed by the Federal Bureau of Investigation (FBI) and a communitybased
    crime prevention activity coordinated with the police. Pittsburgh's
    seeding activities include funding a drug education youth camp that
    provides children with leadership training and fitness tests, and a
    downtown revitalization program. See table 3 for examples of the types of
    activities funded by the Pittsburgh Weed and Seed program.

                          Appendix II: GAO Site Visit Summary

                                    Charleston, S.C.

    Leveraging Efforts

    An important goal of the Weed and Seed program is for sites to leverage
    resources from sources other than EOWS in order to become selfsustaining.
    During our site visit and through documentation obtained from Pittsburgh
    Weed and Seed program staff, we identified several partnerships
    established by the Pittsburgh Weed and Seed program to leverage resources.
    These cooperative arrangements involved partners such as the University of
    Pittsburgh Medical Center and the Pittsburgh Harlequin Rugby Association.
    Table 4 provides specific examples of leveraging efforts that we
    identified.1 Pittsburgh Weed and Seed program staff estimate that they
    coordinated and leveraged approximately $1.1 million in 2002.

    Background

    Charleston became a Weed and Seed site in the spring of 1992. The
    Charleston Weed and Seed target area includes approximately 8 square miles
    on the Charleston Peninsula and in the city of Charleston. The entire Weed
    and Seed site is located in Charleston's Renewal Community. The population
    of the target area, encompassing 13 neighborhoods, is roughly 17,000. In
    fiscal year 2003, the site received $275,000 in Weed and Seed funding. See
    table 2 for Charleston's Weed and Seed funding history.

    Activities

    Charleston's weeding activities include supporting a youth court with a
    Weed and Seed juvenile investigator and funding community policing foot
    patrols. Charleston's seeding activities include supporting a police-youth
    after-school boxing program, and boarding up abandoned and vacant
    buildings to improve the appearance of the area. See table 3 for examples
    of the types of activities funded by the Charleston Weed and Seed program.

    Leveraging Efforts

    An important goal of the Weed and Seed program is for sites to leverage
    resources from sources other than EOWS in order to become selfsustaining.
    During our site visit and through documentation obtained from Charleston,
    we identified several partnerships established by the

    1GAO did not verify the funds leveraged by Weed and Seed sites.

                          Appendix II: GAO Site Visit Summary

                                 North Charleston, S.C.

    Charleston Weed and Seed program to leverage additional resources. These
    cooperative arrangements involved partners such as the FBI and the College
    of Charleston's School of Education. Table 4 provides specific examples of
    leveraging efforts that we identified.2 Charleston Weed and Seed program
    staff estimate that they were able to coordinate and leverage
    approximately $1 million in 2002.

    Background

    North Charleston was originally funded as a Weed and Seed site in August
    1993. The city of North Charleston is located about 7 miles north of the
    city of Charleston. Today, North Charleston has four target neighborhoods:
    Union Heights, Chicora/Cherokee, Accabee, and Liberty Hill. The population
    of the target area is 13,606. In fiscal year 2003, the site received
    $225,000 in Weed and Seed funding. See table 2 for North Charleston's Weed
    and Seed funding history.

    Activities

    North Charleston's weeding activities include participating in Operation
    Cease Fire with the U.S. Attorney's Office, to reduce gun crime in target
    neighborhoods, and partnering with local law enforcement to improve the
    bike patrol program. North Charleston's seeding activities include
    developing the Drug Education for Youth program and supporting an annual
    neighborhood cleanup. See table 3 for examples of the types of activities
    funded by the North Charleston Weed and Seed program.

    Leveraging Efforts

    An important goal of the Weed and Seed program is for sites to leverage
    resources from sources other than EOWS in order to become selfsustaining.
    During our site visit and through documentation obtained from North
    Charleston, we identified several partnerships established by the North
    Charleston Weed and Seed program to leverage resources. These cooperative
    arrangements involved partners such as the United States Air Force and the
    Medical University of South Carolina. Table 4 provides specific examples
    of leveraging efforts that we identified.3 North

    Charleston Weed and Seed program staff estimate that they coordinated and
    leveraged approximately $3.3 million in 2002.

    2GAO did not verify the funds leveraged by Weed and Seed sites. 3GAO did
    not verify funds leveraged by Weed and Seed sites.

                          Appendix II: GAO Site Visit Summary

    Table 3: Examples of Activities Funded by Weed and Seed in Pittsburgh,
    Charleston, and North Charleston

                         Program
          Site     component Activity  Weed and Seed            Description
                                       partner
                     Law enforcement                           Drug violence
                      Safe Streets      U.S. Attorney's    initiative headed by
    Pittsburgh,                          Office,              the FBI.
           Pa.                 program Federal Bureau of
                                         Investigation
                                             (FBI)
                     Community Bike    Community           Community-based crime
                         safety        residents and        prevention activity
                    policing classes    police officers    coordinated with the
                           and                                    police.
                          registration
                                                                About 60 children
                   Prevention, Drug      Boy Scouts of            participated in
                   Education                America,          leadership training
                    intervention, and    U.S. Air Force     and fitness tests.
                        For Youth
                   treatment (DEFY)
                   Camp
                                       Hazelwood          A community-wide effort
                      Neighborhood     Initiative, Inc.,    to reestablish the
                        Improved                                   local
                                       and the Second          business district,
                   restoration housing Avenue               revitalize a 200-acre
                                                                    area, reclaim
                                          Main Street     historic buildings, and
                                            Program         build new housing.

                   Law enforcement                            First-time juvenile
    Charleston, Youth Court     South Carolina Bar      offenders are tried by
                                                                      their peers
          S.C.                     Association, Young    and given an opportunity
                                                       to improve their behavior.
                                    Lawyers Division,
                                   Charleston
                                   Enterprise
                                        Community

               Community   Foot patrols Charleston Weed    Weed and Seed officers
                                        and               patrol the community on
                policing                      Seed Patrol                   foot.
                                                Officers,
                                        Charleston Police
                                               Department
                                 Police                      Participants include
                               Athletic Police Athletic         junior and senior
             Prevention,                League                         boxers and
    intervention, and League Boxing                                   coaches.
               treatment

    Neighborhood Boarding up Charleston Police The city boarded up abandoned
    and vacant

    restoration 	abandoned and Department, Maintenance buildings in Weed and
    Seed neighborhoods. vacant buildings Division

                                                    U.S. Attorney's Office, A gun
          North    Law enforcement Cease Fire       control program that began in
                                                                     January 2003
    Charleston,                             North Charleston Police to reduce
                                      program     gun crime in target areas.
          S.C.                                            Department
                                                  North Charleston Police Two
                                                  officers on bike patrol are
                     Community Bike patrol                 dedicated
                            policing          Department exclusively to the Weed
                                                     and Seed target area.
                                              Charleston Air Force Base Weed and
                   Prevention, DEFY program    Seed staff was developing a DEFY
                       intervention, and        program for participants from the
                                                                    Weed and Seed
                           treatment                     target area.
                                                 Neighborhood residents An annual
                                                 neighborhood cleanup was held in
                   Neighborhood Neighborhood                                April
                      restoration cleanup                    2003.

                           Source: GAO site visits and EOWS.

                          Appendix II: GAO Site Visit Summary

     Table 4: Examples of Leveraging Efforts at Weed and Seed Sites in Pittsburgh,
                            Charleston, and North Charleston

                                 Type of Name of partner
                             partnership       or
        Weed and Seed   or cooperative     cooperative
            site          arrangement      arrangement         Description
                                                         The FBI leads the Safe
                       Federal                 FBI       Streets program in
    Pittsburgh, Pa. government                        Pittsburgh to
                                                          reduce street gang and
                                                          drug related violence.
                                                            State Police officers
                                                           are assisting the U.S.
                       State government   State police          Attorney's Office
                                                                  in implementing
                                                           Justice's Project Safe
                                                              Neighborhoods (PSN)
                                                         initiative to reduce gun
                                                                violence.

    Local government, Allegheny County Health In March and April 2000,
    Pittsburgh began a partnership with

    university Department (ACHD) and ACHD, UPMC, and others to conduct medical
    and dental University of Pittsburgh screening at community events. The
    first major effort of the Medical Center (UPMC) partnership was a lead
    abatement project, which identified and cleaned contaminated dwellings.
    The Pittsburgh Urban Redevelopment Authority and the Housing Authority
    also participated in this effort.

                                           Pittsburgh     The partnership with
                        Local community    Harlequins     the Harlequins began in
                                                          March 2001.
                                                                   The Harlequins
                                        Rugby Association   established two teams
                                                           of 20 children to play
                                                          touch rugby while also
                                                          incorporating training
                                                          in teamwork,
                                                             partnership, and
                                                            discipline into the
                                                                  sport.
                                                          The FBI leads the
    Charleston, S.C. Federal                FBI        Charleston Safe Streets
                        government                        Task Force to
                                                            reduce drug-related
                                                                activities.
                                                           Police officers train
                             Local      Charleston Police      youth in law
                          government                      enforcement procedures
                                                          and tactics as well as
                                           Department     physical training. The
                                                               goal of this
                                                           program is to instill
                                                             responsibility in
                                                            participants and to
                                                             prepare them for
                                                          possible careers in law
                                                               enforcement.
                                                          Graduate interns tutor
                                        College of          students from Safe
                         Local college  Charleston's              Havens.
                                            School of
                                            Education
                                                            The DEFY program will
            North       Federal          U.S. Air Force              be conducted
         Charleston,    government                          year-round, through a
             S.C.                                          partnership with the
                                                                Air Force.
                                        North Charleston      Police officers
                             Local      Police            instruct youth on crime
                          government                         prevention during
                                           Department      school breaks at Safe
                                                                  Havens.
                                                               Through this
                                                               partnership,
                             Local           Medical          information was
                          university      University of       distributed to
                                         South Carolina      Hispanic families in
                                              Crime             the Weed and Seed
                                                              target area about a
                                                          program designed to
                                         Victims Center   provide medical
                                                          services to Hispanic
                                                                 families.

    Source: GAO site visits and EOWS.

    Note: Grantees also receive funds from other federal programs and
    non-federal funds.

    Appendix III: Proposed Legislation Could Affect Self-Sustainability

    Proposed legislation could affect self-sustainability for Weed and Seed
    sites by mandating that EOWS impose more rigorous requirements on its
    grantees, similar to some other Justice programs. Unlike some other
    Justice discretionary grant programs, the law establishing the Weed and
    Seed program currently does not have a matching funding requirement,1 nor
    does it limit the maximum number of years a grantee can receive funding.2
    H.R. 3036, a bill to reauthorize the Department of Justice for fiscal
    years 2004 through 2006, was introduced on September 9, 2003. This bill
    includes a provision that would limit the federal contribution to 75
    percent of the total Weed and Seed program costs. The bill would also
    impose limitations on the duration of grants funded under the Weed and
    Seed program, limiting to 10 the total number of fiscal years a grantee
    may receive grants. Additionally, the bill requires that in order for a
    grantee to be eligible for a grant, it must agree to formulate a timely
    and effective plan to independently sustain the Weed and Seed program when
    federal funding ends.

    Currently there are other federal grant programs that require a funding
    match and a limit on the duration of grant awards.3 For example, the
    Drug-Free Communities Support Program (DFCSP) and Community Oriented
    Policing Services (COPS) both have matching requirements and clearly
    specify the number of years a grantee may be funded. DFCSP is a
    discretionary grant program with a budget of about $60 million in fiscal
    year 2003. DFCSP requires a dollar-for-dollar match, limits grantees to no
    more than 5 years of funding, and reduces awards by 25 percent in the
    fourth and fifth years. The COPS program, with a budget of about $929
    million in fiscal year 2003, generally requires grantees to contribute

    1A matching requirement requires grant recipients to contribute their own
    funds to obtain federal grant funds. The Weed and Seed program is
    authorized and funded by the Department of Justice annual appropriation
    acts. These acts have not included either a requirement for matching funds
    or a limitation on the maximum number of years a grantee may receive
    funding. See for example, Departments of Commerce, Justice, and State, the
    Judiciary, and Related Agencies Appropriations Act, 1998, P.L. 105-119,
    111 Stat. 2440, 2454.

    2As discussed earlier, beginning in 1999, EOWS imposed a 5-year site
    expansion rule, under which a site is awarded funding for 5 years and may
    receive an additional 5 years of funding if the majority of funds are
    shifted to a new site. This is not a requirement imposed by the law.

    3U.S. General Accounting Office, Federal Grants: Design Improvements Could
    Help Federal Resources Go Further, GAO/AIMD-97-7 (Washington, D.C.:
    December 1996).

    Appendix III: Proposed Legislation Could Affect Self-Sustainability

    25 percent of the costs of the program. The federal share of a COPS grant
    that covers more than 1 year must decline year to year.4

    4Grants for hiring and rehiring may be renewed for up to 5 years. Grants
    for other purposes may not cover more than 3 years.

    Appendix IV: Executive Office for Weed and Seed Selected Activities

    Year 1 Year 2 Year 3 Year 4 Year 5

    The local U.S. The steering committee continually
    Attorney's Office educates, guides, and reevaluates
    establishes a steering site management.
    committee, which
    establishes clear
    roles and
    responsibilities for the
    site's organizational
    and management

    a

    structure.

    The steering Steering committee meets at least Steering committee    -> ->
    committee meets monthly.                       and organizational
    monthly and develops                           structure continue to
    a job description for                          function. Steering
    the site coordinator.                          committee meets at
                                                      least quarterly.

    The grantee and the The grantee and/or local U.S. ->
    site coordinator Attorney's Office evaluates the site
    establish procedures coordinator's job performance,
    for tracking grant with input from the steering
    funds and committee. b
    expenditures.

    The site coordinator and the steering committee work together to ensure
    timely expenditure of grant funds.

    Based on the time The site's strategy to achieve The site continues to
    line, goals, and measurable outcomes in the four implement strategy
    objectives required program areas (especially for achieving goals
    established by the law enforcement/community and objectives in all
    site's official policing) is on track. The site four required program
    recognition,c the site continues to collect data related to elements.
    begins implementing goals and objectives.
    strategies and
    developing baseline
    data, addressing the
    four required program
    elements. d

    -> ->

    -> 	The site makes plans to continue community development efforts and
    strategic planning through various funding sources.

    The site begins    The site        The site     The site   The site's
             to           continues      continues     continues  community
                           program
                       publicity and          program program         leaders are
        publicize the  outreach to      publicity and publicity        working to
                       recruit                        and
         program, to    neighborhood      outreach     outreach   sustain efforts
           recruit       leaders and     efforts to   efforts to        and
        neighborhood      mobilize     recruit         mobilize   continue
                         community.    neighborhood    community  community
    leaders, and to                 leaders and to and recruit  mobilization.
                                                          new
        mobilize the                    mobilize the   community
                                                       leaders.
         community.                      community
                                           while
                                        identifying
                                            and
                                       building
                                       sustainable
                                         leadership
                                           among
                                            community
                                             members.

    Appendix IV: Executive Office for Weed and Seed Selected Activities

    Year 1 Year 2 Year 3 Year 4 Year 5

         The site           The site  The Weed and    Community     The Weed and
          begins        continues to      Seed         leaders          Seed
                      coordinate its
    coordinating   efforts with   effort begins       have        community is
            its           local          to be       established      included in
                      initiatives,
    efforts with  particularly                      working     city/community
          related    programs with    included in   relationships       planning.
                     local
           local     parks and       city/community   with local
    initiatives,  recreation      plans.           officials.
                     departments.
    particularly
            law
        enforcement
    resources and
        activities.

    The steering committee develops The special emphasis and implements the
    special program is fully emphasis program based on operational. community
    needs.

    The site submits               ->             ->         ->             ->
    timely grant
    expenditure reports to
    Executive Office for
    Weed and Seed and
    Office of Justice
    Programs.

             The site submits        ->    ->    ->      The site submits a final
             timely progress                               progress report.
            reports, providing
             program outcome
          information related to
            the achievement of
              the goals and
           objectives stated in
         the official recognition
                and grant
              applications.

                      The site communicates regularly      ->      ->
                     with all partners, including U.S.
                    Attorney's Office, Executive Office
                          for Weed and Seed, and
                         community members, using
                     newsletters, faxes, neighborhood
                    visits, e-mail, community meetings,
                               and so forth.

           The steering      -> ->      The steering      The steering committee
         committee and the             committee and site determines if there is
                                                          a
         site coordinator            coordinator review    need to continue the
           evaluate site            previous years' grant    program in other
    progress to determine          applications to      neighborhoods based on
         site's technical             determine plans for      the program.
    assistance needs, if        final year of official
               any.                     recognition.

    Appendix IV: Executive Office for Weed and Seed Selected Activities

    Year 1                    Year 2         Year 3      Year 4         Year 5
    The site develops         ->                    ->     ->               ->
    operating plans for
    next year.

    The site begins          The site establishes a Subcommittee
    developing strategies      subcommittee to look continues to search for
    for long-term self-      for additional funding additional resources to
    sustainment.          ->   or in-kind resources sustain the program.    ->
                                    to sustain the
                                       program.

    The site begins to develop plans The site identifies an The site undergoes
    By the end of the year,

    for evaluating its programs. 	evaluation tool and program evaluation to
    the site releases its full begins developing an assess overall evaluation
    report, evaluation based on effectiveness of official assessing the
    overall Executive Office for recognition strategy program Weed and Seed
    site implementation. accomplishments and its evaluation literature.
    effectiveness.

    Source: EOWS data.

    Note: Arrow indicates action from the previous year continues to the
    current year.

    aA site is a geographically defined area ranging in size from several
    neighborhood blocks to several square miles. With the input of the U.S.
    Attorney's Office and the steering committee, each site develops and
    implements its own Weed and Seed strategy.

    bA grantee is an entity that receives funding from EOWS to implement the
    Weed and Seed program. The grantee distributes the funds to sites and is
    responsible for ensuring that sites comply with the terms of the grant. A
    grantee may have more than one site.

    cOfficial recognition is a designation given to a grantee by EOWS that
    signifies that EOWS has approved the grantee's Weed and Seed strategy in a
    specific site. A grantee must receive official recognition from EOWS
    before a grantee can apply for funding. Official recognition requires,
    among other things, that the site develop a management plan and a
    comprehensive strategy for implementing the Weed and Seed strategy.

    dThe four required program elements that sites are required to address are
    (1) law enforcement; (2) community policing; (3) crime and substance
    prevention, intervention, and treatment; and (4) neighborhood restoration.

    Appendix V: Activity Data Collected by EOWS in Fiscal Year 2003

    Activity data collected by EOWS

    1. Does site include a multijurisdictional task force

    2. Does site participate in Project Safe Neighborhoods

    3. Does site have foot patrols

    4. Does site have bike patrols

    5. Does site have substations

    6. Does site have crime watch

    7. Do police participate in community meetings

    8. 	Number of homicides for 3 preceding calendar years for Weed
    and Seed site

    9. 	Number of homicides for 3 preceding calendar years for entire
    jurisdiction

    a

    10. Number of total drug arrests

    11. Number of heroin drug arrests

    12. Number of cocaine (not including crack) drug arrests

    13. Number of crack cocaine drug arrests

    14. Number of marijuana drug arrests

    15. Number of methamphetamines only drug arrests

    16. Number of safe haven facilities

    17. Number of safe havens receiving EOWS funding

    18. Does site provide academic courses and tutoring

    19. Does site provide mentoring

    20. Does site provide prevention education

    21. Does site provide dispute resolution and mediation

    22. Does site provide recreation and athletics

    23. Does site provide job training

    24. Does site provide job placement

    25. Does site provide antidrug education

    26. Does site provide community police co-located in safe
    haven

    27. Does site provide safe corridors (school escorts for
    children)

    28. Does site provide summer day camp

    29. Does site provide youth leadership training

    30. Does site provide boys and girls club programs

    31. Does site provide scouting programs

    32. Does site provide military cadet training

    33. Does site provide antigang education and training

    Appendix V: Activity Data Collected by EOWS in Fiscal Year 2003

    Activity data collected by EOWS

    34. Does site provide Communities in Schools programs

    35. Does site provide performance or applied arts programs

    36. Does site provide victim assistance programs

    37. Does site provide community projects, such as cleanups

    38. Does site provide general health-screening services

    39. Does site provide lead-poisoning-screening service

    40. Number of persons receiving safe haven services

    41. Number of community development corporations within
    site area

    42. Is site constructing and renovating housing
    developments

    43. Is site constructing and renovating commercial
    developments

    44. Is site constructing and renovating business and
    community partnerships

    45. Number of community cleanups done using EOWS
    support

    46. Do Weed and Seed activities relate to or involve
    Brownfields Assessment Demonstration Pilots

    47. Do Weed and Seed activities relate to or involve
    Brownfields Cleanup and Revolving Load Fund Pilots

    48. Do Weed and Seed activities relate to or involve
    Brownfields Showcase Communities

    49. Do Weed and Seed activities relate to or involve
    Brownfields Tax Incentive

    50. Source of non-EOWS fundingb

    51. Amount of non-EOWS grant funding

    52. Source and type of in-kind contributions
    Source: EOWS data.
    aIn addition to the number of drug arrests, EOWS also requests that each
    site report the number of sale, manufacturing, and possession drug arrests
    for each type of drug.

    bNon-EOWS grant funding includes funding from other federal, state, local,
    and private sources.

    Appendix VI: Description of EOWS's Ongoing and Completed Studies

    Title of study and When results are research partner Purpose of study
    expected

    A Comparison of Compare homicides in Weed and Seed sites with homicides in
    the jurisdictions that Report was expected
    Homicide Trends in sites are located in, to determine whether homicides
    will decline faster in Weed and in January 2004 but
    Local Weed and Seed Seed sites than in the remainder of the jurisdiction.
    was issued in
    Sites Relative to Their November 2003
    Host Jurisdictions,
    1996 to 2001

    Justice Research and
    Statistics Association
    (JRSA)

    Performance Indicator Develop additional performance indicators that will
    be used by EOWS and Weed and March 2004 Study Seed sites to evaluate
    program performance with respect to crime control.

    Urban Institute

    Weed and Seed Crime Summarize the types of crime selected as "target"
    crimes by Weed and Seed sites March 2004 Pattern Data Collection (sites
    choose three crimes) to determine crime reduction strategies. Assess the

    degree of success of Weed and Seed sites by analyzing the percentage of
    reduction Justice Research and in crime compared with the percentage in
    the rest of the jurisdiction that they are

    Statistics Association located in.

           Meta-Analysis        Summarize the completed evaluations of   Mid 2004
                                 individual Weed and Seed sites. The
                                  analysis will include more than 80
             Evaluation          studies. JRSA's preliminary analysis
                                              (March 3,
                              2003) includes 36 evaluations. Of the 36
                              evaluations, JRSA determined that 5 were
        Justice Research and   process, 19 were impact, and 9 were both
                                 process and impact evaluations. The
    Statistics Association      remaining 3 evaluations were not
                                   identified as process or impact.
                              Note: A process evaluation addresses
                              whether the program is working as
                              intended.
                              An impact evaluation isolates the effects
                              of a particular program or factor from all
                              other potential contributing factors that
                                      could also effect change.

    Local Pilot Evaluations	Conduct local evaluations of Weed and Seed sites.
    In fiscal year 2002, EOWS funded Mid 2004 13 local evaluations. Each site
    contracts with its own researcher.

    Local researchers

                        Source: EOWS and research organizations.

    Appendix VII: Comments from the Department of Justice

    Appendix VII: Comments from the Department of Justice

    Appendix VII: Comments from the Department of Justice

    Appendix VII: Comments from the Department of Justice

    Appendix VII: Comments from the Department of Justice

    Appendix VIII: GAO Contacts and Staff Acknowledgments

    GAO Contacts

      Staff Acknowledgments

              Laurie Ekstrand (202) 512-2758 Weldon McPhail (202) 512-8644

    In addition to those named above, the following individuals made key
    contributions to this report: Lisa G. Shibata, Kirstin B. L. Nelson, Cady
    L. Summers, David Alexander, Sidney Schwartz, Kevin L. Jackson, Elizabeth
    Curda, Tom James, Denise Fantone, Jan Montgomery, Ann H. Finley, Katherine
    M. Davis, and Leo Barbour.

    Related GAO Products

    Law Enforcement: Better Performance Measures Needed to Assess Results of
    Justice's Office of Science and Technology, GAO-04-198. Washington, D.C.:
    November 14, 2003.

    Standards for Internal Control in the Federal Government.
    GAO/AIMD-00-21.3.1. Washington, D.C.: November 1999.

    Federal Grants: More Can Be Done to Improve Weed and Seed Program
    Management. GAO/GGD-99-110. Washington, D.C.: July 16, 1999.

    Small Business Administration: Enhancements Needed for Loan Monitoring
    System Benchmark Study. GAO/AIMD-99-165. Washington, D.C.: May 14, 1999.

    Managing for Results: Measuring Program Results That Are Under Limited
    Federal Control. GAO/GGD-99-16. Washington, D.C.: December 11, 1998.

    Managing for Results: Analytic Challenges in Measuring Performance.
    GAO/HEHS/GGD-97-138. Washington, D.C.: May 30, 1997.

    Federal Grants: Design Improvements Could Help Federal Resources Go
    Further. GAO/AIMD-97-7. Washington, D.C.: December 18, 1996.

    Highway Safety: Safety Belts Use Laws Save Lives and Reduce Costs to
    Society. GAO/RCED-92-106. Washington, D.C.: May 1992.

      GAO's Mission

    Obtaining Copies of GAO Reports and Testimony

    The General Accounting Office, the audit, evaluation and investigative arm
    of Congress, exists to support Congress in meeting its constitutional
    responsibilities and to help improve the performance and accountability of
    the federal government for the American people. GAO examines the use of
    public funds; evaluates federal programs and policies; and provides
    analyses, recommendations, and other assistance to help Congress make
    informed oversight, policy, and funding decisions. GAO's commitment to
    good government is reflected in its core values of accountability,
    integrity, and reliability.

    The fastest and easiest way to obtain copies of GAO documents at no cost
    is through the Internet. GAO's Web site (www.gao.gov) contains abstracts
    and fulltext files of current reports and testimony and an expanding
    archive of older products. The Web site features a search engine to help
    you locate documents using key words and phrases. You can print these
    documents in their entirety, including charts and other graphics.

    Each day, GAO issues a list of newly released reports, testimony, and
    correspondence. GAO posts this list, known as "Today's Reports," on its
    Web site daily. The list contains links to the full-text document files.
    To have GAO e-mail this list to you every afternoon, go to www.gao.gov and
    select "Subscribe to e-mail alerts" under the "Order GAO Products"
    heading.

    Order by Mail or Phone 	The first copy of each printed report is free.
    Additional copies are $2 each. A check or money order should be made out
    to the Superintendent of Documents. GAO also accepts VISA and Mastercard.
    Orders for 100 or more copies mailed to a single address are discounted 25
    percent. Orders should be sent to:

    U.S. General Accounting Office 441 G Street NW, Room LM Washington, D.C.
    20548

    To order by Phone: 	Voice: (202) 512-6000 TDD: (202) 512-2537 Fax: (202)
    512-6061

    To Report Fraud,	Contact: Web site: www.gao.gov/fraudnet/fraudnet.htm

      Waste, and Abuse in E-mail: fraudnet@gao.gov

    Federal Programs Automated answering system: (800) 424-5454 or (202)
    512-7470

    Jeff Nelligan, Managing Director, NelliganJ@gao.gov (202) 512-4800

    Public Affairs 	U.S. General Accounting Office, 441 G Street NW, Room 7149
    Washington, D.C. 20548
    *** End of document. ***