    Homeland Security: Federal Protective Service Could Better
    Measure the Performance of Its Control Centers (29-SEP-06,
    GAO-06-1076).

    The Department of Homeland Security's Federal Protective Service
    (FPS) through its control centers (MegaCenters) helps provide for
    the security and protection of federally owned and leased
    facilities. This report (1) identifies the services MegaCenters
    provide, (2) determines how FPS assesses MegaCenter performance
    and whether FPS links MegaCenter performance measures to FPS-wide
    measures, and (3) examines how MegaCenters and selected
    organizations compare in the services they provide. To address
    these issues, GAO reviewed FPS's performance measures and past
    MegaCenter assessments, assessed the MegaCenters' performance
    measures, and interviewed officials and collected relevant
    information at FPS, the four MegaCenters, and nine selected
    security organizations.
    -------------------------Indexing Terms-------------------------
    REPORTNUM:   GAO-06-1076
        ACCNO:   A61609
      TITLE:     Homeland Security: Federal Protective Service Could
    Better Measure the Performance of Its Control Centers
         DATE:   09/29/2006
      SUBJECT:   Facility security
    	     Federal facilities
    	     Federal property management
    	     Homeland security
    	     Monitoring
    	     Performance measures
    	     Physical security
    	     Police
    	     FPS MegaCenters

    ******************************************************************
    ** This file contains an ASCII representation of the text of a  **
    ** GAO Product.                                                 **
    **                                                              **
    ** No attempt has been made to display graphic images, although **
    ** figure captions are reproduced.  Tables are included, but    **
    ** may not resemble those in the printed version.               **
    **                                                              **
    ** Please see the PDF (Portable Document Format) file, when     **
    ** available, for a complete electronic file of the printed     **
    ** document's contents.                                         **
    **                                                              **
    ******************************************************************
    GAO-06-1076

         * Report to the Chairman, Committee on Government Reform, House of
           Representatives
              * September 2006
         * HOMELAND SECURITY
              * Federal Protective Service Could Better Measure the Performance
                of Its Control Centers
         * Contents
              * Results in Brief
              * MegaCenters Provide Alarm and Radio Monitoring and Dispatch from
                Four Locations
              * MegaCenters' and FPS's Performance Measures Are Not Linked
              * Nine Selected Organizations Provide Some of the MegaCenters'
                Services in a Generally Comparable Manner
              * Conclusions
              * Recommendations for Executive Action
              * Agency Comments
         * Briefing for the Committee on Government Reform, U.S. House of
           Representatives, August 14, 2006
         * Comments from the Department of Homeland Security
         * GAO Contact and Staff Acknowledgments
         * PDF6-Ordering Information.pdf
              * Order by Mail or Phone

                     United States Government Accountability Office

    Report to the Chairman, Committee on Government Reform, House of
    Representatives

    September 2006

    HOMELAND SECURITY

     Federal Protective Service Could Better Measure the Performance of Its Control
                                        Centers

                                           a

    GAO-06-1076

    HOMELAND SECURITY

    Federal Protective Service Could Better Measure the Performance of Its
    Control Centers

      What GAO Found

    FPS MegaCenters provide three primary security services-alarm monitoring,
    radio monitoring, and dispatching of FPS police officers and contract
    guards. These and other services are provided around the clock from four
    locations-Battle Creek, Michigan; Denver, Colorado; Philadelphia,
    Pennsylvania; and Suitland, Maryland. With a fiscal year 2006 budget of
    $23.5 million, the MegaCenters monitor alarms at over 8,300 federal
    facilities, covering almost 381 million square feet, and have available
    for dispatch over 7,800 FPS police officers and contract guards.

                       The MegaCenter Emergency Response Process

    Source: GAO analysis of FPS data.

    FPS MegaCenter managers assess MegaCenter operations through a variety of
    means, including reviewing data about volume and timeliness of operations,
    listening to and evaluating a sample of calls between operators and FPS
    police officers and contract guards, and receiving informal feedback about
    customer satisfaction. FPS managers have also developed performance
    measures for assessing MegaCenter operations. However, these measures are
    of limited use because they are not always clearly stated or measurable
    and do not address governmentwide priorities of efficiency, cost of
    service, and outcome-which are among the attributes that GAO has
    identified for successful performance measures. In addition, the
    MegaCenters do not measure a key activity-the time from alarm to officer
    dispatch-that would link MegaCenter performance to an FPS-wide performance
    measure of response time. Without a corresponding measure, FPS is limited
    in its ability to evaluate the MegaCenters' contribution to the FPS-wide
    measure of response time.

    Nine selected security organizations-including federal and local police
    and private entities-offer some of the MegaCenters' services as well as
    provide and assess these services in a manner that is generally similar to
    the MegaCenters. Like the MegaCenters, many of the selected organizations
    have centralized their operations. They also use regular call reviews and
    volume and time measures to assess the quality of the services they
    provide. A major difference between the MegaCenters and some selected
    organizations is the use of a computer-aided dispatch system, which
    enables these organizations to automate many functions.

                     United States Government Accountability Office

    Contents

    Letter                                                                   1
                                Results in Brief                                3
                                MegaCenters Provide Alarm and Radio Monitoring
                                and Dispatch
                                from Four Locations                             5
                                MegaCenters' and FPS's Performance Measures     6
                                Are Not Linked
                                Nine Selected Organizations Provide Some of
                                the MegaCenters'
                                Services in a Generally Comparable Manner       9
                                Conclusions                                    10
                                Recommendations for Executive Action           11
                                Agency Comments                                12
    Appendixes
                    Appendix I: Briefing for the Committee on Government
                                Reform, U.S.
                                House of Representatives, August 14, 2006      14
                  Appendix II:  Comments from the Department of Homeland       61
                                Security
                  Appendix III: GAO Contact and Staff Acknowledgments          63

                                     Abbreviations

    CAD                 computer-aided dispatch
    DHS                 Department of Homeland Security
    FPS                 Federal Protective Service
    GSA                 General Services Administration
    ICE                               U.S. Immigration and Customs Enforcement
    IT                  information technology
    UL                  Underwriters Laboratories

    This is a work of the U.S. government and is not subject to copyright
    protection in the United States. It may be reproduced and distributed in
    its entirety without further permission from GAO. However, because this
    work may contain copyrighted images or other material, permission from the
    copyright holder may be necessary if you wish to reproduce this material
    separately.

    A

    United States Government Accountability Office Washington, D.C. 20548

    September 29, 2006

    The Honorable Tom Davis Chairman Committee on Government Reform House of
    Representatives

    Dear Mr. Chairman:

    Since the 1995 bombing of the Alfred P. Murrah Federal Building in
    Oklahoma City and the September 11, 2001, attacks on the World Trade
    Center and the Pentagon, terrorism has threatened the nation's security,
    including the physical security of federal facilities. The Homeland
    Security Act of 2002 created the Department of Homeland Security (DHS), a
    new federal department with the mission of preventing terrorist attacks
    within the United States, which includes safeguarding federal facilities.1
    DHS, through its Federal Protective Service (FPS), provides law
    enforcement and security services to federal agencies that occupy
    facilities under the jurisdiction of the General Services Administration
    (GSA) and DHS, protecting millions of federal employees, contractors, and
    citizens.2 As part of its approach to facility protection, FPS provides
    support for its law enforcement and security services through four control
    centers known as MegaCenters that are located in Battle Creek, Michigan;
    Denver, Colorado; Philadelphia, Pennsylvania; and Suitland, Maryland.

    Because of the important role MegaCenters play in ensuring the safety of
    federal facilities and their occupants, our objectives were to (1)
    identify the services the MegaCenters provide and how they provide them,
    (2) determine how FPS assesses and measures the performance of MegaCenter
    operations and how FPS links MegaCenter performance measures to FPS-wide
    performance measures, and (3) examine how the MegaCenters compare to
    selected security organizations in the services they provide and in the
    methods they use to provide them.

    To determine the services offered by the MegaCenters and how the
    MegaCenters provide these services, we interviewed managers at the four

    1Federal facilities include government-owned and -leased space.

    2Under agreement, FPS authority can be extended to provide its law
    enforcement and security services to any property with a significant
    federal interest.

    MegaCenters and the MegaCenter branch chief, toured three
    MegaCenters-Denver, Philadelphia, and Suitland, collected information on
    services and workload data from MegaCenter management, and conducted
    document reviews. To determine how FPS assesses MegaCenter performance and
    how FPS links MegaCenter performance measures to FPS-wide performance
    measures, we reviewed documentation on FPS's performance measures and past
    MegaCenter assessments and interviewed MegaCenter management and FPS
    headquarters officials. We also assessed the MegaCenters' 11 performance
    measures to determine whether they were consistent with selected
    attributes of successful performance measures we have identfied, that is,
    that they were linked to agency mission and goals; clearly stated;
    contained measurable targets; sufficiently covered the program's core
    activities; and addressed governmentwide priorities of quality,
    timeliness, efficiency, cost of service, and outcome.3

    To determine how the MegaCenters compare to selected security
    organizations we (1) identified criteria for selecting comparable
    organizations and selected four public and five private organizations; (2)
    interviewed officials at the nine selected organizations; (3) toured four
    of the organizations' control centers-both public and private-to observe
    their security operations, procedures, and technology; (4) collected
    detailed service and workload information from the organizations; (5) used
    the information gathered to compare the MegaCenters and the selected
    organizations with respect to services offered, organizational structure,
    quality assessment practices, and technology utilization; and (6)
    interviewed officials from security industry standard-setting and
    accreditation associations (associations) because these associations were
    identified as having information on security industry organizations,
    operations, quality assessment practices, and technology utilization
    practices.4 We used two approaches for selecting the private and public
    organizations. We selected private organizations from industry lists of
    the top 20 largest security service providers and system integrators in
    terms of 2005 revenue that provided services such as alarm monitoring and
    access

    3We did not assess whether the performance measures were objective,
    reliable, overlapping, or balanced.

    4We interviewed officials from the following associations: Association of
    Public Safety Communications Officials, the Commission on Accreditation
    for Law Enforcement Agencies, the National Burglar and Fire Alarm
    Association, the National Fire Protection Association, Priority Dispatch,
    the Security Industry Association, and Underwriters Laboratories.

    control.5 With the help of one of the associations we interviewed, we were
    able to make contact with the five security organizations that were
    selected as our comparison group.6 For public organization comparisons, we
    selected three federal organizations: U.S. Postal Inspection Service, U.S.
    Customs and Border Protection, and U.S. Park Police. We selected these
    federal organizations because each had a law enforcement branch;
    centralized control center(s) that offered, at a minimum, one of the
    MegaCenters' primary services; and nationwide operations, characteristics
    most similar to those of the MegaCenters. We also selected the Denver
    Police Department to serve as a nonfederal public organization that
    provided dispatch and radio monitoring services through a central control
    center. Because we judgmentally selected the organizations, the
    information we collected from them cannot be generalized. We conducted our
    work from October 2005 through September 2006 in accordance with generally
    accepted government auditing standards.

    This report summarizes the information we provided to your office during
    our August 14, 2006, briefing and, in addition, contains recommendations
    to the Secretary of Homeland Security to improve the MegaCenters'
    performance measures. The briefing slides are included in appendix I.

                                    Results in Brief

    The primary services that FPS MegaCenters provide are remote monitoring of
    building alarm systems as well as radio monitoring and dispatch of FPS
    police officers and contract guards. These and other services are provided
    around the clock from four locations across the country: Battle Creek,
    Denver, Philadelphia, and Suitland. Each MegaCenter has a sister center
    with redundant capabilities that can serve as an emergency backup, and
    each is operated by full-time federal employees and private contractors.
    In fiscal year 2006, the MegaCenters had a budget of $23.5 million-
    accounting for about 5 percent of FPS's total budget-to protect and

    5We looked at two lists: one was in terms of gross revenue, the other was
    in terms of revenue.

    6In specific situations, two of the private security organizations
    dispatched either contract guards or their own guards. However, because
    these were situational circumstances, we did not include them as being
    similar to the MegaCenters' dispatching service. For example, one
    organization only uses contract guards when local jurisdictions will not
    dispatch their own police officers without actual verification of the
    nature of the alarm. In these cases, the organization will have a guard
    check whether there is a burglary in progress, and if so, the organization
    will contact the police.

    monitor over 8,300 federal facilities and dispatch over 7,800 FPS police
    officers and contract guards. To provide these services, the MegaCenters
    rely on a variety of information technology (IT) systems, communications
    systems, and other equipment.

    FPS MegaCenter managers assess MegaCenter activities through a variety of
    means, including reviewing reports on the timeliness and volume of
    operations, listening to and evaluating a sample of calls between
    operators and FPS police officers and contract guards, and receiving
    informal feedback about customer satisfaction. Also, FPS managers have
    developed performance measures for assessing MegaCenter operations. These
    performance measures reflect some of the attributes of successful
    performance measures we have identified, but also contain some weaknesses
    because they are not always clearly stated or measurable and do not
    address the governmentwide priorities of efficiency, cost of service, and
    outcome. In addition, the MegaCenters do not have a performance measure
    that corresponds to the FPS-wide performance measure that is applicable to
    the MegaCenters' operations-the patrol and response time measure, which
    tracks the elapsed time from the receipt of an alarm to a police officer's
    arrival on the scene.7 While the FPS-wide patrol and response time measure
    covers the MegaCenters' activities and reflects their performance, the
    MegaCenters do not have their own measure that covers only the activities
    for which they are responsible-from the receipt of the alarm to the
    officer's dispatch. Without clearly stated and measurable performance
    measures, including a measure that corresponds to FPS's agencywide patrol
    and response time measure, FPS cannot compare the MegaCenters' performance
    over time, assess their contribution to agencywide measures, and identify
    opportunities for their improvement. We are recommending that FPS (1)
    establish MegaCenter performance measures that meet the attributes of
    successful performance measures we have identified; (2) develop a
    performance measure for the MegaCenters that corresponds to the FPS-wide
    patrol and response time measure and covers the MegaCenters' operations,
    from alarm to dispatch; and (3) routinely assess the extent to which the
    MegaCenters meet established performance measures. In commenting on a
    draft of this report, DHS generally agreed with our findings and
    recommendations.

    7In addition to patrol and response time, FPS has three other agencywide
    performance measures (1) timely deployment of countermeasures, (2)
    functionality of countermeasures, and (3) facility security index (an
    average success rate for the other measures).

    Page 4 GAO-06-1076 Federal Protective Service

    The nine selected security organizations offer some of the same security
    services as the MegaCenters, and the services the organizations offer are
    delivered and assessed in a manner generally similar to that of the
    MegaCenters. For example, like the MegaCenters, many organizations have
    centralized their control center operations, have backup capability,
    allocate workload among centers based on geographic location, and use
    regular call reviews as well as volume and time measures to assess the
    quality of the services they provide. A few organizations offer services
    that the MegaCenters do not offer. One major difference between the
    MegaCenters and the organizations is that three organizations use a
    computer-aided dispatch (CAD) system. Selected organizations and
    associations referred to CAD systems as being beneficial for dispatching
    services by allowing for faster operator response, automatic recording of
    all operator actions enabling easier performance analysis, and automatic
    operator access to standard operating procedures and response
    prioritization. MegaCenters have identified a need and developed a plan
    for a CAD system, but FPS has not allocated funding for such a system.

    MegaCenters Provide Alarm and Radio Monitoring and Dispatch from Four
    Locations

    FPS MegaCenters provide federal agencies with three primary security
    services-alarm monitoring, radio monitoring, and dispatch-through four
    locations using a variety of IT systems. MegaCenters monitor intrusion,
    panic, fire/smoke, and other alarms. They also monitor FPS police
    officers' and contract guards' radio communication to ensure their safety
    and to provide information, such as criminal background or license plate
    histories, to officers upon request. In addition, they exercise command
    and control authority by dispatching FPS police officers or contract
    guards. MegaCenters also provide a variety of other services. For example,
    they notify federal agencies regarding national emergencies and facility
    problems and remotely diagnose problems with federal agency alarms. They
    also receive and transcribe FPS police officer incident reports.
    Individual MegaCenters may also provide unique services not provided by
    other MegaCenters, such as facility-specific access control and remote
    programming of alarms via the Internet. One MegaCenter also provides an
    after-hours telephone answering service for the Drug Enforcement
    Administration and for GSA building maintenance emergencies.

    The MegaCenters are located in Battle Creek, Denver, Philadelphia, and
    Suitland. Each MegaCenter has a sister center with redundant capability as
    backup in case of a failure at that MegaCenter. Suitland is paired with
    Battle Creek, and Philadelphia is paired with Denver. A force of 1,014 FPS
    police officers and 6,842 contract guards is available for the MegaCenters
    to dispatch in response to alarms and other emergencies. In fiscal year
    2006, the MegaCenters were supported by a budget of $23.5 million, which
    accounts for about 5 percent of FPS's total budget.8 The MegaCenters are
    operated by 23 full-time federal employees-some of whom manage the
    centers-and about 220 private contractors to provide around the clock
    security services for over 8,300 federal facilities.

    The MegaCenters rely on a variety of IT systems, communications systems,
    and other equipment to provide their security services. The IT systems
    enable MegaCenter staff to, among other activities, monitor alarms and
    radio communications of FPS police officers and contract guards. For
    communications systems, MegaCenters have regional and national toll-free
    numbers for tenants and the public to contact the MegaCenters during
    emergencies. Other equipment includes dictation machines, which enable FPS
    police officers to dictate reports about incidents that occur at
    facilities.

    MegaCenters' and FPS's Performance Measures Are Not Linked

    MegaCenters use various means to assess operations, but their performance
    measures have weaknesses and are not linked to FPS-wide performance
    measures. MegaCenter managers assess MegaCenter operations through a
    variety of means, including reviewing data about volume and timeliness of
    operations, listening to and evaluating a sample of calls between
    operators and FPS police officers and contract guards, and receiving
    informal feedback about customer satisfaction. FPS managers also have
    developed 11 performance measures for assessing MegaCenter operations:

         o distribute emergency notification reports (also known as SPOT reports)
           within 30 minutes of notification;
              * review problem alarm reports daily;
              * o  obtain regular feedback about customer satisfaction from field
                operations;
         o continuously review all SPOT reports and other outgoing information to
           ensure 100 percent accuracy;

    8As of August 2006, MegaCenter officials reported a revised MegaCenter
    budget of approximately $19 million for fiscal year 2006.

    Page 6 GAO-06-1076 Federal Protective Service

         o transcribe dictated offense and incident reports into the database
           management system within 8 hours of receipt of the report;
         o submit reviewed contractor billing reports and time sheets within 7
           business days after the last day of the month;
         o prepare and review contractor reports for quality assurance plan;
         o maintain completely accurate (nonduplicative) case control numbers;
         o meet Underwriters Laboratories (UL) guidelines and requirements
           continuously;9
         o test failover of alarm, radio, and telephone systems weekly;10 and
         o monitor calls and review recorded call content for adherence to
           standard procedures at least monthly.

    The Government Performance and Results Act of 1993 requires federal
    agencies to, among other things, measure agency performance in achieving
    outcome-oriented goals. Measuring performance allows organizations to
    track the progress they are making toward their goals and gives managers
    critical information on which to base decisions for improving their
    progress. We have previously reported on some of the most important
    attributes of successful performance measures. These attributes indicate
    that performance measures should (1) be linked to an agency's mission and
    goals; (2) be clearly stated; (3) have quantifiable targets or other
    measurable values; (4) be reasonably free of significant bias or
    manipulation that would distort the accurate assessment of performance;

    (5) provide areliable way to assess progress; (6) sufficiently cover the
    program's core activities; (7) have limited overlap with other measures;
    (8) have balance or not emphasize one or two priorities at the expense of

    9UL certifies control centers that provide all elements of service
    required by UL's standards, including appropriate operator response to
    fire alarm signals and proper equipment inspection, testing, and
    maintenance.

    10"Failover" is the capability to switch over automatically to a redundant
    or standby system in the event of a system failure. Each MegaCenter tests
    its ability to run its sister-center's operations in case that center has
    system failure.

    Page 7 GAO-06-1076 Federal Protective Service

    others; and (9) address governmentwide priorities of quality, timeliness,
    efficiency, cost of service, and outcome.11

    We assessed the 11 FPS MegaCenter performance measures against selected
    attributes: linkage to mission and goals, clarity, and measurable targets.
    Ten of the 11 MegaCenter performance measures were aligned with FPS's
    mission to protect federal properties and personnel and with the
    MegaCenter program's mission to provide high-quality and standardized
    alarm monitoring, radio monitoring, and dispatch. We found no link between
    timely review of contractor time sheets and billing statements and FPS's
    mission, however, primarily because this measure seems to be related to
    administrative activities. In addition, while 6 of the 11 performance
    measures have measurable targets-a key component for measuring
    performance, none of the MegaCenter performance measures met the clarity
    attribute because FPS could not provide information about how managers
    calculate the measures-a key component in the clarity attribute. For
    example, the performance measure that the centers test the failover
    ability of alarm, radio, and telephone systems weekly is measurable
    because it has a quantifiable target but does not meet the clarity
    attribute because FPS could not describe its methodology for calculating
    it.

    We also assessed whether, collectively, the MegaCenters' 11 performance
    measures sufficiently cover their core program activities (i.e., alarm
    monitoring, radio monitoring, and dispatch) and address governmentwide
    priorities of quality, timeliness, efficiency, cost of service, and
    outcome. Most of the MegaCenter performance measures relate to the three
    core activities. For example, regular feedback on customer service and
    monthly review of operator calls cover aspects of the dispatch and
    radio-monitoring functions. Other performance measures, like distributing
    emergency notification reports in 30 minutes, help fulfill other critical
    support functions. However, two performance measures-reviewing contractor
    quality assurance plans and timely review of contractor time sheets and
    billing statements-relate to administrative activities that are not
    strictly related to MegaCenter core activities. Additionally, the
    MegaCenter performance measures do not collectively address all of the
    governmentwide priorities. The MegaCenter performance measures primarily
    address the governmentwide priorities of quality and timeliness.

    11See GAO, Tax Administration: IRS Needs to Further Refine Its Tax Filing
    Season Performance Measures, GAO-03-143 (Washington, D.C.: Nov. 22, 2002),
    pp.2-3, 46-53.

    Page 8 GAO-06-1076 Federal Protective Service

    For example, the MegaCenter measures pertaining to transcribing reports
    within 8 hours and reviewing recorded calls to see if the operator
    followed standard operating procedures address aspects of service
    timeliness and quality, respectively. None of the measures relate to the
    governmentwide priorities of efficiency, cost of service, and outcome.

    Finally, FPS does not link MegaCenter performance measures to FPS-wide
    performance measures, specifically the patrol and response time measure.
    FPS established FPS-wide performance measures to assess its efforts to
    reduce or mitigate building security risks. The performance measures that
    FPS established were (1) timely deployment of countermeasures, (2)
    functionality of countermeasures, (3) patrol and response time, and (4)
    facility security index. The one measure that relates to the MegaCenters-
    patrol and response time-assesses FPS's ability to respond to calls for
    service and measures the average elapsed time from when a law enforcement
    request is received (e.g., alarm, telephonic request from a building
    tenant, FPS police officer-initiated call) to the time an officer arrives
    at the scene. FPS's goal is to reduce response times by 10 percent in
    fiscal year 2006. The MegaCenters are responsible for part of the patrol
    and response activity that is being measured because the MegaCenters
    receive alarms and emergency calls and dispatch FPS police officers or
    contract guards to the scene. However, although data pertaining to this
    activity exist in the MegaCenters' records management system, they do not
    measure the timeliness of this activity, and FPS has not developed a
    performance measure that would identify the MegaCenters' contribution
    toward meeting FPS's measure.

    Nine Selected Organizations Provide Some of the MegaCenters' Services in a
    Generally Comparable Manner

    The nine selected security organizations generally do not provide all
    three of the MegaCenters' primary services. However, the services these
    organizations offer are provided similarly by the MegaCenters with the
    exception of a CAD system, which three organizations use and the
    MegaCenters do not. The MegaCenters provide three primary services (i.e.,
    alarm monitoring, radio monitoring, and dispatch), and the selected
    organizations provide all or some of these three main services. For
    example, the Park Police provide all three services, while the private
    organizations focus on providing alarm monitoring and offer some services
    the MegaCenters do not. Like the MegaCenters, all of the private
    organizations reviewed have centralized operations: the number of their
    national control centers ranges from two to five. Work allocation (i.e.,
    how incoming alarms and calls are assigned) among centers varies by
    organization but overall is similar to the MegaCenter structure. For
    example, most of the organizations assign calls and alarms to a specific
    center based on the geographic location of the call or signal. However,
    the Postal Inspection Service and one private organization are unique
    because they are able to allocate workload to centers based on demand and
    operator availability. The organizations use a variety of methods to
    measure the quality of their services, many similar to methods used by the
    MegaCenters. For example, like the MegaCenters, most review a sample of
    operator calls on a regular basis. Two entities have established
    measurable performance goals for their centers. While there are
    similarities in the services offered, number of centers, work allocation,
    and service quality appraisals between the organizations reviewed and the
    MegaCenters, three organizations use a CAD system, which the MegaCenters
    do not. A CAD system is a tool used by the Denver Police Department for
    dispatching and officer tracking and by the Postal Inspection Service for
    officer tracking. The Park Police also uses a CAD system with limited
    capabilities at its San Francisco center and plans to purchase and upgrade
    the system for all three of its centers. Selected organizations and
    associations referred to CAD systems as being beneficial for dispatching
    services by allowing for faster operator response, automatic operator
    access to standard operating procedures and response prioritization, and
    automatic recording of operator actions enabling easier performance
    analysis. Since 2003, FPS and DHS both have assessed MegaCenter technology
    and have identified needs for technology upgrades, including the
    installation of a CAD system for the MegaCenters. Our guide on IT
    investment decision making-based on best practices in the public and
    private sector-stresses that part of achieving maximum benefits from an IT
    project requires that decisions be made on a regular basis about the
    status of the project.12 To make these decisions, senior managers need
    assessments of the project's impact on mission performance and future
    prospects for the project. While the MegaCenters have assessed their
    technology on many occasions and have determined that some refreshment is
    needed, FPS has not yet allocated the funding for such upgrades.

    Conclusions

    FPS MegaCenters play a key role in protecting federal facilities, those
    who
    enter these facilities, and the FPS police officers and contract guards
    whose calls the MegaCenters respond to and monitor. How well the
    MegaCenters are fulfilling their role and carrying out their
    responsibilities

    12GAO, Assessing Risks and Returns: A Guide for Evaluating Federal
    Agencies' IT Investment Decision-making, GAO/AIMD-10.1.13 (Washington,
    D.C.: February 1997).

    Page 10 GAO-06-1076 Federal Protective Service

    is uncertain because they do not generate much of the information that
    would be useful for assessing their performance.

    To their credit, the MegaCenters have established performance measures for
    a number of their activities and operations, and these measures are
    aligned with the MegaCenters' mission. However, the measures have
    weaknesses, both individually and collectively, compared with the selected
    attributes of successful performance measures that we have identified.
    Many of the individual measures are neither quantifiable nor clearly
    stated, and collectively the measures do not address the governmentwide
    priorities of efficiency, cost of service, and outcome. As a result, FPS
    cannot compare performance across the MegaCenters or over time, and
    without such information, FPS is limited in its ability to identify
    shortfalls and target improvements.

    Although FPS has established an FPS-wide performance measure for response
    time-from the alarm to the FPS police officer's arrival on the scene-that
    incorporates the MegaCenters' operations, the MegaCenters have not
    established a comparable measure for their operations alone. Without such
    a measure, FPS cannot evaluate the MegaCenters' contribution-from the
    alarm to the FPS police officer's dispatch-to the FPS-wide measure for
    response time and identify opportunities for improvement.

    Recommendations for Executive Action

    We recommend that the Secretary of Homeland Security direct the Director
    of the Federal Protective Service to take the following three actions:

         o establish MegaCenter performance measures that meet the attributes of
           successful performance measures we have identified;
         o develop a performance measure for the MegaCenters that is directly
           linked to the FPS-wide response time measure and covers the scope of
           the MegaCenters' operations, from alarm to dispatch; and
         o routinely assess the extent to which the MegaCenters meet established
           performance measures.

    Agency Comments

    We provided a draft of this report to DHS, the Department of the Interior,
    and the U.S. Postal Service for their review and comment. DHS provided
    comments in a letter dated September 6, 2006, which are summarized below
    and reprinted in appendix II. DHS also provided technical comments, which
    we incorporated into the report where appropriate. The Postal Service
    informed us that it had no comments on this report. The Department of the
    Interior did not provide comments on this report.

    DHS generally agreed with the report's findings and recommendations. DHS
    stated that FPS and the U.S. Immigration and Customs Enforcement (ICE)
    have undertaken a comprehensive review of the MegaCenters to identify,
    among other things, ways in which performance can be better measured. DHS
    noted that through this broad approach, FPS personnel will be able to
    generate and track the kind of information necessary to assess the
    MegaCenters' performance. This one-time review may help FPS identify
    information needed to assess the MegaCenters' performance and, therefore,
    develop appropriate performance measures. In order to reliably assess
    performance over time, FPS should not only establish appropriate
    performance measures, but also routinely assess performance using these
    measures. We therefore clarified our recommendation to include the routine
    use of established performance measures to assess the MegaCenters'
    performance. With regard to the report's discussion of CAD system
    capabilities, DHS said that ICE's Chief Information Officer is currently
    assessing the MegaCenters' technology requirements and recognizes that
    previous studies have identified the need for technology upgrades. DHS
    indicated that the current assessment will have a meaningful impact on
    FPS's technology capabilities.

    As agreed with your office, unless you publicly announce the contents of
    this report earlier, we plan no further distribution until 30 days from
    the report date. At that time, we will send copies of this report to other
    interested congressional committees and the Secretary of Homeland
    Security, and DHS's Assistant Secretary for Immigration and Customs
    Enforcement. We will also make copies available to others upon request. In
    addition, the report will be available at no charge on the GAO Web site at
    http://www.gao.gov.

    If you have any questions regarding this report, please contact me at
    (202) 512-2834 or sciremj@gao.gov. Contact points for our Offices of
    Congressional Relations and Public Affairs may be found on the last page
    of this report. GAO staff who made major contributions to this report are
    listed in appendix III.

    Sincerely yours,

    Mathew J. Scire Acting

    Director, Physical Infrastructure Issues

    Appendix I

    Briefing for the Committee on Government Reform, U.S. House of Representatives,
    August 14, 2006

                 Review of the Federal Protective Service's MegaCenters

                    Briefing for the Committee on Government Reform

                      U.S. House of RepresentativesAugust 14, 2006

         o Introduction and Objectives
         o Scope and Methodology
         o Results in Brief
         o Background
         o Results of GAO Work
         o Since the 1995 bombing of the Alfred P. Murrah Federal Building in
           Oklahoma City and the September 11, 2001, attacks on the World Trade
           Center and Pentagon, terrorism has threatened the nation's security,
           including the physical security of federal facilities.
         o The Homeland Security Act of 2002 created the Department of Homeland
           Security (DHS), a new federal department with the mission of
           preventing terrorist attacks within the United States, which includes
           safeguarding federal facilities.1
         o DHS, through its Federal Protective Service (FPS), provides law
           enforcement and security services to federal agencies that occupy
           almost 9,000 facilities under the jurisdiction of the General Services
           Administration (GSA) and DHS, protecting millions of federal
           employees, contractors, and citizens. Under agreement, FPS authority
           can be extended to provide its law enforcement and security services
           to any property with a significant federal interest.
         o As part of its approach to facility protection, FPS provides support
           for its law enforcement and security services through four control
           centers (known as MegaCenters) located in Battle Creek, Michigan;
           Denver, Colorado; Philadelphia, Pennsylvania; and Suitland, Maryland.

    1Facilities include government owned and lease space.

            * Because of the important role MegaCenters play in assuring the
              safety of federal facilities and their occupants, our objectives
              were to:
                 * Identify the services the MegaCenters provide and how they
                   provide them.
                 * Determine how FPS assesses and measures the performance of
                   MegaCenter operations and how FPS links MegaCenter performance
                   measures to FPS-wide performance measures.
                 * Examine how the MegaCenters compare to selected security
                   organizations in the services they provide and in the methods
                   they use to provide them.
            * Document review: Reviewed the Memorandum of Agreement between GSA
              and FPS and other documentation related to MegaCenter services as
              well as documentation related to (1) FPS's request for a computer
              aided dispatch (CAD) system for the MegaCenters; (2) past FPS
              assessments of MegaCenter operations; (3) FPS's performance
              measures; and
            * (4) FPS'sbudget for the MegaCenters.
         o Interviews: Interviewed FPS officials, including MegaCenter branch
           chief and managers, and staff from the Program Review Office,
           Financial Management Division, and other offices; Immigration and
           Customs Enforcement's (ICE) Budget Enforcement Office; and officials
           from selected public and private organizations; officials from
           security industry standard setting and accreditation associations
           (associations).
            * Selected organizations:
                 o U.S. Customs and Border Protection
                 o U.S. Park Police
                 o U.S. Postal Inspection Service
                 o Denver Police Department
                 o 5 private security companies
         o We conducted our review in accordance with generally accepted
           government auditing standards.
         o Remote monitoring of building alarm systems, radio monitoring, and
           dispatching of FPS police officers and contract guards are the primary
           services FPS MegaCenters provide. These and other services are
           provided around the clock from four locations across the country. Each
           MegaCenter has a sister center with redundant capabilities that can
           serve as an emergency backup and each is operated by full-time federal
           employees and private contractors. In addition, the MegaCenters have a
           fiscal year 2006 budget of $23.5 million and use a variety of
           information technology (IT) systems and other equipment to provide
           their services.
              * FPS MegaCenter managers assess MegaCenter operations through a
                variety of means, including reviewing information on the
                timeliness and volume of operations, listening to and evaluating
                a sample of calls between operators and FPS police officers and
                contract guards, and receiving informal feedback about customer
                satisfaction. FPS managers have also developed performance
                measures for assessing MegaCenter operations. Although these
                MegaCenter measures reflect some attributes of successful
                performance measures, they also contain some weaknesses because
                they are not always clearly stated or measurable, and do not
                address governmentwide priorities of efficiency, cost of service,
                and outcome. In addition, they do not directly measure key
                operations that would link to FPS-wide performance measures,
                which are (1) the timely deployment of countermeasures, (2)
                functionality of countermeasures, (3) patrol and response time,
                and
              * (4) facility security index.
         o The nine selected organizations offer some of the MegaCenters' primary
           services, and they deliver and assess the services they offer in a
           generally similar manner to the MegaCenters. For example, like the
           MegaCenters, many of these organizations have centralized their
           control center operations, have backup capability, allocate workload
           among control centers based on geographic location, and use regular
           call reviews as well as volume and time measures to assess the quality
           of the services they provide. A few organizations offer services the
           MegaCenters do not offer. One difference between the MegaCenters and
           the selected organizations is that three of these organizations use a
           CAD system, which the MegaCenters do not have. The MegaCenters have
           assessed their technology and have identified the need for a CAD;
           however FPS has not allocated funds for such a purchase.
            * FPS operations are solely funded through security fees and
              reimbursements collected from federal agencies for FPS security
              services.
                    * These security fees consist of basic and building-specific
                      security charges.
                         o The basic security charges cover the security services
                           that FPS provides to all federal tenants in
                           FPS-protected buildings, which include such services
                           as patrol, monitoring of building perimeter alarms and
                           dispatching of law enforcement response (MegaCenter
                           operations), criminal investigations, and security
                           surveys.
                         o The building-specific security charges are for FPS
                           security measures that are designed for a particular
                           building and are based on the FPS Building Security
                           Assessment and its designated security level. Such
                           measures include contract guards, X-ray machines,
                           magnetometers, cameras, and intrusion detection
                           alarms. Also, the tenant agencies may request
                           additional security services such as more guards,
                           access control systems, and perimeter barriers.
                         o The above two charges are billed monthly to the tenant
                           agencies. The basic security charge is the same for
                           all tenants regardless of the type of space occupied
                           and is a square footage rate. The building-specific
                           security charge reflects FPS cost recovery for
                           security measures specific to a particular building
                           and the billing is handled differently for single-and
                           multi-tenant buildings. Single tenant buildings-the
                           tenant agency is billed for the total cost of the
                           security measures. Multi-tenant buildings-the tenant
                           agencies are billed based on their pro rata share of
                           the square feet occupied within the respective
                           building.
                 o FPS uses a reimbursable program to charge individual agencies
                   for additional security services and equipment that they
                   request above the level determined for their building.
            * FPS bills the tenant agencies for FPS security fees they have
              incurred.
                 o The agencies pay the fees into an FPS account in the
                   Department of the Treasury, which is administered by FPS.
                   Congress exercises control over the account through the annual
                   appropriations process that sets an annual limit-called
                   obligation authority-on how much of the account FPS can expend
                   for various activities.
                 o FPS uses the security fees to finance its various activities
                   within the limits that Congress sets. The Department of
                   Homeland Security Appropriations Act for fiscal year 2006
                   authorized $487 million in obligation authority for FPS
                   expenses and operations. Through FPS's security fees, funds
                   are to be collected and credited to FPS's account as an
                   offsetting collection from tenant agencies.
         o Under the FPS reimbursable program, agencies request additional
           security services and equipment using a funded Security Work
           Authorization. Once the services are provided and the costs are
           expensed, FPS bills the agency for the costs, and the funds are
           transferred to the FPS account to offset the expenses FPS incurred.
         o The DHS Inspector General reported in 2006 that when FPS was part of
           GSA it budgeted and paid for FPS's annual administrative support costs
           such as financial management, human capital, and IT using funds beyond
           those generated by security fees. GSA estimated these FY 2003 support
           services to cost about $28 million. According to the report, beginning
           in FY 2004, neither DHS's annual budget request nor DHS's
           appropriations set aside funding for FPS's support services. In FY
           2004, as a component of DHS, FPS paid almost $24 million for support
           services using funds from security fees only; a year earlier these
           services had been funded by GSA using funds not derived from fees.
         o Before GSA established the MegaCenters, FPS used regional and
           satellite control centers to monitor alarm systems, dispatch FPS
           police officers and contract guards, and perform criminal background
           checks. In total, there were 22 regional control centers and 12
           satellite control centers, which were located throughout FPS's 11
           regions. Most regions had more than 1 control center.
         o In 1991, GSA conducted an internal review of the control centers. The
           review found that because of significant budgetary and personnel
           constraints over more than a decade, the control centers no longer
           performed well enough to ensure safe, effective, and efficient FPS
           actions to preserve life and property. GSA contracted with Sandia
           National Laboratories- the lead laboratory for U.S. Department of
           Energy security systems- to conduct an in-depth study of the control
           centers' operation and make recommendations.
         o In 1993, Sandia issued its study entitled GSA Control Center Upgrade
           Program. The Sandia study identified serious shortfalls and problems
           that would require a more radical upgrade of the control centers at a
           much higher cost than originally believed. After validating the
           study's findings, GSA determined that a multimillion dollar upgrade of
           all control centers would be prohibitively expensive. The study noted
           that the control centers could be consolidated to almost any level to
           achieve economies of scale. However, the study recommended against a
           single national-level control center because a second center would be
           needed to continue operations under catastrophe or failover
           conditions.
         o GSA concluded that the control center problems that the study
           identified were material weaknesses and reported them to Congress. FPS
           conducted an operational and technical review of the Sandia study's
           findings, which provided a critical assessment of the control centers,
           a high-level concept of operations for the centers, and functional
           specifications for upgrading the centers. GSA decided to upgrade 11
           control centers-one in each region-and address the weaknesses that the
           study had identified.
         o Within GSA, concerns were raised about the cost of upgrading 11
           control centers, how many control centers were really needed, and
           whether the centers' operations could be outsourced. GSA established a
           project team to investigate these concerns. The team contacted several
           public and private sector organizations that operate control centers.
           The team found that the organizations were consolidating their control
           centers but were unable to assume the operations of FPS control
           centers. A decision was made to consolidate additional centers and the
           multi-regional control center or "MegaCenter" concept was developed.
           GSA endorsed the MegaCenter concept. GSA assembled a core project team
           and hired contractors to design, plan, and supervise the construction
           of the centers.
         o In 1994, GSA issued a bid for MegaCenter technical and performance
           specifications and awarded the contract.
         o In 1996, FPS reaffirmed that the MegaCenter concept was the best
           approach for addressing the control center weaknesses. GSA selected
           the MegaCenters sites: Denver, CO; Battle Creek, MI; New York, NY and
           Suitland, MD.2
         o In 1996, construction began on the Denver MegaCenter and design was
           initiated on the Battle Creek MegaCenter.
         o In 1997, the Denver MegaCenter was opened, followed by Battle Creek in
           1999, Suitland in 2000, and Philadelphia in 2001.

    2Site selection for New York, NY was never finalized and eventually was
    switched to a site in Philadelphia, PA.

         o The MegaCenters' mission is to provide the highest quality, nationally
           standardized dispatch, alarm monitoring, and federal law enforcement
           emergency response services. Based on this mission statement, we chose
           to focus on alarm monitoring, radio monitoring, and dispatch as the
           primary services the FPS MegaCenters provide.
              * Primary services:
                   o Alarm monitoring: monitoring intrusion, panic, fire/smoke,
                     elevator, and/or environmental alarms.
                   o Radio monitoring: monitoring FPS police officers' and
                     contract guards' radio communication for safety and
                     providing information upon request.
                   o Dispatch: exercising command and control authority by
                     dispatching FPS police officers and/or contract guards.

    Objective 1: MegaCenters provide other services

            * Other services:
                 o Notifying federal agencies regarding national emergencies and
                   facility problems (also known as SPOT reports)
                 o Checking criminal background histories (including inquiries to
                   the National Crime Information Center database, which the
                   Federal Bureau of Investigation (FBI) maintains) for FPS
                   police officers responding to a call or an alarm or FPS
                   regions and other DHS agencies requesting this assistance
                 o Receiving and transcribing FPS police officer reports
                 o Providing a toll-free help desk line for agency support,
                   including remote diagnosis and service of alarms
                 o Updating quarterly building and emergency contact information
                   from customer agencies to ensure accurate notifications and
                   alerts
            * Unique services (provided by individual MegaCenter):
                 o Monitoring and controlling access to buildings
                 o Using the Internet to program, monitor, and test alarms
                 o Providing after-hours telephone answering service for Drug
                   Enforcement Administration and for GSA maintenance

    Table 1: Services Provided to MegaCenter Clients

                        Services                  GSA, DHS, and      FPS regions,
                                                  federal agency police officers,
                                                     clients         and contract
                                                                      guards
    Monitor remote building alarms
                                                        X               X
    Dispatch FPS police officers, contract
    guards,
    and other response services                      X               X
    Monitor FPS police officers' and contract
    guards' radio communications for their                           X
    safety
    Keep track of FPS police officer and
    contract
    guard locations                                                  X
    Provide warrant and other information from
    the National Crime Information Center            X               X
    Help maintain lists of buildings and
    contacts
    for emergency notifications                      X               X

            * MegaCenters are located in four cities and are responsible for
              providing various services in their respective FPS Regions:
                 o Philadelphia, PA (FPS Regions 1, 2, 3)
                 o Battle Creek, MI (FPS Regions 4, 5, 6)
                 o Denver, CO (FPS Regions 7, 8, 9, 10)
                 o Suitland, MD (FPS Region 11-National Capital Region)
            * Each MegaCenter is paired with a sister center:
                 o Suitland is paired with Battle Creek
                 o Philadelphia is paired with Denver
         o Sister center pairings provide for redundant capability in case of a
           catastrophic failure at any MegaCenter.
         o MegaCenters rely upon contractor staff to carry out dispatch and
           technical support services.
         o Each MegaCenter also has FPS officials located on site to oversee the
           center's overall operations.
         o MegaCenters operate around the clock.

    Table 2: FY 2006 Actual Federal and Contractor Staffing Levels at Each
    MegaCenter

        MegaCenter  Federal staff       Primary Contractor       Contractor staff
    Battle Creek       9        Wackenhut Services Inc.a             74
    Denver             4        Gonzales Consulting Servicesa        56
    Philadelphia       6        Gonzales Consulting Services         49
    Suitland           4        Gonzales Consulting Services         38
              Total      23                                            217

    aThe Battle Creek and Denver MegaCenters have other contracts for
    personnel to provide services, such as technical support, however, these
    contracts are small in terms of the number of personnel provided- three in
    Denver and five in Battle Creek.

    Objective 1: Historical budget and MegaCenter allocations

            * Recent FPS MegaCenter budgets:
                 o Fiscal year 2004 $20.0 million
                 o Fiscal year 2005 $21.5 million
                 o Fiscal year 2006 $23.5 million, which accounts for less than 5
                   percent of FPS's total budget.4
            * Estimated fiscal year 2006 budget for each MegaCenter:
                 o Suitland $6.0 million
                 o Philadelphia $5.0 million
                 o Battle Creek $6.5 million
                 o Denver $6.0 million
         o MegaCenter cost information for FY 2004 and prior years is not
           available since FPS transitioned from GSA's accounting system. FY 2005
           and FY 2006 cost information is also not available because the ICE
           accounting system was not modified to capture costs solely for the
           MegaCenters. According to FPS, it is working with ICE to establish the
           capability in the accounting system to capture MegaCenter costs in the
           future.

    4As of August 2006, MegaCenter officials reported a revised MegaCenter
    budget of approximately $19 million for fiscal year 2006.

    Objective 1: MegaCenters rely on IT systems to deliver services

    o  MegaCenters depend on a variety of IT systems and equipment to deliver
    their primary services.

    Table 3: Information Technology Systems That Support MegaCenters' Primary
    Services

    Information technology systems  Alarm monitoring Radio monitoring Dispatch
    Security Information System and
    software (alarm receivers and
    signal receivers)                      X                X            X
    Facility and building client
    enterprise information systems         X                X            X
    MegaCenter-owned telephone
    exchange system and regional           X                X            X
    and national toll-free
    emergency numbers
    Remote programming software            X
    Radio systems                          X                X            X
    Patrol and dispatch operations
    logs                                   X                X            X
    Voice audio recorders                  X                X            X
    Software that allows
    MegaCenters to access federal                           X            X
    and state law enforcement
    databases

    Source: GAO analysis of FPS data

         o MegaCenters also depend on additional information technology systems,
           such as failover equipment and servers that facilitate sharing files
           with sister centers.
         o Reports on operator activities-FPS MegaCenter supervisors-staff who
           are responsible for overseeing operators-review information about the
           timeliness and volume of operators' activities. For example, they
           review reports that describe how long it took operators to send out
           emergency notifications and transcribe dictated reports and the number
           of problem alarms, among other things.
         o Assessments of operator communications-Supervisors and designated
           quality assurance staff listen to live conversations between operators
           and FPS police officers and contract guards as well as regularly
           listen to a sample of taped conversations to identify whether
           operators are following standard operating procedures. According to
           MegaCenter managers, staff sample taped calls on a monthly basis.
         o Comprehensive reviews of MegaCenter operations-In 2003 and 2004, the
           MegaCenter managers completed a manager review of each MegaCenter.
           These reviews were replaced by more comprehensive program reviews in
           which FPS regional staff with subject area expertise were to review
           each MegaCenter and report on concerns and best practices related to
           MegaCenter management, administration, technology, and equipment. In
           2005, the Philadelphia MegaCenter was the first and only MegaCenter to
           undergo a program review. Program reviews were to be conducted
           annually at each MegaCenter, however, the MegaCenter managers
           suspended these reviews due to budgetary constraints.
         o Feedback on customer satisfaction-MegaCenter managers and supervisors
           use informal means for gathering information about the level of
           customer satisfaction. For example, they receive information from
           conversations with FPS region program staff and FPS police officers
           who, according to MegaCenter managers, do not hesitate to inform them
           of performance concerns. Managers also obtain feedback from federal
           building tenants and agencies during routine activities, such as when
           they update their emergency contact database, and from regional staff
           by attending regional staff meetings.
         o Underwriters Laboratories (UL) control center certification-Although
           not an assessment by the MegaCenter staff themselves, the UL
           certification process involves feedback from UL inspectors about
           whether the MegaCenter meets technical and performance standards. UL
           inspectors complete initial inspections to certify the MegaCenters and
           conduct regular inspections once a center is certified. The Denver
           MegaCenter was UL certified in 2003 and has since had a subsequent
           inspection. The Philadelphia MegaCenter's initial UL inspection is
           scheduled to be completed in August 2006. Initial UL inspections at
           the remaining two MegaCenters have not been scheduled.
         o Performance measurement-FPS has established performance measures for
           the MegaCenters.
              * FPS has identified 11 performance measures for MegaCenter
                operations.
                   o Distribute emergency notification reports (SPOT reports)
                     within 30 minutes of notification.
                   o Review problem alarm reports daily.
                   o Obtain regular feedback about customer satisfaction from
                     field operations.
                   o Continuously review all SPOT reports and other outgoing
                     information to ensure 100 percent accuracy.
                   o Transcribe dictated offense and incident reports into
                     database management system within 8 hours of receipt of the
                     report.
                   o Submit reviewed contractor billing reports and time sheets
                     within 7 business days after the last day of the month.
                   o Prepare and review contractor reports for quality assurance
                     plan.
                   o Maintain completely accurate (non-duplicative) case control
                     numbers.
                   o Meet UL guidelines and requirements continuously.5
                   o Test failover of alarm, radio, and telephone systems
                     weekly.6
                   o Monitor calls and review recorded call content for adherence
                     to standard procedures at least monthly.

    5UL certifies centers that provide all elements of service required by
    UL's standards, including appropriate operator response to fire alarm
    signals and proper equipment inspection, testing, and maintenance.
    6Failover is the capability to switch over automatically to a redundant or
    standby system in the event of a system failure. Each MegaCenter tests its
    ability to run 26 its sister-center's operation in case that center has
    system failure.

         o The Government Performance and Results Act of 1993 requires federal
           agencies to develop strategic plans, link them with outcome-oriented
           goals, and measure agency performance in achieving these goals.
           Measuring performance allows organizations to track the progress they
           are making toward their goals and gives managers critical information
           on which to base decisions for improving their progress.
              * Organizations need to have performance measures that (1)
                demonstrate results, (2) are limited to the vital few,
              * (3) cover multiple program priorities, and (4) provide useful
                information for decision making in order to track how their
                programs and activities can contribute to attaining the
                organization's goals and mission.
              * We have previously reported on some of the most important
                attributes of successful performance measures. These attributes
                indicate that performance measures should:
                   * Be linked to an agency's mission and goals;
                   * Be clearly stated: a measure's name and definition are
                     consistent with the methodology used to calculate it;
                   * Have quantifiable targets or other measurable values;
                   * Be reasonably free of significant bias or manipulation that
                     would distort the accurate assessment of performance;
                   * Provide a reliable way to assess progress;
                   * Sufficiently cover the program's core activities;
                   * Have limited overlap with other measures;
                   * Have balance or not emphasize one or two priorities at the
                     expense of others; and
                   * Address governmentwide priorities of quality, timeliness,
                     efficiency, cost of service, and outcome.
         o We assessed the 11 FPS MegaCenter performance measures against
           selected attributes of successful performance measures: linkage to
           performance goals and mission, clarity, and measurable targets. We
           also assessed whether collectively the 11 performance measures
           sufficiently cover the MegaCenters' core program activities and
           address governmentwide priorities of quality, timeliness, efficiency,
           cost of service, and outcome.
         o Ten of the eleven MegaCenter performance measures were aligned with
           FPS's mission to protect and secure federally owned and leased
           properties and personnel and with the MegaCenter program's mission to
           provide high quality and standardized alarm monitoring, radio
           monitoring, and dispatch. For example, distributing emergency
           notification reports (SPOT reports) to federal agencies within 30
           minutes helps protect federal personnel and properties by alerting
           federal managers of suspicious activities near their locations. We
           could find no link between timely review of contractor time sheets and
           billing statements and FPS's mission primarily because this measure
           seems to be related to administrative activities.
         o MegaCenter performance measures lacked clarity and measurable targets.
           Although some of the measures seemed clearly stated, FPS could not
           provide information about how managers calculate any of the measures-a
           key component in the clarity attribute. Six of eleven performance
           measures have measurable targets-a key component for measuring
           performance. For example, the MegaCenter measure for testing the
           failover ability of alarm, radio, and telephone systems on a weekly
           basis is measurable because it has a quantifiable target but does not
           meet the clarity attribute because FPS could not describe the
           methodology for calculating this measure.
         o Seven of the eleven MegaCenter performance measures relate to the
           MegaCenters' core activities. For example, daily review of problem
           alarm reports and weekly failover of alarm systems relate to the
           MegaCenters' alarm monitoring operation. Regular feedback on customer
           service and monthly review of operator calls cover aspects of the
           radio monitoring and dispatch functions. Other performance measures,
           like distributing emergency notification reports in 30 minutes, help
           fulfill other critical support functions. However, two of the
           performance measures-the timely review of contractor time sheets and
           preparing and reviewing contractor quality assurance plans-seem to
           relate to administrative activities and are not strictly related to
           MegaCenter core activities.
         o The MegaCenter performance measures primarily address the
           governmentwide priorities of quality and timeliness. For example,
           transcribing reports within 8 hours and reviewing recorded calls to
           see if the operator followed standard procedures address aspects of
           service timeliness and quality, respectively. None of the measures
           relate to the other governmentwide priorities of efficiency, cost of
           service, and outcome.
         o FPS has identified four performance measures to assess its efforts to
           reduce or mitigate building security risks.
         o One of these measures-patrol and response time-assesses FPS's ability
           to respond to calls for service within certain time limit goals. It
           measures the average elapsed time from when a MegaCener receives a law
           enforcement request (e.g., an alarm, telephone request from a building
           tenant, FPS officer initiated call) to the time an FPS officer arrives
           at the scene. In fiscal year 2005, FPS reported a national average
           response time of 47 minutes. Its goal is to reduce response time by 10
           percent in fiscal year 2006.

    Table 4: FPS-wide Performance Measures for Facility Protection

    Performance measure                   Purpose
    Timely deployment of countermeasures  To compare actual deployment dates
                                             with planned deployment dates
    Countermeasure functionality (e.g.,   To gauge whether those security
    surveillance cameras, X-ray machines) countermeasures for which FPS is
                                             responsible are working as intended,
                                             once deployed
    Patrol and response time              To assess FPS's ability to respond
                                             to calls for service within certain
                                             time limit goals
    Facility security index               To calculate FPS's average success
                                             rate for the above three performance
                                             measures

    Source: GAO, Homeland Security: Guidance and Standards Are Needed for
    Measuring the Effectiveness of Agencies' Facility Protection Efforts,
    GAO-06-612 (Washington, D.C.: May 2006).

         o FPS does not link the MegaCenter performance measures to the FPSwide
           performance measures, specifically, the FPS-wide patrol and response
           time measure.
         o The MegaCenters are responsible for part of the patrol and response
           activity that is being measured because MegaCenter operators are in
           control of the alarm and call response and dispatch operations.
           However, although some time-related data is recorded in their records
           management system, the MegaCenters do not measure the timeliness of
           this activity and FPS has not developed a performance measure that
           would identify the MegaCenters' contribution toward meeting the
           FPSwide measure.
         o The Park Police was the only selected organization that provides all
           three primary services offered by the MegaCenters. The remaining
           selected organizations provide one or two of these MegaCenter primary
           services.

    Table 5: MegaCenters' Primary Services Offered by Selected Organizations

                Organization           Alarm monitoring Radio monitoring Dispatch
    Federal Protective Services            X                X            X
    U.S. Customs and Border                                 X
    Protection
    U.S. Park Police                       X                X            X
    U.S. Postal Inspection Service         X                X
    Denver Police Department                                X            X
    Private security organizations         X

            * Selected organizations also offer other services the MegaCenters
              offer.
                 o Many organizations record telephone and radio communications
                   in case these communications need to be reviewed; officials
                   from the associations stated that reviewing calls to ensure
                   procedures are followed correctly is an important control
                   center practice.
                 o All public organizations provide information to officers based
                   on inquires to the FBI's National Crime Information Center
                   among other databases.7
                 o All private organizations provide access control services that
                   allow them to track and restrict facility access.
                 o One private organization monitors system conditions separately
                   to distinguish between an intrusion, door propped open,
                   tamper, or long access, which assists in determining the
                   appropriate action to take.
         o A few organizations offer services the MegaCenters do not.

    o  Three private organizations provide audio and/or video monitoring
    services which allow operators to remotely listen and/or view live audio
    and/or video transmission from the secured site. Officials from two
    organizations reported that this type of surveillance technology minimizes
    the number of on-site guards required for their clients. In addition, this
    surveillance technology provides operators with additional intelligence
    information to help them decide what further actions to take.

    7Like the MegaCenters, Customs and Border Protection, Postal Inspection
    Service, and Park Police also retrieve information for officers from the
    National Law Enforcement Telecommunication System and the Denver Police
    Department retrieves information for officers from the Colorado 33 Crime
    Information Center database.

    o  Workloads and staffing vary by public organization. While Postal
    Inspection Service monitors more facilities than the MegaCenters, it does
    not dispatch officers. Park Police, an organization that does dispatch
    officers, has far fewer officers available to dispatch than the
    MegaCenters and, because the Park Police has a different purpose, it
    monitors far fewer facilities than the MegaCenters.

    Table 6: Workload Statistics of Selected Public Organizations' Centers

          Public      Center Purpose    Number  Facilities Dispatchable  Center
    organization                       of    monitored   personnel   staffinga
                                        centers
                    Monitor alarm
    Federal      systems of federal     4      8,328       7,856        240
                    facilities and
                    dispatch
    Protective   FPS police officers
                    and contract guards
                    to these
    Service      facilities, if
                    necessary. Monitor
                    the radios of
                    officers
                    and guards to
                    ensure their safety
                    and to provide
                    information to
                    officers.
                    Monitor radios
    U.S. Customs nationwide for all     1       Not     Not             71
                    customs                                applicable
                    investigations
    and Border   at ports to ensure          applicable
                    officer safety and
                    coordinate
    Protection   response, if
                    necessary.
    U.S. Park    Monitor alarms and
    Police       dispatch Park          3       875b        717         36
                    Police officers and
                    law enforcement
                    rangers to protect
                    national monuments
                    and parks. Monitor
                    officer radios to
                    ensure
                    officer safety and
                    provide information
                    to officers.
                    Monitor postal                         Not
    U.S. Postal  facilities' alarm      2      11,448   applicable      67
                    systems and radio
    Inspection   communication of
                    Postal Inspection
                    police and federal
    Service      agents to ensure
                    officer safety and
                    to provide
                    information to
                    officers.
    Denver       Provide emergency
    Police       services to the        1       Not        1,452       129c
                    city and county of
    Department   Denver.                     applicable

    aThis includes both full-time federal employees and contract staff, where
    applicable.

    bThis is the approximate number of alarms monitored by Park Police's two
    centers in the District of Columbia and San Francisco. The New York
    control center does not monitor alarms. cOne hundred and twenty-nine is an
    approximation. Denver Police Department reported 131 personnel, 4 are part
    time.

    o  The selected organizations have organizational structures similar to
    the MegaCenters.

            * Like the MegaCenters, all five private organizations monitor alarms
              through centralized control centers.
                 o Each of the five private organizations have between 2 to 5
                   national central control centers.
                 o One private organization official stated that centralizing and
                   consolidating monitoring services allows for easier staffing,
                   better customer service, and higher quality technology.
         o Like the MegaCenters, all public and private organizations have backup
           ability, with the exception of Customs and Border Protection.8

    o  Officials from the organizations and associations we interviewed cited
    the importance of center redundancy in the event of catastrophic failure
    at any one control center.

    8Park Police's control center in the District of Columbia has backup
    capabilities for all three primary services. The centers in San Francisco
    and New York are able to backup their radio and call taking functions,
    however, the San Francisco center is not able to backup its alarm
    monitoring function.

            * Similar to the MegaCenters, most of selected organizations assign
              calls and alarms to a specific control center based on the
              geographic location of the call or alarm signal.
            * o  Some organizations reported that this type of work allocation is
              beneficial because it reduces stress for the operator and provides
              better customer service. For example, one private organization
              reported that its operators become familiar with both the people
              who call the control centers and their locations which allows them
              to provide better directions to responders.
            * Postal Inspection Service and one private organization are able to
              allocate workload to their control centers based on demand and
              operator availability. For example, when an alarm signal or
              telephone call comes into Postal Inspection Service, its software
              decides which center should receive the signal or call based on
              center workload.
                    * Both organizations reported efficiencies from this type of
                      work allocation.
                         o Postal Inspection Service reported a reduction in
                           required center staff.
                         o The private organization reported the ability to close
                           some of its control centers during non-peak hours,
                           reducing costs.
                 o Both organizations attribute the ability to allocate work in
                   this manner to more advanced technology.9

    9Postal Inspection Service can only allocate work in this manner for its
    alarm monitoring and call taking service. Radio signals must be assigned
    to a specific center because the technology at the time Postal Inspection
    Service built its control centers was not yet developed to allow radio 36
    signals to be assigned to first available center.

    o  The selected organizations use a variety of methods to ensure centers
    are meeting their goals and providing quality services, many similar to
    the MegaCenters.

    Table 7: Selected Organizations' Methods to Ensure Quality Service

         Organization      Regular   Operator pay Volume      Regular and formal
                         review of a  incentives  and/or time     process for
                          sample of    based on   measures    soliciting customer
                          operators  performance                   feedback
                            calls
    Federal                X                        X
    Protective
    Service
    U.S. Customs and       X                        X
    Border
    Protectiona
    U.S. Park Police
    U.S. Postal            X                        X
    Inspection
    Service
    Denver Police          X           X
    Department
    Private security      Xb           X           Xb              Xb
    organizations

    aCustoms and Border Protection officials reported that they are in the
    process of developing a formal customer survey tool. bIndicates that at
    least one of the selected private security organizations used these
    methods.

            * Regular review of calls
            * o  Most officials from the associations we interviewed agreed that
              competent control centers have a quality assurance process that
              includes regular review of a random sample of calls to ensure that
              the operators asked the correct questions and provided the correct
              information when dispatching and to check the timelines and timing
              of responses. FPS MegaCenter supervisors monitor each of their
              operators calls on a monthly basis to ensure correct procedures are
              followed.
            * Pay incentives
            * o  One private organization not only conducts call reviews but also
              ties pay incentives to its quality assessment program by linking
              employee call review evaluations to spot awards and bonuses. None
              of the federal agencies reviewed, including the MegaCenters, used
              these types of pay incentives.
            * Volume and time measurements
                 o Examples of measurements are the amount of time it takes to
                   answer a call, time taken to act on a call, and number of
                   complaints from responding officers. Although the MegaCenters
                   have volume and time measurements, they do not report the time
                   it takes from receipt of alarm signal to officer dispatch.
                   Some of the measures the MegaCenters use include the time
                   elapsed between an officer calling in an incident and its
                   entry into the records management system and the time it takes
                   to send emergency notification reports to higher management.
                   They also report volume statistics to their regions, such as
                   the number of cases they have opened in a month.
                 o Officials from Customs and Border Protection and the
                   Commission on Accreditation for Law Enforcement Agencies
                   stated that many volume and time measurements cannot be used
                   in law enforcement because timing can depend on circumstances.
                   For example, the time it takes a Customs and Border Protection
                   operator to clear a radio call would not be an appropriate
                   measure because some of their functions are more tactical in
                   nature- for example providing radio support in an undercover
                   investigation- and cannot accurately be measured with time and
                   volume statistics only.
            * Customer satisfaction
                 o Two private organizations regularly and formally survey their
                   customers to find out if operators were pleasant, delivered
                   timely service, and seemed to be well trained and informed.
                 o Other organizations, like MegaCenters, use a more informal
                   process to gauge customer satisfaction. To formalize its
                   customer feedback, the Suitland MegaCenter developed a survey
                   to gauge officer satisfaction, however, it is waiting for
                   approval to use it.
            * o  Customs and Border Protection has hired a private consultant to
              help them develop a customer satisfaction survey and believe its
              use, in conjunction with volume statistics, will provide a more
              meaningful measure of their service quality.
            * Performance measures
                 o One private organization and the Postal Inspection Service not
                   only perform assessment activities but also have established
                   measurable performance goals for their control centers.
                   Examples of the private organization's goals are: resolution
                   of 90 percent of high priority events within 60 seconds and
                   resolution of 85 percent of inbound and technical assistance
                   specialist calls within 18 seconds.
                 o Postal Inspection Service has begun to establish volume and
                   time measurement goals, such as a 25 percent reduction in the
                   number of complaints in which standard operating procedures
                   were not followed and a 10 percent reduction in the time it
                   takes to answer radio calls. They intend to revise their goals
                   in fiscal year 2007 once they have actual experience against
                   which to set a benchmark.
                 o The MegaCenters have created performance measures, however,
                   they lack some attributes of successful performance measures,
                   such as measurable targets.
         o The MegaCenters perform dispatch and incident management functions
           manually while Denver Police Department uses a CAD system to perform
           these functions and the Postal Inspection Service uses a CAD system to
           track officer locations and perform incidentmanagement functions. The
           Park Police's San Francisco center also uses a CAD system with limited
           capabilities, and Park Police has a plan in place to purchase and
           upgrade a system for its three centers.
         o MegaCenter operators keep track of officers on duty and their
           locations by hand or by using an Excel spreadsheet, and when they
           dispatch an officer to a federal building they must enter the address
           location into the database. In comparison, when a PostalInspection
           Service operator tracks an officer to a building, the CAD system
           automatically populates the address field and the system shows all
           contact information for that building. Without a CAD system, the
           operators spend more time retrieving information from different
           sources and entering data, such as client contact information.
         o CAD systems automate dispatch and incident management functions and
           allow for more efficient handling of incidents. Typical CAD system
           functions include management of call routing and prioritization,
           dispatching, and response procedures. For example, a CAD system can
           decide which control center a call should go to based on workload,
           prioritize the call for the operator, and automatically display to the
           operator the actions to take.CAD systems record times of incidents,
           locations, and corresponding actions by dispatchers and officers which
           allows for analysis to be conducted to determine, forexample, response
           times, workload, the types of incidents requiring response,
           andresource allocation needs.
         o Selected organizations and associations stated that CAD systems are
           beneficial for dispatching services by allowing for faster operator
           response, automatic recording of operator actions enabling easier
           performance analysis, and automatic operator access to standard
           operating procedures and response prioritization. For example, when a
           signal comes into a Postal Inspection Service control center, the CAD
           system automatically retrieves and displays the response actions for
           the operator to follow.
         o FPS has repeatedly recognized the need for a CAD system. In 1993, the
           Sandia study of FPS's former control centers recommended that the
           centers have a CAD system for the most effective and time efficient
           dispatch operations while using a minimum center personnel structure.
           A 2003 DHS MegaCenter review also stated that the MegaCenters needed a
           CAD system. In January 2006, FPS issued a request for information for
           a CAD system. No funding has been allocated in fiscal year 2006 for
           the MegaCenters to purchase a CAD system.
         o Our guide on IT investment decision-making- based on best practices in
           the public and private sectors- stresses that part of achieving
           maximum benefits from an IT project requires that decisions be made on
           a regular basis about the status of the project. To make these
           decisions, senior managers need assessments of the project's impact on
           mission performance and future prospects for the project.
         o Senior managers should regularly question whether (1) the current
           system meets organizational needs, (2) the system should be modified
           to better meet these needs, (3) a new system is needed to best meet
           these needs, or (4) the needs could best be met by outsourcing the
           work. Included in the regular review should be costs for operation and
           maintenance of the project, such as hardware upgrades, system software
           changes, and ongoing user training. Successful IT management requires
           that a plan be developed for the continued support and operation of
           every IT project.10
              * While the MegaCenters have assessed their technology on several
                occasions and have determined that some refreshment is needed, no
                funding has been allocated by FPS for this use.
                   o When the MegaCenters were established, FPS intended
                     scheduled replacement of their technical systems after 5
                     years, but this timeline has passed. In addition, a 2003 DHS
                     MegaCenter review found that there were various states of
                     technologies in systems across centers and proposed planning
                     for life cycle equipment replacement. However, according to
                     FPS officials, no technology replacement program has been
                     established, and equipment is replaced on a per MegaCenter,
                     as-needed basis.
                   o A 2005 program review of the Philadelphia MegaCenter found
                     that no strategic plan had been established to guide and
                     lead the MegaCenters into the future in terms of technology
                     and equipment. The review suggested that a national team be
                     assembled to focus on the MegaCenters' communications and
                     technology needs. Loss of IT positions at the MegaCenters
                     has prevented the MegaCenter branch chief from creating this
                     national team. However, FPS's IT program manager conducts
                     weekly teleconferences to discuss IT issues in the
                     MegaCenters, which MegaCenter IT staff attend. In addition,
                     the MegaCenter branch chief has developed a radio coverage
                     plan- his main technology priority for the MegaCenters-
                     which outlines plans to acquire technology systems that will
                     ensure the MegaCenters can receive radio signals for all
                     areas in the regions.11 There has been no funding to
                     implement this plan.

    10GAO, Assessing Risks and Returns: A Guide for Evaluating Federal
    Agencies' IT Investment Decision-making, GAO/AIMD-10.1.13 (Washington,
    D.C.: February 1997).

    11Currently there are some areas where FPS police officers' and contract
    guards' radios cannot transmit signals to the MegaCenters preventing the
    MegaCenter operators from being able to monitor the radios.

         o As part of the budget justification process, FPS has submitted the
           required analysis and requested funding for a technology upgrade
           project for each budget year from 2003 through 2006.12 FPS's proposed
           project is to maintain, consolidate, standardize, and enhance current
           and future FPS systems and integrate it with DHS systems. Included in
           the request for each year is an Operations Reporting and Information
           Network (ORION) system to be used at the MegaCenters that would
           provide enhanced incident capture and tracking, officer tracking, and
           officer safety features and includes a CAD system.
         o Although FPS has developed this investment plan, no funds have been
           allocated for ORION or any other MegaCenter technology improvements.
           An FPS official stated that because of limited funding, the
           MegaCenters are not investing in technology, and the only money being
           spent on MegaCenter technology is for maintenance so they can maintain
           current operations. Another official reported that under GSA, FPS was
           given funds for technology investment but since moving to DHS, FPS has
           not received these extra funds and must take money from its operating
           budget to fund technology purchases.

    12Each year agencies submit to the Office of Management and Budget a
    Capital Asset Plan and Business Case to justify each request for a major
    IT investment. 47

    Appendix II

    Comments from the Department of Homeland Security

    Appendix III

                         GAO Contact and Staff Acknowledgments

    Mathew J. Scire (202) 512-2834 or sciremj@gao.gov.

    GAO Contact

    Other key contributors to this report were Gerald P. Barnes, Assistant

    Staff Acknowledgements

    Director; Deirdre Brown; Bess Eisenstadt; Colin Fallon; Brandon Haller;
    Richard Hung; Alex Lawrence; Gail Marnik; and Josh Ormand.
    *** End of document. ***