Homeland Security: Federal Protective Service Could Better	 
Measure the Performance of Its Control Centers (29-SEP-06,	 
GAO-06-1076).							 
                                                                 
The Department of Homeland Security's Federal Protective Service 
(FPS) through its control centers (MegaCenters) helps provide for
the security and protection of federally owned and leased	 
facilities. This report (1) identifies the services MegaCenters  
provide, (2) determines how FPS assesses MegaCenter performance  
and whether FPS links MegaCenter performance measures to FPS-wide
measures, and (3) examines how MegaCenters and selected 	 
organizations compare in the services they provide. To address	 
these issues, GAO reviewed FPS's performance measures and past	 
MegaCenter assessments, assessed the MegaCenters' performance	 
measures, and interviewed officials and collected relevant	 
information at FPS, the four MegaCenters, and nine selected	 
security organizations. 					 
-------------------------Indexing Terms------------------------- 
REPORTNUM:   GAO-06-1076					        
    ACCNO:   A61609						        
  TITLE:     Homeland Security: Federal Protective Service Could      
Better Measure the Performance of Its Control Centers		 
     DATE:   09/29/2006 
  SUBJECT:   Facility security					 
	     Federal facilities 				 
	     Federal property management			 
	     Homeland security					 
	     Monitoring 					 
	     Performance measures				 
	     Physical security					 
	     Police						 
	     FPS MegaCenters					 

******************************************************************
** This file contains an ASCII representation of the text of a  **
** GAO Product.                                                 **
**                                                              **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced.  Tables are included, but    **
** may not resemble those in the printed version.               **
**                                                              **
** Please see the PDF (Portable Document Format) file, when     **
** available, for a complete electronic file of the printed     **
** document's contents.                                         **
**                                                              **
******************************************************************
GAO-06-1076

     

     * Report to the Chairman, Committee on Government Reform, House of
       Representatives
          * September 2006
     * HOMELAND SECURITY
          * Federal Protective Service Could Better Measure the Performance
            of Its Control Centers
     * Contents
          * Results in Brief
          * MegaCenters Provide Alarm and Radio Monitoring and Dispatch from
            Four Locations
          * MegaCenters' and FPS's Performance Measures Are Not Linked
          * Nine Selected Organizations Provide Some of the MegaCenters'
            Services in a Generally Comparable Manner
          * Conclusions
          * Recommendations for Executive Action
          * Agency Comments
     * Briefing for the Committee on Government Reform, U.S. House of
       Representatives, August 14, 2006
     * Comments from the Department of Homeland Security
     * GAO Contact and Staff Acknowledgments
     * PDF6-Ordering Information.pdf
          * Order by Mail or Phone

                 United States Government Accountability Office

Report to the Chairman, Committee on Government Reform, House of
Representatives

September 2006

HOMELAND SECURITY

 Federal Protective Service Could Better Measure the Performance of Its Control
                                    Centers

                                       a

GAO-06-1076

HOMELAND SECURITY

Federal Protective Service Could Better Measure the Performance of Its
Control Centers

  What GAO Found

FPS MegaCenters provide three primary security services-alarm monitoring,
radio monitoring, and dispatching of FPS police officers and contract
guards. These and other services are provided around the clock from four
locations-Battle Creek, Michigan; Denver, Colorado; Philadelphia,
Pennsylvania; and Suitland, Maryland. With a fiscal year 2006 budget of
$23.5 million, the MegaCenters monitor alarms at over 8,300 federal
facilities, covering almost 381 million square feet, and have available
for dispatch over 7,800 FPS police officers and contract guards.

                   The MegaCenter Emergency Response Process

Source: GAO analysis of FPS data.

FPS MegaCenter managers assess MegaCenter operations through a variety of
means, including reviewing data about volume and timeliness of operations,
listening to and evaluating a sample of calls between operators and FPS
police officers and contract guards, and receiving informal feedback about
customer satisfaction. FPS managers have also developed performance
measures for assessing MegaCenter operations. However, these measures are
of limited use because they are not always clearly stated or measurable
and do not address governmentwide priorities of efficiency, cost of
service, and outcome-which are among the attributes that GAO has
identified for successful performance measures. In addition, the
MegaCenters do not measure a key activity-the time from alarm to officer
dispatch-that would link MegaCenter performance to an FPS-wide performance
measure of response time. Without a corresponding measure, FPS is limited
in its ability to evaluate the MegaCenters' contribution to the FPS-wide
measure of response time.

Nine selected security organizations-including federal and local police
and private entities-offer some of the MegaCenters' services as well as
provide and assess these services in a manner that is generally similar to
the MegaCenters. Like the MegaCenters, many of the selected organizations
have centralized their operations. They also use regular call reviews and
volume and time measures to assess the quality of the services they
provide. A major difference between the MegaCenters and some selected
organizations is the use of a computer-aided dispatch system, which
enables these organizations to automate many functions.

                 United States Government Accountability Office

Contents

Letter                                                                   1 
                            Results in Brief                                3 
                            MegaCenters Provide Alarm and Radio Monitoring 
                            and Dispatch                                   
                            from Four Locations                             5 
                            MegaCenters' and FPS's Performance Measures     6 
                            Are Not Linked                                 
                            Nine Selected Organizations Provide Some of    
                            the MegaCenters'                               
                            Services in a Generally Comparable Manner       9 
                            Conclusions                                    10 
                            Recommendations for Executive Action           11 
                            Agency Comments                                12 
Appendixes                                                              
                Appendix I: Briefing for the Committee on Government       
                            Reform, U.S.                                   
                            House of Representatives, August 14, 2006      14 
              Appendix II:  Comments from the Department of Homeland       61 
                            Security                                       
              Appendix III: GAO Contact and Staff Acknowledgments          63 

                                 Abbreviations

CAD                 computer-aided dispatch                                
DHS                 Department of Homeland Security                        
FPS                 Federal Protective Service                             
GSA                 General Services Administration                        
ICE                               U.S. Immigration and Customs Enforcement 
IT                  information technology                                 
UL                  Underwriters Laboratories                              

This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed in
its entirety without further permission from GAO. However, because this
work may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this material
separately.

A

United States Government Accountability Office Washington, D.C. 20548

September 29, 2006

The Honorable Tom Davis Chairman Committee on Government Reform House of
Representatives

Dear Mr. Chairman:

Since the 1995 bombing of the Alfred P. Murrah Federal Building in
Oklahoma City and the September 11, 2001, attacks on the World Trade
Center and the Pentagon, terrorism has threatened the nation's security,
including the physical security of federal facilities. The Homeland
Security Act of 2002 created the Department of Homeland Security (DHS), a
new federal department with the mission of preventing terrorist attacks
within the United States, which includes safeguarding federal facilities.1
DHS, through its Federal Protective Service (FPS), provides law
enforcement and security services to federal agencies that occupy
facilities under the jurisdiction of the General Services Administration
(GSA) and DHS, protecting millions of federal employees, contractors, and
citizens.2 As part of its approach to facility protection, FPS provides
support for its law enforcement and security services through four control
centers known as MegaCenters that are located in Battle Creek, Michigan;
Denver, Colorado; Philadelphia, Pennsylvania; and Suitland, Maryland.

Because of the important role MegaCenters play in ensuring the safety of
federal facilities and their occupants, our objectives were to (1)
identify the services the MegaCenters provide and how they provide them,
(2) determine how FPS assesses and measures the performance of MegaCenter
operations and how FPS links MegaCenter performance measures to FPS-wide
performance measures, and (3) examine how the MegaCenters compare to
selected security organizations in the services they provide and in the
methods they use to provide them.

To determine the services offered by the MegaCenters and how the
MegaCenters provide these services, we interviewed managers at the four

1Federal facilities include government-owned and -leased space.

2Under agreement, FPS authority can be extended to provide its law
enforcement and security services to any property with a significant
federal interest.

MegaCenters and the MegaCenter branch chief, toured three
MegaCenters-Denver, Philadelphia, and Suitland, collected information on
services and workload data from MegaCenter management, and conducted
document reviews. To determine how FPS assesses MegaCenter performance and
how FPS links MegaCenter performance measures to FPS-wide performance
measures, we reviewed documentation on FPS's performance measures and past
MegaCenter assessments and interviewed MegaCenter management and FPS
headquarters officials. We also assessed the MegaCenters' 11 performance
measures to determine whether they were consistent with selected
attributes of successful performance measures we have identfied, that is,
that they were linked to agency mission and goals; clearly stated;
contained measurable targets; sufficiently covered the program's core
activities; and addressed governmentwide priorities of quality,
timeliness, efficiency, cost of service, and outcome.3

To determine how the MegaCenters compare to selected security
organizations we (1) identified criteria for selecting comparable
organizations and selected four public and five private organizations; (2)
interviewed officials at the nine selected organizations; (3) toured four
of the organizations' control centers-both public and private-to observe
their security operations, procedures, and technology; (4) collected
detailed service and workload information from the organizations; (5) used
the information gathered to compare the MegaCenters and the selected
organizations with respect to services offered, organizational structure,
quality assessment practices, and technology utilization; and (6)
interviewed officials from security industry standard-setting and
accreditation associations (associations) because these associations were
identified as having information on security industry organizations,
operations, quality assessment practices, and technology utilization
practices.4 We used two approaches for selecting the private and public
organizations. We selected private organizations from industry lists of
the top 20 largest security service providers and system integrators in
terms of 2005 revenue that provided services such as alarm monitoring and
access

3We did not assess whether the performance measures were objective,
reliable, overlapping, or balanced.

4We interviewed officials from the following associations: Association of
Public Safety Communications Officials, the Commission on Accreditation
for Law Enforcement Agencies, the National Burglar and Fire Alarm
Association, the National Fire Protection Association, Priority Dispatch,
the Security Industry Association, and Underwriters Laboratories.

control.5 With the help of one of the associations we interviewed, we were
able to make contact with the five security organizations that were
selected as our comparison group.6 For public organization comparisons, we
selected three federal organizations: U.S. Postal Inspection Service, U.S.
Customs and Border Protection, and U.S. Park Police. We selected these
federal organizations because each had a law enforcement branch;
centralized control center(s) that offered, at a minimum, one of the
MegaCenters' primary services; and nationwide operations, characteristics
most similar to those of the MegaCenters. We also selected the Denver
Police Department to serve as a nonfederal public organization that
provided dispatch and radio monitoring services through a central control
center. Because we judgmentally selected the organizations, the
information we collected from them cannot be generalized. We conducted our
work from October 2005 through September 2006 in accordance with generally
accepted government auditing standards.

This report summarizes the information we provided to your office during
our August 14, 2006, briefing and, in addition, contains recommendations
to the Secretary of Homeland Security to improve the MegaCenters'
performance measures. The briefing slides are included in appendix I.

                                Results in Brief

The primary services that FPS MegaCenters provide are remote monitoring of
building alarm systems as well as radio monitoring and dispatch of FPS
police officers and contract guards. These and other services are provided
around the clock from four locations across the country: Battle Creek,
Denver, Philadelphia, and Suitland. Each MegaCenter has a sister center
with redundant capabilities that can serve as an emergency backup, and
each is operated by full-time federal employees and private contractors.
In fiscal year 2006, the MegaCenters had a budget of $23.5 million-
accounting for about 5 percent of FPS's total budget-to protect and

5We looked at two lists: one was in terms of gross revenue, the other was
in terms of revenue.

6In specific situations, two of the private security organizations
dispatched either contract guards or their own guards. However, because
these were situational circumstances, we did not include them as being
similar to the MegaCenters' dispatching service. For example, one
organization only uses contract guards when local jurisdictions will not
dispatch their own police officers without actual verification of the
nature of the alarm. In these cases, the organization will have a guard
check whether there is a burglary in progress, and if so, the organization
will contact the police.

monitor over 8,300 federal facilities and dispatch over 7,800 FPS police
officers and contract guards. To provide these services, the MegaCenters
rely on a variety of information technology (IT) systems, communications
systems, and other equipment.

FPS MegaCenter managers assess MegaCenter activities through a variety of
means, including reviewing reports on the timeliness and volume of
operations, listening to and evaluating a sample of calls between
operators and FPS police officers and contract guards, and receiving
informal feedback about customer satisfaction. Also, FPS managers have
developed performance measures for assessing MegaCenter operations. These
performance measures reflect some of the attributes of successful
performance measures we have identified, but also contain some weaknesses
because they are not always clearly stated or measurable and do not
address the governmentwide priorities of efficiency, cost of service, and
outcome. In addition, the MegaCenters do not have a performance measure
that corresponds to the FPS-wide performance measure that is applicable to
the MegaCenters' operations-the patrol and response time measure, which
tracks the elapsed time from the receipt of an alarm to a police officer's
arrival on the scene.7 While the FPS-wide patrol and response time measure
covers the MegaCenters' activities and reflects their performance, the
MegaCenters do not have their own measure that covers only the activities
for which they are responsible-from the receipt of the alarm to the
officer's dispatch. Without clearly stated and measurable performance
measures, including a measure that corresponds to FPS's agencywide patrol
and response time measure, FPS cannot compare the MegaCenters' performance
over time, assess their contribution to agencywide measures, and identify
opportunities for their improvement. We are recommending that FPS (1)
establish MegaCenter performance measures that meet the attributes of
successful performance measures we have identified; (2) develop a
performance measure for the MegaCenters that corresponds to the FPS-wide
patrol and response time measure and covers the MegaCenters' operations,
from alarm to dispatch; and (3) routinely assess the extent to which the
MegaCenters meet established performance measures. In commenting on a
draft of this report, DHS generally agreed with our findings and
recommendations.

7In addition to patrol and response time, FPS has three other agencywide
performance measures (1) timely deployment of countermeasures, (2)
functionality of countermeasures, and (3) facility security index (an
average success rate for the other measures).

Page 4 GAO-06-1076 Federal Protective Service

The nine selected security organizations offer some of the same security
services as the MegaCenters, and the services the organizations offer are
delivered and assessed in a manner generally similar to that of the
MegaCenters. For example, like the MegaCenters, many organizations have
centralized their control center operations, have backup capability,
allocate workload among centers based on geographic location, and use
regular call reviews as well as volume and time measures to assess the
quality of the services they provide. A few organizations offer services
that the MegaCenters do not offer. One major difference between the
MegaCenters and the organizations is that three organizations use a
computer-aided dispatch (CAD) system. Selected organizations and
associations referred to CAD systems as being beneficial for dispatching
services by allowing for faster operator response, automatic recording of
all operator actions enabling easier performance analysis, and automatic
operator access to standard operating procedures and response
prioritization. MegaCenters have identified a need and developed a plan
for a CAD system, but FPS has not allocated funding for such a system.

MegaCenters Provide Alarm and Radio Monitoring and Dispatch from Four
Locations

FPS MegaCenters provide federal agencies with three primary security
services-alarm monitoring, radio monitoring, and dispatch-through four
locations using a variety of IT systems. MegaCenters monitor intrusion,
panic, fire/smoke, and other alarms. They also monitor FPS police
officers' and contract guards' radio communication to ensure their safety
and to provide information, such as criminal background or license plate
histories, to officers upon request. In addition, they exercise command
and control authority by dispatching FPS police officers or contract
guards. MegaCenters also provide a variety of other services. For example,
they notify federal agencies regarding national emergencies and facility
problems and remotely diagnose problems with federal agency alarms. They
also receive and transcribe FPS police officer incident reports.
Individual MegaCenters may also provide unique services not provided by
other MegaCenters, such as facility-specific access control and remote
programming of alarms via the Internet. One MegaCenter also provides an
after-hours telephone answering service for the Drug Enforcement
Administration and for GSA building maintenance emergencies.

The MegaCenters are located in Battle Creek, Denver, Philadelphia, and
Suitland. Each MegaCenter has a sister center with redundant capability as
backup in case of a failure at that MegaCenter. Suitland is paired with
Battle Creek, and Philadelphia is paired with Denver. A force of 1,014 FPS
police officers and 6,842 contract guards is available for the MegaCenters
to dispatch in response to alarms and other emergencies. In fiscal year
2006, the MegaCenters were supported by a budget of $23.5 million, which
accounts for about 5 percent of FPS's total budget.8 The MegaCenters are
operated by 23 full-time federal employees-some of whom manage the
centers-and about 220 private contractors to provide around the clock
security services for over 8,300 federal facilities.

The MegaCenters rely on a variety of IT systems, communications systems,
and other equipment to provide their security services. The IT systems
enable MegaCenter staff to, among other activities, monitor alarms and
radio communications of FPS police officers and contract guards. For
communications systems, MegaCenters have regional and national toll-free
numbers for tenants and the public to contact the MegaCenters during
emergencies. Other equipment includes dictation machines, which enable FPS
police officers to dictate reports about incidents that occur at
facilities.

MegaCenters' and FPS's Performance Measures Are Not Linked

MegaCenters use various means to assess operations, but their performance
measures have weaknesses and are not linked to FPS-wide performance
measures. MegaCenter managers assess MegaCenter operations through a
variety of means, including reviewing data about volume and timeliness of
operations, listening to and evaluating a sample of calls between
operators and FPS police officers and contract guards, and receiving
informal feedback about customer satisfaction. FPS managers also have
developed 11 performance measures for assessing MegaCenter operations:

     o distribute emergency notification reports (also known as SPOT reports)
       within 30 minutes of notification;
          * review problem alarm reports daily;
          * o  obtain regular feedback about customer satisfaction from field
            operations;
     o continuously review all SPOT reports and other outgoing information to
       ensure 100 percent accuracy;

8As of August 2006, MegaCenter officials reported a revised MegaCenter
budget of approximately $19 million for fiscal year 2006.

Page 6 GAO-06-1076 Federal Protective Service

     o transcribe dictated offense and incident reports into the database
       management system within 8 hours of receipt of the report;
     o submit reviewed contractor billing reports and time sheets within 7
       business days after the last day of the month;
     o prepare and review contractor reports for quality assurance plan;
     o maintain completely accurate (nonduplicative) case control numbers;
     o meet Underwriters Laboratories (UL) guidelines and requirements
       continuously;9
     o test failover of alarm, radio, and telephone systems weekly;10 and
     o monitor calls and review recorded call content for adherence to
       standard procedures at least monthly.

The Government Performance and Results Act of 1993 requires federal
agencies to, among other things, measure agency performance in achieving
outcome-oriented goals. Measuring performance allows organizations to
track the progress they are making toward their goals and gives managers
critical information on which to base decisions for improving their
progress. We have previously reported on some of the most important
attributes of successful performance measures. These attributes indicate
that performance measures should (1) be linked to an agency's mission and
goals; (2) be clearly stated; (3) have quantifiable targets or other
measurable values; (4) be reasonably free of significant bias or
manipulation that would distort the accurate assessment of performance;

(5) provide areliable way to assess progress; (6) sufficiently cover the
program's core activities; (7) have limited overlap with other measures;
(8) have balance or not emphasize one or two priorities at the expense of

9UL certifies control centers that provide all elements of service
required by UL's standards, including appropriate operator response to
fire alarm signals and proper equipment inspection, testing, and
maintenance.

10"Failover" is the capability to switch over automatically to a redundant
or standby system in the event of a system failure. Each MegaCenter tests
its ability to run its sister-center's operations in case that center has
system failure.

Page 7 GAO-06-1076 Federal Protective Service

others; and (9) address governmentwide priorities of quality, timeliness,
efficiency, cost of service, and outcome.11

We assessed the 11 FPS MegaCenter performance measures against selected
attributes: linkage to mission and goals, clarity, and measurable targets.
Ten of the 11 MegaCenter performance measures were aligned with FPS's
mission to protect federal properties and personnel and with the
MegaCenter program's mission to provide high-quality and standardized
alarm monitoring, radio monitoring, and dispatch. We found no link between
timely review of contractor time sheets and billing statements and FPS's
mission, however, primarily because this measure seems to be related to
administrative activities. In addition, while 6 of the 11 performance
measures have measurable targets-a key component for measuring
performance, none of the MegaCenter performance measures met the clarity
attribute because FPS could not provide information about how managers
calculate the measures-a key component in the clarity attribute. For
example, the performance measure that the centers test the failover
ability of alarm, radio, and telephone systems weekly is measurable
because it has a quantifiable target but does not meet the clarity
attribute because FPS could not describe its methodology for calculating
it.

We also assessed whether, collectively, the MegaCenters' 11 performance
measures sufficiently cover their core program activities (i.e., alarm
monitoring, radio monitoring, and dispatch) and address governmentwide
priorities of quality, timeliness, efficiency, cost of service, and
outcome. Most of the MegaCenter performance measures relate to the three
core activities. For example, regular feedback on customer service and
monthly review of operator calls cover aspects of the dispatch and
radio-monitoring functions. Other performance measures, like distributing
emergency notification reports in 30 minutes, help fulfill other critical
support functions. However, two performance measures-reviewing contractor
quality assurance plans and timely review of contractor time sheets and
billing statements-relate to administrative activities that are not
strictly related to MegaCenter core activities. Additionally, the
MegaCenter performance measures do not collectively address all of the
governmentwide priorities. The MegaCenter performance measures primarily
address the governmentwide priorities of quality and timeliness.

11See GAO, Tax Administration: IRS Needs to Further Refine Its Tax Filing
Season Performance Measures, GAO-03-143 (Washington, D.C.: Nov. 22, 2002),
pp.2-3, 46-53.

Page 8 GAO-06-1076 Federal Protective Service

For example, the MegaCenter measures pertaining to transcribing reports
within 8 hours and reviewing recorded calls to see if the operator
followed standard operating procedures address aspects of service
timeliness and quality, respectively. None of the measures relate to the
governmentwide priorities of efficiency, cost of service, and outcome.

Finally, FPS does not link MegaCenter performance measures to FPS-wide
performance measures, specifically the patrol and response time measure.
FPS established FPS-wide performance measures to assess its efforts to
reduce or mitigate building security risks. The performance measures that
FPS established were (1) timely deployment of countermeasures, (2)
functionality of countermeasures, (3) patrol and response time, and (4)
facility security index. The one measure that relates to the MegaCenters-
patrol and response time-assesses FPS's ability to respond to calls for
service and measures the average elapsed time from when a law enforcement
request is received (e.g., alarm, telephonic request from a building
tenant, FPS police officer-initiated call) to the time an officer arrives
at the scene. FPS's goal is to reduce response times by 10 percent in
fiscal year 2006. The MegaCenters are responsible for part of the patrol
and response activity that is being measured because the MegaCenters
receive alarms and emergency calls and dispatch FPS police officers or
contract guards to the scene. However, although data pertaining to this
activity exist in the MegaCenters' records management system, they do not
measure the timeliness of this activity, and FPS has not developed a
performance measure that would identify the MegaCenters' contribution
toward meeting FPS's measure.

Nine Selected Organizations Provide Some of the MegaCenters' Services in a
Generally Comparable Manner

The nine selected security organizations generally do not provide all
three of the MegaCenters' primary services. However, the services these
organizations offer are provided similarly by the MegaCenters with the
exception of a CAD system, which three organizations use and the
MegaCenters do not. The MegaCenters provide three primary services (i.e.,
alarm monitoring, radio monitoring, and dispatch), and the selected
organizations provide all or some of these three main services. For
example, the Park Police provide all three services, while the private
organizations focus on providing alarm monitoring and offer some services
the MegaCenters do not. Like the MegaCenters, all of the private
organizations reviewed have centralized operations: the number of their
national control centers ranges from two to five. Work allocation (i.e.,
how incoming alarms and calls are assigned) among centers varies by
organization but overall is similar to the MegaCenter structure. For
example, most of the organizations assign calls and alarms to a specific
center based on the geographic location of the call or signal. However,
the Postal Inspection Service and one private organization are unique
because they are able to allocate workload to centers based on demand and
operator availability. The organizations use a variety of methods to
measure the quality of their services, many similar to methods used by the
MegaCenters. For example, like the MegaCenters, most review a sample of
operator calls on a regular basis. Two entities have established
measurable performance goals for their centers. While there are
similarities in the services offered, number of centers, work allocation,
and service quality appraisals between the organizations reviewed and the
MegaCenters, three organizations use a CAD system, which the MegaCenters
do not. A CAD system is a tool used by the Denver Police Department for
dispatching and officer tracking and by the Postal Inspection Service for
officer tracking. The Park Police also uses a CAD system with limited
capabilities at its San Francisco center and plans to purchase and upgrade
the system for all three of its centers. Selected organizations and
associations referred to CAD systems as being beneficial for dispatching
services by allowing for faster operator response, automatic operator
access to standard operating procedures and response prioritization, and
automatic recording of operator actions enabling easier performance
analysis. Since 2003, FPS and DHS both have assessed MegaCenter technology
and have identified needs for technology upgrades, including the
installation of a CAD system for the MegaCenters. Our guide on IT
investment decision making-based on best practices in the public and
private sector-stresses that part of achieving maximum benefits from an IT
project requires that decisions be made on a regular basis about the
status of the project.12 To make these decisions, senior managers need
assessments of the project's impact on mission performance and future
prospects for the project. While the MegaCenters have assessed their
technology on many occasions and have determined that some refreshment is
needed, FPS has not yet allocated the funding for such upgrades.

Conclusions

FPS MegaCenters play a key role in protecting federal facilities, those
who
enter these facilities, and the FPS police officers and contract guards
whose calls the MegaCenters respond to and monitor. How well the
MegaCenters are fulfilling their role and carrying out their
responsibilities

12GAO, Assessing Risks and Returns: A Guide for Evaluating Federal
Agencies' IT Investment Decision-making, GAO/AIMD-10.1.13 (Washington,
D.C.: February 1997).

Page 10 GAO-06-1076 Federal Protective Service

is uncertain because they do not generate much of the information that
would be useful for assessing their performance.

To their credit, the MegaCenters have established performance measures for
a number of their activities and operations, and these measures are
aligned with the MegaCenters' mission. However, the measures have
weaknesses, both individually and collectively, compared with the selected
attributes of successful performance measures that we have identified.
Many of the individual measures are neither quantifiable nor clearly
stated, and collectively the measures do not address the governmentwide
priorities of efficiency, cost of service, and outcome. As a result, FPS
cannot compare performance across the MegaCenters or over time, and
without such information, FPS is limited in its ability to identify
shortfalls and target improvements.

Although FPS has established an FPS-wide performance measure for response
time-from the alarm to the FPS police officer's arrival on the scene-that
incorporates the MegaCenters' operations, the MegaCenters have not
established a comparable measure for their operations alone. Without such
a measure, FPS cannot evaluate the MegaCenters' contribution-from the
alarm to the FPS police officer's dispatch-to the FPS-wide measure for
response time and identify opportunities for improvement.

Recommendations for Executive Action

We recommend that the Secretary of Homeland Security direct the Director
of the Federal Protective Service to take the following three actions:

     o establish MegaCenter performance measures that meet the attributes of
       successful performance measures we have identified;
     o develop a performance measure for the MegaCenters that is directly
       linked to the FPS-wide response time measure and covers the scope of
       the MegaCenters' operations, from alarm to dispatch; and
     o routinely assess the extent to which the MegaCenters meet established
       performance measures.

Agency Comments

We provided a draft of this report to DHS, the Department of the Interior,
and the U.S. Postal Service for their review and comment. DHS provided
comments in a letter dated September 6, 2006, which are summarized below
and reprinted in appendix II. DHS also provided technical comments, which
we incorporated into the report where appropriate. The Postal Service
informed us that it had no comments on this report. The Department of the
Interior did not provide comments on this report.

DHS generally agreed with the report's findings and recommendations. DHS
stated that FPS and the U.S. Immigration and Customs Enforcement (ICE)
have undertaken a comprehensive review of the MegaCenters to identify,
among other things, ways in which performance can be better measured. DHS
noted that through this broad approach, FPS personnel will be able to
generate and track the kind of information necessary to assess the
MegaCenters' performance. This one-time review may help FPS identify
information needed to assess the MegaCenters' performance and, therefore,
develop appropriate performance measures. In order to reliably assess
performance over time, FPS should not only establish appropriate
performance measures, but also routinely assess performance using these
measures. We therefore clarified our recommendation to include the routine
use of established performance measures to assess the MegaCenters'
performance. With regard to the report's discussion of CAD system
capabilities, DHS said that ICE's Chief Information Officer is currently
assessing the MegaCenters' technology requirements and recognizes that
previous studies have identified the need for technology upgrades. DHS
indicated that the current assessment will have a meaningful impact on
FPS's technology capabilities.

As agreed with your office, unless you publicly announce the contents of
this report earlier, we plan no further distribution until 30 days from
the report date. At that time, we will send copies of this report to other
interested congressional committees and the Secretary of Homeland
Security, and DHS's Assistant Secretary for Immigration and Customs
Enforcement. We will also make copies available to others upon request. In
addition, the report will be available at no charge on the GAO Web site at
http://www.gao.gov.

If you have any questions regarding this report, please contact me at
(202) 512-2834 or [email protected]. Contact points for our Offices of
Congressional Relations and Public Affairs may be found on the last page
of this report. GAO staff who made major contributions to this report are
listed in appendix III.

Sincerely yours,

Mathew J. Scire Acting

Director, Physical Infrastructure Issues

Appendix I

Briefing for the Committee on Government Reform, U.S. House of Representatives,
August 14, 2006

             Review of the Federal Protective Service's MegaCenters

                Briefing for the Committee on Government Reform

                  U.S. House of RepresentativesAugust 14, 2006

     o Introduction and Objectives
     o Scope and Methodology
     o Results in Brief
     o Background
     o Results of GAO Work
     o Since the 1995 bombing of the Alfred P. Murrah Federal Building in
       Oklahoma City and the September 11, 2001, attacks on the World Trade
       Center and Pentagon, terrorism has threatened the nation's security,
       including the physical security of federal facilities.
     o The Homeland Security Act of 2002 created the Department of Homeland
       Security (DHS), a new federal department with the mission of
       preventing terrorist attacks within the United States, which includes
       safeguarding federal facilities.1
     o DHS, through its Federal Protective Service (FPS), provides law
       enforcement and security services to federal agencies that occupy
       almost 9,000 facilities under the jurisdiction of the General Services
       Administration (GSA) and DHS, protecting millions of federal
       employees, contractors, and citizens. Under agreement, FPS authority
       can be extended to provide its law enforcement and security services
       to any property with a significant federal interest.
     o As part of its approach to facility protection, FPS provides support
       for its law enforcement and security services through four control
       centers (known as MegaCenters) located in Battle Creek, Michigan;
       Denver, Colorado; Philadelphia, Pennsylvania; and Suitland, Maryland.

1Facilities include government owned and lease space.

        * Because of the important role MegaCenters play in assuring the
          safety of federal facilities and their occupants, our objectives
          were to:
             * Identify the services the MegaCenters provide and how they
               provide them.
             * Determine how FPS assesses and measures the performance of
               MegaCenter operations and how FPS links MegaCenter performance
               measures to FPS-wide performance measures.
             * Examine how the MegaCenters compare to selected security
               organizations in the services they provide and in the methods
               they use to provide them.
        * Document review: Reviewed the Memorandum of Agreement between GSA
          and FPS and other documentation related to MegaCenter services as
          well as documentation related to (1) FPS's request for a computer
          aided dispatch (CAD) system for the MegaCenters; (2) past FPS
          assessments of MegaCenter operations; (3) FPS's performance
          measures; and
        * (4) FPS'sbudget for the MegaCenters.
     o Interviews: Interviewed FPS officials, including MegaCenter branch
       chief and managers, and staff from the Program Review Office,
       Financial Management Division, and other offices; Immigration and
       Customs Enforcement's (ICE) Budget Enforcement Office; and officials
       from selected public and private organizations; officials from
       security industry standard setting and accreditation associations
       (associations).
        * Selected organizations:
             o U.S. Customs and Border Protection
             o U.S. Park Police
             o U.S. Postal Inspection Service
             o Denver Police Department
             o 5 private security companies
     o We conducted our review in accordance with generally accepted
       government auditing standards.
     o Remote monitoring of building alarm systems, radio monitoring, and
       dispatching of FPS police officers and contract guards are the primary
       services FPS MegaCenters provide. These and other services are
       provided around the clock from four locations across the country. Each
       MegaCenter has a sister center with redundant capabilities that can
       serve as an emergency backup and each is operated by full-time federal
       employees and private contractors. In addition, the MegaCenters have a
       fiscal year 2006 budget of $23.5 million and use a variety of
       information technology (IT) systems and other equipment to provide
       their services.
          * FPS MegaCenter managers assess MegaCenter operations through a
            variety of means, including reviewing information on the
            timeliness and volume of operations, listening to and evaluating
            a sample of calls between operators and FPS police officers and
            contract guards, and receiving informal feedback about customer
            satisfaction. FPS managers have also developed performance
            measures for assessing MegaCenter operations. Although these
            MegaCenter measures reflect some attributes of successful
            performance measures, they also contain some weaknesses because
            they are not always clearly stated or measurable, and do not
            address governmentwide priorities of efficiency, cost of service,
            and outcome. In addition, they do not directly measure key
            operations that would link to FPS-wide performance measures,
            which are (1) the timely deployment of countermeasures, (2)
            functionality of countermeasures, (3) patrol and response time,
            and
          * (4) facility security index.
     o The nine selected organizations offer some of the MegaCenters' primary
       services, and they deliver and assess the services they offer in a
       generally similar manner to the MegaCenters. For example, like the
       MegaCenters, many of these organizations have centralized their
       control center operations, have backup capability, allocate workload
       among control centers based on geographic location, and use regular
       call reviews as well as volume and time measures to assess the quality
       of the services they provide. A few organizations offer services the
       MegaCenters do not offer. One difference between the MegaCenters and
       the selected organizations is that three of these organizations use a
       CAD system, which the MegaCenters do not have. The MegaCenters have
       assessed their technology and have identified the need for a CAD;
       however FPS has not allocated funds for such a purchase.
        * FPS operations are solely funded through security fees and
          reimbursements collected from federal agencies for FPS security
          services.
                * These security fees consist of basic and building-specific
                  security charges.
                     o The basic security charges cover the security services
                       that FPS provides to all federal tenants in
                       FPS-protected buildings, which include such services
                       as patrol, monitoring of building perimeter alarms and
                       dispatching of law enforcement response (MegaCenter
                       operations), criminal investigations, and security
                       surveys.
                     o The building-specific security charges are for FPS
                       security measures that are designed for a particular
                       building and are based on the FPS Building Security
                       Assessment and its designated security level. Such
                       measures include contract guards, X-ray machines,
                       magnetometers, cameras, and intrusion detection
                       alarms. Also, the tenant agencies may request
                       additional security services such as more guards,
                       access control systems, and perimeter barriers.
                     o The above two charges are billed monthly to the tenant
                       agencies. The basic security charge is the same for
                       all tenants regardless of the type of space occupied
                       and is a square footage rate. The building-specific
                       security charge reflects FPS cost recovery for
                       security measures specific to a particular building
                       and the billing is handled differently for single-and
                       multi-tenant buildings. Single tenant buildings-the
                       tenant agency is billed for the total cost of the
                       security measures. Multi-tenant buildings-the tenant
                       agencies are billed based on their pro rata share of
                       the square feet occupied within the respective
                       building.
             o FPS uses a reimbursable program to charge individual agencies
               for additional security services and equipment that they
               request above the level determined for their building.
        * FPS bills the tenant agencies for FPS security fees they have
          incurred.
             o The agencies pay the fees into an FPS account in the
               Department of the Treasury, which is administered by FPS.
               Congress exercises control over the account through the annual
               appropriations process that sets an annual limit-called
               obligation authority-on how much of the account FPS can expend
               for various activities.
             o FPS uses the security fees to finance its various activities
               within the limits that Congress sets. The Department of
               Homeland Security Appropriations Act for fiscal year 2006
               authorized $487 million in obligation authority for FPS
               expenses and operations. Through FPS's security fees, funds
               are to be collected and credited to FPS's account as an
               offsetting collection from tenant agencies.
     o Under the FPS reimbursable program, agencies request additional
       security services and equipment using a funded Security Work
       Authorization. Once the services are provided and the costs are
       expensed, FPS bills the agency for the costs, and the funds are
       transferred to the FPS account to offset the expenses FPS incurred.
     o The DHS Inspector General reported in 2006 that when FPS was part of
       GSA it budgeted and paid for FPS's annual administrative support costs
       such as financial management, human capital, and IT using funds beyond
       those generated by security fees. GSA estimated these FY 2003 support
       services to cost about $28 million. According to the report, beginning
       in FY 2004, neither DHS's annual budget request nor DHS's
       appropriations set aside funding for FPS's support services. In FY
       2004, as a component of DHS, FPS paid almost $24 million for support
       services using funds from security fees only; a year earlier these
       services had been funded by GSA using funds not derived from fees.
     o Before GSA established the MegaCenters, FPS used regional and
       satellite control centers to monitor alarm systems, dispatch FPS
       police officers and contract guards, and perform criminal background
       checks. In total, there were 22 regional control centers and 12
       satellite control centers, which were located throughout FPS's 11
       regions. Most regions had more than 1 control center.
     o In 1991, GSA conducted an internal review of the control centers. The
       review found that because of significant budgetary and personnel
       constraints over more than a decade, the control centers no longer
       performed well enough to ensure safe, effective, and efficient FPS
       actions to preserve life and property. GSA contracted with Sandia
       National Laboratories- the lead laboratory for U.S. Department of
       Energy security systems- to conduct an in-depth study of the control
       centers' operation and make recommendations.
     o In 1993, Sandia issued its study entitled GSA Control Center Upgrade
       Program. The Sandia study identified serious shortfalls and problems
       that would require a more radical upgrade of the control centers at a
       much higher cost than originally believed. After validating the
       study's findings, GSA determined that a multimillion dollar upgrade of
       all control centers would be prohibitively expensive. The study noted
       that the control centers could be consolidated to almost any level to
       achieve economies of scale. However, the study recommended against a
       single national-level control center because a second center would be
       needed to continue operations under catastrophe or failover
       conditions.
     o GSA concluded that the control center problems that the study
       identified were material weaknesses and reported them to Congress. FPS
       conducted an operational and technical review of the Sandia study's
       findings, which provided a critical assessment of the control centers,
       a high-level concept of operations for the centers, and functional
       specifications for upgrading the centers. GSA decided to upgrade 11
       control centers-one in each region-and address the weaknesses that the
       study had identified.
     o Within GSA, concerns were raised about the cost of upgrading 11
       control centers, how many control centers were really needed, and
       whether the centers' operations could be outsourced. GSA established a
       project team to investigate these concerns. The team contacted several
       public and private sector organizations that operate control centers.
       The team found that the organizations were consolidating their control
       centers but were unable to assume the operations of FPS control
       centers. A decision was made to consolidate additional centers and the
       multi-regional control center or "MegaCenter" concept was developed.
       GSA endorsed the MegaCenter concept. GSA assembled a core project team
       and hired contractors to design, plan, and supervise the construction
       of the centers.
     o In 1994, GSA issued a bid for MegaCenter technical and performance
       specifications and awarded the contract.
     o In 1996, FPS reaffirmed that the MegaCenter concept was the best
       approach for addressing the control center weaknesses. GSA selected
       the MegaCenters sites: Denver, CO; Battle Creek, MI; New York, NY and
       Suitland, MD.2
     o In 1996, construction began on the Denver MegaCenter and design was
       initiated on the Battle Creek MegaCenter.
     o In 1997, the Denver MegaCenter was opened, followed by Battle Creek in
       1999, Suitland in 2000, and Philadelphia in 2001.

2Site selection for New York, NY was never finalized and eventually was
switched to a site in Philadelphia, PA.

     o The MegaCenters' mission is to provide the highest quality, nationally
       standardized dispatch, alarm monitoring, and federal law enforcement
       emergency response services. Based on this mission statement, we chose
       to focus on alarm monitoring, radio monitoring, and dispatch as the
       primary services the FPS MegaCenters provide.
          * Primary services:
               o Alarm monitoring: monitoring intrusion, panic, fire/smoke,
                 elevator, and/or environmental alarms.
               o Radio monitoring: monitoring FPS police officers' and
                 contract guards' radio communication for safety and
                 providing information upon request.
               o Dispatch: exercising command and control authority by
                 dispatching FPS police officers and/or contract guards.

Objective 1: MegaCenters provide other services

        * Other services:
             o Notifying federal agencies regarding national emergencies and
               facility problems (also known as SPOT reports)
             o Checking criminal background histories (including inquiries to
               the National Crime Information Center database, which the
               Federal Bureau of Investigation (FBI) maintains) for FPS
               police officers responding to a call or an alarm or FPS
               regions and other DHS agencies requesting this assistance
             o Receiving and transcribing FPS police officer reports
             o Providing a toll-free help desk line for agency support,
               including remote diagnosis and service of alarms
             o Updating quarterly building and emergency contact information
               from customer agencies to ensure accurate notifications and
               alerts
        * Unique services (provided by individual MegaCenter):
             o Monitoring and controlling access to buildings
             o Using the Internet to program, monitor, and test alarms
             o Providing after-hours telephone answering service for Drug
               Enforcement Administration and for GSA maintenance

Table 1: Services Provided to MegaCenter Clients

                    Services                  GSA, DHS, and      FPS regions, 
                                              federal agency police officers, 
                                                 clients         and contract 
                                                                  guards      
Monitor remote building alarms                            
                                                    X               X         
Dispatch FPS police officers, contract                    
guards,                                                   
and other response services                      X               X         
Monitor FPS police officers' and contract                 
guards' radio communications for their                           X         
safety                                                    
Keep track of FPS police officer and                      
contract                                                  
guard locations                                                  X         
Provide warrant and other information from                
the National Crime Information Center            X               X         
Help maintain lists of buildings and                      
contacts                                                  
for emergency notifications                      X               X         

        * MegaCenters are located in four cities and are responsible for
          providing various services in their respective FPS Regions:
             o Philadelphia, PA (FPS Regions 1, 2, 3)
             o Battle Creek, MI (FPS Regions 4, 5, 6)
             o Denver, CO (FPS Regions 7, 8, 9, 10)
             o Suitland, MD (FPS Region 11-National Capital Region)
        * Each MegaCenter is paired with a sister center:
             o Suitland is paired with Battle Creek
             o Philadelphia is paired with Denver
     o Sister center pairings provide for redundant capability in case of a
       catastrophic failure at any MegaCenter.
     o MegaCenters rely upon contractor staff to carry out dispatch and
       technical support services.
     o Each MegaCenter also has FPS officials located on site to oversee the
       center's overall operations.
     o MegaCenters operate around the clock.

Table 2: FY 2006 Actual Federal and Contractor Staffing Levels at Each
MegaCenter

    MegaCenter  Federal staff       Primary Contractor       Contractor staff 
Battle Creek       9        Wackenhut Services Inc.a             74        
Denver             4        Gonzales Consulting Servicesa        56        
Philadelphia       6        Gonzales Consulting Services         49        
Suitland           4        Gonzales Consulting Services         38        
          Total      23                                            217        

aThe Battle Creek and Denver MegaCenters have other contracts for
personnel to provide services, such as technical support, however, these
contracts are small in terms of the number of personnel provided- three in
Denver and five in Battle Creek.

Objective 1: Historical budget and MegaCenter allocations

        * Recent FPS MegaCenter budgets:
             o Fiscal year 2004 $20.0 million
             o Fiscal year 2005 $21.5 million
             o Fiscal year 2006 $23.5 million, which accounts for less than 5
               percent of FPS's total budget.4
        * Estimated fiscal year 2006 budget for each MegaCenter:
             o Suitland $6.0 million
             o Philadelphia $5.0 million
             o Battle Creek $6.5 million
             o Denver $6.0 million
     o MegaCenter cost information for FY 2004 and prior years is not
       available since FPS transitioned from GSA's accounting system. FY 2005
       and FY 2006 cost information is also not available because the ICE
       accounting system was not modified to capture costs solely for the
       MegaCenters. According to FPS, it is working with ICE to establish the
       capability in the accounting system to capture MegaCenter costs in the
       future.

4As of August 2006, MegaCenter officials reported a revised MegaCenter
budget of approximately $19 million for fiscal year 2006.

Objective 1: MegaCenters rely on IT systems to deliver services

o  MegaCenters depend on a variety of IT systems and equipment to deliver
their primary services.

Table 3: Information Technology Systems That Support MegaCenters' Primary
Services

Information technology systems  Alarm monitoring Radio monitoring Dispatch 
Security Information System and                                            
software (alarm receivers and                                     
signal receivers)                      X                X            X
Facility and building client                                               
enterprise information systems         X                X            X
MegaCenter-owned telephone                                                 
exchange system and regional           X                X            X
and national toll-free                                            
emergency numbers                                                 
Remote programming software            X                          
Radio systems                          X                X            X     
Patrol and dispatch operations                                             
logs                                   X                X            X
Voice audio recorders                  X                X            X     
Software that allows                                                       
MegaCenters to access federal                           X            X
and state law enforcement                                         
databases                                                         

Source: GAO analysis of FPS data

     o MegaCenters also depend on additional information technology systems,
       such as failover equipment and servers that facilitate sharing files
       with sister centers.
     o Reports on operator activities-FPS MegaCenter supervisors-staff who
       are responsible for overseeing operators-review information about the
       timeliness and volume of operators' activities. For example, they
       review reports that describe how long it took operators to send out
       emergency notifications and transcribe dictated reports and the number
       of problem alarms, among other things.
     o Assessments of operator communications-Supervisors and designated
       quality assurance staff listen to live conversations between operators
       and FPS police officers and contract guards as well as regularly
       listen to a sample of taped conversations to identify whether
       operators are following standard operating procedures. According to
       MegaCenter managers, staff sample taped calls on a monthly basis.
     o Comprehensive reviews of MegaCenter operations-In 2003 and 2004, the
       MegaCenter managers completed a manager review of each MegaCenter.
       These reviews were replaced by more comprehensive program reviews in
       which FPS regional staff with subject area expertise were to review
       each MegaCenter and report on concerns and best practices related to
       MegaCenter management, administration, technology, and equipment. In
       2005, the Philadelphia MegaCenter was the first and only MegaCenter to
       undergo a program review. Program reviews were to be conducted
       annually at each MegaCenter, however, the MegaCenter managers
       suspended these reviews due to budgetary constraints.
     o Feedback on customer satisfaction-MegaCenter managers and supervisors
       use informal means for gathering information about the level of
       customer satisfaction. For example, they receive information from
       conversations with FPS region program staff and FPS police officers
       who, according to MegaCenter managers, do not hesitate to inform them
       of performance concerns. Managers also obtain feedback from federal
       building tenants and agencies during routine activities, such as when
       they update their emergency contact database, and from regional staff
       by attending regional staff meetings.
     o Underwriters Laboratories (UL) control center certification-Although
       not an assessment by the MegaCenter staff themselves, the UL
       certification process involves feedback from UL inspectors about
       whether the MegaCenter meets technical and performance standards. UL
       inspectors complete initial inspections to certify the MegaCenters and
       conduct regular inspections once a center is certified. The Denver
       MegaCenter was UL certified in 2003 and has since had a subsequent
       inspection. The Philadelphia MegaCenter's initial UL inspection is
       scheduled to be completed in August 2006. Initial UL inspections at
       the remaining two MegaCenters have not been scheduled.
     o Performance measurement-FPS has established performance measures for
       the MegaCenters.
          * FPS has identified 11 performance measures for MegaCenter
            operations.
               o Distribute emergency notification reports (SPOT reports)
                 within 30 minutes of notification.
               o Review problem alarm reports daily.
               o Obtain regular feedback about customer satisfaction from
                 field operations.
               o Continuously review all SPOT reports and other outgoing
                 information to ensure 100 percent accuracy.
               o Transcribe dictated offense and incident reports into
                 database management system within 8 hours of receipt of the
                 report.
               o Submit reviewed contractor billing reports and time sheets
                 within 7 business days after the last day of the month.
               o Prepare and review contractor reports for quality assurance
                 plan.
               o Maintain completely accurate (non-duplicative) case control
                 numbers.
               o Meet UL guidelines and requirements continuously.5
               o Test failover of alarm, radio, and telephone systems
                 weekly.6
               o Monitor calls and review recorded call content for adherence
                 to standard procedures at least monthly.

5UL certifies centers that provide all elements of service required by
UL's standards, including appropriate operator response to fire alarm
signals and proper equipment inspection, testing, and maintenance.
6Failover is the capability to switch over automatically to a redundant or
standby system in the event of a system failure. Each MegaCenter tests its
ability to run 26 its sister-center's operation in case that center has
system failure.

     o The Government Performance and Results Act of 1993 requires federal
       agencies to develop strategic plans, link them with outcome-oriented
       goals, and measure agency performance in achieving these goals.
       Measuring performance allows organizations to track the progress they
       are making toward their goals and gives managers critical information
       on which to base decisions for improving their progress.
          * Organizations need to have performance measures that (1)
            demonstrate results, (2) are limited to the vital few,
          * (3) cover multiple program priorities, and (4) provide useful
            information for decision making in order to track how their
            programs and activities can contribute to attaining the
            organization's goals and mission.
          * We have previously reported on some of the most important
            attributes of successful performance measures. These attributes
            indicate that performance measures should:
               * Be linked to an agency's mission and goals;
               * Be clearly stated: a measure's name and definition are
                 consistent with the methodology used to calculate it;
               * Have quantifiable targets or other measurable values;
               * Be reasonably free of significant bias or manipulation that
                 would distort the accurate assessment of performance;
               * Provide a reliable way to assess progress;
               * Sufficiently cover the program's core activities;
               * Have limited overlap with other measures;
               * Have balance or not emphasize one or two priorities at the
                 expense of others; and
               * Address governmentwide priorities of quality, timeliness,
                 efficiency, cost of service, and outcome.
     o We assessed the 11 FPS MegaCenter performance measures against
       selected attributes of successful performance measures: linkage to
       performance goals and mission, clarity, and measurable targets. We
       also assessed whether collectively the 11 performance measures
       sufficiently cover the MegaCenters' core program activities and
       address governmentwide priorities of quality, timeliness, efficiency,
       cost of service, and outcome.
     o Ten of the eleven MegaCenter performance measures were aligned with
       FPS's mission to protect and secure federally owned and leased
       properties and personnel and with the MegaCenter program's mission to
       provide high quality and standardized alarm monitoring, radio
       monitoring, and dispatch. For example, distributing emergency
       notification reports (SPOT reports) to federal agencies within 30
       minutes helps protect federal personnel and properties by alerting
       federal managers of suspicious activities near their locations. We
       could find no link between timely review of contractor time sheets and
       billing statements and FPS's mission primarily because this measure
       seems to be related to administrative activities.
     o MegaCenter performance measures lacked clarity and measurable targets.
       Although some of the measures seemed clearly stated, FPS could not
       provide information about how managers calculate any of the measures-a
       key component in the clarity attribute. Six of eleven performance
       measures have measurable targets-a key component for measuring
       performance. For example, the MegaCenter measure for testing the
       failover ability of alarm, radio, and telephone systems on a weekly
       basis is measurable because it has a quantifiable target but does not
       meet the clarity attribute because FPS could not describe the
       methodology for calculating this measure.
     o Seven of the eleven MegaCenter performance measures relate to the
       MegaCenters' core activities. For example, daily review of problem
       alarm reports and weekly failover of alarm systems relate to the
       MegaCenters' alarm monitoring operation. Regular feedback on customer
       service and monthly review of operator calls cover aspects of the
       radio monitoring and dispatch functions. Other performance measures,
       like distributing emergency notification reports in 30 minutes, help
       fulfill other critical support functions. However, two of the
       performance measures-the timely review of contractor time sheets and
       preparing and reviewing contractor quality assurance plans-seem to
       relate to administrative activities and are not strictly related to
       MegaCenter core activities.
     o The MegaCenter performance measures primarily address the
       governmentwide priorities of quality and timeliness. For example,
       transcribing reports within 8 hours and reviewing recorded calls to
       see if the operator followed standard procedures address aspects of
       service timeliness and quality, respectively. None of the measures
       relate to the other governmentwide priorities of efficiency, cost of
       service, and outcome.
     o FPS has identified four performance measures to assess its efforts to
       reduce or mitigate building security risks.
     o One of these measures-patrol and response time-assesses FPS's ability
       to respond to calls for service within certain time limit goals. It
       measures the average elapsed time from when a MegaCener receives a law
       enforcement request (e.g., an alarm, telephone request from a building
       tenant, FPS officer initiated call) to the time an FPS officer arrives
       at the scene. In fiscal year 2005, FPS reported a national average
       response time of 47 minutes. Its goal is to reduce response time by 10
       percent in fiscal year 2006.

Table 4: FPS-wide Performance Measures for Facility Protection

Performance measure                   Purpose                              
Timely deployment of countermeasures  To compare actual deployment dates   
                                         with planned deployment dates        
Countermeasure functionality (e.g.,   To gauge whether those security      
surveillance cameras, X-ray machines) countermeasures for which FPS is     
                                         responsible are working as intended, 
                                         once deployed                        
Patrol and response time              To assess FPS's ability to respond   
                                         to calls for service within certain  
                                         time limit goals                     
Facility security index               To calculate FPS's average success   
                                         rate for the above three performance 
                                         measures                             

Source: GAO, Homeland Security: Guidance and Standards Are Needed for
Measuring the Effectiveness of Agencies' Facility Protection Efforts,
GAO-06-612 (Washington, D.C.: May 2006).

     o FPS does not link the MegaCenter performance measures to the FPSwide
       performance measures, specifically, the FPS-wide patrol and response
       time measure.
     o The MegaCenters are responsible for part of the patrol and response
       activity that is being measured because MegaCenter operators are in
       control of the alarm and call response and dispatch operations.
       However, although some time-related data is recorded in their records
       management system, the MegaCenters do not measure the timeliness of
       this activity and FPS has not developed a performance measure that
       would identify the MegaCenters' contribution toward meeting the
       FPSwide measure.
     o The Park Police was the only selected organization that provides all
       three primary services offered by the MegaCenters. The remaining
       selected organizations provide one or two of these MegaCenter primary
       services.

Table 5: MegaCenters' Primary Services Offered by Selected Organizations

            Organization           Alarm monitoring Radio monitoring Dispatch 
Federal Protective Services            X                X            X     
U.S. Customs and Border                                 X         
Protection                                                        
U.S. Park Police                       X                X            X     
U.S. Postal Inspection Service         X                X         
Denver Police Department                                X            X     
Private security organizations         X                          

        * Selected organizations also offer other services the MegaCenters
          offer.
             o Many organizations record telephone and radio communications
               in case these communications need to be reviewed; officials
               from the associations stated that reviewing calls to ensure
               procedures are followed correctly is an important control
               center practice.
             o All public organizations provide information to officers based
               on inquires to the FBI's National Crime Information Center
               among other databases.7
             o All private organizations provide access control services that
               allow them to track and restrict facility access.
             o One private organization monitors system conditions separately
               to distinguish between an intrusion, door propped open,
               tamper, or long access, which assists in determining the
               appropriate action to take.
     o A few organizations offer services the MegaCenters do not.

o  Three private organizations provide audio and/or video monitoring
services which allow operators to remotely listen and/or view live audio
and/or video transmission from the secured site. Officials from two
organizations reported that this type of surveillance technology minimizes
the number of on-site guards required for their clients. In addition, this
surveillance technology provides operators with additional intelligence
information to help them decide what further actions to take.

7Like the MegaCenters, Customs and Border Protection, Postal Inspection
Service, and Park Police also retrieve information for officers from the
National Law Enforcement Telecommunication System and the Denver Police
Department retrieves information for officers from the Colorado 33 Crime
Information Center database.

o  Workloads and staffing vary by public organization. While Postal
Inspection Service monitors more facilities than the MegaCenters, it does
not dispatch officers. Park Police, an organization that does dispatch
officers, has far fewer officers available to dispatch than the
MegaCenters and, because the Park Police has a different purpose, it
monitors far fewer facilities than the MegaCenters.

Table 6: Workload Statistics of Selected Public Organizations' Centers

      Public      Center Purpose    Number  Facilities Dispatchable  Center   
organization                       of    monitored   personnel   staffinga 
                                    centers                         
                Monitor alarm                                                 
Federal      systems of federal     4      8,328       7,856        240
                facilities and                                      
                dispatch                                            
Protective   FPS police officers                                 
                and contract guards                                 
                to these                                            
Service      facilities, if                                      
                necessary. Monitor                                  
                the radios of                                       
                officers                                            
                and guards to                                       
                ensure their safety                                 
                and to provide                                      
                information to                                      
                officers.                                           
                Monitor radios                                                
U.S. Customs nationwide for all     1       Not     Not             71
                customs                                applicable   
                investigations                                      
and Border   at ports to ensure          applicable              
                officer safety and                                  
                coordinate                                          
Protection   response, if                                        
                necessary.                                          
U.S. Park    Monitor alarms and                                            
Police       dispatch Park          3       875b        717         36
                Police officers and                                 
                law enforcement                                     
                rangers to protect                                  
                national monuments                                  
                and parks. Monitor                                  
                officer radios to                                   
                ensure                                              
                officer safety and                                  
                provide information                                 
                to officers.                                        
                Monitor postal                         Not                    
U.S. Postal  facilities' alarm      2      11,448   applicable      67
                systems and radio                                   
Inspection   communication of                                    
                Postal Inspection                                   
                police and federal                                  
Service      agents to ensure                                    
                officer safety and                                  
                to provide                                          
                information to                                      
                officers.                                           
Denver       Provide emergency                                             
Police       services to the        1       Not        1,452       129c
                city and county of                                  
Department   Denver.                     applicable              

aThis includes both full-time federal employees and contract staff, where
applicable.

bThis is the approximate number of alarms monitored by Park Police's two
centers in the District of Columbia and San Francisco. The New York
control center does not monitor alarms. cOne hundred and twenty-nine is an
approximation. Denver Police Department reported 131 personnel, 4 are part
time.

o  The selected organizations have organizational structures similar to
the MegaCenters.

        * Like the MegaCenters, all five private organizations monitor alarms
          through centralized control centers.
             o Each of the five private organizations have between 2 to 5
               national central control centers.
             o One private organization official stated that centralizing and
               consolidating monitoring services allows for easier staffing,
               better customer service, and higher quality technology.
     o Like the MegaCenters, all public and private organizations have backup
       ability, with the exception of Customs and Border Protection.8

o  Officials from the organizations and associations we interviewed cited
the importance of center redundancy in the event of catastrophic failure
at any one control center.

8Park Police's control center in the District of Columbia has backup
capabilities for all three primary services. The centers in San Francisco
and New York are able to backup their radio and call taking functions,
however, the San Francisco center is not able to backup its alarm
monitoring function.

        * Similar to the MegaCenters, most of selected organizations assign
          calls and alarms to a specific control center based on the
          geographic location of the call or alarm signal.
        * o  Some organizations reported that this type of work allocation is
          beneficial because it reduces stress for the operator and provides
          better customer service. For example, one private organization
          reported that its operators become familiar with both the people
          who call the control centers and their locations which allows them
          to provide better directions to responders.
        * Postal Inspection Service and one private organization are able to
          allocate workload to their control centers based on demand and
          operator availability. For example, when an alarm signal or
          telephone call comes into Postal Inspection Service, its software
          decides which center should receive the signal or call based on
          center workload.
                * Both organizations reported efficiencies from this type of
                  work allocation.
                     o Postal Inspection Service reported a reduction in
                       required center staff.
                     o The private organization reported the ability to close
                       some of its control centers during non-peak hours,
                       reducing costs.
             o Both organizations attribute the ability to allocate work in
               this manner to more advanced technology.9

9Postal Inspection Service can only allocate work in this manner for its
alarm monitoring and call taking service. Radio signals must be assigned
to a specific center because the technology at the time Postal Inspection
Service built its control centers was not yet developed to allow radio 36
signals to be assigned to first available center.

o  The selected organizations use a variety of methods to ensure centers
are meeting their goals and providing quality services, many similar to
the MegaCenters.

Table 7: Selected Organizations' Methods to Ensure Quality Service

     Organization      Regular   Operator pay Volume      Regular and formal  
                     review of a  incentives  and/or time     process for     
                      sample of    based on   measures    soliciting customer 
                      operators  performance                   feedback       
                        calls                             
Federal                X                        X      
Protective                                             
Service                                                
U.S. Customs and       X                        X      
Border                                                 
Protectiona                                            
U.S. Park Police                                       
U.S. Postal            X                        X      
Inspection                                             
Service                                                
Denver Police          X           X                   
Department                                             
Private security      Xb           X           Xb              Xb          
organizations                                          

aCustoms and Border Protection officials reported that they are in the
process of developing a formal customer survey tool. bIndicates that at
least one of the selected private security organizations used these
methods.

        * Regular review of calls
        * o  Most officials from the associations we interviewed agreed that
          competent control centers have a quality assurance process that
          includes regular review of a random sample of calls to ensure that
          the operators asked the correct questions and provided the correct
          information when dispatching and to check the timelines and timing
          of responses. FPS MegaCenter supervisors monitor each of their
          operators calls on a monthly basis to ensure correct procedures are
          followed.
        * Pay incentives
        * o  One private organization not only conducts call reviews but also
          ties pay incentives to its quality assessment program by linking
          employee call review evaluations to spot awards and bonuses. None
          of the federal agencies reviewed, including the MegaCenters, used
          these types of pay incentives.
        * Volume and time measurements
             o Examples of measurements are the amount of time it takes to
               answer a call, time taken to act on a call, and number of
               complaints from responding officers. Although the MegaCenters
               have volume and time measurements, they do not report the time
               it takes from receipt of alarm signal to officer dispatch.
               Some of the measures the MegaCenters use include the time
               elapsed between an officer calling in an incident and its
               entry into the records management system and the time it takes
               to send emergency notification reports to higher management.
               They also report volume statistics to their regions, such as
               the number of cases they have opened in a month.
             o Officials from Customs and Border Protection and the
               Commission on Accreditation for Law Enforcement Agencies
               stated that many volume and time measurements cannot be used
               in law enforcement because timing can depend on circumstances.
               For example, the time it takes a Customs and Border Protection
               operator to clear a radio call would not be an appropriate
               measure because some of their functions are more tactical in
               nature- for example providing radio support in an undercover
               investigation- and cannot accurately be measured with time and
               volume statistics only.
        * Customer satisfaction
             o Two private organizations regularly and formally survey their
               customers to find out if operators were pleasant, delivered
               timely service, and seemed to be well trained and informed.
             o Other organizations, like MegaCenters, use a more informal
               process to gauge customer satisfaction. To formalize its
               customer feedback, the Suitland MegaCenter developed a survey
               to gauge officer satisfaction, however, it is waiting for
               approval to use it.
        * o  Customs and Border Protection has hired a private consultant to
          help them develop a customer satisfaction survey and believe its
          use, in conjunction with volume statistics, will provide a more
          meaningful measure of their service quality.
        * Performance measures
             o One private organization and the Postal Inspection Service not
               only perform assessment activities but also have established
               measurable performance goals for their control centers.
               Examples of the private organization's goals are: resolution
               of 90 percent of high priority events within 60 seconds and
               resolution of 85 percent of inbound and technical assistance
               specialist calls within 18 seconds.
             o Postal Inspection Service has begun to establish volume and
               time measurement goals, such as a 25 percent reduction in the
               number of complaints in which standard operating procedures
               were not followed and a 10 percent reduction in the time it
               takes to answer radio calls. They intend to revise their goals
               in fiscal year 2007 once they have actual experience against
               which to set a benchmark.
             o The MegaCenters have created performance measures, however,
               they lack some attributes of successful performance measures,
               such as measurable targets.
     o The MegaCenters perform dispatch and incident management functions
       manually while Denver Police Department uses a CAD system to perform
       these functions and the Postal Inspection Service uses a CAD system to
       track officer locations and perform incidentmanagement functions. The
       Park Police's San Francisco center also uses a CAD system with limited
       capabilities, and Park Police has a plan in place to purchase and
       upgrade a system for its three centers.
     o MegaCenter operators keep track of officers on duty and their
       locations by hand or by using an Excel spreadsheet, and when they
       dispatch an officer to a federal building they must enter the address
       location into the database. In comparison, when a PostalInspection
       Service operator tracks an officer to a building, the CAD system
       automatically populates the address field and the system shows all
       contact information for that building. Without a CAD system, the
       operators spend more time retrieving information from different
       sources and entering data, such as client contact information.
     o CAD systems automate dispatch and incident management functions and
       allow for more efficient handling of incidents. Typical CAD system
       functions include management of call routing and prioritization,
       dispatching, and response procedures. For example, a CAD system can
       decide which control center a call should go to based on workload,
       prioritize the call for the operator, and automatically display to the
       operator the actions to take.CAD systems record times of incidents,
       locations, and corresponding actions by dispatchers and officers which
       allows for analysis to be conducted to determine, forexample, response
       times, workload, the types of incidents requiring response,
       andresource allocation needs.
     o Selected organizations and associations stated that CAD systems are
       beneficial for dispatching services by allowing for faster operator
       response, automatic recording of operator actions enabling easier
       performance analysis, and automatic operator access to standard
       operating procedures and response prioritization. For example, when a
       signal comes into a Postal Inspection Service control center, the CAD
       system automatically retrieves and displays the response actions for
       the operator to follow.
     o FPS has repeatedly recognized the need for a CAD system. In 1993, the
       Sandia study of FPS's former control centers recommended that the
       centers have a CAD system for the most effective and time efficient
       dispatch operations while using a minimum center personnel structure.
       A 2003 DHS MegaCenter review also stated that the MegaCenters needed a
       CAD system. In January 2006, FPS issued a request for information for
       a CAD system. No funding has been allocated in fiscal year 2006 for
       the MegaCenters to purchase a CAD system.
     o Our guide on IT investment decision-making- based on best practices in
       the public and private sectors- stresses that part of achieving
       maximum benefits from an IT project requires that decisions be made on
       a regular basis about the status of the project. To make these
       decisions, senior managers need assessments of the project's impact on
       mission performance and future prospects for the project.
     o Senior managers should regularly question whether (1) the current
       system meets organizational needs, (2) the system should be modified
       to better meet these needs, (3) a new system is needed to best meet
       these needs, or (4) the needs could best be met by outsourcing the
       work. Included in the regular review should be costs for operation and
       maintenance of the project, such as hardware upgrades, system software
       changes, and ongoing user training. Successful IT management requires
       that a plan be developed for the continued support and operation of
       every IT project.10
          * While the MegaCenters have assessed their technology on several
            occasions and have determined that some refreshment is needed, no
            funding has been allocated by FPS for this use.
               o When the MegaCenters were established, FPS intended
                 scheduled replacement of their technical systems after 5
                 years, but this timeline has passed. In addition, a 2003 DHS
                 MegaCenter review found that there were various states of
                 technologies in systems across centers and proposed planning
                 for life cycle equipment replacement. However, according to
                 FPS officials, no technology replacement program has been
                 established, and equipment is replaced on a per MegaCenter,
                 as-needed basis.
               o A 2005 program review of the Philadelphia MegaCenter found
                 that no strategic plan had been established to guide and
                 lead the MegaCenters into the future in terms of technology
                 and equipment. The review suggested that a national team be
                 assembled to focus on the MegaCenters' communications and
                 technology needs. Loss of IT positions at the MegaCenters
                 has prevented the MegaCenter branch chief from creating this
                 national team. However, FPS's IT program manager conducts
                 weekly teleconferences to discuss IT issues in the
                 MegaCenters, which MegaCenter IT staff attend. In addition,
                 the MegaCenter branch chief has developed a radio coverage
                 plan- his main technology priority for the MegaCenters-
                 which outlines plans to acquire technology systems that will
                 ensure the MegaCenters can receive radio signals for all
                 areas in the regions.11 There has been no funding to
                 implement this plan.

10GAO, Assessing Risks and Returns: A Guide for Evaluating Federal
Agencies' IT Investment Decision-making, GAO/AIMD-10.1.13 (Washington,
D.C.: February 1997).

11Currently there are some areas where FPS police officers' and contract
guards' radios cannot transmit signals to the MegaCenters preventing the
MegaCenter operators from being able to monitor the radios.

     o As part of the budget justification process, FPS has submitted the
       required analysis and requested funding for a technology upgrade
       project for each budget year from 2003 through 2006.12 FPS's proposed
       project is to maintain, consolidate, standardize, and enhance current
       and future FPS systems and integrate it with DHS systems. Included in
       the request for each year is an Operations Reporting and Information
       Network (ORION) system to be used at the MegaCenters that would
       provide enhanced incident capture and tracking, officer tracking, and
       officer safety features and includes a CAD system.
     o Although FPS has developed this investment plan, no funds have been
       allocated for ORION or any other MegaCenter technology improvements.
       An FPS official stated that because of limited funding, the
       MegaCenters are not investing in technology, and the only money being
       spent on MegaCenter technology is for maintenance so they can maintain
       current operations. Another official reported that under GSA, FPS was
       given funds for technology investment but since moving to DHS, FPS has
       not received these extra funds and must take money from its operating
       budget to fund technology purchases.

12Each year agencies submit to the Office of Management and Budget a
Capital Asset Plan and Business Case to justify each request for a major
IT investment. 47

Appendix II

Comments from the Department of Homeland Security

Appendix III

                     GAO Contact and Staff Acknowledgments

Mathew J. Scire (202) 512-2834 or [email protected].

GAO Contact

Other key contributors to this report were Gerald P. Barnes, Assistant

Staff Acknowledgements

Director; Deirdre Brown; Bess Eisenstadt; Colin Fallon; Brandon Haller;
Richard Hung; Alex Lawrence; Gail Marnik; and Josh Ormand.
*** End of document. ***