Information Technology: Immigration and Customs Enforcement Needs
to Fully Address Significant Infrastructure Modernization Program
Management Weaknesses (27-APR-07, GAO-07-565).
The Department of Homeland Security (DHS) fiscal year 2006
appropriations act provided $40.15 million for the Immigration
and Customs Enforcement's (ICE) program to modernize its
information technology (IT) infrastructure. As mandated by the
appropriations act, the department is to develop and submit for
approval an expenditure plan for the program, referred to as
"Atlas," that satisfies certain legislative conditions, including
a review by GAO. In performing its review of the Atlas plan, GAO
(1) determined whether the plan satisfies certain legislative
conditions and (2) provided other observations about the plan and
management of the program. To do this, GAO analyzed the fiscal
year 2006 Atlas expenditure plan and supporting documents against
the legislative conditions, federal requirements, and related
best practices. GAO also interviewed relevant DHS officials.
-------------------------Indexing Terms-------------------------
REPORTNUM: GAO-07-565
ACCNO: A68850
TITLE: Information Technology: Immigration and Customs
Enforcement Needs to Fully Address Significant Infrastructure
Modernization Program Management Weaknesses
DATE: 04/27/2007
SUBJECT: Congressional oversight
Homeland security
Information technology
Performance management
Procurement planning
Program evaluation
Program management
Reporting requirements
Risk assessment
Risk management
Benefit-cost tracking
Program goals or objectives
DHS Atlas Program
******************************************************************
** This file contains an ASCII representation of the text of a **
** GAO Product. **
** **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced. Tables are included, but **
** may not resemble those in the printed version. **
** **
** Please see the PDF (Portable Document Format) file, when **
** available, for a complete electronic file of the printed **
** document's contents. **
** **
******************************************************************
GAO-07-565
* [1]Report to Congressional Committees
* [2]April 2007
* [3]INFORMATION TECHNOLOGY
* [4]Immigration and Customs Enforcement Needs to Fully Address
Significant Infrastructure Modernization Program Management
Weaknesses
* [5]Contents
* [6]Compliance with Legislative Conditions
* [7]Observations on the Expenditure Plan and Management of Atlas
* [8]Conclusions
* [9]Recommendations for Executive Action
* [10]Agency Comments and Our Evaluation
* [11]Briefing to the Staffs of the Subcommittees on Homeland Security,
Senate and House Committees on Appropriations
* [12]Comments from the Department of Homeland Security
* [13]GAO Contact and Staff Acknowledgments
* [14]gao.gov
* [15]GAO-06-296 Homeland Security: Recommendations to Improve
Management of Key Border Security Program Need to Be Implemented
* [16]Report to Congressional Requesters
* [17]February 2006
* [18]HOMELAND SECURITY
* [19]Recommendations to Improve Management of Key Border
Security Program Need to Be Implemented
* [20]Contents
* [21]Results in Brief
* [22]Background
* [23]Acquisition and Implementation Strategy: A
Brief Description
* [24]US-VISIT Is Being Implemented in Four
Increments
* [25]Program Management Roles and Responsibilities
* [26]Our Prior Work Has Resulted in Several
Recommendations
* [27]The Status of DHS's Implementation of Our
Recommendations Is Mixed
* [28]Development and Implementation of a Security
Plan and Performance of a Privacy Impact
Assessment Are Partially Complete
* [29]Security Plan
* [30]Privacy Impact Assessment
* [31]Development and Implementation of Key
Acquisition Controls Are Partially Complete
* [32]Determination and Disclosure of Whether
Increments Produce Mission Value Commensurate with
Costs and Risks Are Partially Complete
* [33]Definition of the Operational Context for
US-VISIT Is in Progress
* [34]Provision of Program Office Resources Is
Partially Complete
* [35]Definition of Program Office Roles and
Responsibilities Has Been Completed
* [36]Development and Implementation of a Human
Capital Strategy Are Partially Complete
* [37]Defining Performance Standards for US-VISIT
Increments Is Partially Complete
* [38]Development and Implementation of a Risk
Management Plan Are Partially Complete
* [39]Development of Test Plans Is Partially
Complete
* [40]Assessment of the Impact of Increment 2B on
Workforce Levels and Facilities Is Partially
Complete
* [41]Implementation of Configuration Management
Practices Is in Progress
* [42]Efforts to Ensure the Independence of the
Verification and Validation Contractor Are
Complete
* [43]Development of a Plan to Address Open
Recommendations Is Partially Complete
* [44]Establishment of Effective Cost-Estimating
Practices Is in Progress
* [45]Reassessment of Plans for Deploying the Exit
Capability Is Partially Complete
* [46]Development and Implementation of Capacity
Management Processes Are in Progress
* [47]Identification of ACE and US-VISIT
Relationships and Dependencies Is in Progress
* [48]Conclusions
* [49]Recommendation for Executive Action
* [50]Agency Comments and Our Evaluation
* [51]Objective, Scope, and Methodology
* [52]Comments from the Department of Homeland Security
* [53]Description of US-VISIT Processes
* [54]Pre-entry Process
* [55]Entry Process
* [56]Status Management Process
* [57]Exit Process
* [58]Analysis Process
* [59]GAO Contact and Staff Acknowledgments
Report to Congressional Committees
April 2007
INFORMATION TECHNOLOGY
Immigration and Customs Enforcement Needs to Fully Address Significant
Infrastructure Modernization Program Management Weaknesses
Contents
April 27, 2007
Letter
The Honorable Robert C. Byrd
Chairman
The Honorable Thad Cochran
Ranking Member
Subcommittee on Homeland Security
Committee on Appropriations
United States Senate
The Honorable David E. Price
Chairman
The Honorable Harold Rogers
Ranking Member
Subcommittee on Homeland Security
Committee on Appropriations
House of Representatives
The 2006 Department of Homeland Security (DHS) appropriations act^1
provided $40.15 million for Immigration and Customs Enforcement's (ICE)
program to modernize its information technology (IT) infrastructure. The
goals of the program--which consists of seven related IT projects and is
referred to by ICE as the "Atlas" program--include improving information
sharing and collaboration, strengthening information security, and
improving workforce productivity. The act provides that none of the funds
may be obligated until the department develops a plan for how the funds
are to be spent that satisfies certain legislative conditions. The
conditions included, among other things, having GAO review the plan. On
November 9, 2006, DHS submitted its fiscal year 2006 expenditure plan to
the Senate and House Appropriations Subcommittees on Homeland Security.
Pursuant to the act, we reviewed the plan. Our objectives were to (1)
determine whether the plan satisfies legislative conditions specified in
the act and (2) provide other observations about the expenditure plan and
the department's management of the Atlas program.
On February 2, 2007, we provided our review results via a briefing to the
staffs of the Senate and House Appropriations Subcommittees on Homeland
Security. This report summarizes our findings, conclusions, and
recommendations from that briefing. The full briefing, including our scope
and methodology, is reprinted in appendix I.
Compliance with Legislative Conditions
DHS's fiscal year 2006 Atlas expenditure plan, including related program
documentation and program officials' stated commitments, satisfies or
partially satisfies key aspects of (1) meeting the capital planning and
investment control review requirements of the Office of Management and
Budget (OMB);^2 (2) complying with the DHS enterprise architecture;^3 (3)
complying with acquisition rules, requirements, guidelines, and systems
acquisition management practices of the federal government; (4) including
certification by the department's chief information officer (CIO) that an
independent verification and validation agent is currently under contract
for the project; and (5) having the plan reviewed and approved by the
department's Investment Review Board, the Secretary of Homeland Security,
and OMB.
Regarding the fourth legislative condition, the department certified that
an independent verification and validation agent is under contract for
Atlas, and the department is in the process of defining how it plans to
use its agent, but unresolved issues remain. For example, we have
previously reported^4 that the independence of a verification and
validation agent is critical to providing management with objective
insight into program processes and associated work products. We also
reported^5 that to be effective, the verification and validation function
should be performed by an entity that is independent of the processes and
products that are being reviewed. When the agent engaged to provide the
service is a contractor, a practice to help ensure independence is to
include in the agent's contract a provision that prohibits the agent from
soliciting, proposing, or being awarded program work other than the
independent verification and validation services and products. However,
the contract that Atlas awarded to its agent did not include such a
provision.
Observations on the Expenditure Plan and Management of Atlas
Our observations address program efforts regarding (1) requirements
development and management, (2) contract management and oversight, (3)
risk management, and (4) performance management. An overview of these
observations follows:
oRigorous practices are not being fully adhered to in developing and
managing system requirements. On its three key projects,^6 the program has
not fully implemented federal IT guidance, relevant best practices, and
agency policies and procedures (i.e., ICE System Lifecycle Management
Handbook) that provide for effective requirements development and
management activities. For example, on its Common Computing Environment
project, while the project established a policy to guide project efforts
in developing and managing requirements, it did not among other things:
odocument stakeholder comments as part of the requirements elicitation
process and review and obtain stakeholder approval on requirements until
approximately 1 year after deploying system capabilities;
ofollow a disciplined change management process for several years (between
2003 and 2006) after the project was initiated in March 2003 (during this
time, requirements were introduced and implemented in an ad hoc fashion);
and
odevelop supporting analysis showing traceability among requirements,
systems design, and the contract.
oKey contract management and oversight practices are not fully
implemented. The program has not fully implemented practices that (1) the
Software Engineering Institute's CMMI^(R)7 has identified as essential to
effectively managing and overseeing contracts that support IT projects and
(2) the program has largely established in its existing policies and
procedures. Specifically, the program has implemented two key practices,
but has not fully implemented four others. For example, the program has
assigned responsibility and authority for performing contract management
and oversight, and has trained staff performing contract management and
oversight. However, it has not, among other things, fully documented each
contract--including clearly identifying the work to be performed and
acceptance criteria--and verified and accepted deliverables.
oKey risk management practices have yet to be implemented. We previously
reported^8 that the program had developed a risk management plan and
process to guide program office staff in managing risks throughout each
project's life cycle and had recently begun to implement the risk
management process. Since then, the program has taken additional steps to
implement risk management, such as hiring a risk management coordinator
and conducting quarterly program management meetings to review the
program's progress in managing risks. However, the program has not fully
implemented key practices essential to effectively managing project risks.
For example, it does not have a transparent, documented, and traceable
process for inventorying and resolving risks. Further, while the risk
management plan defines a process and associated practices for developing
risk mitigation strategies, the program has not fully implemented them. In
addition, although the risk management plan calls for developing a process
for elevating risks to management's attention, the program has not
developed and implemented such a process.
oPerformance management practices are still being implemented and
available data show mixed results and may not be reliable. We reported^9
in July 2006 that Atlas had begun taking steps to implement performance
goals and measures. Since then, the program has taken additional steps to
define and implement them. For example, the program identified 13
goals--for 5 of its 7 projects--that it has been using to measure progress
during fiscal year 2006. However, it has yet to fully implement and
institutionalize its performance goals and measures. For example, the
program has not yet developed performance goals for 2 projects.
In addition, progress against performance goals, as reported by the
program, show mixed results. For example, of the 12 goals, the program has
met 3 of them. Further, underlying data used by the program to measure
performance may not be reliable in all cases. For example, on 1 goal, the
program uses project cost and schedule data from the 2005 Atlas business
case in measuring progress against the goal, even though these data are
out of date (e.g., they do not reflect project cost and schedule slippages
that have occurred since 2005).
Conclusions
The fiscal year 2006 Atlas expenditure plan, in combination with related
program documentation and program officials' statements, satisfies or
partially satisfies the legislative conditions set forth by Congress.
However, this satisfaction is based on plans and commitments that provide
for meeting these conditions, rather than on completed actions to satisfy
the conditions. In addition, while steps are being initiated that are
intended to address significant program management weaknesses, a number of
improvements, including those recommended in our past reports, have yet to
be implemented.
Further, the program has not fully achieved many performance goals that it
set out to accomplish over the past year. The above factors continue to
put the program at risk and call for heightened and sustained management
attention to expeditiously address and resolve the issues. Thus, there is
much that still needs to be accomplished to minimize the risks associated
with the program's capacity to deliver promised IT infrastructure
capabilities and benefits on time and within budget. This includes
demonstrating better progress against established performance goals in the
coming year.
Given that hundreds of millions of dollars are to be invested and the
program is critical to supporting the ICE mission, it is essential that
DHS follow through on its commitments to build the capacity to effectively
manage the program. Proceeding without this capacity introduces
unnecessary risks to the program and potentially jeopardizes its viability
for future investment.
Recommendations for Executive Action
To minimize risks to the Atlas program, we recommend that the Secretary of
Homeland Security direct the Assistant Secretary for Immigration and
Customs Enforcement to ensure that ICE follows through on its commitments
to implement effective management controls and capabilities by
implementing the following five recommendations:
oEmploy practices essential to ensuring that the Atlas program's
independent verification and validation agent is and remains independent,
including incorporating requirements in future contracting actions, such
as the renegotiation or recompetition of the current independent
verification and validation contract, which will prohibit the agent from
soliciting, proposing, or being awarded program work other than providing
independent verification and validation services and products.
oFully adhere to requirements development and management practices,
including those specified in ICE's policies and procedures. This should
also include having the Atlas program manager develop a process
improvement plan for all of the projects that is consistent with the ICE
System Lifecycle Management Handbook and provide for making the program
manager and project managers responsible and accountable for rigorously
adhering to the requirements in the handbook.
oFully implement key contract management and oversight practices,
including those specified in ICE's policies and procedures. This should
also include ensuring that the Atlas program manager, working with the
program's contracting officer's technical representative, follow through
on planned efforts to strengthen the program's compliance with these
practices by (1) revising the acquisition plan by May 2007; (2) revising
the ICE System Lifecycle Management Handbook by June 2007 to incorporate
key contract management activities such as contract tracking and
oversight; and (3) preparing statements of work (beginning in February
2007) for future contracts to clearly define Atlas work statements and
acceptance criteria.
oComplete implementation of planned risk management activities. This
should include (1) updating the risk inventory to include risks for all
projects and risk areas and (2) fully implementing and institutionalizing
procedures for reporting to management on the existence and status of
risks and progress in implementing mitigation strategies.
oImprove program performance management, including developing performance
goals for projects that do not have goals and reporting on their progress
in the fiscal year 2007 expenditure plan. Further, the program should
assess that the data being used to measure progress are reliable,
complete, and accurate.
Agency Comments and Our Evaluation
In DHS's written comments on a draft of this report, signed by the
Director, Departmental GAO/Office of Inspector General Liaison Office, the
department concurred with all of our recommendations and described actions
planned and under way to implement them. The department also orally
provided technical comments updating Atlas obligation amounts and the
milestone for revising the acquisition plan. We incorporated these
comments in the report and briefing as appropriate. DHS's written comments
are reprinted in appendix II.
We are sending copies of this report to the Chairman and Ranking Members
of other Senate and House committees and subcommittees that have
authorization and oversight responsibilities for homeland security. We are
also sending copies to the Secretary of Homeland Security, the Assistant
Secretary for Immigration and Customs Enforcement, and the Director of the
Office of Management and Budget. Copies of this report will also be
available at no charge on our Web site at [60]w ww.gao.gov.
Should you or your offices have any questions on matters discussed in this
report, please contact me at (202) 512-3439 or at [61][email protected] .
Contact points for our Offices of Congressional Relations and Public
Affairs may be found on the last page of this report. Key contributors to
this report are listed in appendix III.
Randolph C. Hite
Director, Information Technology Architecture
and Systems Issues
Appendix I
Briefing to the Staffs of the Subcommittees on Homeland Security, Senate
and House Committees on Appropriations
Appendix II
Comments from the Department of Homeland Security
Appendix III
GAO Contact and Staff Acknowledgments
GAO Contact
Randolph C. Hite (202) 512-3439
Staff Acknowledgments
In addition to the individual named above, Gary Mountjoy, Assistant
Director; Nancy Glover; James Houtz; Jennifer Mills; Freda Paintsil;
Madhav Panwar; Karl Seifert; and Kelly Shaw made key contributions to this
report.
(310640)
www.gao.gov/cgi-bin/getrpt?GAO-07-565 .
To view the full product, including the scope
and methodology, click on the link above.
For more information, contact Randolph C. Hite at (202) 512-3439 or
[email protected].
Highlights of [63]GAO-07-565 , a report to congressional committees
April 2007
INFORMATION TECHNOLOGY
Immigration and Customs Enforcement Needs to Fully Address Significant
Infrastructure Modernization Program Management Weaknesses
The Department of Homeland Security (DHS) fiscal year 2006 appropriations
act provided $40.15 million for the Immigration and Customs Enforcement's
(ICE) program to modernize its information technology (IT) infrastructure.
As mandated by the appropriations act, the department is to develop and
submit for approval an expenditure plan for the program, referred to as
"Atlas," that satisfies certain legislative conditions, including a review
by GAO. In performing its review of the Atlas plan, GAO (1) determined
whether the plan satisfies certain legislative conditions and (2) provided
other observations about the plan and management of the program. To do
this, GAO analyzed the fiscal year 2006 Atlas expenditure plan and
supporting documents against the legislative conditions, federal
requirements, and related best practices. GAO also interviewed relevant
DHS officials.
[64]What GAO Recommends
GAO is recommending that DHS minimize Atlas program risks by fully
adhering to requirements development and management practices,
implementing key contract management and oversight practices, completing
planned risk management activities, and implementing critical performance
management practices. DHS concurred with GAO's recommendations and
described actions planned and under way to implement them.
The fiscal year 2006 Atlas expenditure plan, in combination with related
program documentation and program officials' statements, satisfies or
partially satisfies the legislative conditions set forth by Congress (see
table).
DHS's Satisfaction of Legislative Conditions
Source: GAO.
This satisfaction, however, is based on plans and commitments that provide
for meeting these conditions rather than on completed actions to satisfy
them. For example, to address the legislative condition related to capital
planning and investment control review requirements, the program plans to,
among other things, update its cost-benefit analysis in September 2007 to
reflect emerging requirements and other program changes and to complete a
privacy impact assessment by April 2007. In addition, the program is in
the process of defining how it plans to use its independent verification
and validation agent.
GAO also observed that DHS has not implemented key system management
practices. Specifically,
o rigorous practices are not being fully adhered to in developing
and managing system requirements,
o key contract management and oversight controls have not been
fully implemented,
o planned risk management practices have yet to be implemented,
and
o performance management practices that are critical to measuring
progress against program goals are still being implemented.
Thus, much still needs to be accomplished to minimize the risks associated
with the program's capacity to deliver promised IT infrastructure
capabilities and benefits on time and within budget. Given that hundreds
of millions of dollars are to be invested and the program is critical to
supporting the ICE mission, it is essential that DHS follow through on its
commitments to build the capability to effectively manage the program.
Proceeding without it introduces unnecessary risks to the program and
potentially jeopardizes its viability for future investment.
References
Visible links
60. http://www.gao.gov.
61. [email protected]
63. http://www.gao.gov/cgi-bin/getrpt?GAO-07-565
*** End of document. ***