Homeland Security: Recommendations to Improve Management of Key
Border Security Program Need to Be Implemented (14-FEB-06,
GAO-06-296).
The Department of Homeland Security (DHS) has established a
program--the U.S. Visitor and Immigrant Status Indicator
Technology (US-VISIT)--to collect, maintain, and share
information, including biometric identifiers, on selected foreign
nationals entering and exiting the United States. US-VISIT uses
these identifiers (digital fingerscans and photographs) to screen
persons against watch lists and to verify that a visitor is the
person who was issued a visa or other travel document. Visitors
are also to confirm their departure by having their visas or
passports scanned and undergoing fingerscanning at selected air
and sea ports of entry (POE). GAO has made many recommendations
to improve the program, all of which DHS has agreed to implement.
GAO was asked to report on DHS's progress in responding to 18 of
these recommendations.
-------------------------Indexing Terms-------------------------
REPORTNUM: GAO-06-296
ACCNO: A46797
TITLE: Homeland Security: Recommendations to Improve Management
of Key Border Security Program Need to Be Implemented
DATE: 02/14/2006
SUBJECT: Biometric identification
Cost effectiveness analysis
Data collection
Homeland security
Identity verification
Immigration information systems
Internal controls
Operational testing
Passports
Performance measures
Program evaluation
Program management
Strategic planning
Visas
Information sharing
DHS Visitor and Immigrant Status
Indicator Technology Program
******************************************************************
** This file contains an ASCII representation of the text of a **
** GAO Product. **
** **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced. Tables are included, but **
** may not resemble those in the printed version. **
** **
** Please see the PDF (Portable Document Format) file, when **
** available, for a complete electronic file of the printed **
** document's contents. **
** **
******************************************************************
GAO-06-296
* Report to Congressional Requesters
* February 2006
* HOMELAND SECURITY
* Recommendations to Improve Management of Key Border Security
Program Need to Be Implemented
* Contents
* Results in Brief
* Background
* Acquisition and Implementation Strategy: A Brief Description
* US-VISIT Is Being Implemented in Four Increments
* Program Management Roles and Responsibilities
* Our Prior Work Has Resulted in Several Recommendations
* The Status of DHS's Implementation of Our Recommendations Is
Mixed
* Development and Implementation of a Security Plan and
Performance of a Privacy Impact Assessment Are Partially
Complete
* Security Plan
* Privacy Impact Assessment
* Development and Implementation of Key Acquisition Controls
Are Partially Complete
* Determination and Disclosure of Whether Increments Produce
Mission Value Commensurate with Costs and Risks Are
Partially Complete
* Definition of the Operational Context for US-VISIT Is in
Progress
* Provision of Program Office Resources Is Partially Complete
* Definition of Program Office Roles and Responsibilities Has
Been Completed
* Development and Implementation of a Human Capital Strategy
Are Partially Complete
* Defining Performance Standards for US-VISIT Increments Is
Partially Complete
* Development and Implementation of a Risk Management Plan Are
Partially Complete
* Development of Test Plans Is Partially Complete
* Assessment of the Impact of Increment 2B on Workforce Levels
and Facilities Is Partially Complete
* Implementation of Configuration Management Practices Is in
Progress
* Efforts to Ensure the Independence of the Verification and
Validation Contractor Are Complete
* Development of a Plan to Address Open Recommendations Is
Partially Complete
* Establishment of Effective Cost-Estimating Practices Is in
Progress
* Reassessment of Plans for Deploying the Exit Capability Is
Partially Complete
* Development and Implementation of Capacity Management
Processes Are in Progress
* Identification of ACE and US-VISIT Relationships and
Dependencies Is in Progress
* Conclusions
* Recommendation for Executive Action
* Agency Comments and Our Evaluation
* Objective, Scope, and Methodology
* Comments from the Department of Homeland Security
* Description of US-VISIT Processes
* Pre-entry Process
* Entry Process
* Status Management Process
* Exit Process
* Analysis Process
* GAO Contact and Staff Acknowledgments
Report to Congressional Requesters
February 2006
HOMELAND SECURITY
Recommendations to Improve Management of Key Border Security Program Need
to Be Implemented
Contents
Tables
Figures
February 14, 2006Letter
Congressional Requesters:
The U.S. Visitor and Immigrant Status Indicator Technology (US-VISIT) is a
multibillion-dollar program of the Department of Homeland Security (DHS)
that is intended to record the entry into and exit from the United States
of selected individuals, verify their identity, and confirm their
compliance with the terms of their admission into and stay in the United
States. The goals of the program are to (1) enhance the security of our
citizens and visitors, (2) facilitate legitimate travel and trade,
(3) ensure the integrity of the U.S. immigration system, and (4) protect
the privacy of our visitors.
Since fiscal year 2002, DHS has been legislatively directed to submit
annual expenditure plans for the program, and we have been directed to
review these plans and issue reports. These reports have, among other
things, identified risks that face the department in delivering promised
program capabilities and benefits on time and within cost.1 For example,
we reported that the program office did not have the human capital and
acquisition process discipline needed to effectively manage the program.
Because of the number and severity of program management challenges that
we identified, we concluded that the program was risky.
To address program risks, our reports have included 18 recommendations in
such areas as system acquisition process controls, economic justification,
human capital management, cost estimating, and test management, all of
which DHS has agreed to implement.2 Because of your continued interest in
ensuring that DHS is taking the necessary actions to successfully
implement US-VISIT, you asked us to determine the progress being made in
implementing these recommendations. To achieve this objective, we analyzed
program plans, reports, and system documentation relative to the intent of
each of our recommendations, and we interviewed appropriate DHS and
program officials. (Further details on our objective, scope, and
methodology are provided in app. I.) Our work was performed from August
2005 through December 2005 in accordance with generally accepted
government auditing standards.
Results in Brief
The current status of DHS's implementation of the 18 recommendations is
mixed, but progress in critical areas has been slow. DHS has implemented 2
of the recommendations: it defined program staff positions, roles, and
responsibilities, and it hired an independent verification and validation
contractor. It has also taken steps to implement the other
recommendations, partially completing 11 and beginning to implement
another 5. However, although considerable time has passed since the
recommendations were made, key actions have not yet been taken in such
critical areas as (1) assessing security risks and planning for
cost-effective controls to address the risks, (2) determining-before
US-VISIT increments are deployed-whether each increment will produce
mission value commensurate with cost and risk, and (3) ensuring that each
increment is adequately tested. Of the 11 recommendations that are
partially implemented, 7 are about 2 years old, and 4 are about 10 to 19
months old. Of the 5 that are in progress, 3 are about 10 months old.3
According to the Program Director, the pace of progress is attributable to
competing demands on time and resources. The longer that US-VISIT takes to
implement the recommendations, the greater the risk that the program will
not meet its stated goals on time and within budget.
DHS provided written comments on a draft of this report. In its comments,
the department stated that it agreed with many areas of the report and
that our recommendations had made US-VISIT a stronger program. Further,
the department stated that while it disagreed with certain areas of the
report, it nevertheless concurred with the need to implement our open
recommendations with all due speed and diligence. One area of disagreement
was regarding the program's ability to thoroughly assess the impact of
US-VISIT entry capabilities on the 50 busiest land port of entry (POE)
facilities and staffing levels, an assessment that we called for in our
recommendation. In particular, DHS stated that since US-VISIT was
operational at these POEs, the collection of predeployment baseline
performance data was no longer practical. In light of these comments, we
are making a new recommendation to the Secretary of DHS that recognizes
these facts and circumstances and that replaces the open recommendation
discussed in this report. This recommendation provides for the department
to explore alternative means of assessing the impact of US-VISIT entry
capabilities on land POE facilities and staffing levels. All of DHS's
comments, along with our responses, are discussed in detail in the Agency
Comments and Our Evaluation section of this report. The comments are also
reprinted in their entirety in appendix II.
Background
US-VISIT is a governmentwide program intended to enhance the security of
U.S. citizens and visitors, facilitate legitimate travel and trade, ensure
the integrity of the U.S. immigration system, and protect the privacy of
our visitors. Its scope includes the pre-entry, entry, status, and exit of
hundreds of millions of foreign national travelers who enter and leave the
United States at over 300 air, sea, and land POEs, and the provision of
new analytical capabilities across the overall process.
To achieve its goals, US-VISIT uses biometric information (digital
fingerscans and photographs) to verify identity.4 In many cases, the
US-VISIT process begins overseas at U.S. consular offices, which collect
biometric information from applicants for visas and check this information
against a database of known criminals and suspected terrorists. When a
visitor arrives at a POE, the biometric information is used to verify that
the visitor is the person who was issued the visa. In addition, at certain
sites, visitors are required to confirm their departure by undergoing
US-VISIT exit procedures-that is, having their visas or passports scanned
and undergoing fingerscanning. The exit confirmation is added to the
visitor's travel records to demonstrate compliance with the terms of
admission to the United States. (App. III provides a detailed description
of the pre-entry, entry, status, exit, and analysis processes.)
Key US-VISIT functions include
o collecting, maintaining, and sharing information on certain foreign
nationals who enter and exit the United States;
o identifying foreign nationals who (1) have overstayed or violated the
terms of their admission; (2) may be eligible to receive, extend, or
adjust their immigration status; or (3) should be apprehended or detained
by law enforcement officials;
o detecting fraudulent travel documents, verifying traveler identity, and
determining traveler admissibility through the use of biometrics; and
o facilitating information sharing and coordination within the immigration
and border management community.
In July 2003, DHS established a program office with responsibility for
managing the acquisition, deployment, operation, and sustainment of the
US-VISIT system and its associated supporting people (e.g., Customs and
Border Protection (CBP) officers), processes (e.g., entry/exit policies
and procedures), and facilities (e.g., inspection booths and lanes), in
coordination with its stakeholders (CBP and the Department of State).
As of October 2005, about $1.4 billion has been appropriated for the
program, and, according to program officials, about $962 million has been
obligated.
Acquisition and Implementation Strategy: A Brief Description
DHS plans to deliver US-VISIT capability in four increments, with
Increments 1 through 3 being interim, or temporary, solutions that fulfill
legislative mandates to deploy an entry/exit system, and Increment 4 being
the implementation of a long-term vision that is to incorporate improved
business processes, new technology, and information sharing to create an
integrated border management system for the future. In Increments 1
through 3, the program is building interfaces among existing ("legacy")
systems; enhancing the capabilities of these systems; and deploying these
capabilities to air, sea, and land POEs. These increments are to be
largely acquired and implemented through existing system contracts and
task orders.
In May 2004, DHS awarded an indefinite-delivery/indefinite-quantity5 prime
contract to Accenture and its partners. According to the contract, the
prime contractor will help support the integration and consolidation of
processes, functionality, and data, and it will develop a strategy to
build on the technology and capabilities already available to produce the
strategic solution, while also assisting the program office in leveraging
existing systems and contractors in deploying the interim solutions.
US-VISIT Is Being Implemented in Four Increments
Increment 1 concentrates on establishing capabilities at air and sea POEs.
It is divided into two parts-1 and 1B.
o Increment 1 (air and sea entry) includes the electronic capture and
matching of biographic and biometric information (two digital index
fingerscans and a digital photograph) for selected foreign nationals,
including those from visa waiver countries.6 Increment 1 was deployed on
January 5, 2004, for individuals requiring a nonimmigrant visa to enter
the United States, through the modification of pre-existing systems.7
These modifications accommodated the collection and maintenance of
additional data fields and established interfaces required to share data
among DHS systems in support of entry processing at 115 airports and 14
seaports.
o Increment 1B (air and sea exit) involves the testing of exit devices to
collect biometric exit data for select foreign nationals at 11 airports
and seaports. Three exit alternatives were pilot tested:
o Kiosk-A self-service device (which includes a touch-screen interface,
document scanner, finger scanner, digital camera, and receipt printer)
that captures a digital photograph and fingerprint and prints out an
encoded receipt.
o Mobile device-A hand-held device that is operated by a workstation
attendant;8 it includes a document scanner, finger scanner, digital
camera, and receipt printer and is used to capture a digital photograph
and fingerprint.
o Validator-A hand-held device that is used to capture a digital
photograph and fingerprint, which are then matched to the photograph and
fingerprint captured via the kiosk and encoded in the receipt.
Increment 2 focuses primarily on extending US-VISIT to land POEs. It is
divided into three parts-2A, 2B, and 2C.
o Increment 2A (air, sea, and land) includes the capability to
biometrically compare and authenticate valid machine-readable visas and
other travel and entry documents issued by State and DHS to foreign
nationals at all POEs. Increment 2A was deployed on October 23, 2005,
according to program officials. It also includes the deployment by October
26, 2006, of technology to read biometrically enabled passports from visa
waiver countries.
o Increment 2B (land entry) redesigns the Increment 1 entry solution and
expands it to the 50 busiest land POEs. The process for issuing Form I-949
was redesigned to enable the electronic capture of biographic, biometric
(unless the traveler is exempt), and related travel documentation for
arriving travelers. This increment was deployed to the busiest 50 U.S.
land border POEs as of December 29, 2004. Before Increment 2B, all
information on the Form I-94s was handwritten. The redesigned systems
electronically capture the biographic data included in the travel
document. In some cases, the form is completed by CBP officers, who enter
the data electronically and then print the form.
o Increment 2C is to provide the capability to automatically, passively,
and remotely record the entry and exit of covered individuals using radio
frequency (RF) technology tags at primary inspection and exit lanes.10 An
RF tag that includes a unique ID number is to be embedded in each Form
I-94, thus associating a unique number with a record in the US-VISIT
system for the person holding that Form I-94. In August 2005, the program
office deployed the technology to five border crossings (three POEs) to
verify the feasibility of using passive RF technology to record traveler
entries and exits via a unique ID number embedded in the CBP Form I-94.
The results of this demonstration are to be reported in February 2006.
Increment 3 extended Increment 2B (land entry) capabilities to 104 land
POEs; this increment was essentially completed as of December 19, 2005.11
Increment 4 is the strategic US-VISIT program capability, which program
officials stated will likely consist of a further series of incremental
releases or mission capability enhancements that will support business
outcomes. The program reports that it has worked with its prime
contractor and partners to develop this overall vision for the immigration
and border management enterprise.
Increments 1 through 3 include the interfacing and integration of existing
systems and, with Increment 2C, the creation of a new system, the
Automated Identification Management System (AIDMS). The three main
existing systems are as follows:
o The Arrival Departure Information System (ADIS) stores
o noncitizen traveler arrival and departure data received from air and sea
carrier manifests,
o arrival data captured by CBP officers at air and sea POEs,
o Form I-94 issuance data captured by CBP officers at Increment 2B land
POEs,
o departure information captured at US-VISIT biometric departure pilot
(air and sea) locations,
o pedestrian arrival information and pedestrian and vehicle departure
information captured at Increment 2C POE locations, and
o status update information provided by the Student and Exchange Visitor
Information System (SEVIS) and the Computer Linked Application Information
Management System (CLAIMS 3) (described below).
ADIS provides record matching, query, and reporting functions.
o The passenger processing component of the Treasury Enforcement
Communications System (TECS) includes two systems: Advance Passenger
Information System (APIS), a system that captures arrival and departure
manifest information provided by air and sea carriers, and the Interagency
Border Inspection System, a system that maintains lookout data and
interfaces with other agencies' databases. CBP officers use these data as
part of the admission process. The results of the admission decision are
recorded in TECS and ADIS.
o The Automated Biometric Identification System (IDENT) collects and
stores biometric data on foreign visitors.
US-VISIT also exchanges biographic information with other DHS systems,
including SEVIS and CLAIMS 3. These two systems contain information on
foreign students and foreign nationals who request benefits, such as a
change of status or extension of stay.
Some of the systems previously described, such as IDENT and the new AIDMS,
are managed by the program office, while some systems are managed by
other organizational entities within DHS. For example, TECS is managed by
CBP, SEVIS is managed by Immigration and Customs Enforcement, CLAIMS 3 is
under United States Citizenship and Immigration Services, and ADIS is
jointly managed by CBP and US-VISIT.
US-VISIT also interfaces with other, non-DHS systems for relevant
purposes, including watch list updates and checks to determine whether a
visa applicant has previously applied for a visa or currently has a valid
U.S. visa. In particular, US-VISIT receives biographic and biometric
information from State's Consular Consolidated Database as part of the
visa application process, and returns fingerscan information and watch
list changes.
Program Management Roles and Responsibilities
The US-VISIT program office structure includes nine component offices.
Each of the program offices includes a director and subordinate
organizational units, as established by the director. The responsibilities
for each office are stated below. Figure 1 shows the program office
structure, including its nine offices.
Figure 1: US-VISIT Program Office Structure
The roles and responsibilities for each of the nine offices include the
following:
o Chief Strategist is responsible for developing and maintaining the
strategic vision, strategic documentation, transition plan, and business
case.
o Budget and Financial Management is responsible for establishing the
program's costs estimates; analysis; and expenditure management policies,
processes, and procedures that are required to implement and support the
program by ensuring proper fiscal planning and execution of the budget and
expenditures.
o Mission Operations Management is responsible for developing business and
operational requirements based on strategic direction provided by the
Office of the Chief Strategist.
o Outreach Management is responsible for enhancing awareness of US-VISIT
requirements among foreign nationals, key domestic audiences, and internal
stakeholders by coordinating outreach to media, third parties, key
influencers, Members of Congress, and the traveling public.
o Information Technology Management is responsible for developing
technical requirements based on strategic direction provided by the Office
of the Chief Strategist and business requirements developed by the Office
of Mission Operations Management.
o Implementation Management is responsible for developing accurate,
measurable schedules and cost estimates for the delivery of mission
systems and capabilities.
o Acquisition and Program Management is responsible for establishing and
managing the execution of program acquisition and management policies,
plans, processes, and procedures.
o Administration and Training is responsible for developing and
administering a human capital plan that includes recruiting, hiring,
training, and retaining a diverse workforce with the competencies
necessary to accomplish the mission.
o Facilities and Engineering Management is responsible for establishing
facilities and environmental policies, procedures, processes, and guidance
required to implement and support the program office.
Our Prior Work Has Resulted in Several Recommendations
In response to legislative mandate, we have issued four reports on DHS's
annual expenditure plans for US-VISIT.12 Our reports have, among other
things, assessed whether the plans satisfied the legislative conditions
and provided observations on the plans and DHS's program management. As a
result of our assessments, we made 24 recommendations aimed at improving
both plans and program management, all of which DHS has agreed to
implement. Of these 24 recommendations, 18 address risks stemming from
program management.13
The Status of DHS's Implementation of Our Recommendations Is Mixed
The current status of DHS's implementation of our 18 recommendations on
program risks is mixed, but progress in critical areas has been slow. For
example, over 2 years have passed, and the program office has yet to
develop a security plan consistent with federal guidance or to
economically justify its investment in system increments. According to
the Program Director, the pace of progress is attributable to competing
demands on time and resources.
DHS agreed to implement all 18 recommendations. Of these 18, DHS has
completely implemented 2, has partially implemented 11, and is in the
process of implementing another 5. Of the 11 that are partially
implemented, 7 are about 2 years old, and 4 are about 10 to 19 months old.
Of the 5 that are in progress, 3 are about 10 months old.
These 18 recommendations are aimed at strengthening the program's
management effectiveness. The longer that the program takes to implement
the recommendations, the greater the risk that the program will not meet
its goals on time and within budget.
Figure 2 provides an overview of the extent to which each recommendation
has been implemented. The figure is followed by sections providing
details on each recommendation and our assessment of its implementation
status.
Figure 2: DHS's Progress toward Implementing GAO's 18 Recommendations
aA recommendation is completely implemented when documentation
demonstrated that it had been fully addressed.
bA recommendation is partially implemented when documentation indicated
that actions were under way to implement it.
cA recommendation is in progress when documentation indicated that actions
had been initiated to implement it.
dCarnegie Mellon University Software Engineering Institute, Software
Acquisition Capability Maturity Model, Version 1.03 (March 2002).
eAutomated Commercial Environment is a new trade processing system planned
to support the movement of legitimate imports and exports and to
strengthen border security.
Development and Implementation of a Security Plan and Performance of a
Privacy Impact Assessment Are Partially Complete
In June 2003,14 we reported that the Immigration and Naturalization
Service15 had not developed a security plan and performed a privacy impact
assessment for the entry exit program (as US-VISIT was then known). A
security plan and privacy impact assessment are important to understanding
system requirements and ensuring that the proper safeguards are in place
to protect system data and resources. System acquisition best practices
and federal guidance advocate understanding and defining security and
privacy requirements both early and continuously in a system's life cycle,
and effectively planning for their satisfaction. Accordingly, we
recommended that DHS do the following:
Develop and begin implementing a system security plan, and perform a
privacy impact assessment and use the results of the analysis in near-term
and subsequent system acquisition decision making.
Security Plan
Since we made the system security plan recommendation about 2 1/2 years
ago, its implementation has been slow. For example, we reported in
September 2003 and again in May 2004 that the program office had not
developed a security plan. In February 2005, we reported that the program
office had developed a security plan, dated September 2004, and that this
plan was generally consistent with federal guidance.16 That is, the plan
provided an overview of system security requirements, described the
controls in place or planned for meeting those requirements, referred to
the applicable documents that prescribe the roles and responsibilities for
managing the US-VISIT component systems, and addressed security awareness
and training. However, the program office had not conducted a risk
assessment or included in the plan when an assessment would be completed.
According to guidance from the Office of Management and Budget (OMB), the
security plan should describe the methodology that is used to identify
system threats and vulnerabilities and to assess risks, and it should
include the date the risk assessment was completed.
According to program officials, they completed a programwide risk
assessment in December 2005, but have yet to provide a copy of the
assessment to us. Therefore, we cannot confirm that the assessment has
been done, and done properly. The absence of a risk assessment and a
security plan that reflects this assessment is a significant program
weakness. Risk assessments are critical to establishing effective security
controls because they provide the basis for establishing appropriate
policies and selecting cost-effective controls to implement these
policies. Without such an assessment, US-VISIT does not have adequate
assurance that it knows the risks associated with the program and thus
whether it has implemented effective controls to address them.
Notwithstanding these limitations in the security plan, the program office
has begun to implement aspects of its September 2004 security plan. For
example, the Information Systems Security Manager told us that a security
awareness program is established and key personnel have attended security
training.
Privacy Impact Assessment
Since June 2003, US-VISIT has also developed and periodically updated a
privacy impact assessment. An initial impact assessment was issued in
January 2004, and a revised assessment was issued in September 2004.17 A
more recent assessment, dated July 2005, reflects changes related to
Increments 1B and 2C. Each of these assessments is generally consistent
with OMB guidance.18 That is, each of the assessments addressed most OMB
requirements, including the impact that the system will have on individual
privacy, the privacy consequences of collecting the information, and
alternatives considered to collect and handle information. The most recent
impact assessment, for example, states that three alternatives were
considered for Increment 1B-the kiosk, the mobile device, and the
validator (a combination of the two)-and discusses proposals to mitigate
the privacy risks of all three, such as by limiting the duration of data
retention on the exit devices and using encryption.
However, OMB guidance also requires that privacy impact assessments
developed for systems under development address privacy in relevant system
documentation, including statements of need, functional requirements
documents, and cost-benefit analyses. As we reported about previous
privacy impact assessments, privacy is only partially addressed in system
documentation. For example, the Increment 1B cost-benefit analysis
assesses the privacy risk associated with each exit alternative, and the
Increment 2C business requirements state that all solutions are to be
compliant with privacy laws and regulations and adhere to US-VISIT privacy
policy. However, we did not find privacy in the Increment 1B business
requirements or the Increment 2C functional requirements. Program
officials, including the US-VISIT Privacy Officer, acknowledged that
privacy is not included in the system documentation, but stated that
privacy is considered in the development of the documentation and that the
privacy office reviews key system documentation at relevant times during
the system development life cycle. Nevertheless, we did not find evidence
of privacy being addressed in the system documentation, and program
officials acknowledged that it was not included.
Until the program performs a risk assessment and fully implements a
security plan that reflects this assessment, it cannot adequately ensure
that US-VISIT is cost-effectively safeguarding assets and data. Moreover,
without reflecting privacy in system documentation, it cannot adequately
ensure that privacy needs are being fully addressed.
Development and Implementation of Key Acquisition Controls Are Partially
Complete
We reported in September 200319 that the program office had not defined
key acquisition management controls to support the acquisition of
US-VISIT, and therefore its efforts to acquire, deploy, operate, and
maintain system capabilities were at risk of not satisfying system
requirements and of not meeting benefit expectations on time and within
budget.
The Capability Maturity Model-Integration(R) (CMMI) developed by Carnegie
Mellon University's Software Engineering Institute (SEI) explicitly
defines process management controls that are recognized hallmarks of
successful organizations and that, if implemented effectively, can greatly
increase the chances of successfully acquiring software-intensive
systems.20 SEI's CMMI model uses capability levels to assess process
maturity.21 Because establishing the basic acquisition process
capabilities, according to SEI, can take on average about 19 months, we
recognized the importance of starting early to build effective acquisition
management capabilities by recommending that DHS do the following:
Develop and implement a plan for satisfying key acquisition management
controls, including acquisition planning, solicitation, requirements
management, program management, contract tracking and oversight,
evaluation, and transition to support, and implement the controls in
accordance with SEI guidance.
The program office has recently taken foundational steps to establish key
acquisition management controls. For example, it has developed a process
improvement plan, dated May 16, 2005 (about 20 months after our
recommendation), to define and implement these controls. As part of its
improvement program, the program office is implementing a governance
structure for overseeing improvement activities, consisting of three
groups: a Management Steering Group, an Enterprise Process Group, and
Process Action Teams. Specific roles for each of these groups are
described below.
o The Management Steering Group is to provide policy and procedural
guidance and to oversee the entire improvement program. The steering group
is chaired by the US-VISIT Director, with the Deputy Director and the
functional office directors serving as core members.
o The Enterprise Process Group is to provide planning, management, and
operational guidance in day-to-day process improvement activities. The
group is chaired by the process improvement leader and is composed of
individuals from each functional office.
o Process Action Teams are to provide specific process documentation and
to provide implementation support and training services. These teams are
to be active as long as a particular process improvement initiative is
under way. To date, the program office has chartered five process
teams-configuration management, cost analysis, process development,
communications, and policy.
In addition, the program office has recently completed a self-assessment
of its acquisition process maturity, and it plans to use the assessment
results to establish a baseline of its acquisition process maturity for
improvement. According to program officials, the assessment included 13
key process areas that are generally consistent with the process areas
cited in our recommendation. The program has ranked these 13 process areas
according to their priority, and, for initial implementation, it plans to
focus on the following 6:22
o Configuration management. Establishing and maintaining the integrity of
the products throughout their life cycle.
o Process and product quality assurance. Taking actions to provide
management with objective insight into the quality of products and
processes.
o Project monitoring and control. Tracking the project's progress so that
appropriate corrective actions can be taken when performance deviates
significantly from plans.
o Project planning. Establishing and maintaining plans for work
activities.
o Requirements management. Managing the requirements and ensuring a common
understanding of the requirements between the customer and the product
developers.
o Risk management. Identifying potential problems before they occur so
that they can be mitigated to minimize any adverse impact.
The improvement plan is currently being updated to reflect the results of
the baseline assessment and to include a detailed work breakdown
structure, process prioritization, and resource estimates. According to
the Director, Acquisition and Program Management Office (APMO), the goal
is to conduct a formal SEI appraisal to assess the capability level of
some or all of the six processes by October 2006.
Notwithstanding the recent steps to begin addressing our recommendation,
much work remains to fully implement key acquisition management controls.
Moreover, effectively implementing these controls takes considerable time.
Therefore, it is important that these improvement efforts stay on track.
Until these processes are effectively implemented, US-VISIT will be at
risk of not delivering promised capabilities on time and within budget.
Determination and Disclosure of Whether Increments Produce Mission Value
Commensurate with Costs and Risks Are Partially Complete
In September 2003, we reported that the program had not assessed the costs
and benefits of Increment 1, which is extremely important because the
decision to invest in any capability should be based on reliable analyses
of return on investment. Further, according to OMB guidance, individual
increments of major systems are to be individually supported by analyses
of benefits, cost, and risk.23 Without reliable analyses, an organization
cannot adequately know that a proposed investment is a prudent and
justified use of limited resources. Accordingly, we recommended that DHS
do the following:
Determine whether proposed US-VISIT increments will produce mission value
commensurate with cost and risks and disclose to the Congress planned
actions.
As we reported in September 2003 and again in February 2005,24 the program
office did not justify its planned investment in Increments 1 and 2B,
respectively, based on expected return on investment. Since then, the
program has developed a cost-benefit analysis for Increment 1B.
OMB has issued guidance concerning the analysis needed to justify
investments.25 According to this guidance, such analyses should meet
certain criteria to be considered reasonable. These criteria include,
among other things, comparing alternatives on the basis of net present
value and conducting uncertainty analyses of costs and benefits. DHS has
also issued guidance on such economic analyses that is consistent with
that of OMB.26
The latest cost-benefit analysis for Increment 1B (dated June 23, 2005)
identifies potential costs and benefits for three exit solutions at air
and sea POEs and provides a general rationale for the viability of the
three alternatives described. This latest analysis meets four of eight OMB
economic analysis criteria. However, it does not, for example, include a
complete uncertainty analysis (i.e., both a sensitivity analysis and a
Monte Carlo simulation27) for the three exit alternatives evaluated. That
is, the cost-benefit analysis does include a Monte Carlo simulation, but
it does not include a sensitivity analysis for the three alternatives. An
analysis of uncertainty is important because it provides decision makers
with a perspective on the potential variability of the cost and benefit
estimates should the facts, circumstances, and assumptions change.
Table 1 summarizes our analysis of the extent to which US-VISIT's June 23,
2005, cost-benefit analysis for Increment 1B satisfies eight OMB criteria.
Table 1: US-VISIT Satisfaction of OMB Economic Analysis Criteria
Criterion Explanation Criterion GAO analysis
met?
1. The The analysis should Yes The analysis identifies
cost-benefit clearly explain the the need for the
analysis clearly reason why the investment and identifies
explained why the investment is eight key business
investment was needed, that is, why objectives of the
needed. the status quo is Increment 1B exit
unacceptable. solution.
2. At least two At least two Yes The analysis considers
alternatives to meaningful three alternatives for
the status quo alternatives to the the Increment 1B exit
were considered. status quo should be solution: kiosk, mobile,
examined to help and validator.
ensure that the
alternative chosen
was not preselected.
3. The general The general Yes The assessment includes
rationale for the rationale for the the rationale for the
cost-benefit inclusion of each judgment that the three
analysis, alternative exit alternatives were
including each considered should be viable options.
alternative, was discussed to enable
discussed. reviewers of the
analysis to gain an
understanding of the
context for the
selection of one
alternative over the
others.
4. The quality of The quality of the No The cost estimates are
the cost estimate cost estimate for not complete or reliably
for each each alternative derived. (See later
alternative was should be complete section of this report
reasonable. and reasonable for a for detailed analysis.)
net present value to
be accurate.
5. The quality of The quality of the No Year-by-year benefit
the benefits to benefit estimate for estimates were not
be realized from each alternative reported.
each alternative should be complete
was reasonable. and reasonable for a
net present value to
be calculable and
accurate. According
to OMB Circular
A-94,a year-by-year
estimates should be
reported to promote
independent analysis
and review of those
estimates.
6. Alternatives The net present Yes Net present values were
were compared on value should be calculated for the three
the basis of net calculated because alternatives. However,
present value. it consistently the preferred alternative
allows for the could not be selected on
selection of the this basis, in part
alternative with the because the estimated net
greatest benefit net present value for all
of cost. alternatives was
negative. OMB guidance
presumes that at least
one will be positive, and
that the selected
alternative will have the
greatest total benefit
net of total cost. The
alternative with the more
favorable cost-benefit
was identified on the
basis of its lower labor
intensity (resulting in
lower operating and
maintenance costs) and
lower risk that
personally identifiable
information would be
compromised.
7. The proper OMB Circular A-94 No The analysis does not
discount rate for provides specific explicitly state the
calculating each guidance on the numerical value of the
alternative's net choice of discount discount rate used for
present value rate for evaluating computing the
should be used. projects whose alternatives' net present
benefits and costs values.
will be distributed
over time.
8. A complete Estimates of costs No Although the cost-benefit
uncertainty and benefits are analysis did include
analysis of cost typically uncertain Monte Carlo simulation
and benefit was because of results for the three
included. imprecision in both exit alternatives, no
underlying data and sensitivity analysis was
modeling conducted for those
assumptions. Because alternatives. Instead,
such uncertainty is the cost-benefit analysis
basic to virtually reports sensitivity
any cost-benefit analysis results for the
analysis, its five deployment
effects should be scenarios.
analyzed and
reported. OMB
guidance recommends
both Monte Carlo
simulation and
sensitivity analysis
as uncertainty
analysis techniques.
Source: GAO.
aOMB's Circular A-94 is the general guidance for conducting cost-benefit
analyses for the federal government.
It is important that the program adhere to relevant guidance in developing
its incremental cost-benefit analyses. If this is not done, the
reliability of the analyses is diminished, and an adequate basis for
prudent investment decision making does not exist. Moreover, if the
mission value of a proposed investment is not commensurate with costs, it
is vital that this information be fully disclosed to DHS and congressional
decision makers. The underlying intent of our recommendation is that this
information be available to inform such decisions.
Definition of the Operational Context for US-VISIT Is in Progress
In September 2003, we reported that key aspects of the larger homeland
security environment in which US-VISIT would need to operate had not been
defined. For example, we stated that certain policy and standards
decisions had not been made (e.g., whether official travel documents will
be required for all persons who enter and exit the country, including U.S.
and Canadian citizens, and how many fingerprints are to be collected). In
the absence of this operational context, program officials were making
assumptions and decisions that, if they proved inconsistent with
subsequent policy or standards decisions, would require US-VISIT rework.
To minimize the impact of these changes, we recommended that DHS do the
following:
Clarify the operational context in which US-VISIT is to operate.
After about 27 months, defining this operational context remains a work in
progress. According to the Chief Strategist, an immigration and border
management strategic plan was drafted in March 2005 that shows how
US-VISIT is aligned with DHS's organizational mission and defines an
overall vision for immigration and border management. This official stated
that this vision provides for an immigration and border management
enterprise that unifies multiple internal departmental and other external
stakeholders with common objectives, strategies, processes, and
infrastructures.
Since the plan was drafted, DHS has reported that other relevant
initiatives have been undertaken, such as the Security and Prosperity
Partnership of North America and the Secure Border Initiative. The
Security and Prosperity Partnership is to, among other things, establish a
common approach to securing the countries of North America- the United
States, Canada, and Mexico-by, for example, implementing a border
facilitation strategy to build capacity and improve the legitimate flow of
people and cargo at our shared borders. The Secure Border Initiative is to
implement a comprehensive approach to securing our borders and reducing
illegal immigration. According to the Chief Strategist, while portions of
the strategic plan are being incorporated into these initiatives, these
initiatives and their relationship with US-VISIT are still being defined.
We have yet to receive the US-VISIT strategic plan because, according to
program officials, it had not yet been approved by DHS management.
Until US-VISIT's operational context is fully defined, DHS is increasing
its risk of defining, establishing, and implementing a program that is
duplicative of other programs and not interoperable with them. This in
turn will require rework to address these areas. While this issue was
significant 27 months ago, when we made the recommendation, it is still
more significant now.
Provision of Program Office Resources Is Partially Complete
We reported in September 2003 that the program had not fully staffed its
program office. Our prior experience with major acquisitions like US-VISIT
shows that to be successful, they need, among other things, to have
adequate resources. Accordingly, we recommended that DHS do the following:
Ensure that human capital and financial resources are provided to
establish a fully functional and effective program office.
About 2 years later, US-VISIT had filled 102 of its 115 planned government
positions and all of its planned 117 contractor positions. For the
remaining 13 government positions, 5 positions had been selected (pending
completion of security clearances), and recruitment action was in process
for filling the remaining 8 vacancies. According to the Office of
Administration and Training Manager, funding is available to complete the
hiring of all 115 government employees.
Notwithstanding this progress, in February 2005, US-VISIT completed a
workforce analysis and requested additional positions based on the
results. According to program officials, a revised analysis was submitted
in the summer of 2005, but the request has not yet been approved. Figure 3
shows the program office organization structure and functions and how many
of the 115 positions needed have been filled.
Figure 3: Summary of Program Office Structure, Functions, and Filled and
Vacant Positions
Securing necessary resources will be a continuing challenge and an
essential ingredient to the program's ability to acquire, deploy, operate,
and maintain system capabilities on time and within budget.
Definition of Program Office Roles and Responsibilities Has Been Completed
We reported in September 2003 that the program had not defined specific
roles and responsibilities for its staff. Our prior experience and leading
practices show that for major acquisitions like US-VISIT to be successful,
program staff need, among other things, to understand what they are to do,
how they relate to each other, and how they fit in their organization.
Accordingly, we recommended that DHS do the following:
Define program office positions, roles, and responsibilities.
The program office has developed charters for its nine component offices
that include roles and responsibilities for each. For example, the
Acquisition and Program Management Office is responsible, among other
things, for establishing acquisition and program management policies;
coordinating development of configuration management plans and project
schedules, including the integrated milestone schedule; and developing
policies and procedures for guidance and oversight of systems development
and implementation activities. The program has also defined a set of core
competencies (knowledge, skills, and abilities) for each position. For
example, it has defined critical competencies for program and management
analysts that include, among others, flexibility, interpersonal skills,
organizational awareness, oral communication, problem solving, and
teamwork.
These efforts to define position, roles, and responsibilities should help
in managing the program effectively.
Development and Implementation of a Human Capital Strategy Are Partially
Complete
As previously stated, we reported in September 2003 that US-VISIT had not
fully staffed its program office or defined roles and responsibilities for
its program staff. We observed that prior research and evaluations of
organizations showed that effective human capital management can help
agencies establish and maintain the workforce they need to accomplish
their missions. Accordingly, we recommended that DHS do the following:
Develop and implement a human capital strategy for the program office that
provides for staffing positions with individuals who have the appropriate
knowledge, skills, and abilities.
In February 2005, we reported that the program office, in conjunction with
the Office of Personnel Management (OPM), developed a draft human capital
plan that employed widely accepted human capital planning tools and
principles. The draft plan included, for example, an action plan that
identified activities, proposed completion dates, and the office (OPM or
the program office) responsible for the action. We also reported that the
program office had completed some of the activities, such as designating a
liaison responsible for ensuring alignment between departmental and
program human capital policies.
Since then, the program office has finalized the human capital plan and
completed more activities. For example, program officials told us that
they have
o analyzed the program office's workforce to determine diversity trends,
retirement and attrition rates, and mission-critical and leadership
competency gaps;
o updated the program's core competency requirements to ensure alignment
between the program's human capital and business needs;
o developed an orientation program for new employees; and
o administered competency assessments to incoming employees.
Program officials also told us that they have plans to complete other
activities, such as
o developing a staffing forecast to inform succession planning;
o analyzing workforce data to maintain strategic focus on preserving the
skills, knowledge, and leadership abilities required for the US-VISIT
program's success; and
o developing organizational leadership competency models for the program's
senior executive, managerial, and supervisory levels.
In addition, the officials said that several activities in the plan have
not been completed, such as assessing the extent of any current employees'
competency gaps and developing a competency-based listing of training
courses. These officials said that the reason these activities have not
been completed is that they are related to the department's new human
capital initiative, MAXHR, which is to provide greater flexibility and
accountability in the way employees are paid, developed, evaluated,
afforded due process, and represented by labor organizations. MAXHR is to
include the development of departmentwide competencies. Because of this,
the officials told us that it could potentially impact the program's
ongoing competency-related activities. As a result, these officials said
that they are coordinating these activities closely with the department
as it develops and implements this new initiative, which is currently
being reviewed by the DHS Deputy Secretary for approval.
Until US-VISIT fully implements a comprehensive human capital strategy, it
will continue to risk not having staff with the right skills and abilities
to successfully execute the program.
Defining Performance Standards for US-VISIT Increments Is Partially
Complete
We reported in September 2003 that the operational performance of initial
system increments was largely dependent on the performance of existing
systems that were to be interfaced to create these increments. For
example, we said that the performance of an increment will be constrained
by the availability and downtime of the existing systems that it includes.
Accordingly, we recommended that DHS do the following:
Define performance standards for each increment that are measurable and
reflect the limitations imposed by relying on existing systems.
In February 2005 (17 months later), we reported that several technical
performance standards for Increments 1 and 2B had been defined, but that
it was not clear that these standards reflected the limitations imposed by
the reliance on existing systems. Since then, for the Increment 2C Proof
of Concept (Phase 1), the program office has defined certain other
performance standards. For example, the functional requirements document
for Increment 2C (Phase 1) defines several technical performance
standards, including reliability, recoverability, and availability. For
each, the document states that the performance standard is largely
dependent on those of Increment 2B. More specifically, the document states
that Phase 1 system availability is largely dependent upon the individual
and collective availability of the current systems. The document also
states that the Increment 2C components shall have an aggregated
availability greater than or equal to 97.5 percent. However, the document
does not contain sufficient information to determine whether these
performance standards actually reflect the limitations imposed by reliance
on existing systems.
To further develop performance standards, the program office has prepared
a Performance Engineering Plan, dated March 31, 2005, that links US-VISIT
performance engineering activities to its System Development Life Cycle.
Further, the plan (1) provides a framework to be used to align its
business, application, and infrastructure performance goals and measures;
(2) describes an approach to translate business goals into operational
measures, and then to quantitative metrics; and (3) identifies system
performance measurement areas (effectiveness, efficiency, reliability, and
availability). According to program officials, they intend to establish a
group to develop action plans for implementing the engineering plan, but
did not have a time frame for doing so.
Without defining performance standards that reflect the limitations of the
existing systems upon which US-VISIT relies, the program lacks the ability
to identify and effectively address performance shortfalls.
Development and Implementation of a Risk Management Plan Are Partially
Complete
In September 2003, we reported that US-VISIT was a risky undertaking
because of several factors inherent to the program, such as its large
scope and complexity, as well as because of various program management
weaknesses. We concluded that these risks, if not effectively managed,
would likely cause program cost, schedule, and performance problems.
Risk management is a continuous, forward-looking process that is intended
either to prevent such problems from occurring or to minimize their impact
if they occur by proactively identifying risks, implementing risk
mitigation strategies, and measuring and disclosing progress in doing so.
Because of the importance of effectively managing program risks, we
recommended that DHS do the following:
Develop and implement a risk management plan and ensure that all high
risks and their status are reported regularly to the executive body.
About 2 years later, the program office has developed and has begun
implementing a risk management plan. The plan, which was approved in
September 2005, includes, among other things, a process for identifying,
analyzing, handling, and monitoring risk. It also defines the governance
structure to be used in overseeing and managing the process. The program
also maintains a risk database, which includes, among other things, a
description of the risk, its priority (e.g., high, medium, or low), and
its mitigation strategy. According to program officials, the database is
currently available to program management and staff.
The program has also begun implementing its risk management plan. For
example, it has established a Risk Review Board, Risk Review Council, and
Risk Owners to govern its risk activities. The roles and responsibilities
are described below.
o The Risk Review Board directs all risk governance within the program and
provides the mechanism to escalate/transfer the consideration of risks to
program governing boards and to organizations external to the program.
o The Risk Review Council oversees and manages program-related risks that
are significant, controversial, or cross-project or that may require
escalation to the Risk Review Board.
o Risk Owners analyze, handle, and monitor risks.
However, full implementation of the risk management plan has yet to occur.
As part of its CMMI process maturity baseline self-assessment (previously
discussed), the program office found that the risk management process
detailed in its plan was not being consistently applied across the
program. In response, according to program officials, they have developed
risk management training and began conducting training sessions in
November 2005. These officials also stated that the Risk Review Board,
where risks are reviewed with program executives, has been meeting monthly
since September 2005.
With respect to regular risk reports to program executives, the plan
includes thresholds for escalating risks within the risk governance
structure and to DHS governance entities. For example, risks are to be
elevated to the Risk Review Board when the cost of the project exceeds
more than 5 percent of the project baseline cost, the schedule slippage
exceeds more than 5 percent of the baseline schedule, major areas of scope
are affected, or quality reduction requires approval. However, program
officials stated that these thresholds are not currently being applied.
They further stated that although the plan allows for escalation of risks
to officials outside the program office, doing so is at the discretion of
the Program Director; in addition, according to these officials, although
high risks are not routinely escalated outside the program, selected high
risks have been disclosed to the Assistant Secretary for Policy in weekly
program status reports. As of December 5, 2005, the Program Director
proposed submitting monthly reports of high-priority risks and issues
through the Assistant Secretary for Policy to the Deputy Secretary.
Until US-VISIT fully implements its risk management plan and process, it
cannot be assured that all program risks are being identified and managed
in order to effectively mitigate any negative impact on the program's
ability to deliver promised capabilities on time and within budget.
Development of Test Plans Is Partially Complete
We reported in May 2004, and again in February 2005, that system testing
was not based on well-defined test plans, and thus the quality of testing
being performed was at risk.28 The purpose of system testing is to
identify and correct system defects (i.e., unmet system functional,
performance, and interface requirements) and thereby obtain reasonable
assurance that the system performs as specified before it is deployed and
operationally used. To be effective, testing activities should be planned
and implemented in a structured and disciplined fashion. Among other
things, this includes developing effective test plans to guide the testing
activities and ensuring that test plans are developed and approved before
test execution. According to relevant systems development guidance, an
effective test plan (1) specifies the test environment; (2) describes each
test to be performed, including test controls, inputs, and expected
outputs; (3) defines the test procedures to be followed in conducting the
tests; and (4) provides traceability between the test cases and the
requirements to be verified by the testing. Because these criteria were
not being met, we recommended that DHS do the following:
Develop and approve test plans before testing begins that (1) specify the
test environment; (2) describe each test to be performed, including test
controls, inputs, and expected outputs; (3) define the test procedures to
be followed in conducting the tests; and (4) provide traceability between
test cases and the requirements to be verified by the testing.
About 19 months later, the quality of the system test plans, and thus
system testing, is still problematic. To the program's credit, the test
plans for the Increment 2C Proof of Concept (Phase 1), dated June 28,
2005, satisfied part of our recommendation. Specifically, the test plan
for this increment was approved on June 30, 2005, and, according to
program officials, testing began on July 5, 2005. Further, the test plan
described, for example, the scope, complexity, and completeness of the
test environment, and it described the tests to be performed, including a
high-level description of controls, inputs, and outputs, and it identified
test procedures to be performed.
However, the test plan did not adequately trace between test cases and the
requirements to be verified by testing. For example, 300 of the 438
functional requirements, or about 70 percent of the requirements that we
analyzed, did not have specific references to test cases.
In addition, we identified traceability inconsistencies, including the
following:
o One requirement was mapped to over 50 test cases, but none of the 50
cases referenced the requirement.
o One requirement was mapped to a group of test cases in the traceability
matrix, but several of the test cases to which the requirement was mapped
did not reference the requirement, and several test cases referenced the
requirement and were not included in the traceability matrix.
o One requirement was mapped to all but one of the test cases within a
particular group of test cases, but that test case did refer to the
requirement.
Time and resources were identified as the reasons that test plans have not
been complete. Specifically, program officials stated that milestones do
not permit existing testing/quality personnel the time required to
adequately review testing documents.29 According to these officials, even
when the start of testing activities is delayed because, for example,
requirements definition or product development takes longer than
anticipated, testing milestones are not extended.
Without complete test plans, the program does not have adequate assurance
that the system is being fully tested, and thus unnecessarily assumes the
risk that system defects will not be detected and addressed before the
system is deployed. This means that the system may not perform as intended
when deployed, and defects will not be addressed until late in the systems
development cycle, when they are more difficult and time-consuming to fix.
As we previously reported, this has happened: postdeployment system
interface problems surfaced for Increment 1, and manual work-arounds had
to be implemented after the system was deployed.
Assessment of the Impact of Increment 2B on Workforce Levels and
Facilities Is Partially Complete
We reported in May 2004 that the program had not assessed its workforce
and facility needs for Increment 2B. Because of this, we questioned the
validity of the program's workforce and facility assumptions used to
develop its workforce and facility plans, noting that the program lacked a
basis for determining whether its assumptions and thus its plans were
adequate. Accordingly, we recommended that DHS do the following:
Assess the full impact of Increment 2B on land POE workforce levels and
facilities, including performing appropriate modeling exercises.
Seven months later, the program office evaluated Increment 2B operational
performance. The purpose of the evaluation was to determine the
effectiveness of Increment 2B performance at the 50 busiest land POEs. To
assist in the evaluation, the program office established a baseline for
comparing the average Form I-94 or Form I-94W30 issuance processing times
at 3 of the 50 POEs where processing times were to be evaluated.31 The
program office then conducted two evaluations of the processing times at
the 3 POEs following Increment 2B deployment. The first was in December
2004, after Increment 2B was deployed to these sites as a pilot, and the
second was in February 2005, after Increment 2B was deployed to all 50
POEs. The evaluation results showed that the average processing times
decreased for all 3 sites. Table 2 compares the results of the two
evaluations and the baseline.
Table 2: Reduction in Reported Processing Times for Increment 2B Pilot and
Full Deployment
Pilot site Baseline Pilot: Decrease in Full deployment: Change in
(October 2004) time from baseline time from pilot (February
(December 2004) 2005)
Douglas, 4 minutes, 16 -47 seconds -17 seconds
Arizona seconds
Laredo, Texas 12 minutes, 10 -9 minutes, 37 -15 seconds
seconds seconds
Port Huron, 11 minutes, 42 -1 minutes, 51 +7 seconds
Michigan seconds seconds
Source: GAO analysis of DHS data.
According to program officials, these evaluations supported the workforce
and facilities planning assumption that no additional staff were required
to support deployment of Increment 2B, and that minimal modifications to
interior workspace were required to accommodate biometric capture devices
and printers and to install electrical circuits. These officials stated
that modifications to existing officer training and interior space were
the only changes needed.
However, the scope of the evaluation was too limited to satisfy the
evaluation's stated purpose or our recommendation for assessing the full
impact of Increment 2B. Specifically, program officials stated that the
evaluation focused on the time to process Form I-94s and not on
operational effectiveness, including workforce impacts and traveler
waiting time. Second, the 3 sites were selected, according to program
officials, on the basis of a number of factors, including whether the
sites already had sufficient staff to support the pilot. Selecting sites
on the basis of this factor could affect the results and presupposes that
not all POEs have the staff needed to support Increment 2B. Third,
evaluation conditions were not always held constant. For example, fewer
workstations were used to process travelers in establishing the baseline
processing times at 2 of the POEs-Port Huron (9 versus 14) and Douglas (4
versus 6)-than were used during the pilot evaluations.
Moreover, CBP officials from 1 POE, which was not an evaluation site, told
us that US-VISIT has actually lengthened processing times. (San Ysidro
processes the highest volume of travelers of all land POEs.) While these
officials did not provide specific data to support this statement, it
nevertheless raises questions about the potential impact of Increment 2B
on the 47 sites that were not evaluated.
It is important that the impact of Increment 2B on workforce and
facilities be fully assessed. Since we made our recommendation, Increment
2B deployment and operational facts and circumstances have materially
changed, making the implementation of our recommendation using
predeployment baseline data for the other 47 sites impractical.
Nevertheless, other alternatives, such as surveying officials at these
sites to better understand the increment's impact on workforce levels and
facilities, have yet to be explored. Until they are, the program may not
be able to accurately project resource needs or make required
modifications to achieve its goals of minimizing US-VISIT's impact on POE
processing times.
Implementation of Configuration Management Practices Is in Progress
We reported in May 2004 that US-VISIT had not established effective
configuration management practices. Configuration management establishes
and maintains the integrity of system components and items (e.g.,
hardware, software, and documentation). A key ingredient is a change
control board to evaluate and approve proposed configuration changes.
Accordingly, we concluded that the program did not have adequate assurance
that approved system changes were actually made, and that changes made to
the component systems (for non-US-VISIT purposes) did not interfere with
US-VISIT functionality. Accordingly, we recommended that DHS do the
following:
Implement effective configuration management practices, including
establishing a US-VISIT change control board to manage and oversee system
changes.
After 19 months, US-VISIT has begun implementing configuration management
practices. To its credit, the program recently issued a configuration
management policy (September 2005) and prepared a draft configuration
management plan (August 2005). The policy contains guiding principles,
direction, and expectations for planning and performing configuration
management, and includes activities, authorities, and responsibilities.
The draft plan describes the configuration management
governance structure, including organizational entities and their
responsibilities, the processes and procedures to be applied, and how
controls are to be applied to products. The governance structure includes
the Executive Configuration Control Board and the Configuration Management
Impact Review Team. According to its charter, the configuration control
board is responsible for determining the status of requested configuration
changes and resolving any conflicts related to those changes for
US-VISIT-managed systems (i.e., not for US-VISIT component systems managed
by other DHS organizations). The Impact Review Team, which reports to the
board, is responsible for reviewing requests for system changes and
submitting a recommendation to the appropriate change review authority
(i.e., either the US-VISIT control board or the control board in the DHS
organization that manages the component system). According to program
officials, for US-VISIT-managed systems, the review authority is the
Executive Configuration Control Board. For other systems, such as TECS
(which CBP manages), the US-VISIT review team may submit a recommendation
to the appropriate control board (in this case, the CBP Control Board).
The APMO director stated that the planned configuration management program
is intended to complement rather than replace the configuration management
programs for the legacy systems. That is, change requests approved by the
US-VISIT Executive Configuration Control Board that require changes to a
legacy system will be coordinated with the board having responsibility for
that system. This means, however, that changes to component systems (e.g.,
IDENT, ADIS, and TECS) that are initiated and approved by another DHS
organization, and that could affect US-VISIT performance, are not subject
to US-VISIT configuration management processes and are not also being
examined and approved by the US-VISIT control board. This lack of US-VISIT
control was the impetus for our recommendation.
Although US-VISIT has recently taken steps to begin addressing our
recommendation, the program still does not adequately control changes to
the component systems upon which US-VISIT performance depends. Until
programwide configuration management practices are implemented, the
program does not have an effective means for ensuring that approved system
changes are actually made and that changes made to the component systems
for non-US-VISIT purposes do not compromise US-VISIT functionality and
performance.
Efforts to Ensure the Independence of the Verification and Validation
Contractor Are Complete
We reported in May 2004 that the program office's independent verification
and validation (IV&V) contractor was not independent of the products and
processes that it was verifying and validating. The purpose of IV&V is to
provide management with objective insight into the program's processes and
associated work products. Its use is a recognized best practice for large
and complex system development and acquisition projects like US-VISIT. To
be effective, the verification and validation function is to be performed
by an entity that is independent of the processes and products that are
being reviewed. Accordingly, we recommended that DHS do the following:
Ensure the independence of the IV&V contractor.
In July 2005, the program office issued a new contract for IV&V services.
To ensure the contactor's independence, the program office (1) required
that IV&V contract bidders be independent of the development and
integration contractors; (2) reviewed each of the bidder's affiliations
with the prime contract; (3) included provisions in the contract that
prohibit the contractor from soliciting, proposing, or being awarded work
(other than IV&V services) for the program; (4) required all contractor
personnel to certify that they do not have any conflicts of interest; and
(5) ensured that the contractor's management plan (Oct. 17, 2005)
describes how the contractor will ensure technical, managerial, and
financial independence.
Such steps, if effectively enforced, should adequately ensure that
verification and validation activities are performed in an objective
manner and, thus, should provide valuable assistance to program managers
and decision makers.
Development of a Plan to Address Open Recommendations Is Partially
Complete
We reported in May 2004 that US-VISIT's overall progress on implementing
our recommendations had been slow, and considerable work remained to fully
address them. As we also noted, given that most of our recommendations
focused on fundamental limitations in US-VISIT's ability to manage the
program, it was important to implement the recommendations quickly and
completely. Accordingly, we recommended that DHS do the following:
Develop a plan, including explicit tasks and milestones, for implementing
all of our open recommendations and periodically report to the DHS
Secretary and Under Secretary on progress in implementing this plan; and
report this progress, including reasons for delays, in all future
expenditure plans.
About 19 months after our recommendation, the program assigned
responsibility to specific individuals for preparing a plan, including
specific actions and milestones, to address each recommendation. In
addition, it developed a report that identifies the responsible person for
each recommendation and summarizes progress made in implementing each. The
program office provided this report for the first time to the DHS Deputy
Secretary on October 3, 2005, and plans to forward subsequent reports
every 6 months.
However, the report's description of progress on 4 recommendations is
inconsistent with our assessment, as discussed below:
o First, the report states that the program completed a privacy impact
assessment that is in full compliance with OMB guidance. As previously
discussed, an assessment has been developed, but OMB guidance requires
that these assessments for systems under development (such as Increment
2C) address privacy in the system's documentation. Increment 2C systems
documentation does not address privacy and therefore is not fully
compliant with OMB guidance.
o Second, the report states that a human capital strategy has been
completed. However, as previously discussed, several of the activities in
the human capital plan have yet to be implemented. For example, the
program has not developed a staffing forecast to inform succession
planning.
o Third, the report states that the impact of Increment 2B on land POE
workforce levels and facilities has been fully assessed. However, as we
previously stated, the scope of the evaluations was not sufficient to
satisfy our recommendation. For example, program officials stated that the
evaluation focused on the time to process Form I-94s and not on
operational effectiveness, including workforce impacts and traveler
waiting time. Moreover, officials at the largest land POE told us that the
effect of Increment 2B was the opposite of that reported in the pilot
results.
o Fourth, the report states that the program has partially completed
implementing configuration management practices. However, as previously
discussed, the program office has yet to implement practices or establish
a configuration control board with authority over all changes affecting
US-VISIT functionality and performance, including those made to component
systems for non-US-VISIT purposes, which was the intent of our
recommendation.
In addition, the report does not specifically describe progress against 11
of our other recommendations, so that we could not determine whether the
program's assessment is consistent with ours (described in this report).
For example, we recommended that the program reassess plans for deploying
an exit capability to ensure that the scope of the exit pilot provides for
adequate evaluation of alternative solutions. The report states that the
program office has completed exit testing and has forwarded the exit
evaluation report to the Deputy Secretary for a decision. However, it does
not state whether the program office had expanded the scope or time frames
of the pilot.
Fully understanding and disclosing progress against our recommendations
are essential to building the capability needed to effectively manage the
program, and to ensuring that key decision makers have the information
needed to make well-informed choices among competing investment options.
Establishment of Effective Cost-Estimating Practices Is in Progress
We reported in February 2005 that US-VISIT had not followed effective
practices to develop cost estimates for its system increments, and thus
the reliability of its cost estimates was questionable.32 Such
cost-estimating practices are embedded in the 13 criteria in SEI's
checklist for determining the reliability of cost estimates.33 Of these 13
criteria, we reported in February 2005 that the program's cost estimate
met 2, partially met 6, and did not meet 5. Accordingly, we recommended
that DHS do the following:
Follow effective practices for estimating the costs of future increments.
The latest US-VISIT-related cost estimate is for Increment 1B. This
estimate is in the June 2005 cost-benefit analysis for Increment 1B and
establishes the costs associated with three exit solutions for air and sea
POEs. As was the case for the estimate described in our February 2005
report, this latest estimate also did not meet all 13 criteria, meeting 3
and partially meeting another 5.34 For example, these estimates did not
include a detailed work breakdown structure and omitted important cost
elements, such as system testing. A work breakdown structure serves to
organize and define the work to be performed, so that associated costs can
be identified and estimated. Thus, it provides a reliable basis for
ensuring that the estimates include all relevant costs. In addition, the
uncertainties associated with the Increment 1B cost estimate were not
identified. An uncertainty analysis provides the basis for adjusting these
estimates to reflect unknown facts and circumstances that could affect
costs and identifies the risk associated with the cost estimate. Table 3
summarizes our analysis of the extent to which US-VISIT's Increment 1B
cost estimates satisfy SEI's 13 criteria.
Table 3: Satisfaction of SEI's 13 Cost-Estimating Criteria
Criterion Explanation Criterion GAO analysis
met?a
1. The objectives of The objectives of the Yes The objectives of the
the program are program should be program were clearly
stated in writing. clearly and concisely stated. Specifically,
stated for the cost the objectives are to
estimator to use. provide a more
complete traveler
history and to
capture travelers'
biometric and
biographic data.
2. The life cycle to The life cycle should Partially The life cycle was
which the estimate be clearly defined to not clearly defined
applies is clearly ensure that the full to ensure that the
defined. cost of the program full cost of the
is captured-that is, program was included.
all direct and For example, the
indirect costs for analysis did not
planning, include evidence that
procurement, software maintenance
operations and costs were included
maintenance, and in the cost estimate.
disposal.
3. The task has been An appropriate sizing No The program office
appropriately sized. metric should be used provided no evidence
in the development of to demonstrate that
the estimate, such as an appropriate sizing
the amount of mechanism was used,
software to be and program officials
developed and the stated that they had
amount of software to not collected these
be revised. data.
4. The estimated Estimates should be Partially Officials stated that
cost and schedule validated by being pilot data were used
are consistent with related back to to develop the
demonstrated demonstrated and estimate. They stated
accomplishments on documented they extrapolated
other projects. performance on pilot data to
completed projects. estimate costs for
all Increment 1B
sites; however, they
further stated that
there were no
previous projects
with which to compare
the results to see if
they were consistent.
5. A written summary If a parametric Partially High-level cost
of parameter values equation was used to categories, such as
and their rationales generate the labor, information
accompanies the estimate, the technology,
estimate. parameters that feed facilities, and other
the equation should costs, were
be provided, along identified, but
with an explanation detailed parameters
of why they were used to develop the
chosen. estimate, such as
number of software
lines of code, which
would be relevant to
software maintenance
costs, were not
provided in the
analysis.
6. Assumptions have Assumptions regarding Yes General cost
been identified and issues such as assumptions are
explained. schedule, quantity, identified and
technology, explained, as well as
development assumptions for
processes, workforce,
manufacturing information
techniques, software technology, training,
language, etc., and facilities.
should be understood
and documented.
7. A structured A work breakdown Partially The analysis included
process, such as a structure or similar four high-level cost
template or format, structure that categories (labor,
has been used to organizes, defines, facilities,
ensure that key and graphically operations and
factors have not displays the maintenance, and
been overlooked. individual work units information
to be performed technology), but it
should be used. The did not include a
structure should be detailed work
revised over time as breakdown structure
more information and omitted important
becomes known about cost elements, such
the work to be as system testing.
performed.
8. Uncertainties in For all major cost Partially A risk analysis was
parameter values drivers, an performed, but this
have been identified uncertainty analysis analysis did not
and quantified. should be performed identify detailed
to recognize and parameter values.
reflect the risk
associated with the
cost estimate.
9. If a dictated Managers should be N/A Program officials
schedule has been informed of all stated that the
imposed, an estimate potential cost Increment 1B schedule
of the normal savings associated was not dictated.
schedule has been with alternative
compared to the schedules.
additional
expenditures
required to meet the
dictated schedule.
10. If more than one The primary No No evidence of a
cost model or methodology or cost secondary cost model
estimating approach model results should was included in the
has been used, any be compared with any analysis, and program
differences in secondary methodology officials stated that
results have been (e.g., cross checks) they did not use a
analyzed and to ensure second model.
explained. consistency.
11. Estimators The purpose of an No Program officials
independent of the independent estimate stated that the
performing is to determine the estimate was not
organization reasonableness of the independently
concurred with the parameter values reviewed.
reasonableness of based on an unbiased
the parameter values perspective. This
and estimating approach usually
methodology. results in a more
accurate estimate
because it allows for
better insight into
program risks.
12. Estimates are Estimates are updated Yes Estimates reflected
current. whenever changes to current conditions.
requirements affect
cost or schedule,
constraints, and
resources, or when
priorities change.
13. The results of Plans are reviewed No Program officials
the estimate have and updated whenever stated that the
been integrated with estimates change, and results of the
project planning and estimates used for estimate have not
tracking. project planning are been incorporated
also used as with project
baselines for project planning.
tracking.
Source: GAO.
aWe assessed each of the criteria as satisfied (US-VISIT provided
substantiating evidence for the criterion), partially satisfied (US-VISIT
provided partial evidence, including testimonial evidence, for the
criterion), or not satisfied (no evidence was found for the criterion).
Program officials stated that they recognize the importance of developing
reliable cost estimates and have initiated actions to more reliably
estimate the costs of future increments. For example, as part of its
process improvement program, the program has chartered a cost-analysis
process action team, which is to develop, document, and implement a
cost-analysis policy, process, and plan for the program. Program officials
also stated that they have hired additional contracting staff with
cost-estimating experience.
Strengthening the program's cost-estimating capability is extremely
important. The absence of reliable cost estimates, among other things,
prevents the development of reliable economic justification for program
decisions and impedes effective performance measurement.
Reassessment of Plans for Deploying the Exit Capability Is Partially
Complete
In February 2005, we reported that US-VISIT had not adequately planned for
evaluating the Increment 1B exit alternative because its exit pilot
evaluation's scope and timeline were compressed. Accordingly, we
recommended that DHS do the following:
Reassess plans for deploying an exit capability to ensure that the scope
of the exit pilot provides for adequate evaluation of alternative
solutions and better ensures that the exit solution selected is in the
best interest of the program.
Over the last 10 months, the program office has taken actions to expand
the scope and time frames of the pilot. For example, it extended the pilot
from 5 to 11 POEs-9 airports and 2 seaports.35 It also extended the time
frame for data collection and evaluation to April 2005, which is about 7
months beyond the date for which all exit pilot evaluation tasks were to
be completed. Further, according to program officials, they achieved the
target sample sizes necessary to have a 95 percent confidence level.
Notwithstanding the expanded scope of the pilot, questions remain about
whether the exit alternatives have been evaluated sufficiently to permit
selection of the best exit solution for national deployment. For example,
each of the three exit alternatives was evaluated against three criteria,
including compliance with the US-VISIT exit process (i.e., foreign
travelers providing information as they exit the United States).36
However, across the three alternatives, the average compliance with this
process was only 24 percent, which raises questions as to the
effectiveness of the three alternatives.37 The evaluation report cites
several reasons for the low compliance rate, including that compliance
during the pilot was voluntary. The report further concludes that national
deployment of the exit solution will not have the desired compliance rate
unless the exit process incorporates an enforcement mechanism, such as not
allowing persons to reenter the United States if they do not comply with
the exit process. Although an enforcement mechanism might indeed improve
compliance, program officials stated that no formal evaluation has been
conducted of enforcement mechanisms or their effect on compliance. The
program director stated that he agrees that additional evaluation is
needed to assess the impact of implementing potential enforcement
mechanisms and plans to do so.
Until the program office adequately evaluates the exit alternatives and
knows whether the alternative to be selected will be effective, the
program office will not be in a position to select the exit solution that
is in the best interest of the program. This is very important because
without an effective exit capability, the benefits and the mission value
of US-VISIT are greatly diminished.
Development and Implementation of Capacity Management Processes Are in
Progress
We reported in February 2005 that the overall capacity of the system was
not being effectively managed. At that time, US-VISIT, which comprises
several legacy systems, was relying on the capacity management activities
of these systems. It was not focused on the capacity requirements and
performance of the collective systems that make up US-VISIT. This approach
increases the risk that the system may not be properly designed and
configured for efficient performance, and that it has insufficient
processing and storage capacity for current, future, and unpredictable
workload requirements. Accordingly, we recommended that DHS do the
following:
Develop and implement processes for managing the capacity of the US-VISIT
system.
According to program officials, they have initiated efforts to develop a
capacity management process, including a high-level description of the
necessary steps, such as identifying tools needed to implement the
process. However, a plan, including specific tasks and milestones for
developing and implementing capacity management processes, has not yet
been developed.
Until the program office develops a programwide capacity management
program, it increases the risk that US-VISIT may not be able to adequately
support program mission needs.
Identification of ACE and US-VISIT Relationships and Dependencies Is in
Progress
We reported in February 2005 that the program office recognized that
US-VISIT and the Automated Commercial Environment (ACE)38 have related
missions and operational environments. In addition, US-VISIT and ACE could
potentially develop, deploy, and use common information technology
infrastructures and services. We also reported that managing this
relationship has not been a priority. Accordingly, we recommended that DHS
do the following:
Make understanding the relationships and dependencies between the US-VISIT
and ACE programs a priority matter, and report periodically to the Under
Secretary on progress in doing so.
US-VISIT and ACE managers met in February 2004, to identify potential
areas for collaboration between the two programs and to clarify how the
programs could best support the DHS mission and provide officers with the
information and tools they need. According to program officials, they have
established a US-VISIT/ACE integrated project team to, among other things,
ensure that the two programs are programmatically and technically aligned.
The team has discussed potential areas of focus and agreed to three areas:
RF technology, program control, and data governance. However, it does not
have an approved charter, and it has not developed explicit plans or
milestone dates for identifying the dependencies and relationships between
the two programs. Program officials stated that the team has met three
times and plans to meet on a quarterly basis going forward.
It is important that the relationships and dependencies between these two
programs be managed effectively. The longer it takes for the programs to
understand and exploit their relationships, the more rework will be needed
at a later date to do so.
Conclusions
Over the last 3 years, we have made recommendations aimed at correcting
fundamental limitations in US-VISIT's program management ability and
thereby better ensuring the delivery of mission capability and value on
time and commensurate with costs. While progress on the implementation of
the recommendations is mixed, progress in critical areas has been slow. As
with any program, introducing and institutionalizing the program
management and accountability discipline at which our recommendations are
aimed require investing time and resources while continuing to meet other
program demands. In making such investment choices, it is important to
remember that institutionalizing such program discipline in the near term
will produce long-term payback in a program's ability to meet these other
demands. Accordingly, the longer that US-VISIT takes to implement our
recommendations, the greater the risk that the program will not meet its
stated goals and commitments.
Our open recommendations are all aimed at strengthening US-VISIT program
management and improving DHS's ability to make informed US-VISIT
investment decisions. With the exception of one, these recommendations are
still relevant and applicable. Since we made our recommendation, facts and
circumstances surrounding Increment 2B deployment and operational status
have materially changed, making the collection of Increment 2B
predeployment impractical. Nevertheless, the need remains to better
understand the impact of US-VISIT entry capabilities on all land POEs.
Until this understanding exists, the department will be challenged in its
ability to accurately estimate and provide facilities and staff resource
needs.
Recommendation for Executive Action
To recognize both the need to fully assess the impact of US-VISIT entry
capabilities on staffing levels and facilities at land POEs, as well as
the current operational status of Increment 2B, we are closing our
existing recommendation related to assessing the impact of Increment 2B.
We recommend that the DHS Secretary direct the US-VISIT Program Director
to explore alternative means of obtaining an understanding of the full
impact of US-VISIT at all land POEs, including its impact on workforce
levels and facilities; these alternatives should include surveying the
sites that were not part of the previous assessment.
Agency Comments and Our Evaluation
In its written comments on a draft of this report, signed by the Director,
Departmental GAO/OIG Liaison Office, and reprinted in appendix II, DHS
stated that it agreed with many areas of the report and that our
recommendations had made US-VISIT a stronger program. Further, the
department stated that while it disagreed with certain areas of the
report, it nevertheless concurred with the need to implement our open
recommendations with all due speed and diligence.
DHS commented specifically on 11 of the 18 recommendations discussed in
the report. The recommendations, the department's comments, and our
responses follow:
1.Recommendation: Develop and begin implementing a system security plan,
and perform a privacy impact assessment and use the results of the
analysis in near-term and subsequent system acquisition decision making.
DHS stated that this recommendation has been fully implemented. In
support, it said that it has completed a US-VISIT security plan that is
consistent with National Institute of Standards and Technology (NIST)
guidance, and that it provided the plan to us in September 2004. It also
stated that the security risk assessment aspect of this recommendation was
established in February 2005, 20 months after we made the recommendation,
and thus the age of the recommendation should be shown as 10 months rather
that the 30 months cited in the report.
The department also commented that there is no US-VISIT system, but rather
a US-VISIT program with capabilities delivered by existing interconnected
systems. According to the department, these component systems have been
certified and accredited, consistent with NIST guidance, and as part of
their certification and accreditation, security plans and risk
assessments, as well as risk mitigation strategies, have been developed
for each system. The department stated that it provided us with these
system-level risk assessments, as well as system-specific action plans and
milestones for implementing the mitigation strategies. In addition, the
department noted that it completed a programwide risk assessment in
December 2005 that specifically addresses information security issues that
might not be captured in the system-specific documentation used to certify
and accredit each system. In light of its system-specific certification
and accreditation efforts, existing system-level risk assessments, and the
program-level risk management process (see response 4 for discussion of
the risk management process), DHS commented that it is inaccurate to state
that US-VISIT officials are not in a position to know program risks, and
the recommendation should be closed.
While we agree that we received a copy of the US-VISIT security plan,
dated September 2004, we do not agree that the plan satisfied all relevant
federal guidance and that DHS has fully implemented our recommendation. In
particular, it has not provided us with evidence that a programwide risk
assessment has been done and that a security plan reflective of such an
assessment exists. According to relevant guidance,39 a security plan
should describe, among other things, the methodology that is to be used to
identify system threats and vulnerabilities and to assess risks, and it
should include the date the risk assessment was completed because the
assessment is a necessary driver of the security controls described in the
plan. As we reported in February 2005 and state in this report, the
US-VISIT security plan did not include this information; further, although
DHS stated in its comments that it completed this risk assessment in
December 2005, this statement is contradicted by a statement elsewhere in
its comments that it is still in the process of doing the assessment. In
addition to this contradiction, DHS's comments did not include any
evidence to demonstrate that it has developed a complete risk assessment,
such as a copy of the assessment.
With regard to the age of the recommendation, we do not agree with DHS's
position that we established a new finding regarding the lack of a
programwide risk assessment in our February 2005 report. Rather, as part
of our analysis of actions to implement our prior recommendation to
develop a security plan, which is to include information about the related
security risk assessment, we observed that the plan did not indicate a
date for completing a risk assessment in accordance with federal
guidelines. Therefore, our position that about 30 months had passed from
the time of our initial recommendation (June 2003) is accurate.
With regard to the individual system-level risk assessments, we agree that
we have received them. However, we do not agree that we have received the
action plans and milestones cited in the comments. Regardless, we do not
believe that system-level assessments are a sufficient substitute for a
programwide assessment. Accordingly, our recommendation focused on the
need for an integrated US-VISIT system risk assessment as part of security
planning. While the system-level plans and risk assessments are relevant
and useful, they neither individually nor collectively address the threats
and vulnerabilities imposed as a result of these systems' integration. By
stating in its comments its commitment to having a programwide risk
assessment that identifies and proposes mitigations for security risks
that arise as a result of the interface and integration of the legacy
systems, DHS is agreeing with our position. Moreover, without evidence
that the program has completely assessed its risks, we continue to find no
basis for how program officials would know the full range and degree of
US-VISIT security risks. Our position in this regard has been reinforced
by a recent DHS Inspector General report that identified a number of
US-VISIT security risks.40
To further support its position that this recommendation has been fully
implemented, DHS also commented that it has completed numerous privacy
impact assessments and continues to update them to reflect system changes.
In particular, it said that it updated the privacy impact assessment in
December 2005 to reflect all increments and that it considers the
assessment to be part of US-VISIT system documentation. It further
commented that we appear to be unaware of privacy staff activities to
review system documents and perform privacy risk assessments throughout
the system life cycle. Nevertheless, the department acknowledged that its
privacy work was not always noted within US-VISIT system documentation.
Accordingly, DHS stated that it plans to appropriately reference all
privacy requirements and privacy risk assessments in the program's system
documentation in the future.
We agree that US-VISIT has developed and updated its privacy impact
assessment and would note that our report states this fact. We do not
agree, however, with the comment that we are not aware that the privacy
staff review system documents and perform privacy risk assessments. In
fact, it is because we were aware of these facts that we were careful to
ensure that they were reflected in our report. The point that we are
making is that privacy is not addressed in all relevant systems
documentation, which DHS acknowledged in its comments. With regard to this
point of agreement, we support the department's stated plans to reference
all privacy requirements and any privacy risk assessments in all relevant
system documentation in the future.
2.Recommendation: Develop and implement a plan for satisfying key
acquisition management controls, including acquisition planning,
solicitation, requirements management, program management, contract
tracking and oversight, evaluation, and transition to support, and
implement the controls in accordance with SEI guidance.
DHS commented that the report should reflect that US-VISIT had initially
adopted Carnegie Mellon University's Software Engineering Institute (SEI)
Software Acquisition Capability Maturity Model(R) to guide its
software-related process improvement efforts and that, in December 2004,
it transitioned to SEI's Capability Maturity Model-Integration (CMMI(R)).
As a result, it said that the program's process improvement strategy and
plans, process development, and process appraisals are now aligned to the
most applicable CMMI process areas.
We agree that US-VISIT has transitioned to CMMI. We state in our report
that US-VISIT has done so and that the key process areas it is addressing
in its process improvement strategy and plan are consistent with those
cited in our recommendation. We do not believe that this transition
materially affects our recommendation, however, because even though the
names of the key processes in these two models may in some cases differ,
the processes and respective practices are fundamentally consistent.
3.Recommendation: Clarify the operational context in which US-VISIT is to
operate.
Consistent with our report, DHS commented that the operational context in
which US-VISIT operates is in progress, meaning that it has yet to be
fully established. For example, it said that the mission of DHS, and
therefore the scope of US-VISIT activities to meet the mission, is
continually expanding. Further, it acknowledged that more certainty in the
operational context is desirable. In mitigation of the risks associated
with not having a more stable operational context, DHS made several
statements. For example, it said that the principal role of US-VISIT is to
integrate information and immigration and border management systems across
DHS and the State Department, and to facilitate agencies working toward a
common environment that will eliminate redundancies. It also said that
elements of its draft immigration and border management strategic plan are
being used in current US-VISIT operations. In addition, the department
said that mechanisms to mitigate the risks that we cited have been
developed and are being implemented.
We support DHS's acknowledgment of the importance of having a well-defined
operational context within which to define and implement US-VISIT and
related border security programs. However, we do not believe that DHS's
comments provided any evidence showing that sufficient steps and
activities to mitigate the associated risks have been taken or are
planned.
4.Recommendation: Determine whether proposed US-VISIT increments will
produce mission value commensurate with cost and risks and disclose to the
Congress planned actions.
DHS commented that its cost-benefit analysis (CBA) for Increment 1B
conforms to relevant federal guidance, and noted that our expectations as
to the scope and level of detail of analysis that should be included in
the CBA document are inconsistent with its understanding of OMB Circular
A-94
and DHS's CBA workbook,41 which were used to guide the development of the
CBA analysis. As an example, the department took exception with our
statement that year-by-year benefit estimates were not reported by noting
that the net present value was based on an estimate of annual benefits and
costs, and that net present value could not be estimated without a
year-by-year benefit analysis.
The department further commented that a comprehensive uncertainty analysis
was conducted because it completed a risk analysis, which is more
comprehensive, rigorous, and appropriate than conducting a sensitivity
analysis. In this regard, it added that the results of the risk analysis
provided an indication of Increment 1B's worthiness in light of existing
uncertainty, rather than information on a specific CBA variable or
another. The department further noted that it had provided some of these
supporting analyses to us.
DHS also stated that any investment that has a 5-year life cycle and is
considered interim in nature will face considerable challenge in providing
economic benefits commensurate with cost.
We do not agree that the CBA fully conforms to relevant federal guidance.
As our report states, for example, the analysis does not explicitly state
the numerical value of the discount rate used for calculating each
alternative's net present value, and hence does not conform to OMB
guidance. In addition, the cost estimates used in the analysis were not
complete and reliably derived. In deriving the estimate, for example, the
department did not clearly define the project's life cycle to ensure that
key factors were not overlooked and that the full cost of the program was
included. (See response 10 below for more information on this point.)
Last, while we agree that a year-by-year benefit analysis is a necessary
component of a net present value determination, OMB nevertheless requires
that the year-by-year benefit estimates be reported in the analysis to
promote independent review of the estimates.
Also, we do not agree that DHS performed a complete uncertainty analysis.
According to OMB and DHS guidance, a complete uncertainty analysis should
include both a risk analysis and a sensitivity analysis. However, the
latter was not done. Thus, our point is not, as DHS comments suggest, that
US-VISIT should have performed a sensitivity analysis instead of a risk
analysis, but rather, that both types of analyses are necessary to
completely examine investment uncertainty.
5.Recommendation: Develop and implement a risk management plan and ensure
that all high risks and their status are reported regularly to the
executive body.
DHS commented that US-VISIT began the development and implementation of
its risk management plan in 2004 immediately after we made our
recommendation. It further commented that, as part of a CMMI maturity
internal appraisal that it completed in July 2005, it found that the risk
management process had not been consistently applied across the program.
To address this, the department cited actions that it has taken to fully
implement risk management, such as approving the risk management plan in
September 2005; defining a risk governance structure; establishing and
maintaining a risk database; and developing risk management training and
providing this training to program personnel and contractors beginning in
November 2005.
We support the recent actions that the program cited as having been taken
to strengthen risk management. However, the actions cited do not
demonstrate that the risk management process is being consistently
applied. Until US-VISIT fully implements its risk management plan and
process, it cannot be assured that all program risks are being identified
and managed in order to effectively mitigate any negative impact on the
program's ability to deliver promised capabilities on time and within
budget.
6.Recommendation: Develop and approve test plans before testing begins
that (1) specify the test environment; (2) describe each test to be
performed, including test controls, inputs, and expected outputs;
(3) define the test procedures to be followed in conducting the tests; and
(4) provide traceability between test cases and the requirements to be
verified by the testing.
DHS stated that our report does not accurately reflect the status of the
Increment 2C Phase 1 testing. In particular, it said that the issues
associated with the traceability of requirements to test cases were minor
and that the extent of the discrepancies is far less than what our report
presents. It further stated that the discrepancies in our report are based
on old traceability documentation and do not reflect revised documentation
provided to us on November 9, 2005.
We agree that DHS provided us with revised traceability matrixes after we
had shared with them our analysis of the test plans and traceability
matrixes, dated June 28, 2005, and June 27, 2005, respectively. However,
the revised documentation referenced in DHS's comments was provided in
November 2005, about 4 months after testing began. This means that the
test plans and traceability matrixes available at the time of
testing-which are what we reviewed because they governed the scope and
nature of actual testing performed-did not adequately trace between test
cases and the requirements to be verified. Specifically, 300 of the 438
Increment 2C requirements, or about 70 percent, did not have specific
references to test cases.
7.Recommendation: Implement effective configuration management practices,
including establishing a US-VISIT change control board to manage and
oversee system changes.
DHS commented that a US-VISIT representative attends all configuration
control board meetings for all applicable legacy component systems, and
that any proposed change request from a legacy component control board
that could affect US-VISIT functionality is brought to the attention of
the US-VISIT Executive Configuration Control Board for consideration.
We do not question these statements. However, we do not believe that they
demonstrate that US-VISIT has adequate control over system changes that
could affect the program. That is, they do not ensure that changes to the
component systems that are initiated and approved by another DHS
organization and that could affect US-VISIT performance are subject to
US-VISIT configuration management and approval processes. US-VISIT could
establish explicit and enforceable control over changes to the legacy
systems through such mechanisms as defined and enforced memorandums of
understanding among the affected DHS organizations. It was the lack of
such control that prompted our recommendation.
8.Recommendation: Assess the full impact of Increment 2B on land POE
workforce levels and facilities, including performing appropriate modeling
exercises.
The department stated that, given the imperative to meet the legislatively
mandated time frames, the scope of Increment 2B was limited to only one
part of POE operations-incorporating the collection of a biometric into
the previously manual Form I-94 issuance process. It also stated that wait
times are affected by various factors, including traffic volume, staffing
levels, and availability of officers. Therefore, DHS focused the Increment
2B evaluation on just the change to this process.
The department further commented that given the events since the
evaluation-namely, Increment 2B full operations-it is not practical to
collect and model baseline data for the 47 sites that were not part of the
initial evaluation.
Regarding the 3 pilot sites included in the assessment, the department
stated that the sites were selected based on criteria developed from input
from US-VISIT, as well as CBP operational constraints. The department
further commented that the 3 sites provided a reasonable mix of travelers
and they did not have other constraints that directly impacted the
collection of performance data specific to Form I-94 issuance. DHS also
stated that the I-94 processing times vary by POE, and therefore they are
not easily generalized from one port to another. Further, the department
commented that the number of workstations and officers available to
operate those workstations to process applicants for a Form I-94 do not
impact the time it takes to issue a Form I-94.
We agree that the scope of the Increment 2B evaluation was limited to the
I-94 issuance process, and that it did not address the increment's impact
on the POEs' ability to meet other performance parameters. Our point is
that the limited nature of the evaluation does not satisfy either the
intent of our recommendation or DHS's own stated purpose for the
evaluation, which was to determine the effectiveness of Increment 2B
performance at the 50 busiest land POEs. We also agree that the I-94
processing times vary by POE and cannot be easily generalized. It is for
this reason, among others, that we questioned whether the 3 sites selected
for the assessment were sufficiently representative to satisfy both our
recommendation and the evaluation's stated purpose.
In addition, while we also agree that collecting pre-Increment 2B baseline
data is not practical at this time, the fact remains that the operational
impact of Increment 2B on workforce levels and facilities has not been
adequately assessed, as evidenced by officials at 1 large POE telling us
that processing times have increased and DHS's recognition that each POE
is somewhat different. In light of these new facts and circumstances, we
are closing our existing recommendation and making a new recommendation to
recognize the need for DHS to explore alternative means to assess the
impact of US-VISIT entry capabilities at land POEs. This new
recommendation will be shown as an open recommendation, and the original
recommendation will be closed.
9.Recommendation: Develop a plan, including explicit tasks and milestones,
for implementing all of our open recommendations and periodically report
to the DHS Secretary and Under Secretary on progress in implementing this
plan; and report this progress, including reasons for delays, in all
future expenditure plans.
DHS stated that it is untrue that 19 months had elapsed from the time we
made this recommendation to the time that it assigned responsibilities to
program officials for addressing each of our recommendations. In support,
it commented that it issued its first plan to address our recommendations
on August 18, 2003, and subsequent reports have been issued periodically
that update progress in doing so.
We agree that DHS has assigned responsibilities to specific individuals
for addressing each recommendation. However, we have yet to be provided
any evidence to support its statement that it issued the first report
addressing our recommendations on August 18, 2003. Similarly, we have not
received evidence showing that it has prepared a plan, including specific
actions and milestones, for implementing all of our open recommendations,
which is a focus of this recommendation. We would also observe that we
made this recommendation in May 2004, and at that time the department
stated that it agreed with the recommendation but did not indicate that it
had taken any steps to address it, such as commenting that a report was
issued on August 18, 2003.
10.Recommendation: Follow effective practices for estimating the costs of
future increments.
DHS either tacitly or explicitly agreed with our findings relative to its
satisfaction of 8 of the 13 cost-estimating criteria presented in table 4
(now table 3) of our draft report. For example, it agreed that it did not
clearly define the life cycle to which the cost estimate applies. It also
agreed that it did not include a work breakdown structure, noting that it
used the available project implementation schedule as a proxy for the
activities related to the deployment of the exit alternatives.
Regarding our five findings concerning its satisfaction of cost-estimating
with which DHS disagreed, the department's primary area of disagreement
was with the intended purpose of the Increment 1B CBA that used the cost
estimate, which it said in its comments was to inform decision makers
about the relative worthiness of each of the three exit alternatives
considered for deployment. Hence, DHS stated that the purpose of the CBA
was to analyze only the costs associated with deploying an operational
solution, not to analyze the costs and benefits of both developing and
deploying alternative solutions. DHS further stated that the CBA thus
includes only those costs to be incurred in deploying a selected
alternative, and it does not include costs already incurred in developing
system alternatives (i.e., sunk costs). It further commented that DHS
guidance states that sunk costs are not relevant to the current investment
analysis because "only current decisions can affect the future
consequences of investment alternatives."
DHS also disagreed that the cost estimate in the CBA should have included
nonrecurring development costs, and commented that it did appropriately
size the task described in the cost estimates for each alternative exit
solution, noting that sizing metrics related to software development were
not relevant to deployment of the alternatives because development
activities had already occurred and therefore are sunk costs. The
department added that those sizing metrics that are relevant to the cost
estimate are discussed in the CBA, as are the cost estimating parameters
(i.e., those associated with deployment and not those associated with
development and testing).
In addition, DHS disagreed that DHS's cost estimate excluded important
cost categories, such as system testing, and stated that the estimate
addresses labor, facilities, operations and maintenance, information
technology, travel, and training costs. Once again, DHS emphasized that
since the focus of the CBA was on operational deployment and not system
design and development, system testing costs were not included because
they were not considered relevant. DHS also reiterated its early point
that the uncertainty analysis that it conducted was comprehensive.
We agree that actual sunk costs should not be included in a CBA cost
estimate. However, we disagree that the cost categories that DHS cited as
not relevant are only costs that are associated with predeployment
activities. Testing, for example, is an activity that is normally
performed before, during, and following deployment, and thus the
associated costs would be relevant to the stated purpose of the Increment
1B CBA. However, a testing cost category was missing from the CBA cost
estimate, as was a cost category for software maintenance.
Regarding DHS's statement that it conducted a complete uncertainty
analysis, we reiterate our previous point that a complete uncertainty
analysis should include both a risk analysis and a sensitivity analysis,
and the CBA did not include the latter.
11.Recommendation: Reassess plans for deploying an exit capability to
ensure that the scope of the exit pilot provides for adequate evaluation
of alternative solutions and better ensures that the exit solution
selected is in the best interest of the program.
Concerning the questions we raised about the adequacy of the exit pilots
in light of the 24 percent compliance rate, DHS commented that we failed
to consider the compliance rate of the previous exit pilot program, the
National Security Entry Exit Registration System (NSEERS), which,
according to DHS, had a 75 percent compliance rate. DHS added that NSEERS
achieved this compliance rate with a very limited number of exit
locations, and therefore, any of the three US-VISIT exit alternatives
would have at least a 75 percent compliance rate once national deployment
was completed.
Further, the department commented that Immigration and Customs Enforcement
(ICE) had recently conducted enforcement operations at the Denver
International Airport, and that the compliance rate during these
operations increased from 30 percent to over 90 percent. It then concluded
that the combined results of the exit pilot evaluation, the NSEERS pilot,
and the ICE enforcement activities at the Denver International Airport
lead it to believe that the US-VISIT exit alternatives have been
adequately evaluated.
We do not agree with this conclusion because it is based on unsupported
assumptions. Specifically, DHS did not provide any evidence to support its
claim that that US-VISIT would achieve a comparable compliance rate to the
NSEERS program. Moreover, even if DHS could achieve a 75 percent
compliance rate for US-VISIT exit,that still means that 25 percent of
eligible persons would not be complying with the US-VISIT exit process.
Further, DHS did not provide any information about the recent enforcement
actions conducted by ICE, nor did it provide any evidence that this is a
practical and viable option for the US-VISIT exit solution. While we agree
that enforcement actions may indeed increase the exit compliance rate, DHS
has not yet assessed the impact of such a solution on the US-VISIT exit
process. Further, the US-VISIT program director acknowledged the need to
evaluate the impact of implementing potential enforcement actions on
US-VISIT exit and planned to do so.
We are sending copies of this report to the Chairmen and Ranking Minority
Members of the Senate and House Appropriations Committees, as well as to
the Chairmen and Ranking Minority Members of other Senate and House
committees that have authorization and oversight responsibilities for
homeland security. We are also sending copies to the Secretary of Homeland
Security, Secretary of State, and the Director of OMB. Copies of this
report will also be available at no charge on our Web site at w
ww.gao.gov.
Should you or your offices have any questions on matters discussed in this
report, please contact me at (202) 512-3439 or at h iter@gao.gov. Contact
points for our Offices of Congressional Relations and Public Affairs may
be found on the last page of this report. Key contributors to this report
are listed in appendix IV.
Randolph C. Hite Director, Information Technology Architecture and Systems
Issues
List of Requesters
The Honorable Peter T. King Chairman The Honorable Bennie G. Thompson
Ranking Minority Member Committee on Homeland Security House of
Representatives
The Honorable Bob Filner House of Representatives
The Honorable Raul M. Grijalva House of Representatives
The Honorable Ruben Hinojosa House of Representatives
The Honorable Solomon Ortiz House of Representatives
The Honorable Silvestre Reyes House of Representatives
Objective, Scope, and Methodology Appendix I
Our objective was to determine the progress of the Department of Homeland
Security (DHS) in implementing 18 of our recommendations pertaining to the
U.S. Visitor and Immigrant Status Indicator Technology (US-VISIT) program.
To accomplish this objective, we reviewed and analyzed US-VISIT's most
recent status reports on the implementation of our open recommendations
and related key documents, augmented as appropriate by interviews with
program officials. More specifically, we analyzed relevant systems
acquisition documentation, including the program's process improvement
plan, risk management plan, and configuration management plan. We also
analyzed the US-VISIT security plan, privacy impact assessment,
cost-benefit analysis, cost estimates, test plans, human capital plans,
and related evaluations and assessments. In performing our analyses, we
compared available documentation and program officials' statements with
relevant federal guidance and associated best practices.1 A more detailed
description of our scope and methodology relative to the cost-benefit
analysis, cost estimates, and test plans follows:
o Our analysis of the cost-benefit analysis focused on Increment 1B
because this was the latest cost-benefit analysis and cost estimate
prepared. In doing this analysis, we compared the US-VISIT cost-benefit
analysis to eight criteria in Office of Management and Budget (OMB)
guidance.2
o Our analysis of the cost estimate also focused on Increment 1B for the
same reason previously cited. In doing this analysis, we compared the
estimate to 13 criteria from the Software Engineering Institute3 that we
have previously reported to be the minimum set of actions needed to
develop a reliable cost estimate. We then determined whether the criteria
were satisfied, partially satisfied, or not satisfied using the
definitions given below.
o Our analysis of the test plans focused on Increment 2C because it is the
most recently tested increment. This analysis included determining the
extent to which the test plans for this increment met 4 key criteria that
we have previously reported as essential to effective test plans. In doing
this analysis, we examined Increment 2C systems documentation, including
business and functional requirements and traceability matrixes. We also
independently traced 58 business requirements and 438 functional
requirements to the test cases in the test plan. Further, we independently
traced all test cases to the requirements to determine consistency.
In performing our work, we used the following categories and definitions
in deciding the extent to which each recommendation had been implemented.
Specifically, we considered a recommendation
o completely implemented when documentation demonstrated that it had been
fully addressed,
o partially implemented when documentation indicated that actions were
under way to implement it, and
o in progress when documentation indicated that action had been initiated
to implement it.
These categories and definitions are consistent with those used in our
prior US-VISIT reports.
In determining the amount of time it has taken to implement actions on our
recommendations, we calculated the time from the date the report was
issued through December 2005.
We conducted our audit work at the US-VISIT program office in Rosslyn,
Virginia, from August 2005 through December 2005, in accordance with
generally accepted government auditing standards.
Comments from the Department of Homeland Security Appendix II
Description of US-VISIT Processes Appendix III
US-VISIT involves complex processes governing the stages of a traveler's
visit to the United States (pre-entry, entry, status, and exit) and
analysis of hundreds of millions of foreign national travelers at over 300
air, sea, and land ports of entry (POE). A simplified depiction of these
processes is shown in figure 4.
Figure 4: US-VISIT Process Overview
Pre-entry Process
Pre-entry processing begins with initial petitions for visas, grants of
visa status, or the issuance of travel documentation. When a foreign
national applies for a visa at a U.S. consulate, biographic and biometric
data are collected and shared with border management agencies. The
biometric data are transmitted from the Department of State to DHS, where
the prints are run against the Automated Biometric Identification System
(IDENT) database1 to verify identity and to run a check against the
biometric watch list. The results of the biometric check are transmitted
back to State. A "hit" response prevents State's system from printing a
visa for the applicant until the information is reviewed and cleared by a
consular officer.
Pre-entry also includes transmission by commercial air and sea carriers of
crew and passenger manifests to appropriate immigration officers before
these carriers arrive in the United States.2 These manifests are
transmitted through the Advanced Passenger Information System (APIS). The
APIS lists are run against the biographic lookout system to identify those
arrivals for whom biometric data are available. In addition, POEs review
the APIS list in order to identify foreign nationals who need to be
scrutinized more closely.
Entry Process
When a foreign national arrives at a POE's primary (air and sea) or
secondary (land) inspection booth, the inspector, using a document reader,
scans the machine-readable travel documents. APIS returns any existing
records on the foreign national to the US-VISIT workstation screen,
including manifest data matches and biographic lookout hits. When a match
is found in the manifest data, the foreign national's name is highlighted
and outlined on the manifest data portion of the screen.
Biographic information, such as name and date of birth, is displayed on
the bottom half of the computer screen, along with a photograph obtained
from State's Consular Consolidated Database.3 The inspector at the booth
scans the foreign national's fingerprints (left and right index fingers)
and takes a digital photograph. This information is forwarded to the IDENT
database, where it is checked against stored fingerprints in the IDENT
lookout database. If the foreign national's fingerprints are already in
IDENT, the system performs a match (a comparison of the fingerprint taken
during the primary inspection to the one on file) to confirm that the
person submitting the fingerprints is the person on file. If no prints are
currently in IDENT, the foreign national is enrolled in US-VISIT (i.e.,
biographic and biometric data are entered into IDENT).
During this process, the inspector also questions the foreign national
about the purpose of his or her travel and length of stay. The inspector
adds the class of admission and duration of stay information into the
Treasury Enforcement Communications Systems,4 and stamps the "admit until"
date on the Form I-94.5 If the foreign national is ultimately determined
to be inadmissible, the person is detained, lookouts are posted in the
databases, and appropriate actions are taken.
Status Management Process
The status management process manages the foreign national's temporary
presence in the United States, including the adjudication of benefits
applications and investigations into possible violations of immigration
regulations.
As part of this process, commercial air and sea carriers transmit
departure manifests electronically for each departing passenger. These
manifests are transmitted through APIS and shared with the Arrival
Departure Information System (ADIS).6 ADIS matches entry and exit manifest
data (i.e., each record showing a foreign national entering the United
States is matched with a record showing the foreign national exiting the
United States). ADIS also receives status information from the Computer
Linked Application Information Management System7 and the Student Exchange
Visitor Information System8 on foreign nationals.
Exit Process
The exit process includes the carriers' submission of electronic manifest
data to APIS. This biographic information is transmitted to ADIS, where it
is matched against entry information. At the 11 POEs where the exit
solution is being implemented, the departure is processed by one of three
exit methods. Within each port, one or more of the exit methods may be
used. The three methods are as follows:
o Kiosk: At the kiosk, the traveler, guided by a workstation attendant if
needed, scans the machine-readable travel documents, provides electronic
fingerprints, and has a digital photograph taken. A receipt is printed to
provide documentation of compliance with the exit process and to assist in
compliance on the traveler's next attempted entry to the country. After
the receipt prints, the traveler proceeds to his or her departure gate. At
the conclusion of the transaction, the collected information is
transmitted to IDENT.
o Mobile device: At the departure gate, and just before the traveler
boards the departure craft, either a workstation attendant or law
enforcement officer scans the machine-readable travel documents, scans the
traveler's fingerprints (right and left index fingers), and takes a
digital photograph. A receipt is printed to provide documentation of
compliance with the exit process and to assist in compliance on the
traveler's next attempted entry to the country. The device wirelessly
transmits the captured data in real time to IDENT via the Transportation
Security Administration's Data Operations Center.
If the device is being operated by a workstation attendant, he or she
provides a printed receipt to the traveler, and the traveler then boards
the departure craft. If the mobile device is being operated by a law
enforcement officer, the captured biographic and biometric information is
checked in near real time against watch lists. Any potential match is
returned to the device and displayed visually for the officer. If no match
is found, the traveler is allowed to board the departure craft.
o Validator: Using a kiosk, the traveler, guided by a workstation
attendant if needed, scans the machine-readable travel documents, provides
electronic fingerprints, and has a digital photograph taken.
As with the kiosk, a receipt is printed to provide documentation of
compliance with the exit process and to assist in compliance on the
traveler's next attempted entry to the country. However, this receipt has
biometrics (i.e., the traveler's fingerprints and photograph) embedded on
the receipt. At the conclusion of the transaction, the collected
information is transmitted to IDENT.
The traveler presents his or her receipt to the attendant or law
enforcement officer at the gate or departure area, who scans the receipt
using a mobile device. The traveler's identity is verified against the
biometric data embedded on the receipt. Once the traveler's identity is
verified, he or she is allowed to board the departure craft. The captured
data are not transmitted in real time back to IDENT. Instead, the data are
periodically uploaded through the kiosk to IDENT.
Analysis Process
An analysis capability is to provide for the continuous screening against
watch lists of individuals enrolled in US-VISIT for appropriate reporting
and action. As more entry and exit information becomes available, it is to
be used for analysis of traffic volume and patterns as well as for risk
assessments. The analysis is also to be used to support resource and
staffing projections across POEs, strategic planning for integrated border
management analysis performed by the intelligence community, and
determination of travel use levels and expedited traveler programs.
GAO Contact and Staff Acknowledgments Appendix IV
Randolph C. Hite, (202) 512-3439 or h iter@gao.gov
In addition to the contact named above, the following people made key
contributions to this report: Deborah Davis, Assistant Director; Hal
Brumm; Tonia Brown; Joanna Chan; Barbara Collier; Neil Doherty; Jennifer
Echard; James Houtz; Scott Pettis; Karen Richey; and Karl Seifert.
(310606)
www.gao.gov/cgi-bin/getrpt? GAO-06-296 .
To view the full product, including the scope
and methodology, click on the link above.
For more information, contact Randolph C. Hite at (202) 512-3439 or
hiter@gao.gov.
Highlights of GAO-06-296 , a report to congressional requesters
February 2006
HOMELAND SECURITY
Recommendations to Improve Management of Key Border Security Program Need
to Be Implemented
The Department of Homeland Security (DHS) has established a program-the
U.S. Visitor and Immigrant Status Indicator Technology (US-VISIT)-to
collect, maintain, and share information, including biometric identifiers,
on selected foreign nationals entering and exiting the United States.
US-VISIT uses these identifiers (digital fingerscans and photographs) to
screen persons against watch lists and to verify that a visitor is the
person who was issued a visa or other travel document. Visitors are also
to confirm their departure by having their visas or passports scanned and
undergoing fingerscanning at selected air and sea ports of entry (POE).
GAO has made many recommendations to improve the program, all of which DHS
has agreed to implement. GAO was asked to report on DHS's progress in
responding to 18 of these recommendations.
What GAO Recommends
GAO is closing its existing recommendation related to DHS's assessment of
Increment 2B and recommending that DHS explore alternative means to fully
assess the impact of US-VISIT entry capabilities on land POEs. In its
comments on a draft of this report, DHS stated that it agreed with many
areas of the report and disagreed with others. It also concurred with the
need to quickly implement GAO's open recommendations.
The current status of DHS's implementation of the 18 recommendations is
mixed, but progress in critical areas has been slow. DHS has implemented 2
of the recommendations: it defined program staff positions, roles, and
responsibilities, and it hired an independent verification and validation
contractor. It has also taken steps to implement the other
recommendations, partially completing 11 and beginning to implement
another 5.
o In September 2003, GAO reported that the program had not
assessed the costs and benefits of Increment 1 (which provides
entry capabilities to air and sea POEs) and recommended that the
program determine whether proposed increments will produce mission
value commensurate with cost. In the latest cost-benefit analysis,
dated June 23, 2005, the program identified potential costs and
benefits for three alternatives for an air and sea exit solution.
However, the analysis does not meet key Office of Management and
Budget criteria; for example, it does not include a complete
uncertainty analysis, which helps to provide decision makers with
perspective on the potential variability of the cost and benefit
estimates should circumstances change.
o GAO reported in May 2004 and February 2005 that system testing
was not based on well-defined test plans and recommended that
before testing begins, the program develop and approve test plans
meeting certain criteria. However, although the latest test plan
did cover many required areas (such as the tests to be performed),
it did not adequately trace between test cases and the
requirements to be verified by testing. Without complete and
traceable test plans, the risk is increased that the deployed
system will not perform as intended.
o In May 2004, GAO reported that the program had not assessed its
workforce and facility needs for Increment 2B (which extends entry
capabilities to the 50 busiest land POEs) and recommended that it
do so. Since then, the program evaluated the processing times to
issue and process entry/exit forms at 3 of the 50 busiest POEs and
concluded that the results showed that no additional staff and
only minor facilities modifications were required. However, the
scope of the evaluation was limited. Since then, DHS has deployed
and implemented Increment 2B capabilities to these 50 POEs, making
the collection of predeployment baseline data for these sites
impractical. Nonetheless, other alternatives, such as surveying
site officials about the increment's impacts, have yet to be
explored. Until they are, the program may not be able to
accurately project resource needs or make any needed modifications
to achieve its goals of minimizing US-VISIT's impact on POE
operations, which was the impetus for GAO's recommendation.
DHS attributed the pace of progress to competing demands on time and
resources. The longer that US-VISIT takes to implement the
recommendations, the greater the risk that the program will not meet its
stated goals on time and within budget.
*** End of document. ***