    Homeland Security: Recommendations to Improve Management of Key
    Border Security Program Need to Be Implemented (14-FEB-06,
    GAO-06-296).

    The Department of Homeland Security (DHS) has established a
    program--the U.S. Visitor and Immigrant Status Indicator
    Technology (US-VISIT)--to collect, maintain, and share
    information, including biometric identifiers, on selected foreign
    nationals entering and exiting the United States. US-VISIT uses
    these identifiers (digital fingerscans and photographs) to screen
    persons against watch lists and to verify that a visitor is the
    person who was issued a visa or other travel document. Visitors
    are also to confirm their departure by having their visas or
    passports scanned and undergoing fingerscanning at selected air
    and sea ports of entry (POE). GAO has made many recommendations
    to improve the program, all of which DHS has agreed to implement.
    GAO was asked to report on DHS's progress in responding to 18 of
    these recommendations.
    -------------------------Indexing Terms-------------------------
    REPORTNUM:   GAO-06-296
        ACCNO:   A46797
      TITLE:     Homeland Security: Recommendations to Improve Management
    of Key Border Security Program Need to Be Implemented
         DATE:   02/14/2006
      SUBJECT:   Biometric identification
    	     Cost effectiveness analysis
    	     Data collection
    	     Homeland security
    	     Identity verification
    	     Immigration information systems
    	     Internal controls
    	     Operational testing
    	     Passports
    	     Performance measures
    	     Program evaluation
    	     Program management
    	     Strategic planning
    	     Visas
    	     Information sharing
    	     DHS Visitor and Immigrant Status
    	     Indicator Technology Program

    ******************************************************************
    ** This file contains an ASCII representation of the text of a  **
    ** GAO Product.                                                 **
    **                                                              **
    ** No attempt has been made to display graphic images, although **
    ** figure captions are reproduced.  Tables are included, but    **
    ** may not resemble those in the printed version.               **
    **                                                              **
    ** Please see the PDF (Portable Document Format) file, when     **
    ** available, for a complete electronic file of the printed     **
    ** document's contents.                                         **
    **                                                              **
    ******************************************************************
    GAO-06-296

         * Report to Congressional Requesters
              * February 2006
         * HOMELAND SECURITY
              * Recommendations to Improve Management of Key Border Security
                Program Need to Be Implemented
         * Contents
              * Results in Brief
              * Background
                   * Acquisition and Implementation Strategy: A Brief Description
                   * US-VISIT Is Being Implemented in Four Increments
                   * Program Management Roles and Responsibilities
                   * Our Prior Work Has Resulted in Several Recommendations
              * The Status of DHS's Implementation of Our Recommendations Is
                Mixed
                   * Development and Implementation of a Security Plan and
                     Performance of a Privacy Impact Assessment Are Partially
                     Complete
                        * Security Plan
                        * Privacy Impact Assessment
                   * Development and Implementation of Key Acquisition Controls
                     Are Partially Complete
                   * Determination and Disclosure of Whether Increments Produce
                     Mission Value Commensurate with Costs and Risks Are
                     Partially Complete
                   * Definition of the Operational Context for US-VISIT Is in
                     Progress
                   * Provision of Program Office Resources Is Partially Complete
                   * Definition of Program Office Roles and Responsibilities Has
                     Been Completed
                   * Development and Implementation of a Human Capital Strategy
                     Are Partially Complete
                   * Defining Performance Standards for US-VISIT Increments Is
                     Partially Complete
                   * Development and Implementation of a Risk Management Plan Are
                     Partially Complete
                   * Development of Test Plans Is Partially Complete
                   * Assessment of the Impact of Increment 2B on Workforce Levels
                     and Facilities Is Partially Complete
                   * Implementation of Configuration Management Practices Is in
                     Progress
                   * Efforts to Ensure the Independence of the Verification and
                     Validation Contractor Are Complete
                   * Development of a Plan to Address Open Recommendations Is
                     Partially Complete
                   * Establishment of Effective Cost-Estimating Practices Is in
                     Progress
                   * Reassessment of Plans for Deploying the Exit Capability Is
                     Partially Complete
                   * Development and Implementation of Capacity Management
                     Processes Are in Progress
                   * Identification of ACE and US-VISIT Relationships and
                     Dependencies Is in Progress
              * Conclusions
              * Recommendation for Executive Action
              * Agency Comments and Our Evaluation
         * Objective, Scope, and Methodology
         * Comments from the Department of Homeland Security
         * Description of US-VISIT Processes
              * Pre-entry Process
              * Entry Process
              * Status Management Process
              * Exit Process
              * Analysis Process
         * GAO Contact and Staff Acknowledgments

    Report to Congressional Requesters

    February 2006

    HOMELAND SECURITY

    Recommendations to Improve Management of Key Border Security Program Need
    to Be Implemented

    Contents

    Tables

    Figures

    February 14, 2006Letter

    Congressional Requesters:

    The U.S. Visitor and Immigrant Status Indicator Technology (US-VISIT) is a
    multibillion-dollar program of the Department of Homeland Security (DHS)
    that is intended to record the entry into and exit from the United States
    of selected individuals, verify their identity, and confirm their
    compliance with the terms of their admission into and stay in the United
    States. The goals of the program are to (1) enhance the security of our
    citizens and visitors, (2) facilitate legitimate travel and trade,
    (3) ensure the integrity of the U.S. immigration system, and (4) protect
    the privacy of our visitors.

    Since fiscal year 2002, DHS has been legislatively directed to submit
    annual expenditure plans for the program, and we have been directed to
    review these plans and issue reports. These reports have, among other
    things, identified risks that face the department in delivering promised
    program capabilities and benefits on time and within cost.1 For example,
    we reported that the program office did not have the human capital and
    acquisition process discipline needed to effectively manage the program.
    Because of the number and severity of program management challenges that
    we identified,  we concluded that the program was risky.

    To address program risks, our reports have included  18 recommendations in
    such areas as system acquisition process controls, economic justification,
    human capital management, cost estimating, and test management, all of
    which DHS has agreed to implement.2 Because of your continued interest in
    ensuring that DHS is taking the necessary actions to successfully
    implement US-VISIT, you asked us to determine the progress being made in
    implementing these recommendations. To achieve this objective, we analyzed
    program plans, reports, and system documentation relative to the intent of
    each of our recommendations, and we interviewed appropriate DHS and
    program officials. (Further details on our objective, scope, and
    methodology are provided in app. I.) Our work was performed from August
    2005 through December 2005 in accordance with generally accepted
    government auditing standards.

    Results in Brief

    The current status of DHS's implementation of the 18 recommendations is
    mixed, but progress in critical areas has been slow. DHS has implemented 2
    of the recommendations: it defined program staff positions, roles, and
    responsibilities, and it hired an independent verification and validation
    contractor. It has also taken steps to implement the other
    recommendations, partially completing 11 and beginning to implement
    another 5. However, although considerable time has passed since the
    recommendations were made, key actions have not yet been taken in such
    critical areas as (1) assessing security risks and planning for
    cost-effective controls to address the risks, (2) determining-before
    US-VISIT increments are deployed-whether each increment will produce
    mission value commensurate with cost and risk, and (3) ensuring that each
    increment is adequately tested. Of the 11 recommendations that are
    partially implemented, 7 are about 2 years old, and 4 are about 10 to 19
    months old. Of the 5 that are in progress, 3 are about 10 months old.3
    According to the Program Director, the pace of progress is attributable to
    competing demands on time and resources. The longer that US-VISIT takes to
    implement the recommendations, the greater the risk that the program will
    not meet its stated goals on time and within budget.

    DHS provided written comments on a draft of this report. In its comments,
    the department stated that it agreed with many areas of the report and
    that our recommendations had made US-VISIT a stronger program. Further,
    the department stated that while it disagreed with certain areas of the
    report, it nevertheless concurred with the need to implement our open
    recommendations with all due speed and diligence. One area of disagreement
    was regarding the program's ability to thoroughly assess the impact of
    US-VISIT entry capabilities on the 50 busiest land port of entry (POE)
    facilities and staffing levels, an assessment that we called for in our
    recommendation. In particular, DHS stated that since US-VISIT was
    operational at these POEs, the collection of predeployment baseline
    performance data was no longer practical. In light of these comments, we
    are making a new recommendation to the Secretary of DHS that recognizes
    these facts and circumstances and that replaces the open recommendation
    discussed in this report. This recommendation provides for the department
    to explore alternative means of assessing the impact of US-VISIT entry
    capabilities on land POE facilities and staffing levels. All of DHS's
    comments, along with our responses, are discussed in detail in the Agency
    Comments and Our Evaluation section of this report. The comments are also
    reprinted in their entirety in appendix II.

    Background

    US-VISIT is a governmentwide program intended to enhance the security of
    U.S. citizens and visitors, facilitate legitimate travel and trade, ensure
    the integrity of the U.S. immigration system, and protect the privacy of
    our visitors. Its scope includes the pre-entry, entry, status, and exit of
    hundreds of millions of foreign national travelers who enter and leave the
    United States at over 300 air, sea, and land POEs, and the provision of
    new analytical capabilities across the overall process.

    To achieve its goals, US-VISIT uses biometric information (digital
    fingerscans and photographs) to verify identity.4 In many cases, the
    US-VISIT process begins overseas at U.S. consular offices, which collect
    biometric information from applicants for visas and check this information
    against a database of known criminals and suspected terrorists. When a
    visitor arrives at a POE, the biometric information is used to verify that
    the visitor is the person who was issued the visa. In addition, at certain
    sites, visitors are required to confirm their departure by undergoing
    US-VISIT exit procedures-that is, having their visas or passports scanned
    and undergoing fingerscanning. The exit confirmation is added to the
    visitor's travel records to demonstrate compliance with the terms of
    admission to the United States. (App. III provides a detailed description
    of the pre-entry, entry, status, exit, and analysis processes.)

    Key US-VISIT functions include

    o collecting, maintaining, and sharing information on certain foreign
    nationals who enter and exit the United States;

    o identifying foreign nationals who (1) have overstayed or violated the
    terms of their admission; (2) may be eligible to receive, extend, or
    adjust their immigration status; or (3) should be apprehended or detained
    by law enforcement officials;

    o detecting fraudulent travel documents, verifying traveler identity, and
    determining traveler admissibility through the use of biometrics; and

    o facilitating information sharing and coordination within the immigration
    and border management community.

    In July 2003, DHS established a program office with responsibility for
    managing the acquisition, deployment, operation, and sustainment of the
    US-VISIT system and its associated supporting people (e.g., Customs and
    Border Protection (CBP) officers), processes (e.g., entry/exit policies
    and procedures), and facilities (e.g., inspection booths and lanes), in
    coordination with its stakeholders (CBP and the Department of State).

    As of October 2005, about $1.4 billion has been appropriated for the
    program, and, according to program officials, about $962 million has been
    obligated.

    Acquisition and Implementation Strategy: A Brief Description

    DHS plans to deliver US-VISIT capability in four increments, with
    Increments 1 through 3 being interim, or temporary, solutions that fulfill
    legislative mandates to deploy an entry/exit system, and Increment 4 being
    the implementation of a long-term vision that is to incorporate improved
    business processes, new technology, and information sharing to create an
    integrated border management system for the future. In Increments 1
    through 3, the program is building interfaces among existing ("legacy")
    systems; enhancing the capabilities of these systems; and deploying these
    capabilities to air, sea, and land POEs. These increments are to be
    largely acquired and implemented through existing system contracts and
    task orders.

    In May 2004, DHS awarded an indefinite-delivery/indefinite-quantity5 prime
    contract to Accenture and its partners. According to the contract, the
    prime contractor will help support the integration and consolidation of
    processes, functionality, and data, and it will develop a strategy to
    build on the technology and capabilities already available to produce the
    strategic solution, while also assisting the program office in leveraging
    existing systems and contractors in deploying the interim solutions.

    US-VISIT Is Being Implemented in Four Increments

    Increment 1 concentrates on establishing capabilities at air and sea POEs.
    It is divided into two parts-1 and 1B.

    o Increment 1 (air and sea entry) includes the electronic capture and
    matching of biographic and biometric information (two digital index
    fingerscans and a digital photograph) for selected foreign nationals,
    including those from visa waiver countries.6 Increment 1 was deployed on
    January 5, 2004, for individuals requiring a nonimmigrant visa to enter
    the United States, through the modification of pre-existing systems.7
    These modifications accommodated the collection and maintenance of
    additional data fields and established interfaces required to share data
    among DHS systems in support of entry processing at 115 airports and 14
    seaports.

    o Increment 1B (air and sea exit) involves the testing of exit devices to
    collect biometric exit data for select foreign nationals at 11 airports
    and seaports. Three exit alternatives were pilot tested:

    o Kiosk-A self-service device (which includes a touch-screen interface,
    document scanner, finger scanner, digital camera, and receipt printer)
    that captures a digital photograph and fingerprint and prints out an
    encoded  receipt.

    o Mobile device-A hand-held device that is operated by a workstation
    attendant;8 it includes a document scanner, finger scanner, digital
    camera, and receipt printer and is used to capture a digital photograph
    and fingerprint.

    o Validator-A hand-held device that is used to capture a digital
    photograph and fingerprint, which are then matched to the photograph and
    fingerprint captured via the kiosk and encoded in the receipt.

    Increment 2 focuses primarily on extending US-VISIT to land POEs. It is
    divided into three parts-2A, 2B, and 2C.

    o Increment 2A (air, sea, and land) includes the capability to
    biometrically compare and authenticate valid machine-readable visas and
    other travel and entry documents issued by State and DHS to foreign
    nationals at all POEs. Increment 2A was deployed on October 23, 2005,
    according to program officials. It also includes the deployment by October
    26, 2006, of technology to read biometrically enabled passports from visa
    waiver countries.

    o Increment 2B (land entry) redesigns the Increment 1 entry solution and
    expands it to the 50 busiest land POEs. The process for issuing Form I-949
    was redesigned to enable the electronic capture of biographic, biometric
    (unless the traveler is exempt), and related travel documentation for
    arriving travelers. This increment was deployed to the busiest 50 U.S.
    land border POEs as of December 29, 2004. Before Increment 2B, all
    information on the Form I-94s was handwritten. The redesigned systems
    electronically capture the biographic data included in the travel
    document. In some cases, the form is completed by CBP officers, who enter
    the data electronically and then print the form.

    o Increment 2C is to provide the capability to automatically, passively,
    and remotely record the entry and exit of covered individuals using radio
    frequency (RF) technology tags at primary inspection and exit lanes.10 An
    RF tag that includes a unique ID number is to be embedded in each Form
    I-94, thus associating a unique number with a record in the US-VISIT
    system for the person holding that Form I-94. In August 2005, the program
    office deployed the technology to five border crossings (three POEs) to
    verify the feasibility of using passive RF technology to record traveler
    entries and exits via a unique ID number embedded in the CBP Form I-94.
    The results of this demonstration are to be reported in February 2006.

    Increment 3 extended Increment 2B (land entry) capabilities to 104 land
    POEs; this increment was essentially completed as of December 19, 2005.11

    Increment 4 is the strategic US-VISIT program capability, which program
    officials stated will likely consist of a further series of incremental
    releases or mission capability enhancements that will support business
    outcomes. The program reports that it has  worked with its prime
    contractor and partners to develop this overall vision for the immigration
    and border management enterprise.

    Increments 1 through 3 include the interfacing and integration of existing
    systems and, with Increment 2C, the creation of a new system, the
    Automated Identification Management System (AIDMS). The three main
    existing systems are as follows:

    o The Arrival Departure Information System (ADIS) stores

    o noncitizen traveler arrival and departure data received from air and sea
    carrier manifests,

    o arrival data captured by CBP officers at air and sea POEs,

    o Form I-94 issuance data captured by CBP officers at Increment 2B land
    POEs,

    o departure information captured at US-VISIT biometric departure pilot
    (air and sea) locations,

    o pedestrian arrival information and pedestrian and vehicle departure
    information captured at Increment 2C POE locations, and

    o status update information provided by the Student and Exchange Visitor
    Information System (SEVIS) and the Computer Linked Application Information
    Management System (CLAIMS 3) (described below).

    ADIS provides record matching, query, and reporting functions.

    o The passenger processing component of the Treasury Enforcement
    Communications System (TECS) includes two systems: Advance Passenger
    Information System (APIS), a system that captures arrival and departure
    manifest information provided by air and sea carriers, and the Interagency
    Border Inspection System, a system that maintains lookout data and
    interfaces with other agencies' databases. CBP officers use these data as
    part of the admission process. The results of the admission decision are
    recorded in TECS and ADIS.

    o The Automated Biometric Identification System (IDENT) collects and
    stores biometric data on foreign visitors.

    US-VISIT also exchanges biographic information with other DHS systems,
    including SEVIS and CLAIMS 3. These two systems contain information on
    foreign students and foreign nationals who request benefits, such as a
    change of status or extension of stay.

    Some of the systems previously described, such as IDENT and the new AIDMS,
    are managed by the program office, while some systems are managed  by
    other organizational entities within DHS. For example, TECS is managed by
    CBP, SEVIS is managed by Immigration and Customs Enforcement, CLAIMS 3 is
    under United States Citizenship and Immigration Services, and ADIS is
    jointly managed by CBP and US-VISIT.

    US-VISIT also interfaces with other, non-DHS systems for relevant
    purposes, including watch list updates and checks to determine whether a
    visa applicant has previously applied for a visa or currently has a valid
    U.S. visa. In particular, US-VISIT receives biographic and biometric
    information from State's Consular Consolidated Database as part of the
    visa application process, and returns fingerscan information and watch
    list changes.

    Program Management Roles and Responsibilities

    The US-VISIT program office structure includes nine component offices.
    Each of the program offices includes a director and subordinate
    organizational units, as established by the director. The responsibilities
    for each office are stated below. Figure 1 shows the program office
    structure, including its nine offices.

    Figure 1: US-VISIT Program Office Structure

    The roles and responsibilities for each of the nine offices include the
    following:

    o Chief Strategist is responsible for developing and maintaining the
    strategic vision, strategic documentation, transition plan, and business
    case.

    o Budget and Financial Management is responsible for establishing the
    program's costs estimates; analysis; and expenditure management policies,
    processes, and procedures that are required to implement and support the
    program by ensuring proper fiscal planning and execution of the budget and
    expenditures.

    o Mission Operations Management is responsible for developing business and
    operational requirements based on strategic direction provided by the
    Office of the Chief Strategist.

    o Outreach Management is responsible for enhancing awareness of US-VISIT
    requirements among foreign nationals, key domestic audiences, and internal
    stakeholders by coordinating outreach to media, third parties, key
    influencers, Members of Congress, and the traveling public.

    o Information Technology Management is responsible for developing
    technical requirements based on strategic direction provided by the Office
    of the Chief Strategist and business requirements developed by the Office
    of Mission Operations Management.

    o Implementation Management is responsible for developing accurate,
    measurable schedules and cost estimates for the delivery of mission
    systems and capabilities.

    o Acquisition and Program Management is responsible for establishing and
    managing the execution of program acquisition and management policies,
    plans, processes, and procedures.

    o Administration and Training is responsible for developing and
    administering a human capital plan that includes recruiting, hiring,
    training, and retaining a diverse workforce with the competencies
    necessary to accomplish the mission.

    o Facilities and Engineering Management is responsible for establishing
    facilities and environmental policies, procedures, processes, and guidance
    required to implement and support the program office.

    Our Prior Work Has Resulted in Several Recommendations

    In response to legislative mandate, we have issued four reports on DHS's
    annual expenditure plans for US-VISIT.12 Our reports have, among other
    things, assessed whether the plans satisfied the legislative conditions
    and provided observations on the plans and DHS's program management. As a
    result of our assessments, we made 24 recommendations aimed at improving
    both plans and program management, all of which DHS has agreed to
    implement. Of these 24 recommendations, 18 address risks  stemming from
    program management.13

    The Status of DHS's Implementation of Our Recommendations Is Mixed

    The current status of DHS's implementation of our 18 recommendations on
    program risks is mixed, but progress in critical areas has been slow. For
    example, over 2 years have passed, and the program office has yet to
    develop a security plan consistent with federal guidance or to
    economically justify its investment in system increments.  According to
    the Program Director, the pace of progress is attributable to competing
    demands on time and resources.

    DHS agreed to implement all 18 recommendations. Of these 18, DHS has
    completely implemented 2, has partially implemented 11, and is in the
    process of implementing another 5. Of the 11 that are partially
    implemented, 7 are about 2 years old, and 4 are about 10 to 19 months old.
    Of the 5 that are in progress, 3 are about 10 months old.

    These 18 recommendations are aimed at strengthening the program's
    management effectiveness. The longer that the program takes to implement
    the recommendations, the greater the risk that the program will not meet
    its goals on time and within budget.

    Figure 2 provides an overview of the extent to which each recommendation
    has been implemented.  The figure is followed by sections providing
    details on each recommendation and our assessment of its implementation
    status.

    Figure 2: DHS's Progress toward Implementing GAO's 18 Recommendations

    aA recommendation is completely implemented when documentation
    demonstrated that it had been fully addressed.

    bA recommendation is partially implemented when documentation indicated
    that actions were under way to implement it.

    cA recommendation is in progress when documentation indicated that actions
    had been initiated to implement it.

    dCarnegie Mellon University Software Engineering Institute, Software
    Acquisition Capability Maturity Model, Version 1.03 (March 2002).

    eAutomated Commercial Environment is a new trade processing system planned
    to support the movement of legitimate imports and exports and to
    strengthen border security.

    Development and Implementation of a Security Plan and Performance of a
    Privacy Impact Assessment Are Partially Complete

    In June 2003,14 we reported that the Immigration and Naturalization
    Service15 had not developed a security plan and performed a privacy impact
    assessment for the entry exit program (as US-VISIT was then known). A
    security plan and privacy impact assessment are important to understanding
    system requirements and ensuring that the proper safeguards are in place
    to protect system data and resources. System acquisition best practices
    and federal guidance advocate understanding and defining security and
    privacy requirements both early and continuously in a system's life cycle,
    and effectively planning for their satisfaction. Accordingly, we
    recommended that DHS do the following:

    Develop and begin implementing a system security plan, and perform a
    privacy impact assessment and use the results of the analysis in near-term
    and subsequent system acquisition decision making.

    Security Plan

    Since we made the system security plan recommendation about 2 1/2 years
    ago, its implementation has been slow. For example, we reported in
    September 2003 and again in May 2004 that the program office had not
    developed a security plan. In February 2005, we reported that the program
    office had developed a security plan, dated September 2004, and that this
    plan was generally consistent with federal guidance.16 That is, the plan
    provided an overview of system security requirements, described the
    controls in place or planned for meeting those requirements, referred to
    the applicable documents that prescribe the roles and responsibilities for
    managing the US-VISIT component systems, and addressed security awareness
    and training. However, the program office had not conducted a risk
    assessment or included in the plan when an assessment would be completed.
    According to guidance from the Office of Management and Budget (OMB), the
    security plan should describe the methodology that is used to identify
    system threats and vulnerabilities and to assess risks, and it should
    include the date the risk assessment was completed.

    According to program officials, they completed a programwide risk
    assessment in December 2005, but have yet to provide a copy of the
    assessment to us. Therefore, we cannot confirm that the assessment has
    been done, and done properly. The absence of a risk assessment and a
    security plan that reflects this assessment is a significant program
    weakness. Risk assessments are critical to establishing effective security
    controls because they provide the basis for establishing appropriate
    policies and selecting cost-effective controls to implement these
    policies. Without such an assessment, US-VISIT does not have adequate
    assurance that it knows the risks associated with the program and thus
    whether it has implemented effective controls to address them.

    Notwithstanding these limitations in the security plan, the program office
    has begun to implement aspects of its September 2004 security plan. For
    example, the Information Systems Security Manager told us that a security
    awareness program is established and key personnel have attended security
    training.

    Privacy Impact Assessment

    Since June 2003, US-VISIT has also developed and periodically updated a
    privacy impact assessment. An initial impact assessment was issued in
    January 2004, and a revised assessment was issued in September 2004.17 A
    more recent assessment, dated July 2005, reflects changes related to
    Increments 1B and 2C. Each of these assessments is generally consistent
    with OMB guidance.18 That is, each of the assessments addressed most OMB
    requirements, including the impact that the system will have on individual
    privacy, the privacy consequences of collecting the information, and
    alternatives considered to collect and handle information. The most recent
    impact assessment, for example, states that three alternatives were
    considered for Increment 1B-the kiosk, the mobile device, and the
    validator (a combination of the two)-and discusses proposals to mitigate
    the privacy risks of all three, such as by limiting the duration of data
    retention on the exit devices and using encryption.

    However, OMB guidance also requires that privacy impact assessments
    developed for systems under development address privacy in relevant system
    documentation, including statements of need, functional requirements
    documents, and cost-benefit analyses. As we reported about previous
    privacy impact assessments, privacy is only partially addressed in system
    documentation. For example, the Increment 1B cost-benefit analysis
    assesses the privacy risk associated with each exit alternative, and the
    Increment 2C business requirements state that all solutions are to be
    compliant with privacy laws and regulations and adhere to US-VISIT privacy
    policy. However, we did not find privacy in the Increment 1B business
    requirements or the Increment 2C functional requirements. Program
    officials, including the US-VISIT Privacy Officer, acknowledged that
    privacy is not included in the system documentation, but stated that
    privacy is considered in the development of the documentation and that the
    privacy office reviews key system documentation at relevant times during
    the system development life cycle. Nevertheless, we did not find evidence
    of privacy being addressed in the system documentation, and program
    officials acknowledged that it was not included.

    Until the program performs a risk assessment and fully implements a
    security plan that reflects this assessment, it cannot adequately ensure
    that US-VISIT is cost-effectively safeguarding assets and data. Moreover,
    without reflecting privacy in system documentation, it cannot adequately
    ensure that privacy needs are being fully addressed.

    Development and Implementation of Key Acquisition Controls Are Partially
    Complete

    We reported in September 200319 that the program office had not defined
    key acquisition management controls to support the acquisition of
    US-VISIT, and therefore its efforts to acquire, deploy, operate, and
    maintain system capabilities were at risk of not satisfying system
    requirements and of not meeting benefit expectations on time and within
    budget.

    The Capability Maturity Model-Integration(R) (CMMI) developed by Carnegie
    Mellon University's Software Engineering Institute (SEI) explicitly
    defines process management controls that are recognized hallmarks of
    successful organizations and that, if implemented effectively, can greatly
    increase the chances of successfully acquiring software-intensive
    systems.20 SEI's CMMI model uses capability levels to assess process
    maturity.21 Because establishing the basic  acquisition process
    capabilities, according to SEI, can take on average about 19 months, we
    recognized the importance of starting early to build effective acquisition
    management capabilities by recommending that DHS do the following:

    Develop and implement a plan for satisfying key acquisition management
    controls, including acquisition planning, solicitation, requirements
    management, program management, contract tracking and oversight,
    evaluation, and transition to support, and implement the controls in
    accordance with SEI guidance.

    The program office has recently taken foundational steps to establish key
    acquisition management controls. For example, it has developed a process
    improvement plan, dated May 16, 2005 (about 20 months after our
    recommendation), to define and implement these controls. As part of its
    improvement program, the program office is implementing a governance
    structure for overseeing improvement activities, consisting of three
    groups: a Management Steering Group, an Enterprise Process Group, and
    Process Action Teams. Specific roles for each of these groups are
    described below.

    o The Management Steering Group is to provide policy and procedural
    guidance and to oversee the entire improvement program. The steering group
    is chaired by the US-VISIT Director, with the Deputy Director and the
    functional office directors serving as core members.

    o The Enterprise Process Group is to provide planning, management, and
    operational guidance in day-to-day process improvement activities. The
    group is chaired by the process improvement leader and is composed of
    individuals from each functional office.

    o Process Action Teams are to provide specific process documentation and
    to provide implementation support and training services. These teams are
    to be active as long as a particular process improvement initiative is
    under way. To date, the program office has chartered five process
    teams-configuration management, cost analysis, process development,
    communications, and policy.

    In addition, the program office has recently completed a self-assessment
    of its acquisition process maturity, and it plans to use the assessment
    results to establish a baseline of its acquisition process maturity for
    improvement. According to program officials, the assessment included 13
    key process areas that are generally consistent with the process areas
    cited in our recommendation. The program has ranked these 13 process areas
    according to their priority, and, for initial implementation, it plans to
    focus on the following 6:22

    o Configuration management. Establishing and maintaining the integrity of
    the products throughout their life cycle.

    o Process and product quality assurance. Taking actions to provide
    management with objective insight into the quality of products and
    processes.

    o Project monitoring and control. Tracking the project's progress so that
    appropriate corrective actions can be taken when performance deviates
    significantly from plans.

    o Project planning. Establishing and maintaining plans for work
    activities.

    o Requirements management. Managing the requirements and ensuring a common
    understanding of the requirements between the customer and the product
    developers.

    o Risk management. Identifying potential problems before they occur so
    that they can be mitigated to minimize any adverse impact.

    The improvement plan is currently being updated to reflect the results of
    the baseline assessment and to include a detailed work breakdown
    structure, process prioritization, and resource estimates. According to
    the Director, Acquisition and Program Management Office (APMO),  the goal
    is to conduct a formal SEI appraisal to assess the capability level of
    some or all of the six processes by October 2006.

    Notwithstanding the recent steps to begin addressing our recommendation,
    much work remains to fully implement key acquisition management controls.
    Moreover, effectively implementing these controls takes considerable time.
    Therefore, it is important that these improvement efforts stay on track.
    Until these processes are effectively implemented, US-VISIT will be at
    risk of not delivering promised capabilities on time and within budget.

    Determination and Disclosure of Whether Increments Produce Mission Value
    Commensurate with Costs and Risks Are Partially Complete

    In September 2003, we reported that the program had not assessed the costs
    and benefits of Increment 1, which is extremely important because the
    decision to invest in any capability should be based on reliable analyses
    of return on investment. Further, according to OMB guidance, individual
    increments of major systems are to be individually supported by analyses
    of benefits, cost, and risk.23 Without reliable analyses, an organization
    cannot adequately know that a proposed investment is a prudent and
    justified use of limited resources. Accordingly, we recommended that DHS
    do the following:

    Determine whether proposed US-VISIT increments will produce mission value
    commensurate with cost and risks and disclose to the Congress planned
    actions.

    As we reported in September 2003 and again in February 2005,24 the program
    office did not justify its planned investment in Increments 1 and 2B,
    respectively, based on expected return on investment. Since then, the
    program has developed a cost-benefit analysis for Increment 1B.

    OMB has issued guidance concerning the analysis needed to justify
    investments.25 According to this guidance, such analyses should meet
    certain criteria to be considered reasonable. These criteria include,
    among other things, comparing alternatives on the basis of net present
    value and conducting uncertainty analyses of costs and benefits. DHS has
    also issued guidance on such economic analyses that is consistent with
    that of OMB.26

    The latest cost-benefit analysis for Increment 1B (dated June 23, 2005)
    identifies potential costs and benefits for three exit solutions at air
    and sea POEs and provides a general rationale for the viability of the
    three alternatives described. This latest analysis meets four of eight OMB
    economic analysis criteria. However, it does not, for example, include a
    complete uncertainty analysis (i.e., both a sensitivity analysis and a
    Monte Carlo simulation27) for the three exit alternatives evaluated. That
    is, the cost-benefit analysis does include a Monte Carlo simulation,  but
    it does not include a sensitivity analysis for the three alternatives. An
    analysis of uncertainty is important because it provides decision makers
    with a perspective on the potential variability of the cost and benefit
    estimates should the facts, circumstances, and assumptions change.

    Table 1 summarizes our analysis of the extent to which US-VISIT's June 23,
    2005, cost-benefit analysis for Increment 1B satisfies eight OMB criteria.

    Table 1: US-VISIT Satisfaction of OMB Economic Analysis Criteria

           Criterion         Explanation      Criterion       GAO analysis
                                                met?
    1. The            The analysis should  Yes       The analysis identifies
    cost-benefit      clearly explain the            the need for the
    analysis clearly  reason why the                 investment and identifies
    explained why the investment is                  eight key business
    investment was    needed, that is, why           objectives of the
    needed.           the status quo is              Increment 1B exit
                         unacceptable.                  solution.
    2. At least two   At least two         Yes       The analysis considers
    alternatives to   meaningful                     three alternatives for
    the status quo    alternatives to the            the Increment 1B exit
    were considered.  status quo should be           solution: kiosk, mobile,
                         examined to help               and validator.
                         ensure that the
                         alternative chosen
                         was not preselected.
    3. The general    The general          Yes       The assessment includes
    rationale for the rationale for the              the rationale for the
    cost-benefit      inclusion of each              judgment that the three
    analysis,         alternative                    exit alternatives were
    including each    considered should be           viable options.
    alternative, was  discussed to enable
    discussed.        reviewers of the
                         analysis to gain an
                         understanding of the
                         context for the
                         selection of one
                         alternative over the
                         others.
    4. The quality of The quality of the   No        The cost estimates are
    the cost estimate cost estimate for              not complete or reliably
    for each          each alternative               derived. (See later
    alternative was   should be complete             section of this report
    reasonable.       and reasonable for a           for detailed analysis.)
                         net present value to
                         be accurate.
    5. The quality of The quality of the   No        Year-by-year benefit
    the benefits to   benefit estimate for           estimates were not
    be realized from  each alternative               reported.
    each alternative  should be complete
    was reasonable.   and reasonable for a
                         net present value to
                         be calculable and
                         accurate. According
                         to OMB Circular
                         A-94,a year-by-year
                         estimates should be
                         reported to promote
                         independent analysis
                         and review of those
                         estimates.
    6. Alternatives   The net present      Yes       Net present values were
    were compared on  value should be                calculated for the three
    the basis of net  calculated because             alternatives. However,
    present value.    it consistently                the preferred alternative
                         allows for the                 could not be selected on
                         selection of the               this basis, in part
                         alternative with the           because the estimated net
                         greatest benefit net           present value for all
                         of cost.                       alternatives was
                                                        negative. OMB guidance
                                                        presumes that at least
                                                        one will be positive, and
                                                        that the selected
                                                        alternative will have the
                                                        greatest total benefit
                                                        net of total cost. The
                                                        alternative with the more
                                                        favorable cost-benefit
                                                        was identified on the
                                                        basis of its lower labor
                                                        intensity (resulting in
                                                        lower operating and
                                                        maintenance costs) and
                                                        lower risk that
                                                        personally identifiable
                                                        information would be
                                                        compromised.
    7. The proper     OMB Circular A-94    No        The analysis does not
    discount rate for provides specific              explicitly state the
    calculating each  guidance on the                numerical value of the
    alternative's net choice of discount             discount rate used for
    present value     rate for evaluating            computing the
    should be used.   projects whose                 alternatives' net present
                         benefits and costs             values.
                         will be distributed
                         over time.
    8. A complete     Estimates of costs   No        Although the cost-benefit
    uncertainty       and benefits are               analysis did include
    analysis of cost  typically uncertain            Monte Carlo simulation
    and benefit was   because of                     results for the three
    included.         imprecision in both            exit alternatives, no
                         underlying data and            sensitivity analysis was
                         modeling                       conducted for those
                         assumptions. Because           alternatives. Instead,
                         such uncertainty is            the cost-benefit analysis
                         basic to virtually             reports sensitivity
                         any cost-benefit               analysis results for the
                         analysis, its                  five deployment
                         effects should be              scenarios.
                         analyzed and
                         reported. OMB
                         guidance recommends
                         both Monte Carlo
                         simulation and
                         sensitivity analysis
                         as uncertainty
                         analysis techniques.

    Source: GAO.

    aOMB's Circular A-94 is the general guidance for conducting cost-benefit
    analyses for the federal government.

    It is important that the program adhere to relevant guidance in developing
    its incremental cost-benefit analyses. If this is not done,  the
    reliability of the analyses is diminished, and an adequate basis for
    prudent investment decision making does not exist. Moreover, if the
    mission value of a proposed investment is not commensurate with costs, it
    is vital that this information be fully disclosed to DHS and congressional
    decision makers. The underlying intent of our recommendation is that this
    information be available to inform such decisions.

    Definition of the Operational Context for US-VISIT Is in Progress

    In September 2003, we reported that key aspects of the larger homeland
    security environment in which US-VISIT would need to operate had not been
    defined. For example, we stated that certain policy and standards
    decisions had not been made (e.g., whether official travel documents will
    be required for all persons who enter and exit the country, including U.S.
    and Canadian citizens, and how many fingerprints are to be collected). In
    the absence of this operational context, program officials were making
    assumptions and decisions that, if they proved inconsistent with
    subsequent policy or standards decisions, would require US-VISIT rework.
    To minimize the impact of these changes, we recommended that DHS do the
    following:

    Clarify the operational context in which US-VISIT is to operate.

    After about 27 months, defining this operational context remains a work in
    progress. According to the Chief Strategist, an immigration and border
    management strategic plan was drafted in March 2005 that shows how
    US-VISIT is aligned with DHS's organizational mission and defines an
    overall vision for immigration and border management. This official stated
    that this vision provides for an immigration and border management
    enterprise that unifies multiple internal departmental and other external
    stakeholders with common objectives, strategies, processes, and
    infrastructures.

    Since the plan was drafted, DHS has reported that other relevant
    initiatives have been undertaken, such as the Security and Prosperity
    Partnership of North America and the Secure Border Initiative. The
    Security and Prosperity Partnership is to, among other things, establish a
    common approach to securing the countries of North America- the United
    States, Canada, and Mexico-by, for example, implementing a border
    facilitation strategy to build capacity and improve the legitimate flow of
    people and cargo at our shared borders. The Secure Border Initiative is to
    implement a comprehensive approach to securing our borders and reducing
    illegal immigration. According to the Chief Strategist, while portions of
    the strategic plan are being incorporated into these initiatives, these
    initiatives and their relationship with US-VISIT are still being defined.
    We have yet to receive the US-VISIT strategic plan because, according to
    program officials, it had not yet been approved by DHS management.

    Until US-VISIT's operational context is fully defined, DHS is increasing
    its risk of defining, establishing, and implementing a program that is
    duplicative of other programs and not interoperable with them. This in
    turn will require rework to address these areas. While this issue was
    significant 27 months ago, when we made the recommendation, it is still
    more significant now.

    Provision of Program Office Resources Is Partially Complete

    We reported in September 2003 that the program had not fully staffed its
    program office. Our prior experience with major acquisitions like US-VISIT
    shows that to be successful, they need, among other things, to have
    adequate resources. Accordingly, we recommended that DHS do the following:

    Ensure that human capital and financial resources are provided to
    establish a fully functional and effective program office.

    About 2 years later, US-VISIT had filled 102 of its 115 planned government
    positions and all of its planned 117 contractor positions. For the
    remaining 13 government positions, 5 positions had been selected (pending
    completion of security clearances), and recruitment action was in process
    for filling the remaining 8 vacancies. According to the Office of
    Administration and Training Manager, funding is available to complete the
    hiring of all 115 government employees.

    Notwithstanding this progress, in February 2005, US-VISIT completed a
    workforce analysis and requested additional positions based on the
    results. According to program officials, a revised analysis was submitted
    in the summer of 2005, but the request has not yet been approved. Figure 3
    shows the program office organization structure and functions and how many
    of the 115 positions needed have been filled.

    Figure 3: Summary of Program Office Structure, Functions, and Filled and
    Vacant Positions

    Securing necessary resources will be a continuing challenge and an
    essential ingredient to the program's ability to acquire, deploy, operate,
    and maintain system capabilities on time and within budget.

    Definition of Program Office Roles and Responsibilities Has Been Completed

    We reported in September 2003 that the program had not defined specific
    roles and responsibilities for its staff. Our prior experience and leading
    practices show that for major acquisitions like US-VISIT to be successful,
    program staff need, among other things, to understand what they are to do,
    how they relate to each other, and how they fit in their organization.
    Accordingly, we recommended that DHS do the following:

    Define program office positions, roles, and responsibilities.

    The program office has developed charters for its nine component offices
    that include roles and responsibilities for each. For example, the
    Acquisition and Program Management Office is responsible, among other
    things, for establishing acquisition and program management policies;
    coordinating development of configuration management plans and project
    schedules, including the integrated milestone schedule; and developing
    policies and procedures for guidance and oversight of systems development
    and implementation activities. The program has also defined a set of core
    competencies (knowledge, skills, and abilities) for each position. For
    example, it has defined critical competencies for program and management
    analysts that include, among others, flexibility, interpersonal skills,
    organizational awareness, oral communication, problem solving, and
    teamwork.

    These efforts to define position, roles, and responsibilities should help
    in managing the program effectively.

    Development and Implementation of a Human Capital Strategy Are Partially
    Complete

    As previously stated, we reported in September 2003 that US-VISIT had not
    fully staffed its program office or defined roles and responsibilities for
    its program staff. We observed that prior research and evaluations of
    organizations showed that effective human capital management can help
    agencies establish and maintain the workforce they need to accomplish
    their missions. Accordingly, we recommended that DHS do the following:

    Develop and implement a human capital strategy for the program office that
    provides for staffing positions with individuals who have the appropriate
    knowledge, skills, and abilities.

    In February 2005, we reported that the program office, in conjunction with
    the Office of Personnel Management (OPM), developed a draft human capital
    plan that employed widely accepted human capital planning tools and
    principles. The draft plan included, for example, an action plan that
    identified activities, proposed completion dates, and the office (OPM or
    the program office) responsible for the action. We also reported that the
    program office had completed some of the activities, such as designating a
    liaison responsible for ensuring alignment between departmental and
    program human capital policies.

    Since then, the program office has finalized the human capital plan and
    completed more activities. For example, program officials told us that
    they have

    o analyzed the program office's workforce to determine diversity trends,
    retirement and attrition rates, and mission-critical and leadership
    competency gaps;

    o updated the program's core competency requirements to ensure alignment
    between the program's human capital and business needs;

    o developed an orientation program for new employees; and

    o administered competency assessments to incoming employees.

    Program officials also told us that they have plans to complete other
    activities, such as

    o developing a staffing forecast to inform succession planning;

    o analyzing workforce data to maintain strategic focus on preserving the
    skills, knowledge, and leadership abilities required for the US-VISIT
    program's success; and

    o developing organizational leadership competency models for the program's
    senior executive, managerial, and supervisory levels.

    In addition, the officials said that several activities in the plan have
    not been completed, such as assessing the extent of any current employees'
    competency gaps and developing a competency-based listing of training
    courses. These officials said that the reason these activities have not
    been completed is that they are related to the department's new human
    capital initiative, MAXHR, which is to provide greater flexibility and
    accountability in the way employees are paid, developed, evaluated,
    afforded due process, and represented by labor organizations. MAXHR is to
    include the development of departmentwide competencies. Because of this,
    the officials told us that it could potentially impact the program's
    ongoing competency-related activities. As a result, these officials said
    that they are coordinating  these activities closely with the department
    as it develops and implements this new initiative, which is currently
    being reviewed by the DHS Deputy Secretary for approval.

    Until US-VISIT fully implements a comprehensive human capital strategy, it
    will continue to risk not having staff with the right skills and abilities
    to successfully execute the program.

    Defining Performance Standards for US-VISIT Increments Is Partially
    Complete

    We reported in September 2003 that the operational performance of initial
    system increments was largely dependent on the performance of existing
    systems that were to be interfaced to create these increments. For
    example, we said that the performance of an increment will be constrained
    by the availability and downtime of the existing systems that it includes.
    Accordingly, we recommended that DHS do the following:

    Define performance standards for each increment that are measurable and
    reflect the limitations imposed by relying on existing systems.

    In February 2005 (17 months later), we reported that several technical
    performance standards for Increments 1 and 2B had been defined, but that
    it was not clear that these standards reflected the limitations imposed by
    the reliance on existing systems. Since then, for the Increment 2C Proof
    of Concept (Phase 1), the program office has defined certain other
    performance standards. For example, the functional requirements document
    for Increment 2C (Phase 1) defines several technical performance
    standards, including reliability, recoverability, and availability. For
    each, the document states that the performance standard is largely
    dependent on those of Increment 2B. More specifically, the document states
    that Phase 1 system availability is largely dependent upon the individual
    and collective availability of the current systems. The document also
    states that the Increment 2C components shall have an aggregated
    availability greater than or equal to 97.5 percent. However, the document
    does not contain sufficient information to determine whether these
    performance standards actually reflect the limitations imposed by reliance
    on existing systems.

    To further develop performance standards, the program office has prepared
    a Performance Engineering Plan, dated March 31, 2005, that links US-VISIT
    performance engineering activities to its System Development Life Cycle.
    Further, the plan (1) provides a framework to be used to align its
    business, application, and infrastructure performance goals and measures;
    (2) describes an approach to translate business goals into operational
    measures, and then to quantitative metrics; and (3) identifies system
    performance measurement areas (effectiveness, efficiency, reliability, and
    availability). According to program officials, they intend to establish a
    group to develop action plans for implementing the engineering plan, but
    did not have a time frame for doing so.

    Without defining performance standards that reflect the limitations of the
    existing systems upon which US-VISIT relies, the program lacks the ability
    to identify and effectively address performance shortfalls.

    Development and Implementation of a Risk Management Plan Are Partially
    Complete

    In September 2003, we reported that US-VISIT was a risky undertaking
    because of several factors inherent to the program, such as its large
    scope and complexity, as well as because of various program management
    weaknesses. We concluded that these risks, if not effectively managed,
    would likely cause program cost, schedule, and performance problems.

    Risk management is a continuous, forward-looking process that is intended
    either to prevent such problems from occurring or to minimize their impact
    if they occur by proactively identifying risks, implementing risk
    mitigation strategies, and measuring and disclosing progress in doing so.
    Because of the importance of effectively managing program risks, we
    recommended that DHS do the following:

    Develop and implement a risk management plan and ensure that all high
    risks and their status are reported regularly to the executive body.

    About 2 years later, the program office has developed and has begun
    implementing a risk management plan. The plan, which was approved in
    September 2005,  includes, among other things, a process for identifying,
    analyzing, handling, and monitoring risk. It also defines the governance
    structure to be used in overseeing and managing the process. The program
    also maintains a risk database, which includes, among other things, a
    description of the risk, its priority (e.g., high, medium, or low), and
    its mitigation strategy. According to program officials, the database is
    currently available to program management and staff.

    The program has also begun implementing its risk management plan. For
    example, it has established a Risk Review Board, Risk Review Council, and
    Risk Owners to govern its risk activities. The roles and responsibilities
    are described below.

    o The Risk Review Board directs all risk governance within the program and
    provides the mechanism to escalate/transfer the consideration of risks to
    program governing boards and to organizations external to the program.

    o The Risk Review Council oversees and manages program-related risks that
    are significant, controversial, or cross-project or that may require
    escalation to the Risk Review Board.

    o Risk Owners analyze, handle, and monitor risks.

    However, full implementation of the risk management plan has yet to occur.
    As part of its CMMI process maturity baseline self-assessment (previously
    discussed), the program office found that the risk management process
    detailed in its plan was not being consistently applied across the
    program. In response, according to program officials, they have developed
    risk management training and began conducting training sessions in
    November 2005. These officials also stated that the Risk Review Board,
    where risks are reviewed with program executives, has been meeting monthly
    since September 2005.

    With respect to regular risk reports to program executives, the plan
    includes thresholds for escalating risks within the risk governance
    structure and to DHS governance entities. For example, risks are to be
    elevated to the Risk Review Board when the cost of the project exceeds
    more than 5 percent of the project baseline cost, the schedule slippage
    exceeds more than 5 percent of the baseline schedule, major areas of scope
    are affected, or quality reduction requires approval. However, program
    officials stated that these thresholds are not currently being applied.
    They further stated that although the plan allows for escalation of risks
    to officials outside the program office, doing so is at the discretion of
    the Program Director; in addition, according to these officials, although
    high risks are not routinely escalated outside the program, selected high
    risks have been disclosed to the Assistant Secretary for Policy in weekly
    program status reports. As of December 5, 2005, the Program Director
    proposed submitting monthly reports of high-priority risks and issues
    through the Assistant Secretary for Policy to the Deputy Secretary.

    Until US-VISIT fully implements its risk management plan and process, it
    cannot be assured that all program risks are being identified and managed
    in order to effectively mitigate any negative impact on the program's
    ability to deliver promised capabilities on time and within budget.

    Development of Test Plans Is Partially Complete

    We reported in May 2004, and again in February 2005, that system testing
    was not based on well-defined test plans, and thus the quality of testing
    being performed was at risk.28 The purpose of system testing is to
    identify and correct system defects (i.e., unmet system functional,
    performance, and interface requirements) and thereby obtain reasonable
    assurance that the system performs as specified before it is deployed and
    operationally used. To be effective, testing activities should be planned
    and implemented in a structured and disciplined fashion. Among other
    things, this includes developing effective test plans to guide the testing
    activities and ensuring that test plans are developed and approved before
    test execution. According to relevant systems development guidance, an
    effective test plan (1) specifies the test environment; (2) describes each
    test to be performed, including test controls, inputs, and expected
    outputs; (3) defines the test procedures to be followed in conducting the
    tests; and (4) provides traceability between the test cases and the
    requirements to be verified by the testing. Because these criteria were
    not being met, we recommended that DHS do the following:

    Develop and approve test plans before testing begins that (1) specify the
    test environment; (2) describe each test to be performed, including test
    controls, inputs, and expected outputs; (3) define the test procedures to
    be followed in conducting the tests; and (4) provide traceability between
    test cases and the requirements to be verified by the testing.

    About 19 months later, the quality of the system test plans, and thus
    system testing, is still problematic. To the program's credit, the test
    plans for the Increment 2C Proof of Concept (Phase 1), dated June 28,
    2005, satisfied part of our recommendation. Specifically, the test plan
    for this increment was approved on June 30, 2005, and, according to
    program officials, testing began on July 5, 2005. Further, the test plan
    described, for example, the scope, complexity, and completeness of the
    test environment, and it described the tests to be performed, including a
    high-level description of controls, inputs, and outputs, and it identified
    test procedures to be performed.

    However, the test plan did not adequately trace between test cases and the
    requirements to be verified by testing. For example, 300 of the 438
    functional requirements, or about 70 percent of the requirements that we
    analyzed, did not have specific references to test cases.

    In addition, we identified traceability inconsistencies, including the
    following:

    o One requirement was mapped to over 50 test cases, but none of the 50
    cases referenced the requirement.

    o One requirement was mapped to a group of test cases in the traceability
    matrix, but several of the test cases to which the requirement was mapped
    did not reference the requirement, and several test cases referenced the
    requirement and were not included in the traceability matrix.

    o One requirement was mapped to all but one of the test cases within a
    particular group of test cases, but that test case did refer to the
    requirement.

    Time and resources were identified as the reasons that test plans have not
    been complete. Specifically, program officials stated that milestones do
    not permit existing testing/quality personnel the time required to
    adequately review testing documents.29 According to these officials, even
    when the start of testing activities is delayed because, for example,
    requirements definition or product development takes longer than
    anticipated, testing milestones are not extended.

    Without complete test plans, the program does not have adequate assurance
    that the system is being fully tested, and thus unnecessarily assumes the
    risk that system defects will not be detected and addressed before the
    system is deployed. This means that the system may not perform as intended
    when deployed, and defects will not be addressed until late in the systems
    development cycle, when they are more difficult and time-consuming to fix.
    As we previously reported, this has happened: postdeployment system
    interface problems surfaced for Increment 1, and manual work-arounds had
    to be implemented after the system was deployed.

    Assessment of the Impact of Increment 2B on Workforce Levels and
    Facilities Is Partially Complete

    We reported in May 2004 that the program had not assessed its workforce
    and facility needs for Increment 2B. Because of this, we questioned the
    validity of the program's workforce and facility assumptions used to
    develop its workforce and facility plans, noting that the program lacked a
    basis for determining whether its assumptions and thus its plans were
    adequate. Accordingly, we recommended that DHS do the following:

    Assess the full impact of Increment 2B on land POE workforce levels and
    facilities, including performing appropriate modeling exercises.

    Seven months later, the program office evaluated Increment 2B operational
    performance. The purpose of the evaluation was to determine the
    effectiveness of Increment 2B performance at the 50 busiest land POEs. To
    assist in the evaluation, the program office established a baseline for
    comparing the average Form I-94 or Form I-94W30 issuance processing times
    at 3 of the 50 POEs where processing times were to be evaluated.31 The
    program office then conducted two evaluations of the processing times at
    the 3 POEs following Increment 2B deployment. The first was in December
    2004, after Increment 2B was deployed to these sites as a pilot, and the
    second was in February 2005, after Increment 2B was deployed to all 50
    POEs. The evaluation results showed that the average processing times
    decreased for all 3 sites. Table 2 compares the results of the two
    evaluations and the baseline.

    Table 2: Reduction in Reported Processing Times for Increment 2B Pilot and
    Full Deployment

    Pilot site    Baseline       Pilot: Decrease in Full deployment: Change in
                     (October 2004) time from baseline time from pilot (February
                                    (December 2004)    2005)
    Douglas,      4 minutes, 16  -47 seconds        -17 seconds
    Arizona       seconds
    Laredo, Texas 12 minutes, 10 -9 minutes, 37     -15 seconds
                     seconds        seconds
    Port Huron,   11 minutes, 42 -1 minutes, 51     +7 seconds
    Michigan      seconds        seconds

    Source: GAO analysis of DHS data.

    According to program officials, these evaluations supported the workforce
    and facilities planning assumption that no additional staff were required
    to support deployment of Increment 2B, and that minimal modifications to
    interior workspace were required to accommodate biometric capture devices
    and printers and to install electrical circuits. These officials stated
    that modifications to existing officer training and interior space were
    the only changes needed.

    However, the scope of the evaluation was too limited to satisfy the
    evaluation's stated purpose or our recommendation for assessing the full
    impact of Increment 2B. Specifically, program officials stated that the
    evaluation focused on the time to process Form I-94s and not on
    operational effectiveness, including workforce impacts and traveler
    waiting time. Second, the 3 sites were selected, according to program
    officials, on the basis of a number of factors, including whether the
    sites already had sufficient staff to support the pilot. Selecting sites
    on the basis of this factor could affect the results and presupposes  that
    not all POEs have the staff needed to support Increment 2B. Third,
    evaluation conditions were not always held constant. For example, fewer
    workstations were used to process travelers in establishing the baseline
    processing times at 2 of the POEs-Port Huron (9 versus 14) and Douglas (4
    versus 6)-than were used during the pilot evaluations.

    Moreover, CBP officials from 1 POE, which was not an evaluation site, told
    us that US-VISIT has actually lengthened processing times. (San Ysidro
    processes the highest volume of travelers of all land POEs.) While these
    officials did not provide specific data to support this statement, it
    nevertheless raises questions about the potential impact of Increment 2B
    on the 47 sites that were not evaluated.

    It is important that the impact of Increment 2B on workforce and
    facilities be fully assessed. Since we made our recommendation, Increment
    2B deployment and operational facts and circumstances have materially
    changed, making the implementation of our recommendation using
    predeployment baseline data for the other 47 sites impractical.
    Nevertheless, other alternatives, such as surveying officials at these
    sites to better understand the increment's impact on workforce levels and
    facilities, have yet to be explored. Until they are, the program may not
    be able to accurately project resource needs or make required
    modifications to achieve its goals of minimizing US-VISIT's impact on POE
    processing times.

    Implementation of Configuration Management Practices Is in Progress

    We reported in May 2004 that US-VISIT had not established effective
    configuration management practices. Configuration management establishes
    and maintains the integrity of system components and items (e.g.,
    hardware, software, and documentation). A key ingredient is a change
    control board to evaluate and approve proposed configuration changes.
    Accordingly, we concluded that the program did not have adequate assurance
    that approved system changes were actually made, and that changes made to
    the component systems (for non-US-VISIT purposes) did not interfere with
    US-VISIT functionality. Accordingly, we recommended that DHS do the
    following:

    Implement effective configuration management practices, including
    establishing a US-VISIT change control board to manage and oversee system
    changes.

    After 19 months, US-VISIT has begun implementing configuration management
    practices. To its credit, the program recently issued a configuration
    management policy (September 2005) and prepared a draft configuration
    management plan (August 2005). The policy contains guiding principles,
    direction, and expectations for planning and performing configuration
    management, and includes activities, authorities, and responsibilities.
    The draft plan describes the configuration management

    governance structure, including organizational entities and their
    responsibilities, the processes and procedures to be applied, and how
    controls are to be applied to products. The governance structure includes
    the Executive Configuration Control Board and the Configuration Management
    Impact Review Team. According to its charter, the configuration control
    board is responsible for determining the status of requested configuration
    changes and resolving any conflicts related to those changes for
    US-VISIT-managed systems (i.e., not for US-VISIT component systems managed
    by other DHS organizations). The Impact Review Team, which reports to the
    board, is responsible for reviewing requests for system changes and
    submitting a recommendation to the appropriate change review authority
    (i.e., either the US-VISIT control board or the control board in the DHS
    organization that manages the component system). According to program
    officials, for US-VISIT-managed systems, the review authority is the
    Executive Configuration Control Board. For other systems, such as TECS
    (which CBP manages), the US-VISIT review team may submit a recommendation
    to the appropriate control board (in this case, the CBP Control Board).

    The APMO director stated that the planned configuration management program
    is intended to complement rather than replace the configuration management
    programs for the legacy systems. That is, change requests approved by the
    US-VISIT Executive Configuration Control Board that require changes to a
    legacy system will be coordinated with the board having responsibility for
    that system. This means, however, that changes to component systems (e.g.,
    IDENT, ADIS, and TECS) that are initiated and approved by another DHS
    organization, and that could affect US-VISIT performance, are not subject
    to US-VISIT configuration management processes and are not also being
    examined and approved by the US-VISIT control board. This lack of US-VISIT
    control was the impetus for our recommendation.

    Although US-VISIT has recently taken steps to begin addressing our
    recommendation, the program still does not adequately control changes to
    the component systems upon which US-VISIT performance depends. Until
    programwide configuration management practices are implemented, the
    program does not have an effective means for ensuring that approved system
    changes are actually made and that changes made to the component systems
    for non-US-VISIT purposes do not compromise US-VISIT functionality and
    performance.

    Efforts to Ensure the Independence of the Verification and Validation
    Contractor Are Complete

    We reported in May 2004 that the program office's independent verification
    and validation (IV&V) contractor was not independent of the products and
    processes that it was verifying and validating. The purpose of IV&V is to
    provide management with objective insight into the program's processes and
    associated work products. Its use is a recognized best practice for large
    and complex system development and acquisition projects like US-VISIT. To
    be effective, the verification and validation function is to be performed
    by an entity that is independent of the processes and products that are
    being reviewed. Accordingly, we recommended that DHS do the following:

    Ensure the independence of the IV&V contractor.

    In July 2005, the program office issued a new contract for IV&V services.
    To ensure the contactor's independence, the program office (1) required
    that IV&V contract bidders be independent of the development and
    integration contractors; (2) reviewed each of the bidder's affiliations
    with the prime contract; (3) included provisions in the contract that
    prohibit the contractor from soliciting, proposing, or being awarded work
    (other than IV&V services) for the program; (4) required all contractor
    personnel to certify that they do not have any conflicts of interest; and
    (5) ensured that the contractor's management plan (Oct. 17, 2005)
    describes how the contractor will ensure technical, managerial, and
    financial independence.

    Such steps, if effectively enforced, should adequately ensure that
    verification and validation activities are performed in an objective
    manner and, thus, should provide valuable assistance to program managers
    and decision makers.

    Development of a Plan to Address Open Recommendations Is Partially
    Complete

    We reported in May 2004 that US-VISIT's overall progress on implementing
    our recommendations had been slow, and considerable work remained to fully
    address them. As we also noted, given that most of our recommendations
    focused on fundamental limitations in US-VISIT's ability to manage the
    program, it was important to implement the recommendations quickly and
    completely. Accordingly, we recommended that DHS do the following:

    Develop a plan, including explicit tasks and milestones, for implementing
    all of our open recommendations and periodically report to the DHS
    Secretary and Under Secretary on progress in implementing this plan; and
    report this progress, including reasons for delays, in all future
    expenditure plans.

    About 19 months after our recommendation, the program assigned
    responsibility to specific individuals for preparing a plan, including
    specific actions and milestones, to address each recommendation. In
    addition, it developed a report that identifies the responsible person for
    each recommendation and summarizes progress made in implementing each. The
    program office provided this report for the first time to the DHS Deputy
    Secretary on October 3, 2005, and plans to forward subsequent reports
    every 6 months.

    However, the report's description of progress on 4 recommendations is
    inconsistent with our assessment, as discussed below:

    o First, the report states that the program completed a privacy impact
    assessment that is in full compliance with OMB guidance. As previously
    discussed, an assessment has been developed, but OMB guidance requires
    that these assessments for systems under development (such as Increment
    2C) address privacy in the system's documentation. Increment 2C systems
    documentation does not address privacy and therefore is not fully
    compliant with OMB guidance.

    o Second, the report states that a human capital strategy has been
    completed. However, as previously discussed, several of the activities in
    the human capital plan have yet to be implemented. For example, the
    program has not developed a staffing forecast to inform succession
    planning.

    o Third, the report states that the impact of Increment 2B on land POE
    workforce levels and facilities has been fully assessed. However, as we
    previously stated, the scope of the evaluations was not sufficient to
    satisfy our recommendation. For example, program officials stated that the
    evaluation focused on the time to process Form I-94s and not on
    operational effectiveness, including workforce impacts and traveler
    waiting time. Moreover, officials at the largest land POE told us that the
    effect of Increment 2B was the opposite of that reported in the pilot
    results.

    o Fourth, the report states that the program has partially completed
    implementing configuration management practices. However, as previously
    discussed, the program office has yet to implement practices or establish
    a configuration control board with authority over all changes affecting
    US-VISIT functionality and performance, including those made to component
    systems for non-US-VISIT purposes, which was the intent of our
    recommendation.

    In addition, the report does not specifically describe progress against 11
    of our other recommendations, so that we could not determine whether the
    program's assessment is consistent with ours (described in this report).
    For example, we recommended that the program reassess plans for deploying
    an exit capability to ensure that the scope of the exit pilot provides for
    adequate evaluation of alternative solutions. The report states that the
    program office has completed exit testing and has forwarded the exit
    evaluation report to the Deputy Secretary for a decision. However, it does
    not state whether the program office had expanded the scope or time frames
    of the pilot.

    Fully understanding and disclosing progress against our recommendations
    are essential to building the capability needed to effectively manage the
    program, and to ensuring that key decision makers have the information
    needed to make well-informed choices among competing investment options.

    Establishment of Effective Cost-Estimating Practices Is in Progress

    We reported in February 2005 that US-VISIT had not followed effective
    practices to develop cost estimates for its system increments, and thus
    the reliability of its cost estimates was questionable.32 Such
    cost-estimating practices are embedded in the 13 criteria in SEI's
    checklist for determining the reliability of cost estimates.33 Of these 13
    criteria, we reported in February 2005 that the program's cost estimate
    met 2, partially met 6, and did not meet 5. Accordingly, we recommended
    that DHS do the following:

    Follow effective practices for estimating the costs of future increments.

    The latest US-VISIT-related cost estimate is for Increment 1B. This
    estimate is in the June 2005 cost-benefit analysis for Increment 1B and
    establishes the costs associated with three exit solutions for air and sea
    POEs. As was the case for the estimate described in our February 2005
    report, this latest estimate also did not meet all 13 criteria, meeting 3
    and partially meeting another 5.34 For example, these estimates did not
    include a detailed work breakdown structure and omitted important cost
    elements, such as system testing. A work breakdown structure serves to
    organize and define the work to be performed, so that associated costs can
    be identified and estimated. Thus, it provides a reliable basis for
    ensuring that the estimates include all relevant costs. In addition, the
    uncertainties associated with the Increment 1B cost estimate were not
    identified. An uncertainty analysis provides the basis for adjusting these
    estimates to reflect unknown facts and circumstances that could affect
    costs and identifies the risk associated with the cost estimate. Table 3
    summarizes our analysis of the extent to which US-VISIT's Increment 1B
    cost estimates satisfy SEI's 13 criteria.

    Table 3: Satisfaction of SEI's 13 Cost-Estimating Criteria

            Criterion            Explanation      Criterion     GAO analysis
                                                    met?a
    1. The objectives of The objectives of the Yes       The objectives of the
    the program are      program should be               program were clearly
    stated in writing.   clearly and concisely           stated. Specifically,
                            stated for the cost             the objectives are to
                            estimator to use.               provide a more
                                                            complete traveler
                                                            history and to
                                                            capture travelers'
                                                            biometric and
                                                            biographic data.
    2. The life cycle to The life cycle should Partially The life cycle was
    which the estimate   be clearly defined to           not clearly defined
    applies is clearly   ensure that the full            to ensure that the
    defined.             cost of the program             full cost of the
                            is captured-that is,            program was included.
                            all direct and                  For example, the
                            indirect costs for              analysis did not
                            planning,                       include evidence that
                            procurement,                    software maintenance
                            operations and                  costs were included
                            maintenance, and                in the cost estimate.
                            disposal.
    3. The task has been An appropriate sizing No        The program office
    appropriately sized. metric should be used           provided no evidence
                            in the development of           to demonstrate that
                            the estimate, such as           an appropriate sizing
                            the amount of                   mechanism was used,
                            software to be                  and program officials
                            developed and the               stated that they had
                            amount of software to           not collected these
                            be revised.                     data.
    4. The estimated     Estimates should be   Partially Officials stated that
    cost and schedule    validated by being              pilot data were used
    are consistent with  related back to                 to develop the
    demonstrated         demonstrated and                estimate. They stated
    accomplishments on   documented                      they extrapolated
    other projects.      performance on                  pilot data to
                            completed projects.             estimate costs for
                                                            all Increment 1B
                                                            sites; however, they
                                                            further stated that
                                                            there were no
                                                            previous projects
                                                            with which to compare
                                                            the results to see if
                                                            they were consistent.
    5. A written summary If a parametric       Partially High-level cost
    of parameter values  equation was used to            categories, such as
    and their rationales generate the                    labor, information
    accompanies the      estimate, the                   technology,
    estimate.            parameters that feed            facilities, and other
                            the equation should             costs, were
                            be provided, along              identified, but
                            with an explanation             detailed parameters
                            of why they were                used to develop the
                            chosen.                         estimate, such as
                                                            number of software
                                                            lines of code, which
                                                            would be relevant to
                                                            software maintenance
                                                            costs, were not
                                                            provided in the
                                                            analysis.
    6. Assumptions have  Assumptions regarding Yes       General cost
    been identified and  issues such as                  assumptions are
    explained.           schedule, quantity,             identified and
                            technology,                     explained, as well as
                            development                     assumptions for
                            processes,                      workforce,
                            manufacturing                   information
                            techniques, software            technology, training,
                            language, etc.,                 and facilities.
                            should be understood
                            and documented.
    7. A structured      A work breakdown      Partially The analysis included
    process, such as a   structure or similar            four high-level cost
    template or format,  structure that                  categories (labor,
    has been used to     organizes, defines,             facilities,
    ensure that key      and graphically                 operations and
    factors have not     displays the                    maintenance, and
    been overlooked.     individual work units           information
                            to be performed                 technology), but it
                            should be used. The             did not include a
                            structure should be             detailed work
                            revised over time as            breakdown structure
                            more information                and omitted important
                            becomes known about             cost elements, such
                            the work to be                  as system testing.
                            performed.
    8. Uncertainties in  For all major cost    Partially A risk analysis was
    parameter values     drivers, an                     performed, but this
    have been identified uncertainty analysis            analysis did not
    and quantified.      should be performed             identify detailed
                            to recognize and                parameter values.
                            reflect the risk
                            associated with the
                            cost estimate.
    9. If a dictated     Managers should be    N/A       Program officials
    schedule has been    informed of all                 stated that the
    imposed, an estimate potential cost                  Increment 1B schedule
    of the normal        savings associated              was not dictated.
    schedule has been    with alternative
    compared to the      schedules.
    additional
    expenditures
    required to meet the
    dictated schedule.
    10. If more than one The primary           No        No evidence of a
    cost model or        methodology or cost             secondary cost model
    estimating approach  model results should            was included in the
    has been used, any   be compared with any            analysis, and program
    differences in       secondary methodology           officials stated that
    results have been    (e.g., cross checks)            they did not use a
    analyzed and         to ensure                       second model.
    explained.           consistency.
    11. Estimators       The purpose of an     No        Program officials
    independent of the   independent estimate            stated that the
    performing           is to determine the             estimate was not
    organization         reasonableness of the           independently
    concurred with the   parameter values                reviewed.
    reasonableness of    based on an unbiased
    the parameter values perspective. This
    and estimating       approach usually
    methodology.         results in a more
                            accurate estimate
                            because it allows for
                            better insight into
                            program risks.
    12. Estimates are    Estimates are updated Yes       Estimates reflected
    current.             whenever changes to             current conditions.
                            requirements affect
                            cost or schedule,
                            constraints, and
                            resources, or when
                            priorities change.
    13. The results of   Plans are reviewed    No        Program officials
    the estimate have    and updated whenever            stated that the
    been integrated with estimates change, and           results of the
    project planning and estimates used for              estimate have not
    tracking.            project planning are            been incorporated
                            also used as                    with project
                            baselines for project           planning.
                            tracking.

    Source: GAO.

    aWe assessed each of the criteria as satisfied (US-VISIT provided
    substantiating evidence for the criterion), partially satisfied (US-VISIT
    provided partial evidence, including testimonial evidence, for the
    criterion), or not satisfied (no evidence was found for the criterion).

    Program officials stated that they recognize the importance of developing
    reliable cost estimates and have initiated actions to more reliably
    estimate the costs of future increments. For example, as part of its
    process improvement program, the program has chartered a cost-analysis
    process action team, which is to develop, document, and implement a
    cost-analysis policy, process, and plan for the program. Program officials
    also stated that they have hired additional contracting staff with
    cost-estimating experience.

    Strengthening the program's cost-estimating capability is extremely
    important. The absence of reliable cost estimates, among other things,
    prevents the development of reliable economic justification for program
    decisions and impedes effective performance measurement.

    Reassessment of Plans for Deploying the Exit Capability Is Partially
    Complete

    In February 2005, we reported that US-VISIT had not adequately planned for
    evaluating the Increment 1B exit alternative because its exit pilot
    evaluation's scope and timeline were compressed. Accordingly, we
    recommended that DHS do the following:

    Reassess plans for deploying an exit capability to ensure that the scope
    of the exit pilot provides for adequate evaluation of alternative
    solutions and better ensures that the exit solution selected is in the
    best interest of the program.

    Over the last 10 months, the program office has taken actions to expand
    the scope and time frames of the pilot. For example, it extended the pilot
    from 5 to 11 POEs-9 airports and 2 seaports.35 It also extended the time
    frame for data collection and evaluation to April 2005, which is about 7
    months beyond the date for which all exit pilot evaluation tasks were to
    be completed. Further, according to program officials, they achieved the
    target sample sizes necessary to have a 95 percent confidence level.

    Notwithstanding the expanded scope of the pilot, questions remain about
    whether the exit alternatives have been evaluated sufficiently to permit
    selection of the best exit solution for national deployment. For example,
    each of the three exit alternatives was evaluated against three criteria,
    including compliance with the US-VISIT exit process (i.e., foreign
    travelers providing information as they exit the United States).36
    However, across the three alternatives, the average compliance with this
    process was only 24 percent, which raises questions as to the
    effectiveness of the three alternatives.37 The evaluation report cites
    several reasons for the low compliance rate, including that compliance
    during the pilot was voluntary. The report further concludes that national
    deployment of the exit solution will not have the desired compliance rate
    unless the exit process incorporates an enforcement mechanism, such as not
    allowing persons to reenter the United States if they do not comply with
    the exit process. Although an enforcement mechanism might indeed improve
    compliance, program officials stated that no formal evaluation has been
    conducted of enforcement mechanisms or their effect on compliance. The
    program director stated that he agrees that additional evaluation is
    needed to assess the impact of implementing potential enforcement
    mechanisms and plans to do so.

    Until the program office adequately evaluates the exit alternatives and
    knows whether the alternative to be selected will be effective, the
    program office will not be in a position to select the exit solution that
    is in the best interest of the program. This is very important because
    without an effective exit capability, the benefits and the mission value
    of US-VISIT are greatly diminished.

    Development and Implementation of Capacity Management Processes Are in
    Progress

    We reported in February 2005 that the overall capacity of the system was
    not being effectively managed. At that time, US-VISIT, which comprises
    several legacy systems, was relying on the capacity management activities
    of these systems. It was not focused on the capacity requirements and
    performance of the collective systems that make up US-VISIT. This approach
    increases the risk that the system may not be properly designed and
    configured for efficient performance, and that it has insufficient
    processing and storage capacity for current, future, and unpredictable
    workload requirements. Accordingly, we recommended that DHS do the
    following:

    Develop and implement processes for managing the capacity of the US-VISIT
    system.

    According to program officials, they have initiated efforts to develop a
    capacity management process, including a high-level description of the
    necessary steps, such as identifying tools needed to implement the
    process. However, a plan, including specific tasks and milestones for
    developing and implementing capacity management processes, has not yet
    been developed.

    Until the program office develops a programwide capacity management
    program, it increases the risk that US-VISIT may not be able to adequately
    support program mission needs.

    Identification of ACE and US-VISIT Relationships and Dependencies Is in
    Progress

    We reported in February 2005 that the program office recognized that
    US-VISIT and the Automated Commercial Environment (ACE)38 have related
    missions and operational environments. In addition, US-VISIT and ACE could
    potentially develop, deploy, and use common information technology
    infrastructures and services. We also reported that managing this
    relationship has not been a priority. Accordingly, we recommended that DHS
    do the following:

    Make understanding the relationships and dependencies between the US-VISIT
    and ACE programs a priority matter, and report periodically to the Under
    Secretary on progress in doing so.

    US-VISIT and ACE managers met in February 2004, to identify potential
    areas for collaboration between the two programs and to clarify how the
    programs could best support the DHS mission and provide officers with the
    information and tools they need. According to program officials, they have
    established a US-VISIT/ACE integrated project team to, among other things,
    ensure that the two programs are programmatically and technically aligned.
    The team has discussed potential areas of focus and agreed to three areas:
    RF technology, program control, and data governance. However, it does not
    have an approved charter, and it has not developed explicit plans or
    milestone dates for identifying the dependencies and relationships between
    the two programs. Program officials stated that the team has met three
    times and plans to meet on a quarterly basis going forward.

    It is important that the relationships and dependencies between these two
    programs be managed effectively. The longer it takes for the programs to
    understand and exploit their relationships, the more rework will be needed
    at a later date to do so.

    Conclusions

    Over the last 3 years, we have made recommendations aimed at correcting
    fundamental limitations in US-VISIT's program management ability and
    thereby better ensuring the delivery of mission capability and value on
    time and commensurate with costs. While progress on the implementation of
    the recommendations is mixed, progress in critical areas has been slow. As
    with any program, introducing and institutionalizing the program
    management and accountability discipline at which our recommendations are
    aimed require investing time and resources while continuing to meet other
    program demands. In making such investment choices, it is important to
    remember that institutionalizing such program discipline in the near term
    will produce long-term payback in a program's ability to meet these other
    demands. Accordingly, the longer that US-VISIT takes to implement our
    recommendations, the greater the risk that the program will not meet its
    stated goals and commitments.

    Our open recommendations are all aimed at strengthening US-VISIT program
    management and improving DHS's ability to make informed US-VISIT
    investment decisions. With the exception of one, these recommendations are
    still relevant and applicable. Since we made our recommendation, facts and
    circumstances surrounding Increment 2B deployment and operational status
    have materially changed, making the collection of Increment 2B
    predeployment impractical. Nevertheless, the need remains to better
    understand the impact of US-VISIT entry capabilities on all land POEs.
    Until this understanding exists, the department will be challenged in its
    ability to accurately estimate and provide facilities and staff resource
    needs.

    Recommendation for Executive Action

    To recognize both the need to fully assess the impact of US-VISIT entry
    capabilities on staffing levels and facilities at land POEs, as well as
    the current operational status of Increment 2B, we are closing our
    existing recommendation related to assessing the impact of Increment 2B.
    We recommend that the DHS Secretary direct the US-VISIT Program Director
    to explore alternative means of obtaining an understanding of the full
    impact of US-VISIT at all land POEs, including its impact on workforce
    levels and facilities; these alternatives should include surveying the
    sites that were not part of the previous assessment.

    Agency Comments and Our Evaluation

    In its written comments on a draft of this report, signed by the Director,
    Departmental GAO/OIG Liaison Office, and reprinted in appendix II, DHS
    stated that it agreed with many areas of the report and that our
    recommendations had made US-VISIT a stronger program. Further, the
    department stated that while it disagreed with certain areas of the
    report, it nevertheless concurred with the need to implement our open
    recommendations with all due speed and diligence.

    DHS commented specifically on 11 of the 18 recommendations discussed in
    the report. The recommendations, the department's comments, and our
    responses follow:

    1.Recommendation: Develop and begin implementing a system security plan,
    and perform a privacy impact assessment and use the results of the
    analysis in near-term and subsequent system acquisition decision making.

    DHS stated that this recommendation has been fully implemented. In
    support, it said that it has completed a US-VISIT security plan that is
    consistent with National Institute of Standards and Technology (NIST)
    guidance, and that it provided the plan to us in September 2004. It also
    stated that the security risk assessment aspect of this recommendation was
    established in February 2005, 20 months after we made the recommendation,
    and thus the age of the recommendation should be shown as 10 months rather
    that the 30 months cited in the report.

    The department also commented that there is no US-VISIT system, but rather
    a US-VISIT program with capabilities delivered by existing interconnected
    systems. According to the department, these component systems have been
    certified and accredited, consistent with NIST guidance, and as part of
    their certification and accreditation, security plans and risk
    assessments, as well as risk mitigation strategies, have been developed
    for each system. The department stated that it provided us with these
    system-level risk assessments, as well as system-specific action plans and
    milestones for implementing the mitigation strategies. In addition, the
    department noted that it completed a programwide risk assessment in
    December 2005 that specifically addresses information security issues that
    might not be captured in the system-specific documentation used to certify
    and accredit each system. In light of its system-specific certification
    and accreditation efforts, existing system-level risk assessments, and the
    program-level risk management process (see response 4 for discussion of
    the risk management process), DHS commented that it is inaccurate to state
    that US-VISIT officials are not in a position to know program risks, and
    the recommendation should be closed.

    While we agree that we received a copy of the US-VISIT security plan,
    dated September 2004, we do not agree that the plan satisfied all relevant
    federal guidance and that DHS has fully implemented our recommendation. In
    particular, it has not provided us with evidence that a programwide risk
    assessment has been done and that a security plan reflective of such an
    assessment exists. According to relevant guidance,39 a security plan
    should describe, among other things, the methodology that is to be used to
    identify system threats and vulnerabilities and to assess risks, and it
    should include the date the risk assessment was completed because the
    assessment is a necessary driver of the security controls described in the
    plan. As we reported in February 2005 and state in this report, the
    US-VISIT security plan did not include this information; further, although
    DHS stated in its comments that it completed this risk assessment in
    December 2005, this statement is contradicted by a statement elsewhere in
    its comments that it is still in the process of doing the assessment. In
    addition to this contradiction, DHS's comments did not include any
    evidence to demonstrate that it has developed a complete risk assessment,
    such as a copy of the assessment.

    With regard to the age of the recommendation, we do not agree with DHS's
    position that we established a new finding regarding the lack of a
    programwide risk assessment in our February 2005 report. Rather, as part
    of our analysis of actions to implement our prior recommendation to
    develop a security plan, which is to include information about the related
    security risk assessment, we observed that the plan did not indicate a
    date for completing a risk assessment in accordance with federal
    guidelines. Therefore, our position that about 30 months had passed from
    the time of our initial recommendation (June 2003) is accurate.

    With regard to the individual system-level risk assessments, we agree that
    we have received them. However, we do not agree that we have received the
    action plans and milestones cited in the comments. Regardless, we do not
    believe that system-level assessments are a sufficient substitute for a
    programwide assessment. Accordingly, our recommendation focused on the
    need for an integrated US-VISIT system risk assessment as part of security
    planning. While the system-level plans and risk assessments are relevant
    and useful, they neither individually nor collectively address the threats
    and vulnerabilities imposed as a result of these systems' integration. By
    stating in its comments its commitment to having a programwide risk
    assessment that identifies and proposes mitigations for security risks
    that arise as a result of the interface and integration of the legacy
    systems, DHS is agreeing with our position. Moreover, without evidence
    that the program has completely assessed its risks, we continue to find no
    basis for how program officials would know the full range and degree of
    US-VISIT security risks. Our position in this regard has been reinforced
    by a recent DHS Inspector General report that identified a number of
    US-VISIT security risks.40

    To further support its position that this recommendation has been fully
    implemented, DHS also commented that it has completed numerous privacy
    impact assessments and continues to update them to reflect system changes.
    In particular, it said that it updated the privacy impact assessment in
    December 2005 to reflect all increments and that it considers the
    assessment to be part of US-VISIT system documentation. It further
    commented that we appear to be unaware of privacy staff activities to
    review system documents and perform privacy risk assessments throughout
    the system life cycle. Nevertheless, the department acknowledged that its
    privacy work was not always noted within US-VISIT system documentation.
    Accordingly, DHS stated that it plans to appropriately reference all
    privacy requirements and privacy risk assessments in the program's system
    documentation in the future.

    We agree that US-VISIT has developed and updated its privacy impact
    assessment and would note that our report states this fact. We do not
    agree, however, with the comment that we are not aware that the privacy
    staff review system documents and perform privacy risk assessments. In
    fact, it is because we were aware of these facts that we were careful to
    ensure that they were reflected in our report. The point that we are
    making is that privacy is not addressed in all relevant systems
    documentation, which DHS acknowledged in its comments. With regard to this
    point of agreement, we support the department's stated plans to reference
    all privacy requirements and any privacy risk assessments in all relevant
    system documentation in the future.

    2.Recommendation: Develop and implement a plan for satisfying key
    acquisition management controls, including acquisition planning,
    solicitation, requirements management, program management, contract
    tracking and oversight, evaluation, and transition to support, and
    implement the controls in accordance with SEI guidance.

    DHS commented that the report should reflect that US-VISIT had initially
    adopted Carnegie Mellon University's Software Engineering Institute (SEI)
    Software Acquisition Capability Maturity Model(R) to guide its
    software-related process improvement efforts and that, in December 2004,
    it transitioned to SEI's Capability Maturity Model-Integration (CMMI(R)).
    As a result, it said that the program's process improvement strategy and
    plans, process development, and process appraisals are now aligned to the
    most applicable CMMI process areas.

    We agree that US-VISIT has transitioned to CMMI. We state in our report
    that US-VISIT has done so and that the key process areas it is addressing
    in its process improvement strategy and plan are consistent with those
    cited in our recommendation. We do not believe that this transition
    materially affects our recommendation, however, because even though the
    names of the key processes in these two models may in some cases differ,
    the processes and respective practices are fundamentally consistent.

    3.Recommendation: Clarify the operational context in which US-VISIT is to
    operate.

    Consistent with our report, DHS commented that the operational context in
    which US-VISIT operates is in progress, meaning that it has yet to be
    fully established. For example, it said that the mission of DHS, and
    therefore the scope of US-VISIT activities to meet the mission, is
    continually expanding. Further, it acknowledged that more certainty in the
    operational context is desirable. In mitigation of the risks associated
    with not having a more stable operational context, DHS made several
    statements. For example, it said that the principal role of US-VISIT is to
    integrate information and immigration and border management systems across
    DHS and the State Department, and to facilitate agencies working toward a
    common environment that will eliminate redundancies. It also said that
    elements of its draft immigration and border management strategic plan are
    being used in current US-VISIT operations. In addition, the department
    said that mechanisms to mitigate the risks that we cited have been
    developed and are being implemented.

    We support DHS's acknowledgment of the importance of having a well-defined
    operational context within which to define and implement US-VISIT and
    related border security programs. However, we do not believe that DHS's
    comments provided any evidence showing that sufficient steps and
    activities to mitigate the associated risks have been taken or are
    planned.

    4.Recommendation: Determine whether proposed US-VISIT increments will
    produce mission value commensurate with cost and risks and disclose to the
    Congress planned actions.

    DHS commented that its cost-benefit analysis (CBA) for Increment 1B
    conforms to relevant federal guidance, and noted that our expectations as
    to the scope and level of detail of analysis that should be included in
    the CBA document are inconsistent with its understanding of OMB Circular
    A-94

    and DHS's CBA workbook,41 which were used to guide the development of the
    CBA analysis. As an example, the department took exception with our
    statement that year-by-year benefit estimates were not reported by noting
    that the net present value was based on an estimate of annual benefits and
    costs, and that net present value could not be estimated without a
    year-by-year benefit analysis.

    The department further commented that a comprehensive uncertainty analysis
    was conducted because it completed a risk analysis, which is more
    comprehensive, rigorous, and appropriate than conducting a sensitivity
    analysis. In this regard, it added that the results of the risk analysis
    provided an indication of Increment 1B's worthiness in light of existing
    uncertainty, rather than information on a specific CBA variable or
    another. The department further noted that it had provided some of these
    supporting analyses to us.

    DHS also stated that any investment that has a 5-year life cycle and is
    considered interim in nature will face considerable challenge in providing
    economic benefits commensurate with cost.

    We do not agree that the CBA fully conforms to relevant federal guidance.
    As our report states, for example, the analysis does not explicitly state
    the numerical value of the discount rate used for calculating each
    alternative's net present value, and hence does not conform to OMB
    guidance. In addition, the cost estimates used in the analysis were not
    complete and reliably derived. In deriving the estimate, for example, the
    department did not clearly define the project's life cycle to ensure that
    key factors were not overlooked and that the full cost of the program was
    included. (See response 10 below for more information on this point.)
    Last, while we agree that a year-by-year benefit analysis is a necessary
    component of a net present value determination, OMB nevertheless requires
    that the year-by-year benefit estimates be reported in the analysis to
    promote independent review of the estimates.

    Also, we do not agree that DHS performed a complete uncertainty analysis.
    According to OMB and DHS guidance, a complete uncertainty analysis should
    include both a risk analysis and a sensitivity analysis. However, the
    latter was not done. Thus, our point is not, as DHS comments suggest, that
    US-VISIT should have performed a sensitivity analysis instead of a risk
    analysis, but rather, that both types of analyses are necessary to
    completely examine investment uncertainty.

    5.Recommendation: Develop and implement a risk management plan and ensure
    that all high risks and their status are reported regularly to the
    executive body.

    DHS commented that US-VISIT began the development and implementation of
    its risk management plan in 2004 immediately after we made our
    recommendation. It further commented that, as part of a CMMI maturity
    internal appraisal that it completed in July 2005, it found that the risk
    management process had not been consistently applied across the program.
    To address this, the department cited actions that it has taken to fully
    implement risk management, such as approving the risk management plan in
    September 2005; defining a risk governance structure; establishing and
    maintaining a risk database; and developing risk management training and
    providing this training to program personnel and contractors beginning in
    November 2005.

    We support the recent actions that the program cited as having been taken
    to strengthen risk management. However, the actions cited do not
    demonstrate that the risk management process is being consistently
    applied. Until US-VISIT fully implements its risk management plan and
    process, it cannot be assured that all program risks are being identified
    and managed in order to effectively mitigate any negative impact on the
    program's ability to deliver promised capabilities on time and within
    budget.

    6.Recommendation: Develop and approve test plans before testing begins
    that (1) specify the test environment; (2) describe each test to be
    performed, including test controls, inputs, and expected outputs;
    (3) define the test procedures to be followed in conducting the tests; and
    (4) provide traceability between test cases and the requirements to be
    verified by the testing.

    DHS stated that our report does not accurately reflect the status of the
    Increment 2C Phase 1 testing. In particular, it said that the issues
    associated with the traceability of requirements to test cases were minor
    and that the extent of the discrepancies is far less than what our report
    presents. It further stated that the discrepancies in our report are based
    on old traceability documentation and do not reflect revised documentation
    provided to us on November 9, 2005.

    We agree that DHS provided us with revised traceability matrixes after we
    had shared with them our analysis of the test plans and traceability
    matrixes, dated June 28, 2005, and June 27, 2005, respectively. However,
    the revised documentation referenced in DHS's comments was provided in
    November 2005, about 4 months after testing began. This means that the
    test plans and traceability matrixes available at the time of
    testing-which are what we reviewed because they governed the scope and
    nature of actual testing performed-did not adequately trace between test
    cases and the requirements to be verified. Specifically, 300 of the 438
    Increment 2C requirements, or about 70 percent, did not have specific
    references to test cases.

    7.Recommendation: Implement effective configuration management practices,
    including establishing a US-VISIT change control board to manage and
    oversee system changes.

    DHS commented that a US-VISIT representative attends all configuration
    control board meetings for all applicable legacy component systems, and
    that any proposed change request from a legacy component control board
    that could affect US-VISIT functionality is brought to the attention of
    the US-VISIT Executive Configuration Control Board for consideration.

    We do not question these statements. However, we do not believe that they
    demonstrate that US-VISIT has adequate control over system changes that
    could affect the program. That is, they do not ensure that changes to the
    component systems that are initiated and approved by another DHS
    organization and that could affect US-VISIT performance are subject to
    US-VISIT configuration management and approval processes. US-VISIT could
    establish explicit and enforceable control over changes to the legacy
    systems through such mechanisms as defined and enforced memorandums of
    understanding among the affected DHS organizations. It was the lack of
    such control that prompted our recommendation.

    8.Recommendation: Assess the full impact of Increment 2B on land POE
    workforce levels and facilities, including performing appropriate modeling
    exercises.

    The department stated that, given the imperative to meet the legislatively
    mandated time frames, the scope of Increment 2B was limited to only one
    part of POE operations-incorporating the collection of a biometric into
    the previously manual Form I-94 issuance process. It also stated that wait
    times are affected by various factors, including traffic volume, staffing
    levels, and availability of officers. Therefore, DHS focused the Increment
    2B evaluation on just the change to this process.

    The department further commented that given the events since the
    evaluation-namely, Increment 2B full operations-it is not practical to
    collect and model baseline data for the 47 sites that were not part of the
    initial evaluation.

    Regarding the 3 pilot sites included in the assessment, the department
    stated that the sites were selected based on criteria developed from input
    from US-VISIT, as well as CBP operational constraints. The department
    further commented that the 3 sites provided a reasonable mix of travelers
    and they did not have other constraints that directly impacted the
    collection of performance data specific to Form I-94 issuance. DHS also
    stated that the I-94 processing times vary by POE, and therefore they are
    not easily generalized from one port to another. Further, the department
    commented that the number of workstations and officers available to
    operate those workstations to process applicants for a Form I-94 do not
    impact the time it takes to issue a Form I-94.

    We agree that the scope of the Increment 2B evaluation was limited to the
    I-94 issuance process, and that it did not address the increment's impact
    on the POEs' ability to meet other performance parameters. Our point is
    that the limited nature of the evaluation does not satisfy either the
    intent of our recommendation or DHS's own stated purpose for the
    evaluation, which was to determine the effectiveness of Increment 2B
    performance at the 50 busiest land POEs. We also agree that the I-94
    processing times vary by POE and cannot be easily generalized. It is for
    this reason, among others, that we questioned whether the 3 sites selected
    for the assessment were sufficiently representative to satisfy both our
    recommendation and the evaluation's stated purpose.

    In addition, while we also agree that collecting pre-Increment 2B baseline
    data is not practical at this time, the fact remains that the operational
    impact of Increment 2B on workforce levels and facilities has not been
    adequately assessed, as evidenced by officials at 1 large POE telling us
    that processing times have increased and DHS's recognition that each POE
    is somewhat different. In light of these new facts and circumstances, we
    are closing our existing recommendation and making a new recommendation to
    recognize the need for DHS to explore alternative means to assess the
    impact of US-VISIT entry capabilities at land POEs. This new
    recommendation will be shown as an open recommendation, and the original
    recommendation will be closed.

    9.Recommendation: Develop a plan, including explicit tasks and milestones,
    for implementing all of our open recommendations and periodically report
    to the DHS Secretary and Under Secretary on progress in implementing this
    plan; and report this progress, including reasons for delays, in all
    future expenditure plans.

    DHS stated that it is untrue that 19 months had elapsed from the time we
    made this recommendation to the time that it assigned responsibilities to
    program officials for addressing each of our recommendations. In support,
    it commented that it issued its first plan to address our recommendations
    on August 18, 2003, and subsequent reports have been issued periodically
    that update progress in doing so.

    We agree that DHS has assigned responsibilities to specific individuals
    for addressing each recommendation. However, we have yet to be provided
    any evidence to support its statement that it issued the first report
    addressing our recommendations on August 18, 2003. Similarly, we have not
    received evidence showing that it has prepared a plan, including specific
    actions and milestones, for implementing all of our open recommendations,
    which is a focus of this recommendation. We would also observe that we
    made this recommendation in May 2004, and at that time the department
    stated that it agreed with the recommendation but did not indicate that it
    had taken any steps to address it, such as commenting that a report was
    issued on August 18, 2003.

    10.Recommendation: Follow effective practices for estimating the costs of
    future increments.

    DHS either tacitly or explicitly agreed with our findings relative to its
    satisfaction of 8 of the 13 cost-estimating criteria presented in table 4
    (now table 3) of our draft report. For example, it agreed that it did not
    clearly define the life cycle to which the cost estimate applies. It also
    agreed that it did not include a work breakdown structure, noting that it
    used the available project implementation schedule as a proxy for the
    activities related to the deployment of the exit alternatives.

    Regarding our five findings concerning its satisfaction of cost-estimating
    with which DHS disagreed, the department's primary area of disagreement
    was with the intended purpose of the Increment 1B CBA that used the cost
    estimate, which it said in its comments was to inform decision makers
    about the relative worthiness of each of the three exit alternatives
    considered for deployment. Hence, DHS stated that the purpose of the CBA
    was to analyze only  the costs associated with deploying an operational
    solution, not to analyze the costs and benefits of both developing and
    deploying alternative solutions. DHS further stated that the CBA thus
    includes only those costs to be incurred in deploying a selected
    alternative, and it does not include costs already incurred in developing
    system alternatives (i.e., sunk costs). It further commented that DHS
    guidance states that sunk costs are not relevant to the current investment
    analysis because "only current decisions can affect the future
    consequences of investment alternatives."

    DHS also disagreed that the cost estimate in the CBA should have included
    nonrecurring development costs, and commented that it did appropriately
    size the task described in the cost estimates for each alternative exit
    solution, noting that sizing metrics related to software development were
    not relevant to deployment of the alternatives because development
    activities had already occurred and therefore are sunk costs. The
    department added that those sizing metrics that are relevant to the cost
    estimate are discussed in the CBA, as are the cost estimating parameters
    (i.e., those associated with deployment and not those associated with
    development and testing).

    In addition, DHS disagreed that DHS's cost estimate excluded important
    cost categories, such as system testing, and stated that the estimate
    addresses labor, facilities, operations and maintenance, information
    technology, travel, and training costs. Once again, DHS emphasized that
    since the focus of the CBA was on operational deployment and not system
    design and development, system testing costs were not included because
    they were not considered relevant. DHS also reiterated its early point
    that the uncertainty analysis that it conducted was comprehensive.

    We agree that actual sunk costs should not be included in a CBA cost
    estimate. However, we disagree that the cost categories that DHS cited as
    not relevant are only costs that are associated with predeployment
    activities. Testing, for example, is an activity that is normally
    performed before, during, and following deployment, and thus the
    associated costs would be relevant to the stated purpose of the Increment
    1B CBA. However, a testing cost category was missing from the CBA cost
    estimate, as was a cost category for software maintenance.

    Regarding DHS's statement that it conducted a complete uncertainty
    analysis, we reiterate our previous point that a complete uncertainty
    analysis should include both a risk analysis and a sensitivity analysis,
    and the CBA did not include the latter.

    11.Recommendation: Reassess plans for deploying an exit capability to
    ensure that the scope of the exit pilot provides for adequate evaluation
    of alternative solutions and better ensures that the exit solution
    selected is in the best interest of the program.

    Concerning the questions we raised about the adequacy of the exit pilots
    in light of the 24 percent compliance rate, DHS commented that we failed
    to consider the compliance rate of the previous exit pilot program, the
    National Security Entry Exit Registration System (NSEERS), which,
    according to DHS, had a 75 percent compliance rate. DHS added that NSEERS
    achieved this compliance rate with a very limited number of exit
    locations, and therefore, any of the three US-VISIT exit alternatives
    would have at least a 75 percent compliance rate once national deployment
    was completed.

    Further, the department commented that Immigration and Customs Enforcement
    (ICE) had recently conducted enforcement operations at the Denver
    International Airport, and that the compliance rate during these
    operations increased from 30 percent to over 90 percent. It then concluded
    that the combined results of the exit pilot evaluation, the NSEERS pilot,
    and the ICE enforcement activities at the Denver International Airport
    lead it to believe that the US-VISIT exit alternatives have been
    adequately evaluated.

    We do not agree with this conclusion because it is based on unsupported
    assumptions. Specifically, DHS did not provide any evidence to support its
    claim that that US-VISIT would achieve a comparable compliance rate to the
    NSEERS program. Moreover, even if DHS could achieve a 75 percent
    compliance rate for US-VISIT exit,that still means that 25 percent of
    eligible persons would not be complying with the US-VISIT exit process.

    Further, DHS did not provide any information about the recent enforcement
    actions conducted by ICE, nor did it provide any evidence that this is a
    practical and viable option for the US-VISIT exit solution. While we agree
    that enforcement actions may indeed increase the exit compliance rate, DHS
    has not yet assessed the impact of such a solution on the US-VISIT exit
    process. Further, the US-VISIT program director acknowledged the need to
    evaluate the impact of implementing potential enforcement actions on
    US-VISIT exit and planned to do so.

    We are sending copies of this report to the Chairmen and Ranking Minority
    Members of the Senate and House Appropriations Committees, as well as to
    the Chairmen and Ranking Minority Members of other Senate and House
    committees that have authorization and oversight responsibilities for
    homeland security. We are also sending copies to the Secretary of Homeland
    Security, Secretary of State, and the Director of OMB. Copies of this
    report will also be available at no charge on our Web site at w
    ww.gao.gov.

    Should you or your offices have any questions on matters discussed in this
    report, please contact me at (202) 512-3439 or at h  iter@gao.gov. Contact
    points for our Offices of Congressional Relations and Public Affairs may
    be found on the last page of this report. Key contributors to this report
    are listed in appendix IV.

    Randolph C. Hite Director, Information Technology Architecture and Systems
    Issues

    List of Requesters

    The Honorable Peter T. King Chairman The Honorable Bennie G. Thompson
    Ranking Minority Member Committee on Homeland Security House of
    Representatives

    The Honorable Bob Filner House of Representatives

    The Honorable Raul M. Grijalva House of Representatives

    The Honorable Ruben Hinojosa House of Representatives

    The Honorable Solomon Ortiz House of Representatives

    The Honorable Silvestre Reyes House of Representatives

    Objective, Scope, and Methodology Appendix I

    Our objective was to determine the progress of the Department of Homeland
    Security (DHS) in implementing 18 of our recommendations pertaining to the
    U.S. Visitor and Immigrant Status Indicator Technology (US-VISIT) program.
    To accomplish this objective, we reviewed and analyzed US-VISIT's most
    recent status reports on the implementation of our open recommendations
    and related key documents, augmented as appropriate by interviews with
    program officials. More specifically, we analyzed relevant systems
    acquisition documentation, including the program's process improvement
    plan, risk management plan, and configuration management plan. We also
    analyzed the US-VISIT security plan, privacy impact assessment,
    cost-benefit analysis, cost estimates, test plans, human capital plans,
    and related evaluations and assessments. In performing our analyses, we
    compared available documentation and program officials' statements with
    relevant federal guidance and associated best practices.1 A more detailed
    description of our scope and methodology relative to the cost-benefit
    analysis, cost estimates, and test plans follows:

    o Our analysis of the cost-benefit analysis focused on Increment 1B
    because this was the latest cost-benefit analysis and cost estimate
    prepared. In doing this analysis, we compared the US-VISIT cost-benefit
    analysis to eight criteria in Office of Management and Budget (OMB)
    guidance.2

    o Our analysis of the cost estimate also focused on Increment 1B for the
    same reason previously cited. In doing this analysis, we compared the
    estimate to 13 criteria from the Software Engineering Institute3 that we
    have previously reported to be the minimum set of actions needed to
    develop a reliable cost estimate. We then determined whether the criteria
    were satisfied, partially satisfied, or not satisfied using the
    definitions given below.

    o Our analysis of the test plans focused on Increment 2C because it is the
    most recently tested increment. This analysis included determining the
    extent to which the test plans for this increment met 4 key criteria that
    we have previously reported as essential to effective test plans. In doing
    this analysis, we examined Increment 2C systems documentation, including
    business and functional requirements and traceability matrixes. We also
    independently traced 58 business requirements and 438 functional
    requirements to the test cases in the test plan. Further, we independently
    traced all test cases to the requirements to determine consistency.

    In performing our work, we used the following categories and definitions
    in deciding the extent to which each recommendation had been implemented.
    Specifically, we considered a recommendation

    o completely implemented when documentation demonstrated that it had been
    fully addressed,

    o partially implemented when documentation indicated that actions were
    under way to implement it, and

    o in progress when documentation indicated that action had been initiated
    to implement it.

    These categories and definitions are consistent with those used in our
    prior US-VISIT reports.

    In determining the amount of time it has taken to implement actions on our
    recommendations, we calculated the time from the date the report was
    issued through December 2005.

    We conducted our audit work at the US-VISIT program office in Rosslyn,
    Virginia, from August 2005 through December 2005, in accordance with
    generally accepted government auditing standards.

    Comments from the Department of Homeland Security Appendix II

    Description of US-VISIT Processes Appendix III

    US-VISIT involves complex processes governing the stages of a traveler's
    visit to the United States (pre-entry, entry, status, and exit) and
    analysis of hundreds of millions of foreign national travelers at over 300
    air, sea, and land ports of entry (POE). A simplified depiction of these
    processes is shown in figure 4.

    Figure 4: US-VISIT Process Overview

    Pre-entry Process

    Pre-entry processing begins with initial petitions for visas, grants of
    visa status, or the issuance of travel documentation. When a foreign
    national applies for a visa at a U.S. consulate, biographic and biometric
    data are collected and shared with border management agencies. The
    biometric data are transmitted from the Department of State to DHS, where
    the prints are run against the Automated Biometric Identification System
    (IDENT) database1 to verify identity and to run a check against the
    biometric watch list. The results of the biometric check are transmitted
    back to State. A "hit" response prevents State's system from printing a
    visa for the applicant until the information is reviewed and cleared by a
    consular officer.

    Pre-entry also includes transmission by commercial air and sea carriers of
    crew and passenger manifests to appropriate immigration officers before
    these carriers arrive in the United States.2 These manifests are
    transmitted through the Advanced Passenger Information System (APIS). The
    APIS lists are run against the biographic lookout system to identify those
    arrivals for whom biometric data are available. In addition, POEs review
    the APIS list in order to identify foreign nationals who need to be
    scrutinized more closely.

    Entry Process

    When a foreign national arrives at a POE's primary (air and sea) or
    secondary (land) inspection booth, the inspector, using a document reader,
    scans the machine-readable travel documents. APIS returns any existing
    records on the foreign national to the US-VISIT workstation screen,
    including manifest data matches and biographic lookout hits. When a match
    is found in the manifest data, the foreign national's name is highlighted
    and outlined on the manifest data portion of the screen.

    Biographic information, such as name and date of birth, is displayed on
    the bottom half of the computer screen, along with a photograph obtained
    from State's Consular Consolidated Database.3 The inspector at the booth
    scans the foreign national's fingerprints (left and right index fingers)
    and takes a digital photograph. This information is forwarded to the IDENT
    database, where it is checked against stored fingerprints in the IDENT
    lookout database. If the foreign national's fingerprints are already in
    IDENT, the system performs a match (a comparison of the fingerprint taken
    during the primary inspection to the one on file) to confirm that the
    person submitting the fingerprints is the person on file. If no prints are
    currently in IDENT, the foreign national is enrolled in US-VISIT (i.e.,
    biographic and biometric data are entered into IDENT).

    During this process, the inspector also questions the foreign national
    about the purpose of his or her travel and length of stay. The inspector
    adds the class of admission and duration of stay information into the
    Treasury Enforcement Communications Systems,4 and stamps the "admit until"
    date on the Form I-94.5 If the foreign national is ultimately determined
    to be inadmissible, the person is detained, lookouts are posted in the
    databases, and appropriate actions are taken.

    Status Management Process

    The status management process manages the foreign national's temporary
    presence in the United States, including the adjudication of benefits
    applications and investigations into possible violations of immigration
    regulations.

    As part of this process, commercial air and sea carriers transmit
    departure manifests electronically for each departing passenger. These
    manifests are transmitted through APIS and shared with the Arrival
    Departure Information System (ADIS).6 ADIS matches entry and exit manifest
    data (i.e., each record showing a foreign national entering the United
    States is matched with a record showing the foreign national exiting the
    United States). ADIS also receives status information from the Computer
    Linked Application Information Management System7 and the Student Exchange
    Visitor Information System8 on foreign nationals.

    Exit Process

    The exit process includes the carriers' submission of electronic manifest
    data to APIS. This biographic information is transmitted to ADIS, where it
    is matched against entry information. At the 11 POEs where the exit
    solution is being implemented, the departure is processed by one of three
    exit methods. Within each port, one or more of the exit methods may be
    used. The three methods are as follows:

    o Kiosk: At the kiosk, the traveler, guided by a workstation attendant if
    needed, scans the machine-readable travel documents, provides electronic
    fingerprints, and has a digital photograph taken. A receipt is printed to
    provide documentation of compliance with the exit process and to assist in
    compliance on the traveler's next attempted entry to the country. After
    the receipt prints, the traveler proceeds to his or her departure gate. At
    the conclusion of the transaction, the collected information is
    transmitted to IDENT.

    o Mobile device: At the departure gate, and just before the traveler
    boards the departure craft, either a workstation attendant or law
    enforcement officer scans the machine-readable travel documents, scans the
    traveler's fingerprints (right and left index fingers), and takes a
    digital photograph. A receipt is printed to provide documentation of
    compliance with the exit process and to assist in compliance on the
    traveler's next attempted entry to the country. The device wirelessly
    transmits the captured data in real time to IDENT via the Transportation
    Security Administration's Data Operations Center.

    If the device is being operated by a workstation attendant, he or she
    provides a printed receipt to the traveler, and the traveler then boards
    the departure craft. If the mobile device is being operated by a law
    enforcement officer, the captured biographic and biometric information is
    checked in near real time against watch lists. Any potential match is
    returned to the device and displayed visually for the officer. If no match
    is found, the traveler is allowed to board the departure craft.

    o Validator: Using a kiosk, the traveler, guided by a workstation
    attendant if needed, scans the machine-readable travel documents, provides
    electronic fingerprints, and has a digital photograph taken.

    As with the kiosk, a receipt is printed to provide documentation of
    compliance with the exit process and to assist in compliance on the
    traveler's next attempted entry to the country. However, this receipt has
    biometrics (i.e., the traveler's fingerprints and photograph) embedded on
    the receipt. At the conclusion of the transaction, the collected
    information is transmitted to IDENT.

    The traveler presents his or her receipt to the attendant or law
    enforcement officer at the gate or departure area, who scans the receipt
    using a mobile device. The traveler's identity is verified against the
    biometric data embedded on the receipt. Once the traveler's identity is
    verified, he or she is allowed to board the departure craft. The captured
    data are not transmitted in real time back to IDENT. Instead, the data are
    periodically uploaded through the kiosk to IDENT.

    Analysis Process

    An analysis capability is to provide for the continuous screening against
    watch lists of individuals enrolled in US-VISIT for appropriate reporting
    and action. As more entry and exit information becomes available, it is to
    be used for analysis of traffic volume and patterns as well as for risk
    assessments. The analysis is also to be used to support resource and
    staffing projections across POEs, strategic planning for integrated border
    management analysis performed by the intelligence community, and
    determination of travel use levels and expedited traveler programs.

    GAO Contact and Staff Acknowledgments Appendix IV

    Randolph C. Hite, (202) 512-3439 or h iter@gao.gov

    In addition to the contact named above, the following people made key
    contributions to this report: Deborah Davis, Assistant Director; Hal
    Brumm; Tonia Brown; Joanna Chan; Barbara Collier; Neil Doherty; Jennifer
    Echard; James Houtz; Scott Pettis; Karen Richey; and Karl Seifert.

    (310606)

    www.gao.gov/cgi-bin/getrpt? GAO-06-296 .

    To view the full product, including the scope

    and methodology, click on the link above.

    For more information, contact Randolph C. Hite at (202) 512-3439 or
    hiter@gao.gov.

    Highlights of GAO-06-296 , a report to congressional requesters

    February 2006

    HOMELAND SECURITY

    Recommendations to Improve Management of Key Border Security Program Need
    to Be Implemented

    The Department of Homeland Security (DHS) has established a program-the
    U.S. Visitor and Immigrant Status Indicator Technology (US-VISIT)-to
    collect, maintain, and share information, including biometric identifiers,
    on selected foreign nationals entering and exiting the United States.
    US-VISIT uses these identifiers (digital fingerscans and photographs) to
    screen persons against watch lists and to verify that a visitor is the
    person who was issued a visa or other travel document. Visitors are also
    to confirm their departure by having their visas or passports scanned and
    undergoing fingerscanning at selected air and sea ports of entry (POE).

    GAO has made many recommendations to improve the program, all of which DHS
    has agreed to implement. GAO was asked to report on DHS's progress in
    responding to 18 of these recommendations.

    What GAO Recommends

    GAO is closing its existing recommendation related to DHS's assessment of
    Increment 2B and recommending that DHS explore alternative means to fully
    assess the impact of US-VISIT entry capabilities on land POEs. In its
    comments on a draft of this report, DHS stated that it agreed with many
    areas of the report and disagreed with others. It also concurred with the
    need to quickly implement GAO's open recommendations.

    The current status of DHS's implementation of the 18 recommendations is
    mixed, but progress in critical areas has been slow. DHS has implemented 2
    of the recommendations: it defined program staff positions, roles, and
    responsibilities, and it hired an independent verification and validation
    contractor. It has also taken steps to implement the other
    recommendations, partially completing 11 and beginning to implement
    another 5.

               o  In September 2003, GAO reported that the program had not
               assessed the costs and benefits of Increment 1 (which provides
               entry capabilities to air and sea POEs) and recommended that the
               program determine whether proposed increments will produce mission
               value commensurate with cost. In the latest cost-benefit analysis,
               dated June 23, 2005, the program identified potential costs and
               benefits for three alternatives for an air and sea exit solution.
               However, the analysis does not meet key Office of Management and
               Budget criteria; for example, it does not include a complete
               uncertainty analysis, which helps to provide decision makers with
               perspective on the potential variability of the cost and benefit
               estimates should circumstances change.
               o  GAO reported in May 2004 and February 2005 that system testing
               was not based on well-defined test plans and recommended that
               before testing begins, the program develop and approve test plans
               meeting certain criteria. However, although the latest test plan
               did cover many required areas (such as the tests to be performed),
               it did not adequately trace between test cases and the
               requirements to be verified by testing. Without complete and
               traceable test plans, the risk is increased that the deployed
               system will not perform as intended.
               o  In May 2004, GAO reported that the program had not assessed its
               workforce and facility needs for Increment 2B (which extends entry
               capabilities to the 50 busiest land POEs) and recommended that it
               do so. Since then, the program evaluated the processing times to
               issue and process entry/exit forms at 3 of the 50 busiest POEs and
               concluded that the results showed that no additional staff and
               only minor facilities modifications were required. However, the
               scope of the evaluation was limited. Since then, DHS has deployed
               and implemented Increment 2B capabilities to these 50 POEs, making
               the collection of predeployment baseline data for these sites
               impractical. Nonetheless, other alternatives, such as surveying
               site officials about the increment's impacts, have yet to be
               explored. Until they are, the program may not be able to
               accurately project resource needs or make any needed modifications
               to achieve its goals of minimizing US-VISIT's impact on POE
               operations, which was the impetus for GAO's recommendation.

    DHS attributed the pace of progress to competing demands on time and
    resources. The longer that US-VISIT takes to implement the
    recommendations, the greater the risk that the program will not meet its
    stated goals on time and within budget.
    *** End of document. ***