    Homeland Security: Successes and Challenges in DHS's Efforts to
    Create an Effective Acquisition Organization (29-MAR-05,
    GAO-05-179).

    Department of Homeland Security (DHS) organizations are expected
    to work together to protect the United States from terrorism. To
    support this primary mission, DHS has been acquiring billions of
    dollars worth of goods and services. DHS also has been working to
    integrate the disparate acquisition processes and systems that
    organizations brought with them when DHS was created 2 years ago.
    GAO was asked to identify (1) areas where DHS has been successful
    in promoting collaboration among its various organizations and
    (2) areas where DHS still faces challenges in integrating the
    acquisition function across the department. GAO was also asked to
    assess DHS's progress in implementing an effective review process
    for major, complex investments.
    -------------------------Indexing Terms-------------------------
    REPORTNUM:   GAO-05-179
        ACCNO:   A20237
      TITLE:     Homeland Security: Successes and Challenges in DHS's
    Efforts to Create an Effective Acquisition Organization
         DATE:   03/29/2005
      SUBJECT:   Accountability
    	     Counterterrorism
    	     Federal agency reorganization
    	     Federal procurement
    	     Federal procurement policy
    	     Homeland security
    	     Performance measures
    	     Policy evaluation
    	     Procurement planning
    	     Procurement practices
    	     Risk management
    	     Strategic planning
    	     Policies and procedures

    ******************************************************************
    ** This file contains an ASCII representation of the text of a  **
    ** GAO Product.                                                 **
    **                                                              **
    ** No attempt has been made to display graphic images, although **
    ** figure captions are reproduced.  Tables are included, but    **
    ** may not resemble those in the printed version.               **
    **                                                              **
    ** Please see the PDF (Portable Document Format) file, when     **
    ** available, for a complete electronic file of the printed     **
    ** document's contents.                                         **
    **                                                              **
    ******************************************************************
    GAO-05-179

    United States Government Accountability Office

    GAO

                           Report to Congressional Committees

    March 2005

    HOMELAND SECURITY

      Successes and Challenges in DHS's Efforts to Create an Effective Acquisition
                                      Organization

    GAO-05-179

    [IMG]

    March 2005

    HOMELAND SECURITY

    Successes and Challenges in DHS's Efforts to Create an Effective Acquisition
    Organization

                                     What GAO Found

    DHS's disparate organizations have quickly established collaborative
    relationships to leverage spending for various goods and services without
    losing focus on small businesses. DHS is using strategic sourcing, that
    is, formulating purchasing strategies to meet departmentwide requirements
    for specific commodities, such as office supplies, boats, energy, and
    weapons. By fostering collaboration, DHS has leveraged its buying power
    and savings are expected to grow. Also off to a good start is the small
    business program, whose reach is felt across DHS. Representatives have
    been designated in each DHS procurement office to help ensure that small
    businesses have opportunities to compete for DHS's contract dollars.

    In contrast, lack of clear accountability is hampering DHS's efforts to
    integrate the acquisition functions of its numerous organizations into an
    effective whole. DHS remains a collection of disparate organizations, many
    of which are performing functions with insufficient oversight, giving rise
    to an environment rife with challenges, as shown in the following table.

    Challenges Facing DHS's Efforts to Integrate Acquisition Functions Problem
    areas Challenges

    Overall integration 	New policy emphasizes need for unified, integrated
    acquisition organization but allows U.S. Coast Guard and U.S. Secret
    Service to remain exempt from integration efforts. Possible workload
    imbalances have not been addressed, nor has the lack of enforcement of
    program managers' training and certification.

    Dual accountability	Some of the primary duties delegated to the Chief
    Procurement Officer have also been given to heads of DHS's organizations,
    resulting in confusion over who is ultimately accountable for acquisition
    decisions.

    Chief Procurement Office has lacked sufficient staff to ensure compliance
    with DHS's Officer's oversight staff acquisition regulations and policies.

    Office of Procurement Created almost 1 year after DHS was formed to
    support the Operations' use of organizations that lacked their own
    procurement support, interagency contracting Procurement Operations lacks
    sufficient staff and relies heavily on

    interagency contracting, but missing are management controls to properly
    oversee this activity, including fees paid to other agencies.

    Source: GAO analysis.

    Some of DHS's organizations have major, complex acquisition programs that
    are subject to a multitiered investment review process to help reduce risk
    and increase chances for successful outcomes in terms of cost, schedule,
    and performance. Part of the review process features a knowledge-based
    acquisition approach pioneered by successful commercial firms. DHS's
    adaptation of this best practices approach, however, does not require two
    critical management reviews and is missing some key information before
    decisions are made to invest additional resources. In addition, contractor
    tracking and oversight is not fully incorporated into DHS policy and
    guidance. Finally, some aspects of the review process-which has been under
    revision for many months-need clarification.

                     United States Government Accountability Office

    Contents

    Letter

    Results in Brief
    Background
    Initiatives to Leverage Buying Power and Small Business Program

    Have Helped Foster Collaboration among DHS Organizations
    Goal of Creating an Integrated Acquisition Organization Is
    Hampered by Unclear Policy Decisions and Staffing Disparities

    Despite Adoption of Many Best Practices, Review Process for
    Major Investments Lacks Key Reviews and Some Management
    Controls

    Conclusions
    Recommendations for Executive Action
    Agency Comments and Our Evaluation

                                           1

                                          2 4

                                           7

    11

    21 28 28 30

    Appendix I Scope and Methodology

    Appendix II	Comments from the Department of Homeland Security

    Appendix III Office of Procurement Operations' Customers

    Appendix IV	Summary of Selected Commodity Council Strategic Sourcing
    Initiatives

    Appendix V Selected Acquisition Management Best Practices

    Appendix VI GAO Contacts and Staff Acknowledgments 41

    Contacts 41
    Staff Acknowledgments 41

    Tables

    Table 1: DHS's Principal Organizations and Their Missions 5 Table 2:
    Sampling Error at the 95 Percent Confidence Level for the Sample of
    Interagency Agreements 34 Table 3: Sampling Error at the 95 Percent
    Confidence Level for the Sample of Interagency Agreements Fee Payments 34

    Figures

    Figure 1: DHS's Directorates and Other Principal Organizations and Sources
    of Contracting Support 6 Figure 2: Reported Monthly Monetary Benefits
    Generated by DHS's Strategic Sourcing Program 9 Figure 3: Comparison of
    Fiscal Year 2004 Dollars Obligated per Contracting Staff within Each DHS
    Contracting Office 18 Figure 4: General Depiction of DHS's Investment
    Review Process for Major, Complex Investments 23 Figure 5: Applying the
    Knowledge-Based Approach to DHS's Acquisition Framework 25

    Abbreviations

    DHS Department of Homeland Security
    DOD Department of Defense
    MANPADS man-portable air defense systems
    US-VISIT U.S. Visitor and Immigrant Status Indicator Technology

    This is a work of the U.S. government and is not subject to copyright
    protection in the United States. It may be reproduced and distributed in
    its entirety without further permission from GAO. However, because this
    work may contain copyrighted images or other material, permission from the
    copyright holder may be necessary if you wish to reproduce this material
    separately.

    United States Government Accountability Office Washington, DC 20548

    March 29, 2005

    The Honorable Tom Davis
    Chairman
    Committee on Government Reform
    House of Representatives

    The Honorable Susan M. Collins
    Chairman
    Committee on Homeland Security

    and Governmental Affairs
    United States Senate

    Since it was established, in March 2003,1 the Department of Homeland
    Security (DHS) has been faced with assembling 23 separate federal
    agencies and organizations with multiple missions, values, and cultures
    into one cabinet-level department.2 This mammoth task-one of the
    biggest mergers ever to take place within the federal government-
    involves a variety of transformational efforts, one of which is to design
    and
    implement the necessary management structure and processes for
    acquiring goods and services. In January 2003, we designated DHS's
    implementation and transformation as high-risk because of the size and
    complexity of the effort and the existing challenges faced by the
    components being merged into the department.3 As it progresses through
    the early stages of its merger and transformation, DHS has an opportunity
    to put into place the necessary elements to become a 21st century federal
    department with a high-quality acquisition organization that effectively
    supports its critically important missions.

    DHS has some of the most extensive acquisition needs within the U.S.
    government. In fiscal year 2004, the department obligated $9.8 billion to
    acquire a wide range of goods and services-such as information systems,
    new technologies, weapons, aircraft, ships, and professional services. At

    1The President signed legislation to create DHS on November 25, 2002.
    Homeland Security Act of 2002, Pub. L. No. 107-296, 116 stat. 2135, Nov.
    25, 2002.

    2When the department was established, 22 agencies and organizations were
    brought in; Plum Island Animal Disease Center joined DHS afterward.

    3GAO, High-Risk Series: An Update, GAO-03-119 (Washington, D.C.: January
    2003).

                        the same time, DHS is working to integrate the many
                        acquisition processes and systems that the disparate
    					agencies and organizations brought with them. Given DHS's
    					complex merger, you asked us to review the department's
    					progress in establishing an effective acquisition
    					organization. In response to your request, we (1)
    					identified areas where DHS has promoting
    					collaboration among its various organizations and (2)
    					identified  areas where DHS still faces challenges in
    					integrating the acquisition  function across the department.
    					We also assessed the department's progress in implementing
    					an effective review process for its major, complex investments.
                        To conduct this work, we assessed information from DHS
                        headquarters and the department's principal organizations
    					and compared the information against what our previous work
    					be best acquisition practices. We reviewed agency directives,
                        memorandums, and other documentation; interviewed agency
    					officials; and analyzed agency systems and processes.
    					For the purposes of this report, the term acquisition
    					refers to the overall effort to acquire goods and services and
                        involves a host of activities. Acquisition begins when an
                        agency establishes its requirements and moves through a
    					process that involves soliciting providers of goods and
    					services, awarding contracts, monitoring performance, and
    					handling various contract administration duties. In some
                        cases, acquisition can include development of a new
                        product, such as a weapon system or a database system.
    					Acquisition also can be the simple purchase of existing
    					products. The Federal Acquisition Regulation, which
                        governs acquisitions within the federal government,
                        defines the term procurement as being synonymous with
    					acquisition. 4 Appendix I presents our scope and
    					methodology in more detail. We conducted our review from
                        March 2004 through February 2005 in accordance with
                        generally accepted government auditing standards.

    4Federal Acquisition Regulation 2.1, Definitions.

    Results in Brief

    In the relatively short time since its inception, DHS has
    demonstrated some successes in implementing a strategic
    sourcing program to leverage the department's buying power and in creating a small
    business program.5

    5Strategic sourcing is a process used by leading commercial companies and
    a small number of federal agencies to establish an organizationwide
    approach to leveraging the organization's buying power and fostering new
    ways of doing business.

    Both of these efforts have fostered an environment in which DHS's various
    organizations work collaboratively toward a common goal. The cornerstone
    of the strategic sourcing program is the use of commodity councils, each
    consisting of a cross-functional group of acquisition-related personnel
    and specialists from the various organizations. These councils are charged
    with formulating purchasing strategies to meet departmentwide requirements
    for specific commodities, such as office supplies and the department's
    vehicle fleet. Even though the councils have faced challenges in gathering
    accurate spending data and in managing workload, they have helped spur
    collaboration and cooperation across the department. Moreover, DHS already
    has reported $14 million in dollar savings as a result of leveraging
    resources across DHS, and officials expect the savings to grow over time.
    Another area of early success is DHS's small business program, which has
    been proactive in working with each organization in the department to
    ensure that small businesses have the opportunity to compete for DHS
    dollars. In fiscal year 2004, DHS reported that 35 percent of its prime
    contract dollars went to small businesses, exceeding its goal of 23
    percent.

    Notwithstanding these initial successes, DHS's progress in creating a
    unified acquisition organization has been slowed by policy decisions that
    create ambiguity and by procurement staffing disparities within the
    department. An October 2004 management directive emphasizes the need for a
    unified, integrated acquisition organization but relies on a system of
    dual accountability between the Chief Procurement Officer and the heads of
    the department's organizations to make this happen. The Chief Procurement
    Officer has been delegated the responsibility to manage, administer, and
    oversee all acquisition activity across DHS, but in practice enforcement
    of these activities is spread throughout the department with unclear
    accountability. Further, the directive states that the U.S. Coast Guard
    and U.S. Secret Service are statutorily exempt from its application.
    Although the Homeland Security Act provides that both the Coast Guard and
    the Secret Service shall be maintained as distinct entities within the
    department, we found no reasonable basis to conclude that the directive
    could not be made applicable to them. Rather, it appears to be a policy
    decision that is likely to hamper efforts to effectively integrate the
    acquisition function in DHS. To a great extent, the various acquisition
    organizations within the department are still operating in a disparate
    manner, with oversight of acquisition activities left primarily up to each
    individual organization. Significant disparities in the dollar value of
    contracting staff's workloads across DHS have only recently begun to be
    addressed. Staffing shortages have led one organization, which handles
    about $2 billion in obligations for various departmental organizations, to

    rely extensively on outside agencies for contracting support-often for a
    fee. We found that this office lacked adequate internal controls to
    provide oversight of this interagency contracting activity.

    To protect its major, complex investments, DHS's Office of the Chief
    Financial Officer has put in place a review process that adopts many best
    practices-that is, proven methods, processes, techniques, and activities-
    to help the department reduce risk and increase the chances for successful
    outcomes in terms of cost, schedule, and performance. One best practice is
    an overarching, knowledge-based acquisition approach pioneered by
    successful commercial companies. A general principle of this approach is
    that program managers should provide sufficient knowledge about important
    aspects of their programs at key points in the acquisition process, so
    that senior leaders are prepared to make a well-informed investment
    decision before an acquisition moves forward. DHS's framework includes key
    tenets of this approach but does not require two critical management
    reviews to ensure that resources match customer needs and that design
    performs as expected. Also missing is important information to help reduce
    risk and meet cost and delivery targets for major investments. In
    addition, DHS's review process does not fully address how program managers
    will conduct effective contractor tracking and oversight. We also found
    that program managers lacked sufficient guidance about how to navigate
    investment reviews. The review process has been under revision for many
    months, and DHS officials could not tell us when the process would be
    finalized.

    In this report, we are making recommendations to the Secretary of Homeland
    Security to help ensure that the department's strategic sourcing program
    maintains its current momentum, that acquisition integration efforts
    continue, and that DHS leadership has the information it needs to
    proactively manage risks that arise during the acquisition of major,
    complex systems. In written comments on a draft of this report, DHS
    concurred with the recommendations. DHS's comments are included in their
    entirety in appendix II.

    Background 	The Homeland Security Act of 2002 created DHS, effective March
    1, 2003, by merging agencies and organizations that specialize in one or
    more aspects of homeland security. Some of those specialties are
    intelligence analysis, law enforcement, border security, transportation
    security, biological research, critical infrastructure protection, and
    disaster recovery. The intent behind DHS's merger and transformation was
    to improve coordination, communication, and information sharing among the

    multiple federal agencies responsible for protecting the homeland.
    Critical to performing the homeland security mission is the effective
    interaction between and integration of these agencies and organizations.
    Table 1 shows DHS's eight principal organizations and their missions.

    Table 1: DHS's Principal Organizations and Their Missions

                           Principal organizationsa Missions

    Border and Transportation Security Directorate Ensures security of U.S.
    borders and transportation systems

    Enforces the nation's immigration laws

    Manages and coordinates port-of-entry activities and oversees protection
    of government buildings

    Emergency Preparedness and Response Directorate 	Prepares for catastrophes
    Oversees federal government's national response and recovery strategy

    Information Analysis and Infrastructure Protection Identifies and assesses
    threats

    Directorate 	Recommends measures necessary to protect key resources and
    critical infrastructure

    Science and Technology Directorate Coordinates DHS's efforts in research
    and development

    Management Directorate 	Administers DHS's budget, financial management
    systems, procurement activities, human resources functions, information
    technology systems, facilities management, and performance measurement
    efforts

    U.S. Secret Service 	Protects U.S. President and other designated
    personnel, as well as the country's currency and financial infrastructure,
    and provides security for designated national events

    U.S. Coast Guard 	Protects the public, the environment and U.S. economic
    interests in the nation's ports and waterways, coasts, international
    waters, or any maritime region as required to support national security;
    has terrorism, counternarcotics border protection roles; and prevents
    illegal incursion of U.S. exclusive economic zone

    U.S. Citizenship and Immigration Services 	Directs immigration benefit
    system and promotes citizenship values by providing immigration services,
    such as immigrant and nonimmigrant sponsorship; adjustment of status; work
    authorization and other permits; naturalization of qualified applicants;
    and asylum or refugee processing

    Sources: DHS (data); GAO (analysis).

    aThis table does not show the organizations that fall under each of the
    five directorates. This table also does not show all organizations that
    report directly to the DHS Secretary and Deputy Secretary, such as
    executive secretary, legislative affairs, public affairs, chief of staff,
    inspector general, and general counsel.

    Of the 23 entities that joined DHS from other agencies, only 7 came with
    their own procurement support. Providing support to the other entities- as
    well as a number of newly created entities, such as the offices of the
    Chief Information Officer and Chief Financial Officer-is an eighth office,
    the Office of Procurement Operations (Procurement Operations). That office
    was not created until January 2004, almost a year after DHS came

    into being. Appendix III lists all of the DHS organizations that receive
    contracting support from Procurement Operations. Figure 1 shows the
    sources of contracting support for DHS's principal organizations.

     Figure 1: DHS's Directorates and Other Principal Organizations and Sources of
                                  Contracting Support

    Source: DHS (data); GAO (presentation).

    To carry out acquisition effectively across a large federal organization
    requires an integrated structure with standardized policies and processes,
    the appropriate placement of the acquisition function within the
    department, leadership that fosters good acquisition practices, and a
    general framework that delineates the key phases along the path for a
    major acquisition. An effective acquisition organization has in place
    knowledgeable personnel who work together to meet cost, quality, and
    timeliness goals while adhering to guidelines and standards for federal
    acquisition.

      Initiatives to Leverage Buying Power and Small Business Program Have Helped
      Foster Collaboration among DHS Organizations

    In the 2 years since its creation, DHS has realized some successes in
    opening the lines of communication among the various organizations within
    the department through its strategic sourcing and small business programs.
    Both of these efforts have involved every principal organization in DHS,
    along with strong involvement from the Chief Procurement Officer, and both
    have yielded positive results. DHS already has begun to demonstrate that
    its strategic sourcing program can foster collaboration across the
    department and at the same time maximize the department's overall buying
    power. DHS's small business program has a presence departmentwide, and
    according to DHS officials, the department exceeded its 23 percent small
    business goal for fiscal year 2004.6

    Strategic Sourcing Program Has Encouraged Partnerships across DHS and
    Begun to Realize Savings

    Under the authority of the Chief Procurement Officer, DHS created a
    strategic sourcing group in October 2003 to leverage departmentwide
    spending for various commodities. The group brought together diverse
    expertise from throughout DHS. To identify commodities with the most
    potential for savings, strategic sourcing officials conducted a spend
    analysis using available acquisition databases, such as the Federal
    Procurement Data System, and input from DHS senior management.7 The

    6The Small Business Reauthorization Act of 1997 directed the President to
    establish a goal of not less than 23 percent of the federal government's
    prime contracting dollars to be awarded to small businesses each fiscal
    year. The Small Business Administration is charged with ensuring that
    federal agencies' goals, in the aggregate, meet or exceed the 23 percent
    goal. Pub. L. No. 105-135, 111 stat. 2592, Dec. 2, 1997.

    7Spend analysis is a tool that organizations use to acquire knowledge
    about how much is being spent for what goods and services, who are the
    buyers, and who are the suppliers. GAO, Best Practices: Using Spend
    Analysis to Help Agencies Take a More Strategic Approach to Procurement,
    GAO-04-870 (Washington, D.C.: Sept. 16, 2004).

    following 15 commodities were identified as having potential to leverage
    the department's buying power:

    o  aviation,

    o  boats,

    o  business wireless communications,

    o  copiers,

    o  energy,

    o  enterprise software agreements,

    o  facilities,

    o  facilities security,

    o  vehicle fleets,

    o  mail,
    o  office supplies,

    o  professional services,

    o  training,

    o  uniforms, and

    o  weapons.

    Consistent with best practices, the strategic sourcing group then
    established commodity councils composed of representatives from across
    DHS. The commodity councils were assigned responsibility for further
    collection and refinement of historical procurement data in order to
    better assess future purchasing strategies. Typically, members from the
    strategic sourcing group and the DHS organization with the most expertise
    in a particular commodity serve as council cochairs. For example, a Coast
    Guard official is a cochair for the boats commodity council.

    Commodity council cochairs said they were willing to devote time to the
    strategic sourcing initiatives because they recognized the unique
    opportunity DHS had to move forward to leverage buying power across the
    department. Further, the cochairs were virtually unanimous in telling us
    that the councils enable stakeholders to build awareness of a particular
    commodity and develop strong relationships throughout DHS. They said the
    councils foster a sense of community in which the various organizations
    can share information, participate in forums, find commonalities, engage
    in open and productive communication, and make smarter and more
    collaborative business decisions. For example, the weapons commodity
    council routinely shares information on ammunition. When one DHS
    organization is low on ammunition, others help meet the need. Appendix IV
    contains more detail on initiatives that several commodity councils have
    undertaken.

    In fiscal year 2004, 4 commodity councils-office supplies, boats, energy,
    and weapons-reported approximately $14.1 million in cost savings and cost
    avoidances, and department officials expect the savings to continue to
    grow.8 The savings have resulted from DHS negotiating lower rates with
    suppliers and leveraging resources across the department. Figure 2 depicts
    the savings trend over a 12-month period. The September 2004 surge
    resulted when authorized DHS employees began purchasing pistols through
    two large contracts, a strategy spearheaded by the weapons commodity
    council.

    Figure 2: Reported Monthly Monetary Benefits Generated by DHS's Strategic
    Sourcing Program

                                 Dollars in millions 6

    Source: DHS (data); GAO (presentation).

    Some councils are encountering a problem faced by many federal departments
    and agencies, namely, a shortage of comprehensive data upon which to draw
    an accurate and detailed picture of what is being spent on certain
    commodities over time. Strategic sourcing officials and commodity council
    members told us that they cannot take full advantage of spend analyses,
    nor can they accurately chart historical spending,

    8Our review found that the department's strategic sourcing cost savings
    methodologies appear reasonable. However, we did not verify the accuracy
    of any strategic procurement cost savings reported to us.

                                           5

                                           4

                                           3

                                           2

                                           1

                                  0 4 43 3 44 4 4 43 4

    v. 200

    oNn. 200

    aJg. 200

    uAc. 200

    eD

    ly 200uJ

    r. 200

    aMr. 200

    pA

    y 200aM

    ne 200uJ

    t. 200

    cO

                                         . 200

    beF

    Baseline Data and Workload Demands Have Presented Challenges

    because DHS's acquisition databases do not contain enough procurement
    data. The problem is compounded by the fact that when parts of existing
    agencies, such as the Immigration and Naturalization Service from the
    Department of Justice, joined DHS, detailed information on its spending
    history was not available. Without an accurate analysis of how DHS
    organizations historically purchased a commodity, council members will
    likely continue to rely on a patchwork of estimates, as well as
    information from suppliers, to glean information on spending history and
    develop purchasing strategies for the future.

    In addition, some commodity council members have found it challenging to
    balance council duties with the demands and responsibilities of their
    full-time positions within DHS. Officials told us that council meetings
    and activities sometimes stall because council members must shift
    attention to their full-time positions. Many commodity councils did not
    make much progress during the last month of fiscal year 2004, we were
    told, because council members' time was diverted to year-end priorities.
    Our prior work on strategic sourcing shows that leading commercial
    companies often establish full-time commodity managers to more effectively
    manage commodities.9 Commodity managers help commodity councils define
    requirements with internal clients, negotiate with potential vendors, and
    resolve performance or other issues arising after a contract is awarded
    and can help maintain consistency, stability, and a long-term strategic
    focus.

    Small Business Program Off to Good Start

    DHS's small business program has also had initial successes, reporting
    that 35 percent of fiscal year 2004 obligations were awarded to small
    business prime contractors, exceeding the department's goal of 23 percent.
    Although reporting directly to the Deputy Secretary of DHS, the Director
    of the Office of Small and Disadvantaged Business Utilization works
    closely with the Chief Procurement Officer to emphasize throughout the
    department the important public policy objective of small business
    inclusion in acquisition activities. The small business office, in
    conjunction with the procurement staff across the department, has created
    an outreach program that advises small businesses on ways to market goods
    and services to DHS. Small business representatives have been designated
    in each DHS procurement office, and each office is required to

    9GAO, Best Practices: Taking a Strategic Approach Could Improve DOD's
    Acquisition of Services, GAO-02-230 (Washington, D.C.: Jan. 18, 2002).

    submit a forecast of upcoming contract opportunities above $100,000. DHS
    posts this information on a Web site so that small businesses can identify
    opportunities to do business with the department. The small business
    office has conducted extensive outreach to DHS's business partners through
    regular seminars and has established a mentor-protege program that is
    designed to motivate and encourage large businesses to provide mutually
    beneficial developmental assistance to small businesses.

    The Director of the Office of Small and Disadvantaged Business Utilization
    and his staff have also been directly involved in DHS's strategic sourcing
    efforts to help ensure that, even as the department leverages its buying
    power, small businesses continue to have opportunities to compete for
    contracts. Several commodity councils have developed strategies to address
    this issue. The office supplies council has worked out an arrangement for
    DHS employees to purchase office supplies from the Department of Defense's
    (DOD) Web-based Emall, where employees can easily identify and order from
    small businesses.10 The head of procurement in the Immigration and Customs
    Enforcement organization, working closely with the weapons commodity
    council, awarded a contract for half of the largest pistol procurement in
    the history of U.S. law enforcement to a small business. According to the
    chair of the boats council, the council plans to consult with the Coast
    Guard's small business specialist to explore future possibilities for
    providing opportunities to small businesses.

    DHS's goal of integrating the acquisition function more broadly across the
    department has not been accomplished, and the introduction of a new policy
    has been unsuccessful in breaking down barriers to effective
    departmentwide management. An integrated acquisition organization is
    essential to the department's success in executing policies and processes
    to effectively obligate and administer billions of dollars in acquiring
    what DHS needs to accomplish its mission. An October 2004 DHS management
    directive emphasizes the need for an integrated acquisition organization
    and reiterates the Chief Procurement Officer's responsibility to manage,
    administer, and oversee all acquisition activity across DHS and to
    establish a qualified acquisition workforce. In practice, these
    responsibilities are

      Goal of Creating an Integrated Acquisition Organization Is Hampered by Unclear
      Policy Decisions and Staffing Disparities

    10DOD Emall is an Internet-based marketplace that allows DOD and other
    federal purchasers to access DOD's wide variety of vendors and catalogs
    and acquire off-the-shelf items from the commercial marketplace.

    spread throughout the department with unclear accountability. Further
    hampering efforts to effectively integrate the acquisition function, the
    directive provides that the Coast Guard and the Secret Service are
    statutorily exempt from its application. We found no reasonable basis to
    conclude that the directive could not be made applicable to them. The
    various organizations within DHS continue to operate in a largely
    disparate manner, with a lack of centralized oversight of compliance with
    the department's acquisition regulation and policies. Staffing disparities
    across the procurement organizations have only recently begun to be
    addressed. We found that staffing shortfalls led Procurement Operations to
    rely extensively on outside agencies for contracting support-often for a
    fee-and that this office lacked adequate internal controls to properly
    manage this interagency contracting activity. Because of the risks
    associated with interagency contracting, we recently designated this
    approach as a high-risk issue.11

    Not Clear How Recent Management Directive Will Drive Integration of the
    Acquisition Organization

    In October 2004, the Secretary of DHS signed a management directive
    entitled "Acquisition Line of Business Integration and Management." This
    directive, the department's principal guidance for "leading, governing,
    integrating, and managing" the acquisition function, states that DHS will
    standardize acquisition policies and procedures and continue to
    consolidate and integrate the number of systems supporting the acquisition
    function. It directs managers from each organization to commit resources
    to training, development, and certification of acquisition professionals.
    The directive also highlights the Chief Procurement Officer's broad
    authority, including

    o  	management, administration, and oversight of departmentwide
    acquisition, financial assistance, strategic sourcing, and competitive
    sourcing programs;

    o  	promotion of career development and establishment of qualifications,
    training, and certification standards for the acquisition and financial
    assistance workforce;

    o  	development and publication of departmentwide acquisition and
    financial assistance regulations, directives, policies, and procedures;

    o  designation of all heads of contracting activities; and

    11GAO, High-Risk Series: An Update, GAO-05-207 (January 2005, Washington,
    D.C.).

    o  	development and maintenance of contracting officer warrant and
    financial assistance officer programs, including designation of qualified
    persons as contracting officers and financial assistance officers.

    However, the directive may not achieve its goal of creating an integrated
    acquisition organization, because it creates unclear working relationships
    between the Chief Procurement Officer and heads of DHS's principal
    organizations.12 For example, the Chief Procurement Officer and the
    director of Immigration and Customs Enforcement share responsibility for
    recruiting and selecting key acquisition officials, preparing performance
    ratings for the top manager of the contracting office, and providing
    appropriate resources to support the Chief Procurement Officer's
    initiatives. The policy leaves unclear how the responsibilities will be
    implemented or what enforcement authority the Chief Procurement Officer
    has to ensure that initiatives are carried out. In addition, directors are
    only required to "consider" the allocation of resources to meet
    procurement staffing levels in accordance with the Chief Procurement
    Officer's analysis. Agreements have not been developed on how the
    resources to train, develop, and certify acquisition professionals in the
    principal organizations will be identified or funded. In a relatively new
    department like DHS, which is still in the process of instituting
    procedures, developing a strong organizational culture, and establishing
    clear roles and missions, this concept of dual accountability-absent
    effective implementing guidance-may not accomplish the intended goals.

    The October 2004 management directive does not apply to the Coast Guard or
    the Secret Service, further hampering efforts to integrate the acquisition
    organization. The Coast Guard is one of the largest organizations within
    DHS, with obligations accounting for about $2.1 billion in fiscal year
    2004, nearly 23 percent of the department's total. According to the
    directive, the Coast Guard is exempted by statute. We disagree. We are not
    aware of any explicit statutory exemption that would prevent the
    application of this directive. While several provisions in the Homeland
    Security Act would limit the range of management initiatives concerning
    the Coast Guard, none of them would appear to be applicable

    12Two prior GAO reports, Transportation Security Administration:
    High-Level Attention Needed to Strengthen Acquisition Function, GAO-04-544
    (Washington, D.C.: May 28, 2004), and Contract Management: INS Contracting
    Weaknesses Need Attention from the Department of Homeland Security,
    GAO-03-799 (Washington, D.C.: July 25, 2003), address the importance of
    having clear lines of authority to ensure that contracting activity is
    effective and enables the department to get best value on goods and
    services.

    in this case. For example, the Homeland Security Act requires the Coast
    Guard to be maintained as a distinct entity within the department.13 Other
    limitations prevent the transfer of assets,14 alteration of missions,15
    and changes in reporting relationships.16 The act also expressly provides
    that "the authorities, functions, and capabilities of the Coast Guard to
    perform its missions shall be maintained intact and without significant
    reduction after the transfer of the Coast Guard to the department." We
    find nothing in the directive that contravenes these limitations. Nothing
    in the document would reasonably appear to threaten the status of the
    Coast Guard as a distinct entity or otherwise impair its ability to
    perform statutory missions.

    We raised the question of statutory exemption with the DHS General
    Counsel, who shared our assessment concerning the explicit statutory
    exemptions. He viewed the applicability of the management directive as a
    policy matter, noting that "the determination of whether the application
    of all or part of this [management directive] would impact the [Coast
    Guard's] ability to perform its mission is not a legal matter and is more
    appropriately made by DHS policy officials." We agree that DHS officials,
    with sufficient reasons, could make a policy decision that a particular
    management directive impacts the Coast Guard's ability to perform its
    missions. In this instance, however, we found no evidence that such a
    decision had been made.

    The directive also provides that the Secret Service is exempted by
    statute. As with the Coast Guard, we are unaware of any specific statutory
    exemption that would prevent the application of the directive. The
    Homeland Security Act requires the Secret Service to be maintained as a
    distinct entity within the department.17 The 2005 Homeland Security
    Appropriations Act reiterates this requirement and imposes additional

    13Pub. L. No. 107-296, S: 888(b) November 25, 2002.
    14Id. at S: 888(d).
    15Pub. L. No. 107-296, S: 888(e), November 25, 2002.
    16Id. at S:S:104 and 888(g).
    17Pub. L. 107-296, S: 521, November 25, 2002.

    limitations on altering reporting relationships.18 Given the nature of the
    management directive, we also conclude that there is no apparent reason to
    exempt the Secret Service from its application.19

    In Environment of Largely Disparate Procurement Organizations, Oversight
    of Acquisition Activity Lacking

    DHS's principal organizations are, to a large extent, still functioning
    much as they did in premerger days with regard to acquisition-related
    functions. Embedded within seven of the procurement organizations are, for
    the most part, the same contracting staffs that joined DHS from their
    former agencies. The eighth organization, Procurement Operations, created
    to meet the needs of the many DHS organizations that do not have colocated
    procurement support, has a direct reporting chain to the Chief Procurement
    Officer.

    Until recently, the Chief Procurement Officer, whom DHS's top leadership
    delegated with the key responsibility of ensuring compliance with the
    department's acquisition regulation and policies, had only two staff
    members to carry out this duty. Consequently, the department's acquisition
    oversight program relies extensively on self-assessments by personnel in
    each procurement organization. A component of the oversight program is a
    recent initiative to review DHS acquisition plans, according to dollar
    thresholds, and to perform on-site evaluations of each procurement
    organization. The fiscal year 2005 budget provided the Chief Procurement
    Officer with five additional staff, but it is too soon to tell whether
    this number will be adequate to effectively implement the oversight
    program. Further, it remains unclear what the result would be if an
    organization were found not to be in compliance with DHS's acquisition
    regulation and policies.

    Our prior work shows that in a highly functioning acquisition
    organization, the chief procurement officer is in a position to oversee
    compliance with

    18Pub. L. 108-334, S: 528, October 18, 2004. "None of the funds available
    in this Act shall be available to maintain the United States Secret
    Service as anything but a distinct entity within the Department of
    Homeland Security and shall not be used to merge the United States Secret
    Service with any other department function, cause any personnel and
    operational elements of the United States Secret Service to report to an
    individual other than the Director of the United States Secret Service, or
    cause the Director to report directly to any individual other than the
    Secretary of Homeland Security." Department of Homeland Security
    Appropriations Act, 2005, Pub. L. No. 108-334, S: 521, 118 stat. 1298,
    Oct. 18, 2004.

    19Given the similarity of the issue pertaining to the Coast Guard, we did
    not separately seek the view of the DHS General Counsel concerning the
    exemption of the Secret Service.

    acquisition policies and processes by implementing strong oversight
    mechanisms.20 Adequate oversight of acquisition activities across DHS is
    imperative, in light of the department's mission and the problems that
    have been reported by us and inspectors general for some of the large
    agencies and organizations within the department. These reports have
    highlighted the lack of important management controls for monitoring
    contractors and ensuring efficiencies and effectiveness in the acquisition
    process. For example, the Department of Homeland Security Inspector
    General reported that during its first year of operation, the
    Transportation Security Administration relied extensively on contractors
    to accomplish its mission. It also found that contracting officers wrote
    contracts without clearly defined deliverables, and on occasion,
    contractors themselves were permitted to determine requirements and define
    deliverables. As a result, the cost of those initial contracts
    ballooned.21 The Transportation Security Administration is in the process
    of devising policies and procedures that require adequate procurement
    planning, contract structure, and contract oversight. We have also
    reported that the former U.S. Immigration and Naturalization Service did
    not have the basic infrastructure-including oversight of procurement
    activities-to ensure that its contracting office was effective. DHS's
    Inspector General recently reported that the Federal Emergency Management
    Agency discovered it has not been reporting or tracking procurements
    handled by its field offices.22

    Challenges to Effective In July 2003, we recommended that DHS develop a
    data-driven assessment Implementation of of the department's acquisition
    personnel, resulting in a workforce plan Acquisition Workforce that would
    identify the number, location, skills, and competencies of the Plan
    workforce.23 DHS concurred with the recommendation and has drafted a

    plan, based on best practices, that defines the acquisition workforce,
    focuses on the need for continuous training, and implements a
    certification program for contracting officials, program managers, and

    20GAO-02-230 and GAO-04-544.

    21DHS Office of the Inspector General, Semiannual Report to the Congress,
    (Washington, D.C.: April 30, 2003), and Review of the Status of Department
    of Homeland Security Efforts to Address Its Major Management Challenges,
    OIG-04-21 (Washington, D.C.: March 2004).

    22OIG-04-21.

    23GAO-03-799.

    contracting officers' technical representatives.24 However, the department
    faces challenges in implementing the plan.

    As part of its acquisition workforce planning efforts, the department has
    not conducted an assessment of whether contracting staff within DHS are
    appropriately distributed for the varying workloads in each procurement
    organization. Our analysis shows that some disparities may exist. We
    divided the obligated fiscal year 2004 dollars for each contracting office
    by the number of contracting staff. While this approach is limited in that
    it does not take into account the complexity of the acquisitions being
    performed, it can provide senior leadership with an indication of whether
    disparities may exist in the contracting workforce. Figure 3 shows the
    amount of contracting obligations per contracting staff within DHS
    contracting activities.

    24Contracting officers' technical representatives represent the
    contracting officer in monitoring the contractor's performance.

    Figure 3: Comparison of Fiscal Year 2004 Dollars Obligated per Contracting
    Staff within Each DHS Contracting Office

    Dollars in millions 110

    100 90

    80

    70

    60

    50

    40

    30

    20

    10 0

    Customs and Customs EnfderBor

    ansportation rT

    Security Administration

                       otection ency Preparedness Secret Service

    cementor

    Enfaining CenterrT

    wcement

    oror

    ment - Office ofeUndersecretar g

                         Immigration and dCoast Guar Federal La

    y f

                                           Pr

    S..U

    S..U

    ManaocurPr

    gEmer

    Source: DHS (data); GAO (analysis and presentation).

    As of September 2004, Procurement Operations had only 19 contracting staff
    to support a number of DHS organizations that, taken together, accounted
    for about 21 percent, or almost $2 billion, of DHS's fiscal year 2004
    obligations. That year, Procurement Operations contracting staff on the
    average handled $101 million per employee, whereas the contracting staff
    for the Federal Law Enforcement Training Center on the average handled
    close to $2.7 million per employee. Disparities such as this may indicate
    the need to assess the numbers of contracting staff across the department
    to determine whether imbalances exist and whether actions are needed to
    correct the imbalances.

    Another challenge to effectively implementing the acquisition workforce
    plan pertains to the lack of enforcement of DHS's certification program.
    While the plan calls for program managers responsible for acquisitions to
    be certified in accordance with DHS's established training and experience
    requirements, in practice, the means to enforce compliance is lacking. In

    January 2005, the Director for the Acquisition Workforce Program, within
    the Office of the Chief Procurement Officer, determined that only 22
    percent of the identified programs in the department had program managers
    that had documented that they had the training and experience requirements
    for certification. While the Office of the Chief Procurement Officer has
    issued a management directive requiring program mangers to meet the
    department's certification and training requirements-and the number of
    certified program managers has been increasing- accountability for
    complying with the certification program rests with the principal
    organizations to whom the program managers report. At present, according
    to DHS officials, no mechanism is in place to ensure that program managers
    take the required training and obtain certification from the Chief
    Procurement Officer.

    Struggling with Staffing Shortfalls, Procurement Operations Has Relied on
    Interagency Agreements but Failed to Follow DHS Guidance in Doing So

    Established almost 1 year after DHS was formed and tasked with providing
    contracting support to the department organizations that did not have
    their own contracting support, Procurement Operations has struggled to
    manage its almost $2 billion workload because of staffing shortfalls.
    Lacking in-house capability, Procurement Operations has turned extensively
    to interagency contracting, and we found that management controls were not
    in place to effectively oversee this activity. Interagency contracting
    occurs when a federal agency obtains supplies or services through another
    federal agency, either by placing orders on existing contracts that have
    already been awarded by the other agency, or by asking the other agency to
    award and administer contracts or issue and administer task orders on its
    behalf. Use of these contracts demands a high degree of business acumen
    and flexibility on the part of the federal acquisition workforce.

    We found that Procurement Operations had transferred almost 90 percent of
    its obligations to other federal agencies through interagency agreements
    in fiscal year 2004. For example, DHS transferred $12 million to the
    Department of the Interior's National Business Center to obtain contractor
    operations and maintenance services at the Plum Island Animal Disease
    Center. Interior charged DHS $62,000 for this assistance. While some of
    the interagency agreements were for contracting support, others were for
    program support, such as sending funds to Department of Energy
    laboratories for providing a threat and capability assessment.

    DHS has issued a management directive that sets forth a number of
    requirements meant to ensure that internal controls are in place when
    using interagency contracting. Based on a random sample of

    136 interagency agreements between Procurement Operations and outside
    agencies, we found that the office had not complied with the requirements
    in the directive.25 We can project, for example, that in fiscal year 2004:

    o  	94 percent of Procurement Operations' files did not document that the
    contracting staff had conducted the required analysis of alternatives to
    justify the decision to pay an outside agency for contracting support;

    o  	47 percent of the files did not identify the contracting officer's
    technical representative, although this information is required to be in
    the files;

    o  	35 percent of the files did not contain the required determination and
    findings;26 and

    o  	96 percent of the files lacked an indication that contractor oversight
    had been performed.

    Further, we found that Procurement Operations was not tracking how much it
    is paying in fees to other agencies for contracting support. On the basis
    of our sample, we found that the office had spent $12.9 million in fees in
    fiscal year 2004.

    While the oversight problems we identified are in large part due to the
    staffing shortages in the office, we also found evidence that contracting
    staff lacked basic information on how to use interagency contracting. For
    example, a memo from Procurement Operations to DHS's Director of
    Acquisition Policy and Oversight requested clarification on what
    documentation is required in order to use another agency for contracting
    support. In September 2004, DHS's Office of General Counsel reviewed 20 of
    Procurement Operations' interagency agreements and found that 16 were not
    legally sufficient. For example, 13 agreements appeared to require
    performance by a contractor, but appropriate documentation was not
    included in the contract files. Three were insufficient because it was
    unclear whether the servicing agency would perform cost/price analysis and
    trade-off, or whether Procurement Operations had conducted a technical
    competition and expected the servicing agency to award without a
    trade-off. According to a Procurement Operations official, a fiscal year

    25See appendix I for details on our methodology and sampling error rates.

    26Determination and findings is a written approval by an authorized
    official to take certain contract actions. The determination is a
    conclusion or decision supported by the findings. The findings are
    statements of fact or rationale essential to support the determination.

    2005 initiative for the office is to establish a policy for processing
    interagency agreements that provides clear guidance to staff.

    Procurement Operations to Use Working Capital Fund to Increase Staffing
    Levels

      Despite Adoption of Many Best Practices, Review Process for Major Investments
      Lacks Key Reviews and Some Management Controls

    Since January 2004, Procurement Operations has increased its staffing
    level from 7 to 42 employees, and it plans to build to 127 staff in fiscal
    year 2005. Rather than use direct appropriations to fund the additional
    positions, the office plans to require the DHS organizations that rely on
    its contracting services to contribute to a working capital fund, to be
    replenished on a no-profit basis with payments based on the extent to
    which the various organizations use the office's support. Although the DHS
    budget for fiscal year 2005 includes $8.9 million to add contracting staff
    through the fund, we found that the mechanics of making the fund viable
    have not been worked out. Currently, Procurement Operations is negotiating
    agreements with each organization it supports to determine the terms and
    conditions of support and the dollar level to be contributed to the fund.
    According to DHS officials, this negotiation process has been problematic.
    For example, at the time of our review, the Science and Technology
    Directorate and Procurement Operations were having difficulty reaching a
    decision about who would be responsible for hiring the contracting staff
    that would support the directorate. As of January 2005, no agreements had
    been reached.

    Some DHS organizations have large, complex, and high-cost acquisition
    programs that need to be closely managed. For example, the Bureau of
    Customs and Border Protection's Automated Commercial Environment system,
    which is a new trade processing system intended to improve the movement of
    goods imported into the United States, is projected to cost $5 billion,
    and the Coast Guard's Deepwater Program is expected to cost $17 billion
    and take 2 to 3 decades to complete.27 To review major, complex
    investments such as these (referred to as level 1 investments), DHS's
    Office of the Chief Financial Officer has put in place a multitiered
    process. DHS has taken positive steps in creating a knowledge-based
    framework, or philosophy, for managing major investments; however, it
    still lacks key reviews and deliverables-both best practices-within this
    framework to ensure that cost and schedule estimates for major investments
    are as accurate as possible. These reviews take place at

    27At the time of our review, the Coast Guard had revised its requirements
    for the Deepwater program based on the new homeland security mission. We
    did not have information on the new cost of the program.

    critical junctures in the process and include demonstrating knowledge
    about technologies, design, and manufacturing processes. In addition, we
    found that contractor oversight, an important tool for managing programs,
    is not receiving high-level attention in the review process. Finally, we
    identified several areas of confusion surrounding the mechanics of
    implementing the process from a program manager's perspective. The
    management directive on the review process has been under revision for
    many months, and DHS officials could not tell us when the directive would
    be finalized.

    Key Elements of DHS's Investment Review Process

    DHS's investment review process involves several different levels of
    review, depending on the dollar threshold and risk level of the program.
    The Investment Review Board makes decisions on level 1 investments with
    prior review and input from the Joint Requirements Council, which in turn
    seeks input from other DHS specialists who have expertise in such areas as
    asset management and information technology. In classifying investments as
    level 1, DHS considers the following criteria: contract costs; importance
    to DHS strategic and performance plans; high development, operating, or
    maintenance costs; high risk; high return; and significance in resource
    administration. Investments classified as levels 2, 3, or 4 are considered
    lower-level acquisitions and follow different investment review processes.
    In addition, many of the major DHS organizations, such as the
    Transportation Security Administration and the Coast Guard, have their own
    review processes, which occur prior to higher-level review in the
    department. Figure 4 illustrates who is involved in the decision-making
    process and the levels of review.

    Figure 4: General Depiction of DHS's Investment Review Process for Major,
    Complex Investments

    Source: DHS (data); GAO (analysis and presentation).

    Although Policy Governing the Review Process Has Established a Solid Risk
    Management Framework, Additional Best Practices Would Benefit DHS

    DHS has adopted several best practices from lessons learned from leading
    commercial companies and successful federal programs that, if applied
    consistently, could refine its ability to reduce risk to meet cost and
    delivery targets for major investments. One of the best practices is a
    knowledge-based approach, or framework, for managers to hold reviews at
    key decision points in order to reduce risk before investing resources in
    the next phase of a program's development. The investment review policy
    provides guidance to program managers to provide knowledge about important
    aspects of a product at key points in the acquisition process and
    encourages them to reduce technology risk through demonstration prior to
    beginning a project. The policy also encourages program managers to
    demonstrate a product's design with critical design reviews and reduce
    manufacturing risk prior to a production decision. However, we found,
    based on our extensive body of work on this knowledge-based approach, that
    additional program reviews and knowledge deliverables could be
    incorporated into the process as internal controls to better position DHS
    to make well-informed decisions on its major, complex investments.

    Figure 5 generally depicts the major phases of the knowledge-based
    approach and the positioning of three knowledge points, or gates, when key
    reviews are scheduled. The figure applies the knowledge-based approach to
    DHS's investment review framework and displays an exclamation mark where
    key reviews or information are missing.

    Figure 5: Applying the Knowledge-Based Approach to DHS's Acquisition
    Framework

    Start of Start of program production

    Knowledge point 1

    Technologies, schedule, funding, and other resources match customer needs

    Knowledge point 2

    Design performs as expected

    Knowledge point 3

    Product must be producible within cost, schedule, and quality targets and
    demonstrated to be reliable before production begins

    Start of Start of program production

    Knowledge point 1

    Aligns with knowledge-based approach but some key information for matching
    resources to customer needs is missing

    Knowledge point 2

    Missing is the knowledge point that ensures design performs as expected;
    also missing is some of the information needed before deciding to proceed
    to production

    Knowledge point 3

    Knowledge point is discretionary; a required review does not occur until
    after low-rate production is well underway

                  Source: DHS (data); GAO (analysis and presentation).

    As shown in figure 5, DHS review points do not fully align with the
    knowledge-based approach. For example, DHS does not require a review to
    ensure that an investment's design performs as expected before investing
    in a prototype. In addition, DHS's mandatory review to proceed to
    production does not occur until after low-rate initial production is well
    under way. DHS does have a review for low-rate initial production;
    however, it is at the discretion of the Investment Review Board. Our past
    work has shown that successful investments reduce risk by ensuring that
    high levels of knowledge are achieved at these key points of development.
    We found that investments that were not reviewed at the appropriate points
    faced problems-such as redesign-that resulted in cost increases and
    schedule delays.

    We also found that some critical information is not addressed in DHS's
    investment review policy or the guidance provided to program managers. In
    other cases, it is made optional. For example, before program start
    (knowledge point 1) is approved, DHS policy requires program managers to
    identify an acquisition's key performance requirements and to have
    technical solutions in place. This information is then used to form cost
    and schedule estimates for the product's development to ensure that a
    match exists between requirements and resources. However, DHS policy does
    not establish cost and schedule estimates for the acquisition based on
    knowledge from preliminary designs. At knowledge point 2, while DHS policy
    requires program managers to identify and resolve critical operational
    issues before proceeding to production, initial reviews-such as the system
    and subsystem reviews-are not mandatory.

    Not all investments require the use of every piece of information included
    under a knowledge-based approach. Many of DHS's major investments use
    commercial, off-the-shelf products that do not require the same level of
    review as a complex, developmental investment would. However, DHS is
    investing in a number of major, complex systems, such as the Coast Guard's
    Deepwater Program and the U.S. Visitor and Immigrant Status Indicator
    Technology (US-VISIT),28 which incorporate new technology and therefore
    require greater adherence to the knowledge-based approach in order to
    ensure risk is reduced before committing to the next phase of the
    investment. In addition, the added reviews and information included in the
    knowledge-based approach may still be required for programs that use
    existing technology, such as the Counter-MANPADS program, which involves
    placing military technology on commercial aircraft.29

    In addition to the knowledge-based approach and its associated controls,
    DHS's investment review policy and guidance adopt a number of other
    important acquisition management practices, such as requiring program

    28US-VISIT is a governmentwide program to collect, maintain, and share
    information on foreign nationals. The program's goals are to enhance
    national security, facilitate legitimate trade and travel, contribute to
    the integrity of the U.S. immigration system, and adhere to U.S. privacy
    laws and policies.

    29Counter-MANPADS (man-portable air defense systems) is a protective
    system for U.S. commercial aircraft against shoulder-fired missiles. We
    recently issued a report that recommended additional information be
    provided at key decision points. See GAO, The Department of Homeland
    Security Needs to Fully Adopt a Knowledge-based Approach to Its
    Counter-MANPADS Development Program, GAO-04-341R (Washington, D.C.: Jan.
    30, 2004).

    managers to submit acquisition plans and project management plans.
    However, a key practice, contractor tracking and oversight, is not fully
    incorporated in the policy and guidance. We have cited the need for
    increased contractor tracking and oversight for several large DHS
    programs. For example, we previously reported that the Coast Guard's
    Deepwater program needed increased management and contractor oversight.
    One activity of contract tracking and oversight requires that a
    quantitative set of software and system metrics are used to define and
    measure product quality and contractor performance. The Coast Guard had
    not developed measurable performance goals or adhered to effective
    procedures for holding the contractor accountable for its ongoing
    performance. In addition, we previously recommended that the US-VISIT
    program should improve attention to implementing acquisition best
    practices. While DHS agreed, and the US-VISIT program office has assigned
    responsibility for implementing the recommended management controls, the
    department has not yet developed explicit plans or time frames for
    defining and implementing acquisition best practices. A list of selected
    acquisition management practices and required activities is in appendix V.

    As Review Process Matures, Some Mechanics Still to Be Worked Out

    The investment review process has been under revision for many months.
    According to DHS officials, the changes will include shifting
    responsibilities of some tiers in the review process and increasing the
    dollar threshold for level 1 investments. To date, the new process has not
    been finalized, and officials could not provide us with a time frame for
    completion. In the meantime, we found unclear guidance and confusion about
    several aspects of the process. In some cases, the confusion has resulted
    in key stakeholders, such as the Chief Procurement Officer, not receiving
    materials in time to conduct a thorough review and provide meaningful
    feedback prior to investment review meetings.

    The issues we found include the following:

    o  	Program managers have been provided with only draft guidance regarding
    the information they are required to submit and the time frames for
    submissions. This draft guidance is, in some cases, unclear.

    o  	Some DHS officials noted that their submissions to the review board
    had been rejected on an inconsistent basis with no explanation.

    o  	Program managers have not received formal training on the investment
    review process. Officials told us that some program managers have been
    unaware of when to submit information about their programs for review.

    Conclusions

    Recommendations for Executive Action

    o  	In practice, major investments in services are exempt from the review
    process and are only reviewed when done as part of a capital investment.
    Officials from the Office of the Chief Financial Officer who are in charge
    of the process told us that services investments are reviewed only when
    they are part of a capital investment because these acquisitions are not
    complex and therefore do not need the same level of scrutiny reserved for
    the acquisition of goods.

    Currently, program managers receive assistance in developing their review
    board submissions from a small number of staff in the Chief Financial
    Officer's Office of Program Analysis and Evaluation at the beginning of a
    program and at DHS's key decision points. However, because of limited
    resources, the office has only been able to provide limited support to
    programs to assist them in completing their submissions.

    In the 2 years since its merger, DHS has taken strides toward putting in
    place an acquisition organization that contains many promising elements,
    but the steps taken so far are not enough to ensure that the department is
    effectively managing the acquisition of the multitude of goods and
    services it needs to meet its mission. More needs to be done to fully
    integrate the department's acquisition function, to pave the way for the
    Chief Procurement Officer to fully carry out his responsibilities in a
    modern-day acquisition organization, and to put in place the strong
    internal controls needed to effectively manage interagency contracting
    activity and large, complex investments. Unless DHS's top leaders address
    these challenges, the department is at risk of continuing to exist with a
    fragmented acquisition organization that provides stopgap, ad hoc
    solutions. DHS has an opportunity, while it is still involved in
    transformational efforts, to avoid the complications that plague
    acquisition efforts in other longestablished federal departments.

    To help ensure that DHS receives the goods and services it needs at the
    best value to the government, we recommend that the Secretary of Homeland
    Security take the following six actions:

    o  	establish a structure to ensure continued support for commodity
    councils, such as appointing full-time dedicated commodity managers, to
    ensure that the commodity councils develop long-term strategies, maintain
    momentum, and continue to realize savings;

    o  	provide the Office of the Chief Procurement Officer with sufficient
    resources and enforcement authority to enable effective, departmentwide
    oversight of acquisition policies and procedures;

    o  	conduct a departmentwide assessment of the number of contracting staff
    and, if a workload imbalance is found, take steps to correct it by
    re-aligning resources;

    o  	direct higher-level management attention to the implementation of the
    working capital fund (which is to be used to fund contracting staff for
    the Office of Procurement Operations) by, for example, determining the
    level of contracting support needed by the organizations relying on this
    office, ensuring that appropriate funds are committed to hire needed
    contracting staff, and ensuring that funds are available on an ongoing
    basis for continuity;

    o  	revise the October 2004 management directive "Acquisition Line of
    Business Integration and Management" to eliminate reference to the Coast
    Guard and Secret Service being statutorily exempt from complying; and

    o  	ensure that DHS's management directive on interagency agreements is
    followed and that fees paid to other agencies are tracked.

    To help ensure that DHS leadership is aware of risks as they arise during
    the acquisition of major, complex systems, we recommend that the Secretary
    of Homeland Security take the following seven actions:

    o  in making revisions to the investment review policy:

    o  	require for all complex, developmental investments a formal design
    review between the integration and demonstration of a program to ensure
    that the design is stable and has been demonstrated through prototype
    testing;

    o  	require for all complex, developmental investments a review before
    initial production;

    o  	require that program managers supply additional information-such as
    cost and schedule estimates based on results of a preliminary design
    review and critical design review-when their major, complex programs are
    reviewed; and

    o  	require program managers to specifically address contractor oversight
    in their submissions to investment review boards.

    o  	ensure that stakeholders, including acquisition officials in the
    Office of the Chief Procurement Officer, have adequate time to review
    investment submissions and provide formal input to decision-making review
    boards;

    o  	implement training for program managers on the investment review
    process that emphasizes the importance of a knowledge-based approach; and

    o  require that major acquisitions of services be subject to oversight by
    the

      Agency Comments
      and Our Evaluation

    investment review board.

    We provided a draft of this report to DHS for review and comment. In
    written comments, DHS generally agreed with our facts and conclusions and
    concurred with all of our recommendations. Regarding three of our
    recommendations on the investment review process, DHS stated that the
    actions exist in current directives and are already being done. We
    disagree. Our work demonstrated that DHS's review points do not fully
    align with the knowledge-based approach. For example, DHS's mandatory
    review to proceed into production does not occur until after low-rate
    initial production is well under way, and the review for starting low-rate
    initial production only occurs at the discretion of the Investment Review
    Board. Also, DHS's framework lacks the knowledge deliverables necessary at
    each key review to ensure that cost and schedule estimates for major
    investments are as predictable as possible. Our past work has shown that
    investments that were not reviewed at the appropriate points faced
    problems that resulted in cost increases and schedule delays.

    The department's comments are reprinted in appendix II.

    As arranged with your offices, unless you publicly announce its contents
    earlier, we plan no further distribution of this report until 30 days from
    its issue date. We are sending copies of this report to other interested
    congressional committees, the Secretary of the Department of Homeland
    Security, and the Director of the Office of Management and Budget. In
    addition, the report will be available on GAO's Web site at
    http://www.gao.gov.

    If you or your staff have any questions concerning this report, please
    contact me at (937) 258-7915. Staff making major contributions to this
    report are listed in appendix VI.

    Michael J. Sullivan, Director Acquisition and Sourcing Management

                           Appendix I: Scope and Methodology

    To identify the areas where the Department of Homeland Security (DHS) has
    been successful in promoting collaboration among its various
    organizations, we interviewed senior acquisition officials at DHS
    headquarters and analyzed pertinent documents. We obtained information
    from the Office of Strategic Sourcing and Acquisition Systems to analyze
    how DHS has used strategic sourcing to leverage the department's buying
    power. We interviewed senior strategic sourcing officials at DHS
    headquarters to obtain information on how they identified the commodities
    with potential for savings. We reviewed studies, policies, guidance, and
    other documents related to ongoing or proposed strategic sourcing
    initiatives that leveraged buying power, cut costs, or achieved other
    performance benefits. Our review found that the department's strategic
    sourcing cost savings methodologies appear reasonable; however, we did not
    verify the accuracy of any strategic procurement costs savings reported to
    us. We asked senior strategic sourcing and commodity council participants
    about potential barriers to employing strategic sourcing at DHS. We
    interviewed the Director, Office of Small and Disadvantaged Business
    Utilization, and procurement officials within the department. We obtained
    policies, memorandums, and other documents from the small business office,
    procurement organizations, and the U.S. Small Business Administration.

    To determine areas where DHS faces challenges in integrating the
    acquisition function, we reviewed DHS organizational charts to gain
    insight into where the procurement offices fall in the hierarchy and to
    determine the lines of responsibility and authority between the various
    stakeholders in the acquisition process. We reviewed DHS policies,
    guidance, and procedures governing acquisition and analyzed internal audit
    reports, when available, to determine if those policies and procedures
    were being followed. We obtained statistics from DHS's Office of the Chief
    Procurement Officer on the department's procurements and the acquisition
    workforce. We interviewed procurement policy officials at headquarters and
    conducted interviews with personnel in each of the procurement
    organizations. To assess DHS's effectiveness in managing its acquisition
    workforce, we interviewed contracting and human resource officials at DHS
    headquarters. We analyzed DHS's processes and procedures for certifying
    program managers, warranting contracting officers, and tracking the
    acquisition workforce training. Lastly, we reviewed previous GAO work
    regarding best acquisition practices for organizational alignment and
    oversight.

    Because the Office of Procurement Operations has, by far, the highest
    level of interagency contracting activity in the department, we examined

    Appendix I: Scope and Methodology

    the office's database of contracts to gain an understanding of the general
    scope of activity involving interagency agreements. We randomly selected
    and then reviewed 136 interagency agreement contract files. We interviewed
    officials responsible for the data to ensure that the system they use to
    track procurement activity was adequate for identifying the sample
    population. We reviewed the interagency agreement contract files to assess
    key aspects of the acquisition process-such as a signed determination and
    findings in accordance with the Federal Acquisition Regulation,
    acquisition planning, and interagency agreement contract administration,
    including contractor oversight. We held follow-on discussions with the
    Office of Procurement Operations to discuss discrepancies noted in the
    files. Further information on our methodology and sampling error rates is
    as follows:

    o  	In fiscal year 2004, the Office of Procurement Operations
    predominately obtained contracting services for its customers through the
    issuance of interagency agreements with other government agencies. We drew
    our sample from an original population of 984 agreements. Of the 150 items
    we initially sampled, the department deleted 14 of these agreements and
    withdrew them from the data. As a result, we adjusted the population of
    agreements to which we are making estimates to 892 agreements. All
    estimates are to these 892 agreements based upon a sample size of 136
    agreements. In addition, 4 files were missing and were not able to be
    reviewed, and one agreement was a duplicate. This resulted in only 131
    files being reviewed. As we conducted our reviews, the agency became
    better at recording the agreements that should be included in its
    database. Currently, Procurement Operations reports that 1,104 agreements
    are contained in its records and total $1.8 billion.

    o  	We reviewed the files and documents for these 131 agreements, using as
    our criteria the department's directive on interagency agreements. The
    following projections are made from our sample of 136 agreements. The
    estimates from a statistical sample are always subject to some uncertainty
    because the entire collection is not reviewed. This uncertainty is called
    sampling error. Tables 2 and 3 show the sampling errors for certain
    factors relating to the issuance of interagency agreements.

    Appendix I: Scope and Methodology

      Table 2: Sampling Error at the 95 Percent Confidence Level for the Sample of
             Interagency Agreements Number Percentage Observation Estimate

    Sampling error Estimate

    Sampling error

                   Analysis of alternatives not conducted             840 36 94.1
    Contracting officer's technical representatives not identified 420 76 47.1
                  Determination and findings not prepared             308 72 34.6
             No indication that contractor oversight performed        853 31 95.6

                            Source: GAO (data and analysis).

      Table 3: Sampling Error at the 95 Percent Confidence Level for the Sample of
    Interagency Agreements Fee Payments Number Dollars Observation Estimate Sampling
                             error Estimate Sampling error

        Fees paid on use of interagency agreements 433 76 $12,991,079 $2,280,189

    Source: GAO (data and analysis).

    To assess the department's progress in implementing a review process for
    major, complex systems, we compared DHS's acquisition policies for major
    acquisitions to our knowledge-based approach. We used information from
    several of our prior reports that examine how commercial best practices
    can improve outcomes for acquisition programs. Specifically, we compared
    and contrasted DHS's investment review process with the best practices for
    commercial acquisitions identified in our past reports. Our analysis
    focused on whether DHS's policies contained the measurable criteria and
    management controls necessary for minimizing cost, schedule, and
    performance risks. To clarify the content of the investment review
    process, we met with various DHS officials from the Office of the Chief
    Procurement Officer and the Office of Program Evaluation and Analysis
    within the Office of the Chief Financial Officer. We also discussed the
    review process with officials from DHS procurement organizations and
    collected reports and analyzed available program data on the current
    status of major acquisitions being carried out by DHS.

    We conducted our review from March 2004 through February 2005 in
    accordance with generally accepted government auditing standards.

    Appendix II: Comments from the Department of Homeland Security

    Appendix II: Comments from the Department of Homeland Security

    Appendix II: Comments from the Department of Homeland Security

    Appendix III: Office of Procurement Operations' Customers

    Chief Financial Officer
    Chief Human Capital Officer
    Chief Information Officer
    Chief Procurement Officer
    Chief of Administrative Services
    Chief of Staff
    Citizenship and Immigration Services
    Citizenship and Immigration Services Ombudsman
    Civil Rights and Civil Liberties
    Communications Director
    Counter Narcotics Officer
    Deputy Secretary
    Executive Secretary
    General Counsel
    Headquarters Operational Integration Staff
    Homeland Security Advisory Council
    Homeland Security Operations Center
    Immigration Statistics
    Information Analysis and Infrastructure Protection
    International Affairs
    Office of the Secretary
    Legislative Affairs
    National Capital Region
    Office of the Under Secretary--Border and Transportation Security
    Press Secretary
    Privacy Officer
    Private Sector
    Public Affairs
    Science and Technology
    Security
    State and Local Government Coordination (includes Office of

    Domestic Preparedness)
    Under Secretary for Management
    White House Liaison

    Appendix IV: Summary of Selected Commodity Council Strategic Sourcing
    Initiatives

                   Commodity council Strategic sourcing actions taken

    Office Supplies Leverages DHS's purchasing power and reduces amount spent
    on office supplies:

    o  	For its main strategic sourcing initiative, this council partnered
    with the Department of Defense's Electronic Mall (DOD Emall). DOD Emall is
    an Internet-based catalog ordering system that provides 24hour-a-day,
    7-day-a-week access to over 200 small and large office supply vendors.

    o  	Because many military services also purchase office supplies through
    DOD Emall, DHS can take advantage of large volume discounts to increase
    its buying power.

    o  	As of June 1, 2004, DHS mandated all of its purchase cardholders to
    exclusively use DOD Emall to buy office supplies. As a result, DHS has
    already become the third-largest federal government user of DOD Emall. For
    the 4-month period starting June 1, 2004, DHS spent over $14.5 million on
    DOD Emall, representing 16.7 percent of the total dollars spent on Emall.

    o  	Before the full implementation of DOD Emall, the council also
    generated savings by using an office supplies blanket purchase agreement
    that the Transportation Security Administration had in place to provide
    office supplies to other DHS principal organizations.

    o  Through September 30, 2004, the council estimated cost savings of $8
    million.

    Boats 	Establishes standard boat procurements to decrease life cycle
    costs, facilitate interoperability, reduce training burden, and provide
    immediate cost savings:

    o  	For example, Customs and Border Patrol needed six boats in fiscal year
    2004. Instead of creating a new contract, the council explored whether
    there were any existing contract vehicles within DHS that could satisfy
    the need.

    o  	The council purchased the six boats through an existing Coast Guard
    contract and took advantage of large volume discounts.

    o  	As a result of these strategic sourcing efforts, the council was able
    to acquire each boat for $50,000 less than anticipated, resulting in a
    total of $300,000 in cost savings.

    Weapons Identifies and consolidates emerging firearms and ammunition
    requirements for DHS:

    o  	In fiscal year 2004, the council planned to acquire pistols for DHS
    organizations to meet critical, mission-driven requirements.

    o  	The council surveyed all DHS organizations interested in weapons,
    factored in the end users' requirements, and established a list of
    potential vendors that could provide best value for pistols and satisfy
    requirements.

    o  	As a result, DHS awarded contracts to two vendors for 65,000 pistols
    each over 5 years-the largest pistol acquisition in the history of U.S.
    law enforcement.

    o  	As of September 20, 2004, the council has reported cost savings of
    over $4.1 million and $915,000 in cost avoidance.

    Energy Identifies strategies for aggregating and centralizing DHS's energy
        procurements to take advantage of economies of scale and negotiate more
                           competitive prices with suppliers:

    o  	Although DHS spent approximately $58 million for electricity in fiscal
    year 2003, only $9 million of the spending was for negotiable energy
    services in states with deregulated electricity industries. Nevertheless,
    the council estimates more than $705,000 in cost savings through September
    30, 2004.

                        Source: DHS (data); GAO (presentation).

    Appendix V: Selected Acquisition Management Best Practices

                                Best practices Activity

    Acquisition planning Plans are prepared during acquisition planning and To
    ensure that maintained throughout the acquisition.

    reasonable planning for Planning addresses the entire acquisition process,
    as well as all parts of the life cycle support of the products being
    acquired.

    acquisition is conducted 	The acquisition organization has a written
    policy for planning the acquisition.

    Responsibility for acquisition planning activities is designated.

    Contract tracking and The acquiring organization has sufficient insight
    into the oversight contractor's activities to manage and control the
    contractor To ensure that contract and ensure that contract requirements
    are met.

    activities are performed The acquiring organization and contractor
    maintain ongoing in accordance with communication; commitments are agreed
    to and implemented contractual requirements by both parties.

    All contract changes are managed throughout the life of the contract.

    The acquisition organization has a written policy for contract tracking
    and oversight.

    Responsibility for contract tracking and oversight activities is
    designated.

    The acquiring organization involves contracting specialists in the
    execution of the contract.

    A quantitative set of software and system metrics is used to define and
    measure product quality and contractor performance.

    In addition to incentives for meeting cost and schedule estimates,
    measurable, metrics-based product quality incentives are explicitly stated
    in the contract.

    Risk management Projectwide participation in the identification and
    mitigation of

    To ensure that risks are risks is encouraged. proactively identified and
    The defined acquisition process provides for the identification,
    systematically mitigated analysis, and mitigation of risks.

    Milestone reviews include the status of identified risks.

    The acquisition organization has a written policy for managing acquisition
    risk.

    Responsibility for acquisition risk management activities is designated.

    Source: GAO (data); GAO (presentation).

    Note: See GAO, Information Technology: DoD's Acquisition Policies and
    Guidance Need to Incorporate Additional Best Practices and Controls,
    GAO-04-722 (Washington, D.C.: July 30, 2004) for a complete list of the
    acquisition management practices and required activities.

    Appendix VI: GAO Contacts and Staff Acknowledgments

    Contacts

      Staff Acknowledgments

    (120328)

    Michael J. Sullivan, (937) 258-7915 Michele Mackin, (202) 512-4309

    In addition to those named above, Daniel Chen, Lily Chin, Benjamin
    Federlein, Arthur James Jr., John Krump, Jose Ramos, and Russell Reiter
    made key contributions to this report.

        GAO's Mission

    Obtaining Copies of GAO Reports and Testimony

    The Government Accountability Office, the audit, evaluation and
    investigative arm of Congress, exists to support Congress in meeting its
    constitutional responsibilities and to help improve the performance and
    accountability of the federal government for the American people. GAO
    examines the use of public funds; evaluates federal programs and policies;
    and provides analyses, recommendations, and other assistance to help
    Congress make informed oversight, policy, and funding decisions. GAO's
    commitment to good government is reflected in its core values of
    accountability, integrity, and reliability.

    The fastest and easiest way to obtain copies of GAO documents at no cost
    is through GAO's Web site (www.gao.gov). Each weekday, GAO posts newly
    released reports, testimony, and correspondence on its Web site. To have
    GAO e-mail you a list of newly posted products every afternoon, go to
    www.gao.gov and select "Subscribe to Updates."

    Order by Mail or Phone 	The first copy of each printed report is free.
    Additional copies are $2 each. A check or money order should be made out
    to the Superintendent of Documents. GAO also accepts VISA and Mastercard.
    Orders for 100 or more copies mailed to a single address are discounted 25
    percent. Orders should be sent to:

    U.S. Government Accountability Office 441 G Street NW, Room LM Washington,
    D.C. 20548

    To order by Phone: 	Voice: (202) 512-6000 TDD: (202) 512-2537 Fax: (202)
    512-6061

        To Report Fraud, Contact:

    Waste, and Abuse in Web site: www.gao.gov/fraudnet/fraudnet.htm

    E-mail: fraudnet@gao.govFederal Programs Automated answering system: (800)
    424-5454 or (202) 512-7470

    Gloria Jarmon, Managing Director, JarmonG@gao.gov (202)
    512-4400Congressional U.S. Government Accountability Office, 441 G Street
    NW, Room 7125 Relations Washington, D.C. 20548

    Public Affairs 	Paul Anderson, Managing Director, AndersonP1@gao.gov (202)
    512-4800 U.S. Government Accountability Office, 441 G Street NW, Room 7149
    Washington, D.C. 20548

                                     RECYCLED PAPER
    *** End of document. ***