FDA Import Automation: Serious Management and Systems Development
Problems Persist (Letter Report, 09/28/95, GAO/AIMD-95-188).

The Food and Drug Administration (FDA) oversees imports of food, drugs,
cosmetics, medical devices, and other products to ensure that the public
is protected from goods of questionable quality or that make misleading
claims. In 1987, FDA began developing an automated system to improve its
import entry clearance process, which required extensive paperwork.
Despite an investment of eight years and $13.8 million to automate its
process for inspecting imported goods, the new system contains major
deficiencies. This is due mainly to inadequate top management oversight
and a management team that lacked expertise and skills in system
development. FDA has implemented a portion of the system--the
Operational and Administrative System for Import Support (OASIS)--to
enhance its ability to regulate imports and to relieve importers and FDA
personnel of some of the paperwork burdens associated with import
processing. In developing OASIS, FDA did not follow generally accepted
systems development practices for validating software; conducting user
acceptance testing; developing a security plan to safeguard its computer
facilities, equipment, and data; and conducting a cost-benefit analysis.
The resulting shortcomings mean that OASIS may not perform as needed and
that unsafe products could enter the country.

--------------------------- Indexing Terms -----------------------------

 REPORTNUM:  AIMD-95-188
     TITLE:  FDA Import Automation: Serious Management and Systems 
             Development Problems Persist
      DATE:  09/28/95
   SUBJECT:  Information resources management
             Systems design
             Import regulation
             Management information systems
             Safety regulation
             Agricultural products
             Import restriction
             Monitoring
             Contaminated foods
             Information systems
IDENTIFIER:  FDA Import Support and Information System
             Customs Service Automated Commercial System
             
**************************************************************************
* This file contains an ASCII representation of the text of a GAO        *
* report.  Delineations within the text indicating chapter titles,       *
* headings, and bullets are preserved.  Major divisions and subdivisions *
* of the text, such as Chapters, Sections, and Appendixes, are           *
* identified by double and single lines.  The numbers on the right end   *
* of these lines indicate the position of each of the subsections in the *
* document outline.  These numbers do NOT correspond with the page       *
* numbers of the printed product.                                        *
*                                                                        *
* No attempt has been made to display graphic images, although figure    *
* captions are reproduced. Tables are included, but may not resemble     *
* those in the printed version.                                          *
*                                                                        *
* A printed copy of this report may be obtained from the GAO Document    *
* Distribution Facility by calling (202) 512-6000, by faxing your        *
* request to (301) 258-4066, or by writing to P.O. Box 6015,             *
* Gaithersburg, MD 20884-6015. We are unable to accept electronic orders *
* for printed documents at this time.                                    *
**************************************************************************


Cover
================================================================ COVER


Report to the Ranking Minority Member, Committee on Commerce, House
of Representatives

September 1995

FDA IMPORT AUTOMATION - SERIOUS
MANAGEMENT AND SYSTEMS DEVELOPMENT
PROBLEMS PERSIST

GAO/AIMD-95-188

FDA Import Automation

(511172)


Abbreviations
=============================================================== ABBREV

  ACS - Automated Commercial System
  CFO - Chief Financial Officer
  EEPS - Electronic Entry Processing System
  FDA - Food and Drug Administration
  HHS - Health and Human Services
  IRM - information resources management
  ISIS - Import Support and Information System
  OASIS - Operational and Administrative System for Imports
  OIRM - Office of Information Resources Management
  ORA - Office of Regulatory Affairs
  PHS - Public Health Service

Letter
=============================================================== LETTER


B-261934

September 28, 1995

The Honorable John D.  Dingell
Ranking Minority Member
Committee on Commerce
House of Representatives

Dear Mr.  Dingell: 

The Food and Drug Administration (FDA), a component of the Public
Health Service (PHS) within the Department of Health and Human
Services (HHS), regulates the importation of selected foods, drugs,
cosmetics, biological products, medical devices, and electronic
products to ensure that the public is protected from products that
make fraudulent or misleading claims or that threaten public health
and safety.  In 1987, FDA began developing a support and information
system to automate and improve its import entry clearance process,
which required extensive manual examination of paperwork.  Through an
automated interface with the U.S.  Customs Service, FDA has
implemented a portion of this system, now known as the Operational
and Administrative System for Import Support (OASIS), to enhance its
ability to regulate imported products and to relieve importers and
FDA personnel of some of the paperwork burdens associated with
processing imported products. 

This report responds to your request that we assess the progress of
and identify any problems associated with FDA's implementation of
OASIS, as well as systems development areas needing improvement. 


   RESULTS IN BRIEF
------------------------------------------------------------ Letter :1

Although some operational improvements have been made to import
operations, FDA has not completed a fully functional system after 8
years and an estimated $13.8 million in system development costs
(hardware and software acquisition, telecommunications, and other
systems costs).  This is due primarily to inadequate top management
oversight and an OASIS management team that lacked expertise and
skills in systems development. 

A 1994 self-assessment review\1 of this systems effort performed
jointly by HHS, PHS, and FDA, found that this project was at a high
risk for failure and recommended suspending development until a
comprehensive review of the system was completed.  We found that
although FDA has begun to address some of the problems identified in
the self-assessment, others have not been corrected.  In addition,
performance measures have not been established and project costs have
not been properly accounted for.  In its approach to developing
OASIS, FDA did not follow generally accepted systems development
practices for validating software; conducting user acceptance
testing; developing a security plan to safeguard its computer
facilities, equipment, and data; and conducting a cost-benefit
analysis.  The resulting deficiencies introduce potential risks that
OASIS, which is partially implemented, may not perform as needed and
that unsafe products could enter the country. 

FDA initially halted deployment of all but the initial portion of
OASIS until the completion of a system design review.  The review,
completed in June 1995,\2 concluded that OASIS was not ready for
national implementation and recommended an immediate reengineering
effort.  The problems identified by us, the self-assessment report,
and the system design review must be resolved if FDA is to
successfully complete its automated import system.  As indicated by
the best practices of leading public and private organizations for
managing information and their related technologies, FDA's success in
achieving improved operational performance with a fully functional
automated import support system will depend on better planning and
top management involvement in system design, development, and
deployment. 


--------------------
\1 Review of the Food and Drug Administration Import Support and
Information System, Department of Health and Human Services, July 14,
1994. 

\2 Review of the Import Support and Information System (ISIS), System
Design Review Committee, June 21, 1995. 


   BACKGROUND
------------------------------------------------------------ Letter :2

FDA's overall mission is to protect the public from selected domestic
or imported foods, drugs, cosmetics, biological products, and medical
devices and from products that make fraudulent or misleading claims
that might threaten public health and safety.  On matters relating to
its import operations, FDA's Office of Regulatory Affairs provides
guidance and systems support, and performs planning, budgeting, and
reporting activities for 6 regional offices, 21 district offices, and
about 130 resident inspection posts.  Imported products can enter the
United States at seaports, airports, courier hubs, and border
crossings.  The volume of import entries subject to FDA regulations
has been increasing over the last 20 years from about 590,000 entries
in 1975 to about 1.6 million entries currently, and is expected to
reach 2 million entries by 2000. 

Products imported into the United States must be cleared first by
Customs, whose responsibilities include assessing and collecting
revenues from imports, enforcing customs and related laws, and
assisting in the administration and enforcement of other provisions
of laws and regulations on behalf of 60 federal agencies.  Import
brokers act as agents for importers and process the information
required to bring products into the United States.  Brokers can
electronically transmit data on their products to Customs through an
automated interface with Customs' Automated Commercial System (ACS). 
If Customs determines that a product requires FDA approval before
being released into the domestic market, such as for regulated food
and drugs, the broker is to forward entry information to FDA for
review. 

Under FDA's manual entry and review process, brokers must submit
entry documents (an FDA-701,\3 invoice, and associated
certifications) to FDA for each shipment.  Using these documents, FDA
inspectors at the port of entry decide whether to release the
shipment for entry, examine the shipment by inspecting it there,
perform paper or laboratory examination of it for possible refusal
due to violations of laws and regulations, or detain it until the
broker furnishes additional information.  Entry documents can range
from a few pages to as many as 40 pages depending on the type and
volume of goods in a shipment.  The time interval between when the
broker submits the documents to FDA and when the broker receives a
release or examination decision from FDA averages 2 days. 

As the volume of imports continued to grow, FDA recognized a need to
automate and expedite its entry and review process.  Also, FDA
envisioned that an automated system would provide a method to capture
and share historical data to bring uniformity to its enforcement
decisions for detecting and preventing "port shopping" by importers. 
FDA found that because of its heavy workload or less interest in
particular products at some ports, some importers tended to use the
port of entry that provided them with the best opportunity for
receiving FDA approval.  In 1987, the FDA Commissioner formed a task
force to develop a new automated system as recommended in a
contractor-prepared feasibility study.\4 This system, now known as
OASIS, was intended to increase the efficiency and effectiveness of
FDA's program for monitoring imported products.  In general, OASIS
was expected to (1) increase the productivity of investigations
personnel through automated interfaces with the laboratories, brokers
and/or Customs, (2) improve screening of imports by providing
suggestions for actions likely to result in discovery of violations,
(3) provide faster turnaround for processing of importer's entries
and faster and more consistent responses, (4) provide national and
district uniformity in processing of entries, and (5) maintain a base
of information for generation of reports.  OASIS was initially
planned to be fully implemented in September 1989. 


--------------------
\3 This document includes a general description of the shipment
(quantity, packaging, and items), identification of the port of
entry, country of origin, importer/broker, shipper, manufacturer, and
value of entry in U.S.  dollars, etc. 

\4 A feasibility study (1) analyzes automation objectives,
requirements, and system concepts, (2) evaluates alternative
approaches, and (3) describes a proposed approach. 


   SCOPE AND METHODOLOGY
------------------------------------------------------------ Letter :3

We interviewed FDA and Customs officials in the Washington, D.C.,
area, and Seattle, Washington, to determine the operational
objectives and timeframe for implementing OASIS.  We also reviewed
systems documentation provided by FDA, such as the system design,
functional requirements, capacity analysis, risk assessment, regional
contingency plan, implementation schedules, software support
contracts, task orders, interagency agreement between FDA and
Customs, and security and information resources management policies
and procedures.  We assessed FDA's efforts to design, develop, and
implement OASIS against GAO's executive guide on the best practices
of leading private and public organizations for strategic information
management,\5 and federal guidelines, such as the Federal Information
Processing Standards Publications.  In addition, we reviewed the 1994
joint self-assessment report on OASIS, the contractor's cost-benefit
analysis, and the System Design Review Committee's report. 

To monitor the implementation of OASIS, we visited and interviewed
officials in FDA district offices and ports of entry in Seattle
(system pilot location); Miami, Florida; Buffalo and New York, New
York; and Detroit, Michigan.  We also conducted telephone interviews
with FDA officials in several other district offices.  Interviews
with FDA import managers and inspectors at these sites provided us
with observations and examples of entries processed both manually and
electronically at major FDA air, sea, and border ports.  We also
interviewed Customs officials and import brokers at the sites visited
to obtain their perspectives on how the system has improved the
import process. 

Further, we interviewed officials at HHS, who were involved with the
self-assessment and system design reviews.  We also interviewed one
of the prior software development contractors and the current
contractor regarding their roles and responsibilities for the OASIS
project. 

We performed our work from April 1994 through June 1995, in
accordance with generally accepted government auditing standards.  We
requested official comments on a draft of this report from the
Secretary of Health and Human Services on August 10, 1995.  As of
September 18, 1995, we had not received any comments to include in
the final version of this report. 


--------------------
\5 Executive Guide:  Improving Mission Performance Through Strategic
Information Management and Technology (GAO/AIMD-94-115, May 1994). 


   AUTOMATED IMPORT SYSTEM REMAINS
   INCOMPLETE
------------------------------------------------------------ Letter :4

The development of OASIS is taking considerably longer than FDA
officials expected.  As shown in figure 1, after 8 years and three
software development contractors, FDA still does not have a fully
functional automated import system. 

   Figure 1:  Timeline of Major
   Events

   (See figure in printed
   edition.)

The original design, which was called the Import Support and
Information System (ISIS), was for a large, nationwide, on-line,
real-time, distributed FDA system.  This system was modified
following its 1991 pilot test and FDA's agreement with Customs to
include an automated interface with Customs' ACS.  As shown in figure
1, the modified system, known as OASIS, was pilot tested in Seattle,
Washington, in 1992 and expanded to Portland, Oregon, and Blaine,
Washington, in 1993. 

OASIS, as implemented in the Seattle District locations, provides FDA
inspectors the ability to (1) receive import entry data
electronically from import brokers through interface with ACS, (2)
receive results of preliminary processing against FDA's selectivity
criteria screening file, which is installed on ACS, that the shipment
"May Proceed," must be "Detained" for sampling, or must be held for
"FDA Review," (3) be alerted to potential problem areas with each
line item of an entry, make follow-up screening decisions, and
transmit these electronically to the broker through interface with
ACS, (4) track actions taken and maintain historical data on all
electronic import entries, and (5) eliminate many of the paper
transactions among FDA, Customs, and import brokers.  In addition,
import brokers who interface with ACS receive preliminary and
subsequent screening decisions relating to their electronic entries
simultaneously with FDA. 

However, software design problems experienced at the pilot locations
made OASIS difficult to use.  Such problems included slow response
times when receiving and printing electronic data, moving from
computer screen to computer screen, or going in and out of other
systems while processing entries.  These OASIS development problems
prompted FDA to assemble a team of information resources management
(IRM) representatives from HHS, PHS, and within FDA to pilot a
self-assessment tool to analyze risks associated with the development
of OASIS. 

In June 1994, as a result of the exit briefing by the self-assessment
team on its results and pending actions under way by FDA to replace
the expiring OASIS contract, FDA's Director of the Office of
Information Resources Management called for the termination of both
the development and deployment of all but the front-end portion of
OASIS known as the electronic entry processing system or EEPS.  FDA
decided to maintain and refine OASIS in the Seattle District.  In its
July 1994 report, the self-assessment team concluded that the OASIS
project was at high risk for system failure due to the lack of
senior-level management involvement, project planning, and basic
development processes as well as system design flaws, an insufficient
budget, and a skeleton staff lacking adequate system design and
implementation expertise. 

In contrast to the OASIS functions described previously, EEPS allows
FDA inspectors to receive the broker's entry data from ACS, but only
allows inspectors and brokers to receive the preliminary
admissibility messages of either "May Proceed" or "FDA Review" for
the entire entry.  It is not capable of processing or transmitting
any follow-up line-item decisions from FDA to the brokers.  EEPS was
deployed to 114 ports between March 1994 and June 1995, with 103
additional ports expected to be automated by the end of 1995. 

According to import brokers and FDA inspectors we interviewed, even
EEPS' limited capability provides them quicker notifications as to
the admissibility of imported shipments and reduces the amount of
paperwork required from brokers.  For "May Proceed" decisions, paper
entry documentation is generally eliminated.  For example, during the
month of June 1995, FDA reported that of 2,520 brokers who interfaced
with Customs' ACS in the Seattle District and EEPS ports, 1,585, or
63 percent, used electronic filing for 178,412 FDA entries, and that
78 percent of the 1,585 electronic filers were not required to submit
entry documentation for "May Proceed" entries.  Table 1 below
compares the traditional manual entry process to EEPS. 



                                Table 1
                
                 Manual and Automated Portion of FDA's
                          Import Entry Process

Manual Process                      EEPS
----------------------------------  ----------------------------------
For FDA-regulated shipment, import  Broker electronically sends
broker submits entry document       required shipment information
paperwork to an FDA import office.  through interface with Customs'
                                    ACS for each FDA-regulated product
                                    in shipment.

Entry documents are reviewed by     FDA's selectivity criteria file
FDA inspectors, who decide if       installed in Customs' ACS screens
shipment should be released,        information sent by the broker and
examined, detained, or verified     sends an electronic message of
for other compliance requirements.  "May Proceed" or "FDA Review"
                                    simultaneously to broker's
                                    terminal and FDA.

FDA inspector stamps "May Proceed"
for released items or "Exam/
Notify" for all others, on the
documents and leaves them in a
broker's box for courier pickup
and hand delivery to broker.

"May Proceed" shipments are         Broker and FDA receive screening
released into the country. "Exam"   message. "May Proceed" shipments
shipments require brokers to        are released into the country.
notify FDA in writing of location   "FDA Review" shipments that are
of shipment for sample purposes.    not reviewed on-screen require
                                    brokers to submit entry documents
                                    for FDA manual review. Based on a
                                    review of the shipment
                                    information, FDA decides whether
                                    to release or hold the shipment
                                    for further examination.

FDA inspector physically examines   For entries selected for
and/or collects sample at location  examination, FDA inspector
specified by broker.                physically examines and/or
                                    collects sample at location
                                    specified by broker.

Sampled items and lab results are   Sampled items and lab results are
entered into a district database    entered into a district database
maintained by individual districts  maintained by individual districts
to track the status and maintain a  to track the status and maintain a
historical database of items        historical database of items
selected for sampling.              selected for sampling.
----------------------------------------------------------------------
As recommended in 1994 by the self-assessment team, HHS, PHS, and FDA
formed a systems design review committee to determine if (1) the
OASIS design adequately meets the user requirements, (2) FDA computer
hardware, or platform, is adequate for the system, (3) real-time
access is necessary, and (4) telecommunications are adequate.  The
committee's June 1995 report addressed the first three items.  FDA's
telecommunications management branch is conducting an agencywide
study on the telecommunications and network capacity and capabilities
needed.  The results of this study are expected in February 1996. 

The committee's June 1995 report stated that (1) the OASIS system
design contains significant deficiencies, (2) the adequacy of the
agency platform cannot be determined because certain stress and
system load tests have not been performed or documented, and (3)
real-time access is not necessary. 


   SERIOUS MANAGEMENT AND SYSTEMS
   DEVELOPMENT PROBLEMS PERSIST
------------------------------------------------------------ Letter :5

Although further development and deployment of the OASIS system is on
hold, completion of a successful import system remains a major
information resource management goal for FDA.  We previously
reported\6 on FDA's need to address systems development problems and
implementation delays, and our current review identified many of the
same problems reported by the self-assessment team.  In addition, we
found that the OASIS project lacked necessary cost and performance
information and did not consider some proven best practices of
leading organizations that help ensure successful systems
development.  These problems must be resolved if FDA is to complete
its automation of import operations. 

Beginning in late September 1994 with the award of a new agencywide
strategic information systems support contract, FDA began to address
some of the systems development process problems identified but
continued to lack effective senior-level management and direction, as
well as a systems project management team with information technology
expertise.  In addition, FDA has made little progress in implementing
basic systems development procedures, including conducting user
acceptance testing and a risk assessment.  Recent developments
include the completion of a system design review which concluded that
OASIS was not ready for national implementation and recommended an
immediate reengineering effort. 


--------------------
\6 See ADP Systems:  FDA Can Reduce Development Risks for Its Import
Information System (GAO/IMTEC-88-42, Sept.  30, 1988); Pesticide
Monitoring:  FDA's Automated Import Information System Is Incomplete
(GAO/RCED-92-42, Dec.  31, 1991); and Pesticides:  Status of FDA's
Efforts To Improve Import Monitoring and Enforcement
(GAO/T-RCED-93-55, June 16, 1993). 


      OVERSIGHT AND PROJECT
      MANAGEMENT EXPERTISE ARE
      CRITICAL
---------------------------------------------------------- Letter :5.1

FDA top management did not adequately oversee the OASIS project and
did not provide clear direction and appropriate resources needed to
support the project.  We found that this situation was largely due to
an IRM structure that did not clearly define control and lines of
accountability for the OASIS project.  In addition, we found that the
OASIS project was directed by managers who lacked the systems
development training and expertise to successfully design, develop,
deploy, and maintain an information system. 


         MANAGEMENT OVERSIGHT
-------------------------------------------------------- Letter :5.1.1

The Deputy Commissioner for the Office of Management and Systems is
both the chief financial officer and the senior IRM official for FDA. 
IRM activities on the OASIS project are the responsibility of the
Office of Information Resources Management (OIRM) under the Deputy
Commissioner for the Office of Management and Systems and the Office
of Regulatory Affairs (ORA) under the Deputy Commissioner for the
Office of Operations.  As shown in figure 2, many FDA offices and
divisions have some involvement with the OASIS project. 

   Figure 2:  Organizational
   Structure for OASIS

   (See figure in printed
   edition.)

The responsibilities of OIRM include (1) ensuring that the agency's
5-year strategic plan for acquisition and development of information
resources is prepared and implemented, (2) ensuring that the most
cost-effective approach is applied when acquiring information
technology, and (3) approving acquisitions and ensuring that IRM
goals and strategies are achieved. 

Since the OASIS project began, its planning, design, development,
implementation, and contractor acquisition and interaction resided
primarily within the divisions of import operations and information
systems in ORA.  However, ORA's requests for procurement authority
for OASIS were and continue to be reviewed and approved by OIRM.  The
Deputy Commissioner for Management and Systems told us that since the
award of the current strategic information systems contract in
September 1994, the Associate Commissioner for OIRM has been charged
with providing ORA with continuous technical consultation and
scrutiny of all contractor task orders and deliverables prepared
under ORA's direction.  The OASIS project manager is the director of
the strategic initiatives staff, which is part of ORA, and does not
report directly to OIRM officials. 

In addition, some oversight has been provided at the department
level.  In accordance with the Paperwork Reduction Act, as amended,\7
the HHS Secretary designated a senior official who is responsible for
ensuring agency compliance with and prompt, efficient, and effective
implementation of the information policies and IRM responsibilities
under the Act.  The designated senior official at HHS is the
Assistant Secretary for Management and Budget, who has delegated
certain authorities--such as for procurement--to agencies within the
Public Health Service, including FDA.  The HHS Deputy Assistant
Secretary for IRM, who reports directly to the designated senior
official, is responsible for management and operation of the
department's IRM program.  It is at this level that HHS has provided
FDA with assistance on both the self-assessment team and system
design review committee. 

The joint FDA/PHS/HHS self-assessment report indicated that FDA
senior-level management needed to be closely involved with OASIS due
to the visibility of the system and the troubled system development
history.  The report stated that the Commissioner or other top FDA
officials did not receive regularly scheduled progress reports on the
project.  We found several memoranda dating back to 1989 in which
OIRM raised concerns to ORA about the cost, complexity, and lack of
well-defined requirements, alternatives, and planning regarding
OASIS.  Nonetheless, the project continued under the direction of ORA
until June 1994, when the OIRM director called for the termination of
further development based upon the results of the self-assessment
team. 

It is critical that senior-level oversight of this automation effort
be established to ensure that information technology is acquired,
used, and managed to improve the performance of FDA's public health
and safety mission, and that responsibility and accountability are
improved.  As discussed in GAO's May 1994 publication on the best
practices of leading private and public organizations for strategic
information management, these organizations have found that without
senior executives recognizing the value of improving information
management, meaningful change is slow and sometimes nearly
impossible. 


--------------------
\7 44 U.S.C.  3506. 


         PROJECT MANAGEMENT
-------------------------------------------------------- Letter :5.1.2

As discussed above, ORA administered the day-to-day management of the
OASIS project.  We found, however, that ORA did not have the systems
development expertise in-house to perform these functions.  Our
review of the experience and qualification statements of OASIS
project management showed that the ORA Deputy Associate
Commissioner--the senior project official, the project manager, and
the project officer did not have any systems development training or
experience.  The OASIS project manager concurred with our finding in
a February 1995 memorandum, which stated that ORA did not have
employees with adequate knowledge and experience in life-cycle
methodology and related skills, all of which were important to a
system of OASIS' complexity.  The memorandum stated that ORA planned
to use its current software development and support contractor to
address this deficiency in systems development and hardware
acquisition expertise. 

During our review, the self-assessment team recommended in its July
1994 report that ORA request and accept assistance from another FDA
component, PHS, or HHS to address deficiencies in staff knowledge. 
As stated previously, ORA receives oversight from OIRM for task order
review and approval, but not day-to-day assistance from this or other
sources as recommended.  ORA still does not have someone with the
system development expertise to oversee the OASIS project and monitor
the contractor's work. 

A best practice that can lead to improved mission performance is to
ensure that skills and knowledge of line and information management
professionals are upgraded.  Also useful is establishing
customer/supplier relationships internally and defining roles between
line managers and information management support professionals to
maximize management processes.  Lastly, the chance of a breakdown
between the agency and contractors is great when the agency does not
have information management professionals with the needed expertise
to assist line management in evaluating and supervising contractor
performance. 


      COST AND PERFORMANCE
      INFORMATION ARE NEEDED
---------------------------------------------------------- Letter :5.2

We found that FDA has not been effective in controlling costs or
monitoring the progress of OASIS.  FDA officials informed us that
they did not have a cost accounting system that would enable them to
clearly identify the costs of the OASIS project.  They said that some
of this cost was commingled with other information systems projects. 
For example, despite our repeated attempts to obtain the systems
life-cycle cost for OASIS from its inception through the current
fiscal year, FDA did not provide us with cost data until July 1995. 
This information was prepared by FDA's contractor and submitted in
June 1995, as part of a cost-benefit analysis requested by FDA. 
According to information contained in the contractor's report, the
OASIS systems development costs were estimated to be $13.8 million
from fiscal year 1987 through April 1995.  We did not independently
verify these estimates. 

In addition, the agency did not properly account for or match OASIS
costs with outcomes to determine if OASIS would meet FDA's needs
within its budget allocation.  Accurate accounting of all project
costs will be crucial since FDA is supportive of legislation that
would allow the agency to collect user fees for imports processed
through the automated system to offset the costs of developing,
deploying and supporting the system.  Also, the importance of an
import screening system to FDA's operations and the import community
warrants the maintenance of reliable cost and performance information
to keep congressional appropriations and oversight committees
informed of the status of any systems development effort. 

ORA officials we interviewed told us that they did not establish any
baseline measures to assess current and expected OASIS operational
and technical performance.  As discussed in GAO's best practices
publication, standard performance measurement practices focus on
benefits, costs, and risks and, in most cases, include program
outcomes, resource consumption, and elapsed time (cycle time) of
specific work processes, activities, or transactions.  Performance
measures act as a common focus, allowing management to target problem
areas, highlight successes, and generally increase the rate of
performance improvement through enhanced learning.  Such measures
would allow top management to assess and manage the risk associated
with its import automation effort, and to control the trade-offs
between continued funding of existing operations and developing new
performance capabilities. 


      BASIC SYSTEMS DEVELOPMENT
      PROCEDURES NOT FOLLOWED
---------------------------------------------------------- Letter :5.3

We found that FDA did not follow sound systems development
procedures, such as those outlined in federal guidelines,\8

when developing OASIS because its project management team lacked
expertise and training in systems development.  Specifically, FDA did
not (1) validate its criteria for electronically screening import
entries, (2) conduct user acceptance testing, (3) conduct a risk
assessment or prepare a security plan to address contingencies or
backup procedures to be used in the event of disasters or threats to
FDA's computer facilities, equipment, and data, and (4) conduct a
cost-benefit analysis.  Many of these problems were brought to FDA's
attention as early as 1988.\9 The following systems development
problems must be resolved if FDA is to avoid continued criticism of
its attempts to complete an automated import system. 

  No FDA validation of screening criteria.  FDA had not validated the
     import admissibility screening criteria that reside in Customs'
     ACS.  Validation is essential to ensure that import entries are
     processed accurately and that potentially unsafe products are
     properly identified for "FDA Review." OASIS project officials in
     ORA said that they did not have access to the criteria in ACS
     and could only validate information contained in the
     ACS-generated error reports.  Moreover, these officials stated
     that they did not know if Customs corrected all the errors they
     identified.  The joint self-assessment report also concluded
     that FDA did not have an adequate verification and validation
     process for its software and documentation. 

  Did not conduct user acceptance testing.  The self-assessment
     report stated that FDA did not have written acceptance criteria
     or test plans.  For example, FDA did not conduct nor participate
     with Customs in user acceptance testing prior or subsequent to
     implementing the ACS interface.  ORA's Deputy Associate
     Commissioner told us that it relied on and trusted Customs to
     ensure that the screening criteria database was functioning as
     intended. 

  Security plan not developed.  Until recently, FDA had not conducted
     a risk assessment or developed a disaster recovery plan for
     EEPS, as required by federal guidelines.\10 In 1992, FDA
     declared OASIS a "record system" subject to the requirements of
     the Privacy Act of 1974.\11 Thereafter, FDA considered OASIS a
     critical-sensitive system.  Also, the Computer Security Act of
     1987\12 requires agencies to establish security plans and
     perform vulnerability assessments for all computer systems that
     contain sensitive information.\13

In February 1995, FDA issued a risk assessment of EEPS at FDA
headquarters and a contingency plan to address backup procedures for
the Pacific Region, which runs the regional computer facility in the
Seattle District office.  However, we found that the risk assessment
was incomplete and did not address major portions of EEPS.  In
addition, the contingency plan was not viable because FDA moved the
OASIS processing function from Seattle to the larger processing
facility in its headquarters in Rockville, Maryland.  FDA does not
have a contingency plan for ORA's headquarters computer center. 
However, it plans to obtain a risk assessment of ORA's information
systems and contents through an interagency agreement with the
Department of Transportation.  As of May 1995, FDA could not tell us
when a risk assessment and contingency plan would be performed at FDA
headquarters to address security concerns for this mission-critical
system. 

  No cost-benefit analysis conducted.  At the beginning of our
     review, we learned that no one had performed a cost-benefit
     analysis for the OASIS project.  This deficiency was also later
     reported by the self-assessment team.  A cost-benefit analysis
     describes the development and operational costs of each
     alternative, and of nonrecurring (improved system operations and
     resource utilization) and recurring (operations and maintenance,
     including personnel) benefits that could be attained through the
     development of each proposed alternative.  Such an analysis is
     useful to managers, users, and designers for analyzing
     alternative systems and will be essential to any decisions for
     further development of an automated import system. 

ORA officials told us that they did not ask for such an analysis in
the past.  In February 1995, the current contractor was tasked with
conducting a cost-benefit analysis, which was completed in June 1995. 
However, FDA did not request that the contractor perform an
alternatives analysis.  The current effort was limited to an analysis
of OASIS' historical costs from fiscal year 1987 through April 1995,
which were estimated to be $13.8 million\14 as well as projected
costs from May 1995 through fiscal year 2001, which were estimated to
be $26.2 million.\15 The contractor also analyzed the costs and
benefits of automation as compared to the current manual process. 


--------------------
\8 Federal Information Processing Standards Publication 64,
Guidelines for Documentation of Computer Programs and Automated Data
Systems for the Initiation Phase, and Publication 38, Guidelines for
Documentation of Computer Programs and Automated Data Systems,
National Institute of Standards and Technology, Department of
Commerce.  Also, Model Framework for Management Control Over
Automated Information Systems, President's Council on Management
Improvement and the President's Council on Integrity and Efficiency,
January 1988. 

\9 GAO/IMTEC-88-42, Sept.  30, 1988. 

\10 OMB Circular A-130, appendix III, and Federal Information
Processing Standards Publication 38. 

\11 5 U.S.C.  552a. 

\12 Public Law 100-235, section 6, 101 Stat.  1724, 1729 (1988). 

\13 Vulnerability (risk) assessments are most useful when applied
during the system design phase so that potential losses may be
identified and security requirements defined from the start.  They
are also useful in designing an approach for reducing the loss of
personnel efficacy, information, equipment, and processing
capability. 

\14 This estimate included (1) nonrecurring costs for systems
development and implementation, ADP hardware and software, and
training and (2) recurring costs for ADP hardware and software,
telecommunications, systems operations, the interface with Customs. 
Also, the contractor's estimate identified an additional $351 million
in recurring costs for FDA import personnel FTEs during this period. 

\15 This estimate included (1) nonrecurring costs for a backup site
and facility, systems development and implementation, ADP hardware
and software, and training and (2) recurring costs for ADP hardware
and software, telecommunications, and systems operations.  Also, the
contractor's estimate identified an additional $321 million in
recurring costs for FDA import personnel FTEs during this period. 
The contractor used a 4.6 percent discount rate to calculate the
total present value cost in 1995 dollars at $288 million. 


      RECENT DEVELOPMENTS
---------------------------------------------------------- Letter :5.4

In June 1995, the System Design Review Committee issued its report on
OASIS which concluded that the system is not ready for national
implementation because of significant system deficiencies, including
inconsistent user interface design and the lack of automated
configuration management and version control.  Consequently, the
committee recommended that a reengineering effort begin immediately
to design a system that would incorporate all customers' needs, take
advantage of modern technology and the strategic direction in which
FDA is heading, and position FDA for the future. 

In a July 10, 1995, meeting with FDA's Deputy Commissioner for the
Office of Management and Systems, we were told that FDA will not
implement OASIS nationwide and will begin a reengineering effort.  In
addition, FDA agreed to the recommendations of the committee as
stated in a July 12, 1995, correspondence from the Deputy
Commissioner to ORA officials.  However, the details of the
reengineering effort have not yet been documented so it is not clear
who will lead this effort, what will it involve, and how long will it
take. 

Reengineering is a formidable undertaking that requires an
organization's managers and employees to change the way they think
and work.  For example, after senior management recognizes the need
for change and commits to reengineering, it then must direct the
effort.  Existing business processes should be described and
analyzed, and measurable improvement goals should be set.  In
addition, senior management must also support the reengineering
effort by identifying training needs and determining whether outside
expertise is necessary.  New business processes should then be
designed and the organizational culture, structure, roles, and
responsibilities should be changed to support these new processes. 
Finally, new business processes should be implemented by acquiring
and installing new technology or redesigning existing technology to
support the new processes. 

FDA, though, has not yet clearly defined its reengineering effort and
how it plans to link this effort to its information technology
initiatives.  This is critical if FDA is to achieve dramatic changes
in overall performance and customer satisfaction. 


   CONCLUSIONS
------------------------------------------------------------ Letter :6

A thorough understanding of the factors that led to FDA's failure
over the past 8 years to develop and implement an import system to
meet its mission critical needs is crucial to help ensure that
similar problems and obstacles are avoided in the future.  As FDA
plans its reengineering effort, it is presented with an opportunity
to identify and correct its long-standing systems development
problems.  Because these problems can be attributed to a lack of top
management oversight, systems expertise, and reliable cost and
performance information, continued attention by FDA and HHS is vital
to the success of this automation effort.  It is crucial that FDA
follow sound system development procedures, in conjunction with a
well-defined reengineering strategy, if it is to successfully
implement an import system and achieve its public health and safety
mission. 


   RECOMMENDATIONS
------------------------------------------------------------ Letter :7

We recommend that the Secretary of Health and Human Services direct
the Assistant Secretary for Management and Budget and the
Commissioner of the Food and Drug Administration to ensure that

  continuous top management oversight and systems expertise are
     provided to FDA as it proceeds with its import automation
     effort;

  FDA develops and maintains reliable cost and performance
     information; and

  FDA follows sound systems development practices, including
     validating systems software, conducting user acceptance testing,
     developing a security plan, and conducting a cost-benefit
     analysis that includes an assessment of alternative systems. 

We also recommend that the Secretary direct the Assistant Secretary
and the Commissioner to clearly define how FDA plans to reengineer
its import operations.  At a minimum, FDA should (1) identify and
analyze existing business processes and work flows, (2) obtain the
necessary technical assistance and training to support its
reengineering efforts, and (3) determine new information needs,
application system requirements, and technology requirements
necessary to support the new business processes. 


---------------------------------------------------------- Letter :7.1

As agreed with your office, unless you publicly announce the contents
of this report earlier, we plan no further distribution until 15 days
from the date of this letter.  We will then send copies of this
report to the Secretary of Health and Human Services, the
Commissioner of the Food and Drug Administration, the Director of the
Office of Management and Budget, and other interested parties. 
Copies also will be made available to others upon request.  This
report was prepared under the direction of Patricia T.  Taylor,
Associate Director.  You or your staff can reach me at (202)
512-6252, or Ms.  Taylor at (202) 512-5539, if there are any
questions on the report.  Other major contributors are listed in
appendix I. 

Sincerely yours,

Frank W.  Reilly
Director, Information Resources Management/
  Health, Education, and Human Services Issues


MAJOR CONTRIBUTORS TO THIS REPORT
=========================================================== Appendix I


   ACCOUNTING AND INFORMATION
   MANAGEMENT DIVISION,
   WASHINGTON, D.C. 
--------------------------------------------------------- Appendix I:1

Helen Lew, Assistant Director
William D.  Hadesty, Technical Assistant Director
Karen D.  Bell, Project Manager
Francine M.  DelVecchio, Communications Analyst


   SEATTLE REGIONAL OFFICE
--------------------------------------------------------- Appendix I:2

Susan T.  Chin, Evaluator-in-Charge


*** End of document. ***