Hazardous Waste: Benefits of EPA's Information System Are Limited (Letter
Report, 08/22/95, GAO/AIMD-95-167).

Pursuant to a congressional request, GAO provided information on whether
the Environmental Protection Agency's (EPA) Resource Conservation and
Recovery Information System (RCRIS) is meeting its primary objectives
and its effect on implementation of the Resource Conservation and
Recovery Act (RCRA).

GAO found that: (1) RCRIS does not meet its overall goal of helping EPA
and the states manage the hazardous waste program; (2) most users of
RCRIS do not rely on the system because they find data entry and
retrieval to be too difficult; (3) RCRIS data are not reliable because
the system requires users to manually verify data; and (4) RCRIS
shortfalls have not significantly affected RCRA implementation because
users do not consider the system to be the key tool for program
management and have developed workarounds.

--------------------------- Indexing Terms -----------------------------

 REPORTNUM:  AIMD-95-167
     TITLE:  Hazardous Waste: Benefits of EPA's Information System Are 
             Limited
      DATE:  08/22/95
   SUBJECT:  Hazardous substances
             Environmental law
             Information analysis operations
             Computerized information systems
             Environmental policies
             Data integrity
             Quality assurance
             Federal/state relations
IDENTIFIER:  EPA Resource Conservation and Recovery Information System
             EPA Hazardous Waste Data Management System
             
**************************************************************************
* This file contains an ASCII representation of the text of a GAO        *
* report.  Delineations within the text indicating chapter titles,       *
* headings, and bullets are preserved.  Major divisions and subdivisions *
* of the text, such as Chapters, Sections, and Appendixes, are           *
* identified by double and single lines.  The numbers on the right end   *
* of these lines indicate the position of each of the subsections in the *
* document outline.  These numbers do NOT correspond with the page       *
* numbers of the printed product.                                        *
*                                                                        *
* No attempt has been made to display graphic images, although figure    *
* captions are reproduced. Tables are included, but may not resemble     *
* those in the printed version.                                          *
*                                                                        *
* A printed copy of this report may be obtained from the GAO Document    *
* Distribution Facility by calling (202) 512-6000, by faxing your        *
* request to (301) 258-4066, or by writing to P.O. Box 6015,             *
* Gaithersburg, MD 20884-6015. We are unable to accept electronic orders *
* for printed documents at this time.                                    *
**************************************************************************


Cover
================================================================ COVER


Report to the Ranking Minority Member, Committee on Governmental
Affairs, U.S.  Senate

August 1995

HAZARDOUS WASTE - BENEFIT'S OF
EPA'S INFORMATION SYSTEM ARE
LIMITED

GAO/AIMD-95-167

EPA's Hazardous Waste Information System


Abbreviations
=============================================================== ABBREV

  EPA - Environmental Protection Agency
  GAO - General Accounting Office
  OSW - Office of Solid Waste
  RCRA - Resource Conservation and Recovery Act
  RCRIS - Resource Conservation and Recovery Information System
  WIN - Waste Information Needs

Letter
=============================================================== LETTER


B-261668

August 22, 1995

The Honorable John Glenn
Ranking Minority Member
Committee on Governmental Affairs
United States Senate

Dear Senator Glenn: 

As you know, one of the Environmental Protection Agency's (EPA)
responsibilities is to administer a national program for the safe
management of hazardous waste.  Pursuant to the Resource Conservation
and Recovery Act (RCRA), EPA has granted authority to most states to
implement key requirements of the act, including permitting,
inspecting, monitoring, and enforcement. 

EPA developed, and in 1991 began operating, the Resource Conservation
and Recovery Information System (RCRIS) with the goal of helping both
the federal government and the states better manage the hazardous
waste program.  Intended to overcome serious concerns about the
system it replaced, RCRIS' primary objectives were to (1) provide
individual databases for each state to use in managing the program,
(2) be user friendly, (3) be flexible enough to meet federal
requirements, as well as the specific needs of individual states and
EPA regions, and (4) provide a mechanism for maintaining highly
reliable data. 

In your October 13, 1994, letter to us, you expressed concern about
whether RCRIS is meeting the needs of EPA and the states.  As agreed
with your office, our objectives were to determine whether the system
is meeting its primary objectives and, if not, the effect any
shortfalls have had on RCRA implementation.  Details on our scope and
methodology are provided in appendix I. 


   RESULTS IN BRIEF
------------------------------------------------------------ Letter :1

RCRIS falls short of meeting its overall goal of helping EPA and the
states manage the hazardous waste program because it has not met
three of its primary objectives.  While RCRIS gives the states their
own individual databases, data entry and retrieval are so difficult
that most users do not rely on the system.  Moreover, while the
system was intended to be flexible enough to meet individual needs,
most users do not attempt to do this because of the difficulties
using and modifying the system.  Furthermore, the data in RCRIS are
not reliable, forcing users to manually verify the data before it can
be used. 

RCRIS' shortfalls have not significantly affected RCRA
implementation, primarily because the system is not relied on as a
key tool for managing the program.  Instead, users rely on systems
they have independently developed as well as other work-arounds that
have been created.  Recognizing that federal and state users' needs
are not being met through RCRIS, EPA recently began reassessing
information needs and systems support for the RCRA program. 


   BACKGROUND
------------------------------------------------------------ Letter :2

Congress enacted RCRA to protect human health and the environment and
to conserve energy and material resources.  In implementing RCRA, EPA
is responsible for developing federal hazardous waste management and
disposal requirements and for ensuring compliance with these
requirements.  Two offices within EPA share responsibility for
RCRA--the Office of Solid Waste (OSW) and the Office of Enforcement
and Compliance Assurance. 

EPA may authorize states\1 to implement their own RCRA programs, or
portions thereof, provided that their programs meet federal
requirements.  Currently, 46 states, the District of Columbia, and
Guam, have been authorized to implement at least some portion of
RCRA.  States that are authorized receive annual grants from EPA and
are subject to EPA oversight.  In fiscal year 1994, EPA gave
approximately $92 million in RCRA grant money to these states and
territories. 

To help manage RCRA, EPA and authorized states collect and manage
data on more than 250,000 hazardous waste handlers, including 210,900
small quantity generators; 15,300 large quantity generators; 19,700
transporters; and 4,300 treatment, storage, and disposal facilities. 
Day-to-day RCRA activities include assigning hazardous waste handler
identification numbers; issuing permits; tracking inspections,
violations, and enforcement actions; and maintaining other related
data to assess the extent to which facilities are complying with RCRA
regulations. 

In 1985, to overcome problems with the information system it was
using, known as the Hazardous Waste Data Management System, EPA began
developing RCRIS.  Both the states and EPA had reported that the
Hazardous Waste Data Management System was not adequately supporting
the program in many areas, including permitting, compliance
monitoring, and enforcement.  In addition, the system was criticized
as cumbersome, difficult to use, and confusing, and the database was
criticized as excessively complex. 

The overall goal of RCRIS was to help both the federal government and
the states better manage the hazardous waste program.  The National
Governors' Association assisted EPA in identifying state needs for
RCRIS.  EPA determined, based on information collected by the
Association, that for RCRIS to be successful, the system had to meet
the following four key objectives:  (1) provide individual databases
for each state, (2) be user friendly, (3) be flexible enough to meet
federal requirements, as well as the specific needs of individual
states, and (4) provide a mechanism for maintaining highly reliable
data. 

RCRIS became operational nationwide in 1991.  The system comprises
four key modules that mirror the major RCRA program activities. 
Table 1 provides the name and purpose of these modules. 



                           Table 1
           
            Key RCRIS Modules and Intended Purpose

Module                         Purpose
-----------------------------  -----------------------------
Handler Identification         Maintain basic data
                               identifying and describing
                               handlers so they can be
                               tracked and monitored

Permitting, Closure,\a and     Maintain detailed information
Post-Closure                   about hazardous waste
                               treatment; storage and
                               disposal processes; and
                               permitting, closure, and
                               post-closure activities

Compliance Monitoring and      Maintain information on
Enforcement                    inspections, violations, and
                               enforcement actions

Corrective Action              Track specific corrective
                               action information needed to
                               regulate facilities with
                               hazardous waste releases
------------------------------------------------------------
\a Closure occurs when hazardous wastes are no longer accepted and
the facility is prepared to cease operations. 

Although OSW did not maintain information about RCRIS operational
costs, a recent report by the EPA Inspector General stated that
fiscal year 1993 operational costs were $7.5 million.\2 This figure,
however, excludes EPA's and states' personnel costs for time spent
(1) entering, retrieving, and manipulating data in RCRIS and (2)
developing and operating other systems and work-arounds to compensate
for RCRIS inadequacies.  Officials in OSW said they did not know
these costs. 


--------------------
\1 These include all 50 states as well as the District of Columbia
and five territories. 

\2 Management of Application Software Maintenance at EPA
(E1NMF3-15-0072-5100240, Mar.  31, 1995). 


   SCOPE AND METHODOLOGY
------------------------------------------------------------ Letter :3

Our work was performed at several offices at EPA headquarters,
including the Office of Solid Waste, the Office of Enforcement and
Compliance Assurance, and Office of Inspector General.  These offices
were located in Washington, D.C., and Arlington, Virginia.  We also
worked at EPA regional offices in Atlanta, Georgia; Dallas, Texas;
and San Francisco, California; and at state offices responsible for
RCRA in Tallahassee, Florida; Jackson, Mississippi; Little Rock,
Arkansas; Austin, Texas; Carson City, Nevada; and Sacramento,
California. 

We conducted our review from August 1994 through May 1995, in
accordance with generally accepted government auditing standards.  We
requested comments on a draft of this report from the Administrator
of EPA.  The Director for the Office of Solid Waste provided us with
comments on July 18, 1995.  We have incorporated these comments where
appropriate. 


   RCRIS IS NOT MEETING USER NEEDS
------------------------------------------------------------ Letter :4

RCRIS falls short of meeting its overall goal of helping EPA and the
states manage the hazardous waste program because it has not met
three of its primary objectives.  While RCRIS provides individual
databases for states to use to maintain their data, the system is
difficult to use, difficult to tailor to meet individual state and
EPA regional needs, and does not provide a mechanism for maintaining
highly reliable data. 


      STATES CAN MAINTAIN THEIR
      OWN RCRIS DATABASES
---------------------------------------------------------- Letter :4.1

RCRIS provides two databases for each state--one maintained by the
state and one maintained by the EPA regional office.  These databases
are intended to support day-to-day RCRA activities carried out by the
states and EPA regional offices.  Extracts from these databases are
merged to construct a database of required information for regional
oversight.  A national oversight database for EPA headquarters and
national reporting is then formed by merging extracts from the
regional merged databases. 


      RCRIS IS DIFFICULT TO USE
---------------------------------------------------------- Letter :4.2

RCRIS is not easy to use, particularly with regard to data entry and
retrieval.  Data entry is cumbersome largely because of the design of
the four key RCRIS modules.  For example, each module uses numerous
data entry screens, many of which allow the user to modify data in
only a few data fields at one time.  The RCRIS user manual shows that
a user could possibly interact with up to 150 data entry screens to
add, modify, or delete data. 

Because of this design, users are often required to switch between
modules, screens, and files--even when making simple changes to the
data.  For example, to modify a hazardous waste handler's information
and add data for an inspection of a specific facility, a user must
access the handler identification module, enter the handler's
identification number, update the handler information, exit the
handler module, enter the compliance monitoring and enforcement
module, reenter the handler's identification number, and then enter
the inspection information. 

Data entry is also difficult because codes are used extensively to
represent much of the information that is recorded.  These codes are
used to describe various aspects of a hazardous waste handler, such
as the process code that specifies the current waste treatment,
storage, or disposal process being used.  RCRIS' four key modules
include about 875 codes.  The RCRIS user manual notes that
understanding the majority of the codes is essential for successful
data entry, maintenance, processing, and reporting. 

Users have also found data retrieval difficult.  While users can
obtain several standard reports from the system, these reports are
voluminous.  For example, up to four cardboard boxes are needed to
hold one standard report on facilities.  Consequently, users need to
query the system to obtain specific information, but creating
customized reports is very difficult.  For example, to obtain
information on a specific facility's history or status directly from
the system, a user must first know which files contain data on the
facility.  The user must then query each file to obtain the desired
data.  Finally, the user must know how to merge the extracted data to
produce a single report.  Users in the six states and the three EPA
regions we visited said that creating these customized reports is
very difficult. 


      RCRIS DOES NOT SATISFY
      INDIVIDUAL STATE AND EPA
      REGIONAL NEEDS
---------------------------------------------------------- Letter :4.3

Another primary objective for RCRIS was that it be flexible enough to
allow states and EPA regions to tailor the system to meet their
individual program needs.  However, users in five of the six state
offices and all three of the regional offices we visited stated that
RCRIS has not met their individual needs.  Many of these users said
they have no desire to tailor RCRIS to meet their needs because the
system is just too difficult to use. 

We identified users in two EPA regions who said they had been able to
better meet some individual needs using RCRIS.  For example, because
RCRIS' standard reports do not include the precise information
needed, these users developed customized query programs and reports,
and have made them available to other users across the country. 
These users said they were able to do this because they invested a
significant amount of time in learning the intricacies of the system
and had become expert in using it. 

In addition, because almost all the states were using or developing
their own systems, EPA wanted them to be able to extract the required
data from their systems and submit it to RCRIS electronically. 
Although states can do this, state users told us that creating
conversion software to reformat their data into RCRIS' data structure
and then modifying the conversion software each time EPA changes
RCRIS' data structure can be extremely difficult.  Consequently, only
one state transmits any data to RCRIS electronically. 


      RCRIS DATA ARE NOT RELIABLE
---------------------------------------------------------- Letter :4.4

RCRIS does not provide a mechanism for maintaining highly reliable
data.  While EPA does not require data quality reviews of the data in
RCRIS, the few ad hoc reviews that have been conducted have uncovered
significant problems.  For example, a Region VI (Dallas) review of
facility demographic data on about 45,000 hazardous waste handlers
identified over 15,000 errors on the location of the handlers. 
Similarly, Region IV (Atlanta) found anecdotal evidence of
discrepancies between the data in RCRIS and the information in hard
copy files.  According to EPA officials, states have corrected most
of these discrepancies.  However, users in all regional and state
offices we visited consistently expressed concerns about the quality
of RCRIS data. 

In addition, as part of a review of whether land disposal facilities
were complying with groundwater monitoring requirements, we found
RCRIS data to be unreliable.  We selected a random sample of 30
facilities out of RCRIS' universe of 1,427 classified by EPA as land
disposal facilities.  For each sampled facility, we reviewed data in
14 data fields that were most relevant to groundwater monitoring
requirements.  We estimated minimum error rates ranging from 8 to 37
percent for eight of the data fields.\3 These data provided
information on the types of violations detected and enforcement
actions taken. 

These data reliability problems can be attributed, in part, to a lack
of controls over the quality of data when EPA converted from the
predecessor system to RCRIS.  Although EPA recognized that the
conversion would result in inaccurate data, it did not require
verification and correction of the data--a generally accepted
practice.  Instead, EPA loaded the inaccurate data and then left it
up to each state and region to perform a post-conversion assessment
of data quality.  Further, although EPA (1) provided software to help
the states and regions detect data quality errors after the
conversion, (2) set aside 5 percent of state grant funding to support
data cleanups, and (3) set a 90-percent accuracy goal for the
converted data, the agency allowed users to choose which data they
would check and did not determine whether states and regions had met
the accuracy goal.  Thus, officials in all three regions we visited
had concerns about lingering data quality problems because of
unverified data from the predecessor system. 

A second factor contributing to data quality problems is EPA
headquarters' failure to provide precise definitions for certain data
elements in the system.  Officials in all regions and five of the six
states we visited said that RCRA data definitions are interpreted
differently and have led to differences among states in how data in
the same fields are recorded in RCRIS.  For example, while EPA
requires users to identify facilities that burn hazardous waste, the
agency does not specify whether only a facility that burns hazardous
waste as a regular part of its operations should be considered a burn
facility, or if all facilities that have ever burned waste,
regardless of frequency, should be included.  Headquarters officials
agreed that this is a critical problem and acknowledged that no one
has taken control of data standardization issues. 

Another factor contributing to unreliable data is that RCRIS'
front-end edit checks do not effectively help ensure that data are
accurate when they are entered.  They do not check whether data that
are entered are consistent with other related data fields.  For
example, while violations typically result from an inspection,
violation data can be entered into the RCRIS violation file even if
the inspection data were never entered because users are not prompted
to enter the related inspection.  Therefore, incomplete information
may remain in the system. 

Finally, EPA does not have a data quality assurance program that
would include establishing reliability standards and require periodic
reviews of RCRIS data to determine if they meet these standards.  EPA
headquarters officials said they do not require regular data quality
reviews because they believe that it is the regions' and states'
responsibility to ensure data in the system are reliable.  EPA
officials in the three regions we visited told us that they use a
RCRIS software feature to verify certain data, but that overall data
quality reviews have only been done on an ad hoc basis, and that
these reviews had focused on limited data elements. 


--------------------
\3 See appendix II for details on the error rates. 


   RCRIS SHORTFALLS HAVE NOT
   SIGNIFICANTLY AFFECTED THE RCRA
   PROGRAM
------------------------------------------------------------ Letter :5

Although RCRIS has not met its original objectives, system problems
have not significantly affected the RCRA program because users do not
rely on it as a primary management tool.  However, RCRIS' problems
have placed additional burdens on EPA and the states to develop work
arounds and additional systems.  All of the regions and states we
visited were using at least one PC-based system to supplement RCRIS. 
In some cases, these systems did not address unique needs, but rather
duplicated the same functions as RCRIS in a format that was easier to
use. 


      EPA OFFICIALS DO NOT RELY ON
      RCRIS DATA
---------------------------------------------------------- Letter :5.1

While EPA officials stated that RCRIS has improved some aspects of
systems support, officials in all three regions we visited and
headquarters, including the Director of the Office of Solid Waste,
told us that the system is not reliable enough to strategically
manage the RCRA program.  Some data in RCRIS are used for program
planning, state grant workload models, and regulatory development. 
However, rather than relying on information directly out of the
system, EPA headquarters officials said they routinely send national-
level RCRIS reports back to regional offices and the states for
subsequent verification and correction before they accept and use the
data.  EPA program officials and staff also told us that RCRIS does
not provide all the information they need.  For example, these
officials pointed out that universe information--the total number of
hazardous waste handlers in particular classes--cannot be obtained
from RCRIS.  As a result of these problems, the Director stated that
he believes it is necessary to rethink RCRA's information needs,
particularly with regard to data needed for environmental and
management indicators. 


      STATES USE THEIR INDIVIDUAL
      SYSTEMS
---------------------------------------------------------- Letter :5.2

All six of the state locations we visited use their own systems to
help them meet their information needs.  State officials said they
enter data into RCRIS primarily to satisfy their RCRA grant
commitments, which require them to devote specific resources to
supporting RCRIS and entering required data into the system.  The
officials also said they maintain their own systems because these
systems are easier to use and they meet the state's individual needs
much better than RCRIS.  For example, California maintains separate
databases for managing its regulated universe of hazardous waste
handlers because its data needs are more detailed and broader in
scope than EPA's. 


      EPA EFFORTS TO ADDRESS
      SHORTFALLS
---------------------------------------------------------- Letter :5.3

During our review, EPA acknowledged RCRIS' many problems.  Further,
OSW admits that it does not know who the system's users are or how
frequently the system is used.  In addition, as previously noted, EPA
does not know how much money is being spent on RCRIS field support or
on developing and maintaining supplemental systems. 

Recognizing RCRIS' shortcomings, OSW has begun a new strategic
effort--the Waste Information Needs (WIN) initiative--to determine
the information and technology needs of EPA headquarters, EPA
regions, and states.  EPA intends to produce a plan addressing these
needs by September 1995.  In the interim, EPA plans to continue
operating RCRIS even though (1) most states we visited do not use it
to meet their needs and (2) EPA acknowledged that RCRIS data are
unreliable. 


   CONCLUSIONS
------------------------------------------------------------ Letter :6

RCRIS is providing few of the benefits that were originally intended. 
The system is difficult to use and its data are unreliable. 
Consequently, many of the users RCRIS was created to serve do not
rely on the system or the information in it.  Instead, states operate
their own systems, and EPA uses other means to get reliable data. 
Until EPA identifies what information it needs to manage RCRA, EPA
will continue to operate a national system with unknown costs, few
benefits, and documented burdens. 


   RECOMMENDATIONS
------------------------------------------------------------ Letter :7

We recommend, as a part of the recently initiated reassessment of
RCRA information needs, that the Administrator of the Environmental
Protection Agency require the Assistant Administrator of the Office
of Solid Waste and Emergency Response to

  determine what information EPA needs to oversee states'
     implementation of RCRA,

  develop clear data definitions for all EPA required data,

  develop a data quality assurance program that establishes data
     reliability standards and methods to ensure data reliability,
     and

  develop and implement a cost-effective solution for meeting
     identified needs. 

In the interim, given the burden RCRIS imposes on the states and the
lack of EPA's reliance on the system, we recommend that the
Administrator of the Environmental Protection Agency direct the
Assistant Administrator of the Office of Solid Waste and Emergency
Response to assess RCRIS data reporting requirements and eliminate
those that are identified as nonessential. 


   AGENCY COMMENTS AND OUR
   EVALUATION
------------------------------------------------------------ Letter :8

In commenting on a draft of this report, the Director for the Office
of Solid Waste stated that EPA generally agreed with our analyses,
findings, and recommendations.  However, the Director requested
clarification on two of our recommendations. 

First, our draft report proposed, as part of the WIN initiative, that
EPA define the information needs of the states that require system
support from EPA to manage their programs.  The Director stated that
EPA did not know if we were recommending that the agency only define
information needs or if we were recommending that it also supply
system support (development and/or operational) to those states.  The
Director said EPA realizes that it cannot build one automated system
to support all users and that EPA wants to get out of the business of
supplying system support to the states. 

We made this proposal because of our concern that in conducting the
WIN initiative, EPA would again gather information requirements from
state and EPA users, and then develop a system that it hoped would
satisfy all of these requirements.  Our intent was for EPA to
explicitly identify which states require systems support in order to
narrow the requirements for the new system to only those users who
will rely on the system.  We have deleted our recommendation based on
the Director's recognition that EPA cannot build one system to meet
federal and state needs; however, we caution EPA as it proceeds with
the WIN initiative to determine and make clear to the states
specifically what level of system support will be provided. 

Secondly, we proposed that EPA consider eliminating, while it
completed the WIN initiative, the requirement for states to supply
data to RCRIS.  The Director stated that EPA realizes that the
burdens associated with maintaining RCRIS need to be reduced, but
noted that EPA relies on such data for program implementation and
management.  EPA did agree to consider eliminating, or significantly
reducing, the requirement for states to supply data to RCRIS. 

We did not intend for EPA to take an all-or-nothing approach to
RCRIS.  In EPA's consideration of the option, we expected that
certain data elements would be identified as essential while others
could be eliminated.  We have modified our recommendation to reflect
this. 


---------------------------------------------------------- Letter :8.1

As arranged with your office, unless you publicly announce the
contents of this report earlier, we plan no further distribution
until 30 days from the date of this letter.  At that time, we will
send copies to the Administrator of the Environmental Protection
Agency, Director of the Office of Management and Budget, and
interested congressional committees.  Copies will also be made
available to others upon request. 

Please call me at (202) 512-6253 if you or your staff have any
questions concerning this report.  Other major contributors are
listed in appendix III. 

Sincerely yours,

Joel C.  Willemssen
Director, Information Resources
 Management/Resources, Community,
 and Economic Development


SCOPE AND METHODOLOGY
=========================================================== Appendix I

To address our objectives, we used a structured interview document to
discuss use of the system and its information with regional and state
users.  We interviewed officials and staff at EPA regional offices in
Dallas, Texas; Atlanta, Georgia; and San Francisco, California.  We
chose these locations because they are three of the largest RCRA
regions responsible for 17 states and 3 territories, and because
headquarters officials believed they would adequately represent the
diversity in use of RCRIS. 

We also interviewed representatives of six state environmental
protection offices--the largest in each of the regions--Texas,
Florida, and California--and the smallest--Arkansas, Mississippi, and
Nevada.  At each location, we met with the RCRIS database
administrators and other system staff, as well as officials who were
responsible for managing and implementing the RCRA program.  We also
reviewed numerous documents related to RCRIS use, operation, and
output. 

We reviewed detailed information on the RCRA program, EPA's previous
attempts to collect hazardous waste data through automation, and the
development and operation history of RCRIS; and examined prior GAO
and EPA Inspector General reports that discussed weaknesses in EPA's
information systems environment.  We interviewed EPA headquarters
officials from the Office of Solid Waste and the Office of
Enforcement and Compliance Assurance.  These officials included the
Director of the Office of Solid Waste, the RCRIS system manager, and
staff involved in RCRA and RCRIS activities.  We also interviewed
representatives from the National Governor's Association and reviewed
documents pertaining to its involvement in assisting EPA to identify
the objectives and requirements in RCRIS. 

In addition, we used relevant segments of the information systems
audit methodology published by the EDP Auditors Foundation to
evaluate the extent to which RCRIS' development and operation met
generally accepted systems practices.\1 We reviewed and analyzed
documents on EPA's actions, including documentation on user
requirements, system feasibility, functional requirements, change
control procedures, and data management.  We also reviewed EPA
planning documents including information on an Office of Solid Waste
strategic initiative. 

Our work was performed at several offices at EPA headquarters
including the Office of Solid Waste, the Office of Enforcement and
Compliance Assurance, and Office of Inspector General.  These offices
were located in Washington, D.C., and Arlington, Virginia.  We also
worked at EPA regional offices in Atlanta, Georgia; Dallas, Texas;
and San Francisco, California; and at state offices responsible for
RCRA in Tallahassee, Florida; Jackson, Mississippi; Little Rock,
Arkansas; Austin, Texas; Carson City, Nevada; and Sacramento,
California. 

We conducted our review from August 1994 through May 1995, in
accordance with generally accepted government auditing standards.  We
requested comments on a draft of this report from the Administrator
of EPA.  The Director for the Office of Solid Waste provided us with
comments on July 18, 1995.  We have incorporated these comments where
appropriate. 


--------------------
\1 Computerized Information Systems Audit Manual, EDP Auditors
Foundation, Inc., 1992. 


ERROR RATES FOR 14 SELECTED
RESOURCE CONSERVATION AND RECOVERY
INFORMATION SYSTEM DATA ELEMENTS
========================================================== Appendix II


                                         Actua
                                             l
                                  Numbe  error
                                   r of   rate   From     To
                                  error  perce  perce  perce
Data element                          s     nt     nt     nt
--------------------------------  -----  -----  -----  -----
1. Compliance evaluation             16     53  36.68  69.52
 inspections
2. Comprehensive groundwater          6     20   8.48  36.49
 monitoring evaluations
3. Operation and maintenance          6     20   8.48  36.49
 inspections
4. Cumulative groundwater            11     37  21.68  53.71
 violations
5. Informal enforcement actions      13     43  27.48  60.18
6. Formal enforcement actions         7     23   10.9  40.08
7. Assessed penalties                 7     23   10.9  40.08
8. Outstanding groundwater            9     30   16.1  47.00
 violations
9. Facility operational status        5     17   6.21  32.85
10. Environmental ranking             1      3   0.12  16.57
11. Stabilization evaluation          2      6   0.91  21.02
12. RCRA facility assessment          4     13   4.13  29.07
 completed
13. RCRA facility investigation       4     13   4.13  29.07
 needed
14. Corrective action status          2      6   0.91  21.02
------------------------------------------------------------
\a Lower and upper bounds of the 95-percent confidence interval for
percent of entries in error. 


MAJOR CONTRIBUTORS TO THIS REPORT
========================================================= Appendix III

ACCOUNTING AND INFORMATION
MANAGEMENT DIVISION, WASHINGTON,
D.C. 

Ronald W.  Beers, Assistant Director
Patricia J.  Macauley, Senior Information Systems Analyst
Shane D.  Hartzler, Communications Analyst

DALLAS REGIONAL OFFICE, DALLAS,
TEXAS

William H.  Thompson, Evaluator-in-Charge
Luis Escalante, Jr., Senior Evaluator
Shannon Q.  Cross, Senior Computer Specialist
