    [From the U.S. Government Printing Office, www.gpo.gov]

       *~~~~~~~~~~~~~~~~DM OIMk*G

    TD
    4 20
    1 992

                                               United States
       GAO                                 General Accounting Office       Property of csc Library
                                               Washington, D.C. 20548

                                               Information Management and
                                               Technology Division

                                               B-248721

                                               June 22, 1992
                                                                             U. D. F4FTFNT )F COMMERCE NOAA
                                               The Honorable Max Baucus COM   T      '   tCE $ CE N T ER
                                               Chairman, Subcommittee on 2234 SOiUTH nOBSON AVENUE
                                                 Environmental Protection CHAL ESTON, SC  29405-2413
                                               Committee on Environment and
                                                 Public Works
                                               United States Senate

                                               Dear Mr. Chairman:

                                               To assist you in your efforts writing proposed amendments to the Clean
                                               Water Act, we reviewed specific capabilities of the Environmental
                                               Protection Agency's (EPA) Permit Compliance System (PCS). PCS is the
                                               information system that supports the National Pollutant Discharge
                                               Elimination System (NPDES), a Clean Water Act program that issues
                                               permits and tracks facilities that discharge pollutants into our navigable
                                               waters.

                                               You expressed concern that PCS may not have adequate capabilities to
                                               maintain various data elements or perform analyses, and that it does not
                                               include detailed data' on all facilities, both major and minor,2 regulated
                                               under NPDES. Without these capabilities and data, a clear picture of what is
                                               being put into our waters cannot be formed.

                                              As agreed with your office, we determined (1) whether PCs had capabilities
                                               to maintain various data elements and perform analyses, (2) if EPA had
                                               assessed any alternative technologies to the existing data entry process,
                                               (3) the number of facilities in PCS with detailed data, particularly minors,
                                               (4) the estimated resource expenditures for minor facility data entry and
                                               update, and (5) the approximate operating costs of PCs.

                                              We provided the results of our work in a formal briefing to your office in
                                               May 1992, and agreed to prepare this report containing the charts used in
                                               the briefing (see app. I).

    -    =,.                               l'Detailed data are all data that are required to monitor a facility's water discharge. Detailed data consist
    �> . .-a                               of facility name and location, permit discharge limits, discharge monitoring report information, and
                                              related enforcement action and inspection data.
                                              2EPA classifies a facility as either a major or minor discharger depending on established criteria such as
                                              the flow of the discharge or the potential impact the facility will have on a water body or public health.

                                              Page 1                      GAO/IMTEC-92-58BR EPA's Permit Compliance System

                                          B-248721

                                          The NPDES program regulates facilities that discharge pollutants into our
    Background
                                           navigable waters. Implementation authority lies with EPA's 10 regional
                                           offices and 39 delegated states. The regional offices also retain
                                           implementation authority for nondelegated states in their region.
                                           Delegated authorities issue permits, set effluent limits and monitoring
                                           requirements, track compliance, and carry out enforcement actions.

                                          According to EPA records, there are approximately 7,100 major and 57,000
                                           minor facilities regulated under NPDES nationwide. To monitor compliance,
                                           delegated authorities conduct inspections and screen facility-submitted
                                           discharge monitoring reports (DMR) that include pollutant discharge
                                           measurements. When a facility discharges above a permitted limit or does
                                           not report all required data, enforcement actions may follow.

                                           PCs was developed in 1974 to support the NPDES program. It resides on a
                                           mainframe computer at EPA's National Computer Center (Ncc) in North
                                           Carolina. Data are entered into PCS by states or EPA regional offices either
                                           by direct entry or batch updates. Delegated authorities are required to
                                           enter and maintain detailed data on all major facilities in Pcs. For minor
                                           facility dischargers, EPA only requires the facility name and address, permit
                                           event data, and inspection data to be maintained in the system, although
                                           some authorities choose to enter detailed data. PCs is used to monitor
                                           compliance for major facilities and can be used for minor facilities if
                                           detailed data are in the system. EPA uses PCS to produce reports of major
                                           facility discharge and compliance activities under the NPDES program.

                                           PCs has capabilities that enable EPA to monitor and track compliance of
                                           permitted facilities. However, many PCS users said that some of the
                                           capabilities are difficult to use because the system is not user-friendly. In
                                           addition, use of PCS varies among delegated authorities. Of the 64,227
                                           NPDES facilities, we found that all major facilities (7,139) and 21,187 minor
                                           facilities have detailed data in PCS. EPA is considering using new
                                           technologies to facilitate more efficient data entry. We estimate that
                                           currently 76 full-time equivalents (FTE) are required to enter detailed data
                                           for the 21,187 minor facilities in Pcs. We also determined the total
                                           operating cost for PCs, including state data entry costs, to be
                                           approximately $5.9 million in fiscal year 1991.

                                           Page 2                      GAO/IMTEC-92-58BR EPA's Permit Compliance System

    B-248721

    As agreed with your office, we did not obtain official comments on a draft
    of this report. However, we discussed this information with the Acting
    Chief of the Information Management Section within the Office of
    Wastewater Enforcement and Compliance (OWEc). We have incorporated
    her comments where appropriate.

    We performed our work in accordance with generally accepted
    government auditing standards between February 1992 and May 1992. We
    interviewed system managers and coordinators responsible for maintaining
    and overseeing use of PCs at headquarters, regional, and state levels. We
    spoke with officials responsible for 27 of the 39 delegated state
    environmental offices across the country and all IO EPA regional offices.
    We also met with personnel in EPA Region 1 and state environmental
    personnel in Rhode Island, Pennsylvania, and Ohio. We observed use of PCs
    at two of these locations and EPA headquarters. In addition, we evaluated
    system documents and output reports from the system.

    We are providing copies of this report to the Administrator of EPA;
    interested congressional commnittees; the Director, Office of Management
    and Budget; and other interested parties. We will also make copies
    available to others on request. Should you have any questions, please
    contact me at (202) 512-6416. Major contributors are listed in appendix
    Ill.

    Sincerely yours,

      JayttaZ.Hecker
    Director, Resources, Community, and Economic
    Development Information Systems

    Page 3                      GAO/IMTEC-92-58BR EPA's Permit Compliance System

    Contents

    Letter                                                                                                            1

    Appendix I                                                                                                        6
    Briefing Slides and                    PCS CapabilitiesAssessed                                                   7
        Explanatory Narrative             Potential Added Data to PCS                                                9
                                         PCS Use Varies                                                           11
                                         Optical Character Recognition                                            13
                                         Electronic Data Interchange                                              15
                                         Number of Facilities Tracked in PCS                                      17
                                         Differences in PCS Data                                                  19
                                         Estimated Data Entry Resource Requirements                               21
                                          PCS Operating Costs                                                      23
                                          Conclusions                                                              25

    Appendix II                                                                                                      26
    Methodology for
    Estimating Data Entry
    Resource Requirements

    Appendix III                                                                                                     30
    Major Contributors to
    This Report

    Tables                                 Table 1.1: Survey Results                                                 17
                                          Table 1.2: EPA Reported Data                                             17
                                          Table 1.3: PCS Operating Costs                                           23

                                          Page 4                        GAO/IMTEC-92-58BR EPA's Permit Compliance System

    Contents

    Abbreviations

    DMR      discharge monitoring report
    EDI      electronic data interchange
    EPA      Environmental Protection Agency
    FTE      full-time equivalent
    GAO      General Accounting Office
    GSC      General Sciences Corporation
    IMTEC    Information Management and Technology Division
    NCC      National Computer Center
    NPDES   National Pollutant Discharge Elimination System
    OCR      optical character recognition
    OWEC    Office of Wastewater Enforcement and Compliance
    Pcs      Permit Compliance System
    REI      Recognition Equipment Incorporated

    Page 5                      GAO/flMTEC-92-58BR EPA's Permit Compliance System

    Appendix I
    Briefing Slides and Explanatory Narrative

          GAO   Permit Compliance System -
                Capabilities Assessed

                PCS can maintain:
                *previous permit data
                *terminated discharge data
                *detailed data
                *noncompliance status until
                 final correction

                Related applications can:
                *track trends
                *calculate total loadings and
                 basic statistics
                *produce graphics

                        Page 6        GAOJIMTEC-92-58BR EPA's Permit Compliance System

                                           Appendix I
                                           Briefing Slides and Explanatory Narrative

    PCS  Capabilities                    You questioned whether PCS could perform various functions, such as
                                           maintaining: (1) individual entries for each NPDES permit issued, including
    Assessed                              data concerning any permit renewals or permits previously issued for the
                                           same discharge; (2) data concerning any permitted facility that no longer
                                           discharges for a period of 5 years; (3) essential permit information,
                                           including the holder of the permit, the location of the discharge, the water
                                           body receiving the discharge, the nature of the discharge, and the limits
                                           placed on the discharge; (4) permit compliance data, including the date of
                                           any inspection, the date and nature of any violation, and the nature and
                                           schedule of any corrective action; and (5) a facility's noncompliance status
                                           until final corrective action is implemented. In addition, you wanted to
                                           know if PCs has the ability to do analyses, such as tracking compliance
                                           trends over time.

                                           Through observations of PCs and conversations with PCS users, we found
                                           that all the capabilities and analyses stated above do exist. However, since
                                           EPA does not require detailed data to be maintained on minor facilities,
                                           compliance and discharge data for all NPDES facilities are not available.
                                           Therefore, nationwide NPDES reports reflect major facility activities only.

                                           A separate application developed by EPA Region 2 uses PCS data to perform
                                           analyses such as calculating total discharge loadings on water bodies and
                                           tracking trends. This application resides on the mainframe computer at
                                           NCC and is available through the PCS menu. Some states have also
                                           developed additional applications to further enhance their ability to obtain
                                           and use information from PCs. Some of these enhancements include
                                           making PCs reports easier to read and analyzing data in ways PCS does not.

                                          Page 7                       GAO/IMTEC-92-58BR EPA's Permit Compliance System

                    Appendix I
                    Briefing Slides and Explanatory Narrative

    GAO Potential Added
           Data to PCS

           Sludge disposal data
           *about 16,000 current permits
            affected
           *about 3,000 new permits
            required

           Stormwater runoff data
           *about 100,000 to 300,000
            new permits may be required

                    Page 8        GAO/IMTEC-92-58BR EPA's Permit Compliance System

                                           Appendix I
                                           Briefing Slides and Explanatory Narrative

    Potential Added Data                 Because of provisions added to the Clean Water Act in 1987, EPA is
                                           considering whether to add stormwater runoff and sludge disposal data to
    to PCS                               PCs. However, when we met with EPA, no decisions had been made on
                                            either issue.

                                            Currently there are about 16,000 NPDES permits that include sludge data.
                                           In addition, an enforcement official in OWEC told us she anticipates that
                                           about 3,000 new sludge permits will be issued as a result of legislative
                                           provisions. In 1991 EPA conducted a sludge feasibility study to determine
                                           what to do with sludge data. However, the study assumed that existing PCS
                                           hardware and software would be used. Several state environmental officials
                                           told us they do not want to add sludge disposal data to PCS because sludge
                                           is already regulated by state solid waste programs. If required to enter this
                                            data into PCS, some states may have to rekey data that are already entered
                                            into a state sludge system.

                                            Four states included in the sludge feasibility study wanted to use a separate
                                           sludge monitoring report rather than including sludge disposal data in
                                            DMRS. However, this issue was not addressed. According to the study, it is
                                           likely that sludge monitoring information will be reported through
                                           facilities' existing DMRS. In providing comments on the facts contained in
                                           this report, EPA informed us that an automated system for sludge tracking
                                           was being tested for implementation into PCS. It is scheduled to be available
                                           to users in June 1992, although specific reporting requirements have not
                                           been determined.

                                            Headquarter's consideration of building a separate system for data on
                                            stormwater also runs opposite to several state officials' opinion on what to
                                            do with the data. According to some state environmental officials,
                                           stormwater seems to be a natural addition to a data base, such as PCs, that
                                           tracks discharges into water. In addition, many of these officials agreed
                                           that stormwater may be amenable to use of a general permit,3 thus
                                            reducing the data entry impact of an estimated addition of 100,000 to
                                            300,000 permits. EPA will be conducting a feasibility study to determine the
                                           permitting and compliance needs for a stormwater system. The study will
                                            examine different system options such as using PCs or PC-based systems.

                                           3A general permit can be used when two or more facilities have the same effluent limits and monitoring
                                           requirements. Thus, one permit is written, but by appending the facility name and address, it can be
                                           issued to any number of permittees.

                                           Page 9                      GAO/IMTEC-92-58BR EPA's Permit Compliance System

                    Appendix I
                     Briefing Slides and Explanatory Narrative

    GAO PCS Use Varies

           PCS use varies among
           delegated authorities due to
           differences in:

           *enforcement and
            environmental priorities

           *user knowledge

           *resource levels

                     Page 10          GAO/IMTEC-92-58BR EPA's Permit Compliance System

    Appendix I
    Briefing Slides and Explanatory Narrative

    While PCs has many useful reporting capabilities, it is not consistently used
    by delegated authorities. A major reason for this is that EPA does not
    require detailed data on minor facilities to be maintained in the system. In
    addition, there are other causes for the inconsistency, including differences
    in priorities, and in levels of user knowledge and resources.

    Differences in enforcement and environmental priorities among states and
    regions allow some authorities to focus on specific geographic areas and
    enter detailed data on minor facilities. However, other states only monitor
    compliance of major facilities and do not have the time to focus on special
    geographic initiatives or enter detailed minor facility data. Many authorities
    we spoke with said that it would be beneficial to have data for all facilities
    in PCS. They agreed that without all the data, a complete picture of what is
    being discharged into our waters cannot be formed.

    Differences in PCS use also occur because of differing levels of user
    knowledge. PCS is difficult to use primarily because users must know some
    500 field name codes and acronyms in order to put data in the system
    accurately and to retrieve reports. For example, we observed one query
    that required 18 acronyms to identify facilities in the District of Columbia
    that had been in noncompliance since October 1991. Many officials noted
    that the codes and acronyms in the system are cumbersome and that PCS
    and the analytical application developed by Region 2 are not user-friendly.
    Although PCS is difficult to use, EPA recommends that users have some
    experience with the system prior to attending training. In addition, official
    training on the Region 2 application has not been given because it was
    regionally developed and is an option for PCS users. One user we spoke
    with was unaware that the Region 2 application exists. Another user told us
    that PCS documentation is inadequate.

    Resource levels also influence PCS use, especially at the state level. States
    that designate more resources to PCS activities typically spend more time
    using the system and enter detailed data for at least some minor facilities.
    States with limited resources generally devoted most of their time
    maintaining data for major facilities. Many authorities, including those with
    minimal resources, developed applications on personal computers to make
    their use of PCS easier and to manipulate PCS data further.

    Page 11                      GAOIIMTEC-92-58BR EPA's Permit Compliance System

                    Appendix I
                    Briefing Slides and Explanatory Narrative

    GAO   Optical Character Recognition
            (OCR)

            OCR study completed in 1991
            oonly one system evaluated
            *focus on automating data
             entry
           *no cost/benefit analysis

            EPA is moving away from OCR

            States free to use OCR

                    Page 12       GAO/IMTEC-92-58BR EPA's Permit Compliance System

                                          Appendix I
                                          Briefing Slides and Explanatory Narrative

    Otical Character                    You asked us to determine what data entry alternatives EPA has considered
               Charactr .for PCS. Optical character recognition (ocR) is one alternative the agency
    Recognition                         has studied. OCR allows printed data to be entered into a computer without
                                          manually keying in the data. Text and numeric information on paper is
                                          scanned electronically and converted to an electronic format by a
                                          computer program that interprets each character. Federal agencies that
                                          use OCR include the U.S. Postal Service and the Internal Revenue Service.

                                          While OCR primarily reduces data entry effort, data entry via OCR is not an
                                          error-free process. Read success rates per character can be high for typed
                                          and computer-printed information, but can vary with print quality,
                                          character alignment, and other factors. Read success rates are also
                                          significantly lower for hand-printed information. As a result, any data entry
                                          method involving OCR would still require human operators to review
                                          scanned information and correct errors.

                                          EPA has formally considered OCR as an option for entering DMRs into PCS for
                                          several years. The agency first examined the technology in 1985. However,
                                          the state-of-the-art technology at that time would have increased the size of
                                          the DMR from one to eight pages, so the method was not pursued. More
                                          recently, EPA identified a system produced by Recognition Equipment
                                          Incorporated (REI) that had potential to meet its data entry needs.

                                          EPA contracted with General Sciences Corporation (GSC) to perform a
                                          study of OCR. However, GSC's analysis, completed in 1991, was not
                                          comprehensive. First, the study only examined the REI system, instead of
                                          studying several systems and comparing their costs and capabilities.
                                          Second, the study focused only on automating the PCs data input process
                                          rather than attempting to improve the process by which facilities submit
                                          DMRS. Last, the study did not present any cost/benefit analysis of OCR
                                          comparing it to manual or other data entry methods and EPA cannot show
                                          that any such analysis was performed.

                                          EPA is moving away from OCR as a data entry alternative for PCs. However,
                                          state regulatory agencies are free to implement OCR systems if they choose
                                          to. For example, Ohio's Environmental Protection Agency currently uses
                                          an OCR system to input DMRS into its state permit system. Other authorities
                                          we spoke with are also considering OCR as a DMR data entry option.

                                          Page 13                     GAO/IMTEC-92-58BR EPA's Permit Compliance System

                     Appendix I
                     Briefing Slides and Explanatory Narrative

    GAo Electronic Data Interchange
           (EDI)

           EPA is studying EDI

           *1990: Agencywide policy
            statement

            1991:  EDI alternatives
            study for PCS

           *1 992: PCS pilot studies

                     Page 14          GAO/IMTEC-92-58BR EPA's Permit Compliance System

                                          Appendix I
                                          Briefing Slides and Explanatory Narrative

    Electronic Data                      In addition to OCR, EPA has considered electronic data interchange (EDI) as
                                          a data entry alternative for PCS. EDI is a means of transferring information
    Interchange                          from one computer to another without printing out and rekeying
                                          information. It involves the direct transfer of information between
                                          computers via telecommunication links or magnetic storage media.
                                          Potential benefits of EDI include increased data accuracy, reduced data
                                          entry costs, reduced mailing costs, more rapid availability of data, and
                                          reduced paperwork.

                                          Many businesses in the private sector have implemented EDI as a means of
                                          exchanging information, and the General Services Administration and
                                          Office of Management and Budget already give preferred status to EDI
                                          businesses. While the transition to EDI has not been painless for most
                                          businesses, the benefits are often significant. These businesses'
                                          experiences emphasize the need for proper planning, reasonable goals, and
                                          commitment to change throughout all levels of an organization.

                                          EPA is leaning toward EDI as a method of DMR data entry for PCS and other
                                          programs. In 1990 the agency issued an agencywide policy statement on
                                          standards that EPA programs must follow if they use EDI. In 1991 EPA OWEC
                                          completed a study of EDI as a data entry alternative for PCS. The OWEC
                                          report identified four components necessary in an EDI system: (1) a data
                                          format, (2) a data transmission method, (3) a data certification method,
                                          and (4) a security protocol. The report examined multiple alternatives for
                                          the first three components and provided cost and ease of use data for those
                                          alternatives. The report recommended a base scenario for implementing
                                          EDI, but did not contain a total cost figure for implementing that scenario.
                                          As a result, the specifics of the alternatives proposed by the base scenario
                                          and how they are implemented need further clarification before an
                                          accurate and comprehensive EDI cost figure can be developed. The report
                                          also contained no cost/benefit analysis that examined current data entry
                                          methods and costs.

                                          EPA has used EDI on a limited basis with at least one NPDES facility that
                                          wanted to submit its data electronically. EPA OWEC has also planned at least
                                          three NPDES pilot implementation studies to begin in Spring 1992.
                                          However, these pilot studies have not been designed to provide
                                          cost/benefit data on the use of EDI for PCS.

                                          Page 15                     GAO/IMTEC-92-58BR EPA's Permit Compliance System

                                   Appendix I
                                   Briefing Slides and Explanatory Narrative

    GAO Number of Facilities

              Tracked in PCS

                  NPOES Pmnlitted FKaclllts
                                                                             Mhir not Identified Ia In a� Istm

                                                             11       -    MajonM witM dmetied data In PCS

                                                                 A    33#% M- Mlnm it dlaad dta hIn PCS

                                                                             qrstems only

                  Mkh' FOaclHtls Only
                                                                         Minors with detailed data In state
                                                                         systems only

                                                    13%    <

                                                               50L% c+-  Minors not Identified as In a system
                                                37%

                                                                          Minors with detailed data In PCS

                                    Page 16                              GAO/IMTEC-92-58BR EPA's Permit Compliance System

                                           Appendix I
                                           Briefing Slides and Explanatory Narrative

    Number of Facilities                   At the time of our review, EPA OWEC officials said they could not provide us
                                           with the number of minor facilities with detailed data in PCS although they
    Tracked in PCS                         estimated it was around 25 percent. As a result, we conducted a survey of
                                           officials responsible for Pcs in all 10 EPA regional offices and 27 of the 39
                                           delegated state environmental offices. In providing comments, EPA
                                           informed us that they had recently determined the number of minor
                                           facilities with detailed data in PCS. Both sets of numbers are presented
                                           below (see tables 1.1 & 1.2).

                                           Based on our survey, we estimate that 44 percent of all major and minor
                                           NPDES dischargers have detailed data in Pcs. The survey showed that 37
                                           percent of the minor facilities had detailed data in PCS. The survey also
                                           confirmed that 90 percent of major facilities have detailed data in Pcs. We
                                           assumed that the remaining major facilities are also maintained in the
                                           system, since EPA requires this information.

    Table 1.1: Survey Results
                                                                                           Majors     Minors       Total
                                           Total number of NPDES permits as of April 1992  7,139      57,088     64,227
                                           Facilities covered in survey                    6,412      50,925     57,337
                                           Facilities having detailed data in PCS          7,139      21,187     28,326

    Table 1.2: EPA Reported Data
                                                                                           Majors     Minors       Total
                                           Facilities having detailed data in PCS          7,139      14,386     21,525

                                           Page 17                       GAO/IMTEC-92-58BR EPA's Permit Compliance System

                    Appendix I
                     Briefing Slides and Explanatory Narrative

    GAO Differences in PCS Data

           Major facilities typically have
           more data to enter than minors

           Quality control checks on
           minor data not as rigorous

           Some states enter detailed
           minor data into their own
           systems, but not into PCS

                     Page 18        GAO/IMTEC-92-58BR EPA's Permit Compliance System

                                             Appendix I
                                             Briefing Slides and Explanatory Narrative

    Differences lini PCS                   There are several differences between data for major and minor facilities in
                                             Pcs. Major facilities generally have more detailed data to enter than minor
    Data                                   facilities. On average, DMRS for major facilities cover more water discharge
                                             points and monitored substances than DMRS for minor facilities. This
                                             requires a greater proportion of resources be devoted to entering
                                             monitoring information for major facilities.

                                             Because EPA requires detailed data on major facilities to be entered into
                                             PCs, these data are more rigorously checked for quality.4 However, the
                                             extent of quality control checks of minor facility data varies among
                                             different authorities. Consequently, the quality of all the minor facility data
                                             currently in PCs is probably not equal to that of major facility data. Some
                                             state agencies do not differentiate between major and minor facilities and
                                             therefore quality check minor facility data to the same extent as major
                                             facility data. Some delegated authorities are in the process of reviewing
                                             their minor facility data.

                                             We found that several state agencies enter detailed major and minor facility
                                             data into their state systems, but only upload their major facility data.
                                             Minor facility data in state systems account for approximately 13 percent
                                             of all minor facilities. Before the minor facility data in state systems can be
                                             submitted to PCs, the data must be in a format compatible with PCs and
                                             should be complete.

                                             4In this context, quality control checks consist of making sure all necessary fields on DMRs submitted
                                             by facilities have been completed and that the values in the fields fall within logical ranges (e.g., pH
                                            values cannot physically exceed 14, so verification activities would identify a value of 15).

                                            Page 19                       GAO/IMTEC-92-SSBR EPA's Permit Compliance System

                    Appendix I
                    Briefing Slides and Explanatory Narrative

    GAO Estimated Data Entry
           Resource Requirements

           76 FTEs to enter detailed
           data for minor facilities

           109 additional FTEs at an
           estimated cost of $3.3 million
           needed to enter remaining
           facilities not in PCS

           Cautions and assumptions
           should be noted with the
           estimates

                     Page 20         GAO/IMTEC-92-58BR EPA's Permit Compliance System

                                           Appendix I
                                           Briefing Slides and Explanatory Narrative

    Estimated Data Entry                  On the basis of our discussions with state and regional officials, we
                                           estimate that it takes approximately 76 full-time equivalents (FTE) to enter
    Resource                              detailed data for minor facilities and 54 FrEs to enter detailed data for
    Requirements                          major facilities into PCs. We also estimate that 109 additional FTEs at a cost
                                           of $3.3 million would be needed to enter detailed data for the remaining
                                           minor facilities we identified as not in Pcs. (See app. II for the methodology
                                           used to determine these figures.)

                                           We believe our estimate of 109 additional FTEs to enter detailed data for
                                           the remaining minor facilities is on the high side for several reasons:

                                        * our estimate may significantly overemphasize the effort required to enter
                                           minor facility data,
                                        * data for other minor facilities that we did not identify may already be in
                                           PCS,
                                        � alternative data entry methods could reduce data entry resource
                                           requirements, and
                                        * the remaining facilities are all minors and should require less total data
                                           entry effort than those in PCs.

                                           Several cautions should be kept in mind when interpreting these estimated
                                           resource requirements. The quality of any estimate is only as good as the
                                           data used in developing the estimate. We were not able to independently
                                           verify the accuracy of the data because it would have required extensive
                                           examination of data at the facility level for thousands of permits. This was
                                           not possible given the time constraints of the job.

                                           These estimates also assume the quality of all minor facility data currently
                                           tracked is acceptable. This may not be the case due to differences in states'
                                           efforts to ensure the quality of minor facility data. If authorities are
                                           required to enter discharge monitoring data of additional minor facilities,
                                           EPA will need to assess the quality of minor facility data already in PCS and
                                           take action to "clean up" any data that are not acceptable or complete.
                                           This may result in a one-time FTE expenditure and additional FTEs on a
                                           regular basis for maintaining the information in the proper form.

                                           In comments provided after our audit work was completed, EPA reported
                                           that 14,386 minor facilities have detailed data in PCS. Using this number
                                           and our data entry resource requirement estimate, we calculated that
                                           approximately 130 additional FTEs would be needed to enter the 42,702
                                           minor facilities that EPA reported are not in PCs.

                                           Page 21                       GAO/IMTEC-92-68BR EPA's Permit Compliance System

                   Appendix I
                    Briefing Slides and Explanatory Narrative

    GAO PCS Operating Costs
          (dollars in thousands)

          Regional data entry and report
          generation:      $974

           Enhancements, operations,
          and maintenance: $842

          Training and headquarters
           user support:    $134

           State data entry
           and other:      $3,940

                    Page 22       GAO/IMTEC-92-58BR EPA's Permit Compliance System

                                            Appendix I
                                            Briefing Slides and Explanatory Narrative

    PCS  Operating Costs                    We estimate the total operating cost for PCs in fiscal year 1991 to be
                                            approximately $5.9 million. This includes EPA's reported expenditures for
                                            PCs data entry, maintenance, and other items as identified in table 1.3
                                            below. In addition, included as the first line item is the state data entry cost
                                            we estimated. The total cost will be higher, however, because states also
                                            allocate funds for operations and maintenance to support their use of PCs.
                                            We did not obtain estimates for this number.

    Table 1.3: PCS Operating Costs (dollars in
    thousands)                              Activity                                                           Expenditures
                                            State data entry                                                         $3,270
                                            Data entry including user support by EPA regions and report
                                            generation                                                                  974
                                            PCS system enhancements, operations, and maintenance                       842
                                            Training and headquarters user support                                      134
                                            EDI procedures and pilots                                                   180
                                            Latitude and longitude coordinates                                          150
                                            Data quality procedures                                                     150
                                            Permit-writers' software                                                     80
                                            Quality assurance guide                                                      60
                                            Public access                                                                50
                                            Total                                                                    $5,890

                                            EPA OWEC officials also reported a total of 15.6 FrEs that it provides to
                                            regional offices for data entry, user support, and report generation. These
                                            15.6 FTEs are included in the 130 FTEs that we estimated. Additionally,
                                            OWEC supports the activities in items 3 through 10 above with a total of 5.5
                                            FTEs, which are not a part of our estimated 130 FrEs.

                                            Page 23                        GAO/IMTEC-92-58BR EPA's Permit Compliance System

                      Appendix I
                      Briefinrg Slides and Explanatory Narrative

    GAO Conclusions

           PCS is not user-friendly

           EPA does not know cost
           savings of data entry
           alternatives

           PCS can provide a more
           comprehensive picture

                      Page 24        GAOIIMTEC-92-58BR EPA's Permnit Compliance System

                                 Appendix I
                                 Briefing Slides and Explanatory Narrative

    ~Co,\ncolu rsi o~nsPCS has many capabilities to monitor and track compliance of permitted
    Conclu.~sionsv    ~facilities. However, these capabilities are not fully utilized because the
                                system is difficult to use and all available data are not included in
                                nationwide compliance reports. As a result, some states have developed
                                their own systems and specialized applications.

                                EPA has assessed OCR and EDI as ways to improve DMR data entry into PCS.
                                However, its analyses did not include cost/benefit studies. It is necessary to
                                compare the costs of the existing method of entering data to the costs of an
                                alternative entry method in order to decide which is the best option.
                                Without knowing how much it currently costs to enter data, EPA cannot
                                determine whether the costs of implementing OCR or EDI would be
                                reasonable.

                                Although EPA does not require detailed permit and discharge data to be in
                                PCS for minor facilities, we found that detailed data for approximately 50
                                percent of all minor facilities are maintained in PCS and state systems.
                                Without discharge data on all NPDES facilities, a clear picture of what is
                                being put into our nation's waters cannot be formed. Approximately twice
                                the number of resources currently expended on PCS data entry would be
                                needed to enter detailed data for all facilities.

                                Page 25                     GAOIIMTEC-92-SSBR EPA's Permit Compliance System

    Appendix II

    Methodology for Estimating Data Entry

    Resource Requirements

                                           In order to estimate the data entry resources required to enter detailed data
                                           for the remaining minor facilities, we needed to estimate the data entry
                                           resources (represented in s'Es) allocated to the entry of detailed data for
                                           minor facilities. This is straightforward for majors because we had data
                                           from authorities that enter detailed major facility data only. Calculating this
                                           number for minors is difficult because no NPDES authority enters only
                                           minor facility data; all authorities enter either majors only or majors and
                                           some minors. Because detailed data are generally greater for majors than
                                           for minors, we hypothesized that less effort (FTEs) is needed to enter
                                           detailed minor data. We therefore needed to estimate the average effort
                                           devoted to the entry of detailed minor facility data.

                                           We used the following procedure to estimate the average effort used to
                                           enter detailed minor facility data and the total number of additional FrEs
                                           required to enter detailed data for the remaining minor facilities:

                                           (1) We estimated the average number of major facilities that could be
                                           entered by one rFTE. We did this by dividing the total number of major
                                           facilities (530) by the total number of FrEs (3.8) for offices that enter
                                           detailed data for major facilities only.5 This figure produced an estimate of
                                           141.3 major facilities per FTE.

                                           (530 majors)/(3.8 major F'ES) = 141.3 majors/rE

                                           (2) For offices that entered detailed data for both major and minor
                                           facilities, we estimated the amount of their total FrEs expended on entry of
                                           their major facility data. We determined the total number of affected major
                                           facilities (2,813) and divided this number by our estimate of the number of
                                           major facilities that could be entered by one FTE (141.3). This figure
                                           produced an estimate of 19.9 FTrEs.

                                           (2,813 majors)/(141.3 majors/FrE) = 19.9 major FrEs

                                           (3) On the basis of the resources allocated to data entry for major facilities,
                                           we estimated the remaining FrEs attributable to only entry of minor facility
                                           data for those authorities that enter both major and minor facility data. We
                                           subtracted the FTEs for entry of majors (19.9) from the total FTES
                                           expended by the affected authorities (77.6). The result was 57.7 FrEs.

                                           50f the 37 data points we obtained, we excluded 16 from use in estimating additional resource
                                           requirements because the data were not comparable due to special circumstances (i.e. nonmanual data
                                           entry methods, inexperienced data entry personnel, etc.).

                                           Page 26                       GAO/IMTEC-92-58BR EPA's Permit Compliance System

    Appendix H
    Methodology for Estimating Data Entry
    Resource Requirements

    (77.6 total FTEs) - (19.9 major FrEs) = 57.7 minor FTrES

    (4) We then estimated the average number of minor facilities that could be
    entered by one FTE. This was done by dividing the total number of minor
    facilities (19,032) by the estimated number of FrTEs allocated to the entry of
    minor facility data (57.7). This figure produced an estimate of 329.9 minor
    facilities per FrE.

    (19,032 minors)/(57.7 minor FrEs) = 329.9 minors/FTE

    (5) Using this average number of minor facilities per  rE (329.9) and the
    total number of uncovered minors (35,901), we estimated that those
    uncovered minor facilities could be entered with approximately 109
    additional FTEs. Assuming an average cost of $30,000 per FrE, we
    calculated the total cost for the additional FrEs to be approximately $3.3
    million.

    (35,901 minors)/(329.9 minors/FrE) = 109 FTrEs

    (109 FrEs)*($30,000/FiTE) = $3.27 million

    (6) On the basis of our interviews with state and regional authorities, we
    calculated that they spend a total of approximately 130 FrEs entering
    detailed data for major and minor facilities. Using the effort levels from (1)
    and (4), we calculated the theoretical number of MrEs for both major and
    minor facilities.

    (6,412 majors)/(141.3 majors/vFrE) = 45.4 major MTES

    (21,187 minors)/(329.9 minors/FrE) = 64.2 minor FrEs

    This results in 110 total FrEs which is less than the 130 FrEs reported
    because of the excluded data points. Therefore, we calculated the relative
    effort of the 110 FrEs and applied it to the 130 FrEs reported. This yielded
    76 FrEs required to enter detailed data on minor facilities and 54 FrEs
    required to enter detailed data on major facilities.

    (45.4 major FTEs/110 FrEs)*(130 FTEs) = 54 FrEs

    Page 27                      GAO/IMTEC-92-58BR EPA's Permit Compliance System

    Appendix H
    Methodology for Estimating Data Entry
    Resource Requirements

    (64.2 minor FTEs/110 FTEs) * 130 FTEs = 76 FTEs

    (7) After EPA provided us with their estimate of the number of minor
    facilities currently not in PCs (42,702), we calculated how much effort
    would be required to enter the data into PCS. Based on our estimate of
    329.9 minors/FTE, we found that approximately 130 FT'Es would be
    required to cover this many minor facilities.

    (42,702 minors)/(329.9 minors/frE) = 130 minor FIs

    We believe our estimate of 109 additional FTES to enter detailed data for
    the remaining minor facilities is on the high side for several reasons:

    (1) Our estimate may significantly overemphasize the effort required to
    enter minor facility data. One state that enters detailed data for all its major
    and minor facilities found that the amount of detailed minor facility data
    was approximately 20 percent of the detailed data for major facilities. Our
    estimates were 141.3 majors/FrE and 329.9 minors/FrE, which indicate that
    detailed minor facility data require approximately 42 percent of the effort
    that major facilities require. We therefore assume that more effort is
    required to enter detailed minor facility data than a certain state authority
    assumes.

    (2) Other minors that we did not identify may already be in Pcs. States we
    did not speak with cover 6,163 minor facilities, so a maximum of 11
    percent of all NPDES facilities could have detailed data in pCS in addition to
    the 44 percent currently covered.

    (3) Alternative data entry methods could reduce data entry resource
    requirements. The TrEs reported are based on manual data entry for most
    offices. In addition, the projected FrEs required to cover all currently
    uncovered facilities are based strictly on the assumption that manual data
    entry will be used. As OCR, EDI, and other nonmanual data entry techniques
    are used, both the current and projected data entry resource requirements
    could decline. For example, Ohio implemented an OCR system and reduced
    its data entry requirements from 9 to 4 FTES. There will be an initial
    investment cost for converting to OCR or EDI, but if either method requires
    less overall effort than the current manual data entry method, that cost
    would be recovered over a period of time.

    (4) The remaining facilities are all minors and should require less total data
    entry effort than those in PCs. Currently, 44 percent (almost half) of all

    Page 28                     GAO/IMTEC-92-58BR EPA's Permit Compliance System

    Appendix II
    Methodology for Estimating Data Entry
    Resource Requirements

    facilities are maintained with approximately 130 FrEs. A significant portion
    of those facilities tracked are major facilities, which require more effort to
    maintain. As a result, it is expected that additional FrEs required to cover
    the remaining 56 percent of the facilities should not exceed the current
    130 FTEs expended on data entry. The 109 additional FTEs calculated meet
    this condition.

    Page 29                       GAO/IMTEC-92-58BR EPA's Permit Compliance System

    Appendix III
    Major Contributors to This Report

            ~~Information      ~      David L. McClure, Project Director
                                    Patricia J. Macauley, Project Manager
    Management and                     Heather A. Wmand, Computer Scientist
    Technology Division,               Scott F. Robohn, Staff Evaluator
    Washington, D.C.

    (510812)                           Page 30                     GAO/IMTEC-92-58BR EPA's Permit Compliance System