[From the U.S. Government Printing Office, www.gpo.gov]











   *~~~~~~~~~~~~~~~~DM OIMk*G





























TD
4 20
1 992





                                           United States
   GAO                                 General Accounting Office       Property of csc Library
                                           Washington, D.C. 20548

                                           Information Management and
                                           Technology Division

                                           B-248721

                                           June 22, 1992
                                                                         U. D. F4FTFNT )F COMMERCE NOAA
                                           The Honorable Max Baucus COM   T      '   tCE $ CE N T ER
                                           Chairman, Subcommittee on 2234 SOiUTH nOBSON AVENUE
                                             Environmental Protection CHAL ESTON, SC  29405-2413
                                           Committee on Environment and
                                             Public Works
                                           United States Senate

                                           Dear Mr. Chairman:

                                           To assist you in your efforts writing proposed amendments to the Clean
                                           Water Act, we reviewed specific capabilities of the Environmental
                                           Protection Agency's (EPA) Permit Compliance System (PCS). PCS is the
                                           information system that supports the National Pollutant Discharge
                                           Elimination System (NPDES), a Clean Water Act program that issues
                                           permits and tracks facilities that discharge pollutants into our navigable
                                           waters.

                                           You expressed concern that PCS may not have adequate capabilities to
                                           maintain various data elements or perform analyses, and that it does not
                                           include detailed data' on all facilities, both major and minor,2 regulated
                                           under NPDES. Without these capabilities and data, a clear picture of what is
                                           being put into our waters cannot be formed.

                                          As agreed with your office, we determined (1) whether PCs had capabilities
                                           to maintain various data elements and perform analyses, (2) if EPA had
                                           assessed any alternative technologies to the existing data entry process,
                                           (3) the number of facilities in PCS with detailed data, particularly minors,
                                           (4) the estimated resource expenditures for minor facility data entry and
                                           update, and (5) the approximate operating costs of PCs.

                                          We provided the results of our work in a formal briefing to your office in
                                           May 1992, and agreed to prepare this report containing the charts used in
                                           the briefing (see app. I).




-    =,.                               l'Detailed data are all data that are required to monitor a facility's water discharge. Detailed data consist
ï¿½> . .-a                               of facility name and location, permit discharge limits, discharge monitoring report information, and
                                          related enforcement action and inspection data.
                                          2EPA classifies a facility as either a major or minor discharger depending on established criteria such as
                                          the flow of the discharge or the potential impact the facility will have on a water body or public health.


                                          Page 1                      GAO/IMTEC-92-58BR EPA's Permit Compliance System





                                      B-248721








                                      The NPDES program regulates facilities that discharge pollutants into our
Background
                                       navigable waters. Implementation authority lies with EPA's 10 regional
                                       offices and 39 delegated states. The regional offices also retain
                                       implementation authority for nondelegated states in their region.
                                       Delegated authorities issue permits, set effluent limits and monitoring
                                       requirements, track compliance, and carry out enforcement actions.

                                      According to EPA records, there are approximately 7,100 major and 57,000
                                       minor facilities regulated under NPDES nationwide. To monitor compliance,
                                       delegated authorities conduct inspections and screen facility-submitted
                                       discharge monitoring reports (DMR) that include pollutant discharge
                                       measurements. When a facility discharges above a permitted limit or does
                                       not report all required data, enforcement actions may follow.

                                       PCs was developed in 1974 to support the NPDES program. It resides on a
                                       mainframe computer at EPA's National Computer Center (Ncc) in North
                                       Carolina. Data are entered into PCS by states or EPA regional offices either
                                       by direct entry or batch updates. Delegated authorities are required to
                                       enter and maintain detailed data on all major facilities in Pcs. For minor
                                       facility dischargers, EPA only requires the facility name and address, permit
                                       event data, and inspection data to be maintained in the system, although
                                       some authorities choose to enter detailed data. PCs is used to monitor
                                       compliance for major facilities and can be used for minor facilities if
                                       detailed data are in the system. EPA uses PCS to produce reports of major
                                       facility discharge and compliance activities under the NPDES program.


                                       PCs has capabilities that enable EPA to monitor and track compliance of
                                       permitted facilities. However, many PCS users said that some of the
                                       capabilities are difficult to use because the system is not user-friendly. In
                                       addition, use of PCS varies among delegated authorities. Of the 64,227
                                       NPDES facilities, we found that all major facilities (7,139) and 21,187 minor
                                       facilities have detailed data in PCS. EPA is considering using new
                                       technologies to facilitate more efficient data entry. We estimate that
                                       currently 76 full-time equivalents (FTE) are required to enter detailed data
                                       for the 21,187 minor facilities in Pcs. We also determined the total
                                       operating cost for PCs, including state data entry costs, to be
                                       approximately $5.9 million in fiscal year 1991.







                                       Page 2                      GAO/IMTEC-92-58BR EPA's Permit Compliance System






B-248721









As agreed with your office, we did not obtain official comments on a draft
of this report. However, we discussed this information with the Acting
Chief of the Information Management Section within the Office of
Wastewater Enforcement and Compliance (OWEc). We have incorporated
her comments where appropriate.

We performed our work in accordance with generally accepted
government auditing standards between February 1992 and May 1992. We
interviewed system managers and coordinators responsible for maintaining
and overseeing use of PCs at headquarters, regional, and state levels. We
spoke with officials responsible for 27 of the 39 delegated state
environmental offices across the country and all IO EPA regional offices.
We also met with personnel in EPA Region 1 and state environmental
personnel in Rhode Island, Pennsylvania, and Ohio. We observed use of PCs
at two of these locations and EPA headquarters. In addition, we evaluated
system documents and output reports from the system.

We are providing copies of this report to the Administrator of EPA;
interested congressional commnittees; the Director, Office of Management
and Budget; and other interested parties. We will also make copies
available to others on request. Should you have any questions, please
contact me at (202) 512-6416. Major contributors are listed in appendix
Ill.

Sincerely yours,





  JayttaZ.Hecker
Director, Resources, Community, and Economic
Development Information Systems













Page 3                      GAO/IMTEC-92-58BR EPA's Permit Compliance System






Contents






Letter                                                                                                            1

Appendix I                                                                                                        6
Briefing Slides and                    PCS CapabilitiesAssessed                                                   7
    Explanatory Narrative             Potential Added Data to PCS                                                9
                                     PCS Use Varies                                                           11
                                     Optical Character Recognition                                            13
                                     Electronic Data Interchange                                              15
                                     Number of Facilities Tracked in PCS                                      17
                                     Differences in PCS Data                                                  19
                                     Estimated Data Entry Resource Requirements                               21
                                      PCS Operating Costs                                                      23
                                      Conclusions                                                              25

Appendix II                                                                                                      26
Methodology for
Estimating Data Entry
Resource Requirements

Appendix III                                                                                                     30
Major Contributors to
This Report

Tables                                 Table 1.1: Survey Results                                                 17
                                      Table 1.2: EPA Reported Data                                             17
                                      Table 1.3: PCS Operating Costs                                           23












                                      Page 4                        GAO/IMTEC-92-58BR EPA's Permit Compliance System






Contents







































Abbreviations

DMR      discharge monitoring report
EDI      electronic data interchange
EPA      Environmental Protection Agency
FTE      full-time equivalent
GAO      General Accounting Office
GSC      General Sciences Corporation
IMTEC    Information Management and Technology Division
NCC      National Computer Center
NPDES   National Pollutant Discharge Elimination System
OCR      optical character recognition
OWEC    Office of Wastewater Enforcement and Compliance
Pcs      Permit Compliance System
REI      Recognition Equipment Incorporated


Page 5                      GAO/flMTEC-92-58BR EPA's Permit Compliance System



Appendix I
Briefing Slides and Explanatory Narrative





      GAO   Permit Compliance System -
            Capabilities Assessed

            PCS can maintain:
            *previous permit data
            *terminated discharge data
            *detailed data
            *noncompliance status until
             final correction

            Related applications can:
            *track trends
            *calculate total loadings and
             basic statistics
            *produce graphics








                    Page 6        GAOJIMTEC-92-58BR EPA's Permit Compliance System






                                       Appendix I
                                       Briefing Slides and Explanatory Narrative








PCS  Capabilities                    You questioned whether PCS could perform various functions, such as
                                       maintaining: (1) individual entries for each NPDES permit issued, including
Assessed                              data concerning any permit renewals or permits previously issued for the
                                       same discharge; (2) data concerning any permitted facility that no longer
                                       discharges for a period of 5 years; (3) essential permit information,
                                       including the holder of the permit, the location of the discharge, the water
                                       body receiving the discharge, the nature of the discharge, and the limits
                                       placed on the discharge; (4) permit compliance data, including the date of
                                       any inspection, the date and nature of any violation, and the nature and
                                       schedule of any corrective action; and (5) a facility's noncompliance status
                                       until final corrective action is implemented. In addition, you wanted to
                                       know if PCs has the ability to do analyses, such as tracking compliance
                                       trends over time.

                                       Through observations of PCs and conversations with PCS users, we found
                                       that all the capabilities and analyses stated above do exist. However, since
                                       EPA does not require detailed data to be maintained on minor facilities,
                                       compliance and discharge data for all NPDES facilities are not available.
                                       Therefore, nationwide NPDES reports reflect major facility activities only.

                                       A separate application developed by EPA Region 2 uses PCS data to perform
                                       analyses such as calculating total discharge loadings on water bodies and
                                       tracking trends. This application resides on the mainframe computer at
                                       NCC and is available through the PCS menu. Some states have also
                                       developed additional applications to further enhance their ability to obtain
                                       and use information from PCs. Some of these enhancements include
                                       making PCs reports easier to read and analyzing data in ways PCS does not.




















                                      Page 7                       GAO/IMTEC-92-58BR EPA's Permit Compliance System




                Appendix I
                Briefing Slides and Explanatory Narrative






GAO Potential Added
       Data to PCS


       Sludge disposal data
       *about 16,000 current permits
        affected
       *about 3,000 new permits
        required

       Stormwater runoff data
       *about 100,000 to 300,000
        new permits may be required












                Page 8        GAO/IMTEC-92-58BR EPA's Permit Compliance System






                                       Appendix I
                                       Briefing Slides and Explanatory Narrative








Potential Added Data                 Because of provisions added to the Clean Water Act in 1987, EPA is
                                       considering whether to add stormwater runoff and sludge disposal data to
to PCS                               PCs. However, when we met with EPA, no decisions had been made on
                                        either issue.

                                        Currently there are about 16,000 NPDES permits that include sludge data.
                                       In addition, an enforcement official in OWEC told us she anticipates that
                                       about 3,000 new sludge permits will be issued as a result of legislative
                                       provisions. In 1991 EPA conducted a sludge feasibility study to determine
                                       what to do with sludge data. However, the study assumed that existing PCS
                                       hardware and software would be used. Several state environmental officials
                                       told us they do not want to add sludge disposal data to PCS because sludge
                                       is already regulated by state solid waste programs. If required to enter this
                                        data into PCS, some states may have to rekey data that are already entered
                                        into a state sludge system.

                                        Four states included in the sludge feasibility study wanted to use a separate
                                       sludge monitoring report rather than including sludge disposal data in
                                        DMRS. However, this issue was not addressed. According to the study, it is
                                       likely that sludge monitoring information will be reported through
                                       facilities' existing DMRS. In providing comments on the facts contained in
                                       this report, EPA informed us that an automated system for sludge tracking
                                       was being tested for implementation into PCS. It is scheduled to be available
                                       to users in June 1992, although specific reporting requirements have not
                                       been determined.

                                        Headquarter's consideration of building a separate system for data on
                                        stormwater also runs opposite to several state officials' opinion on what to
                                        do with the data. According to some state environmental officials,
                                       stormwater seems to be a natural addition to a data base, such as PCs, that
                                       tracks discharges into water. In addition, many of these officials agreed
                                       that stormwater may be amenable to use of a general permit,3 thus
                                        reducing the data entry impact of an estimated addition of 100,000 to
                                        300,000 permits. EPA will be conducting a feasibility study to determine the
                                       permitting and compliance needs for a stormwater system. The study will
                                        examine different system options such as using PCs or PC-based systems.





                                       3A general permit can be used when two or more facilities have the same effluent limits and monitoring
                                       requirements. Thus, one permit is written, but by appending the facility name and address, it can be
                                       issued to any number of permittees.


                                       Page 9                      GAO/IMTEC-92-58BR EPA's Permit Compliance System




                Appendix I
                 Briefing Slides and Explanatory Narrative






GAO PCS Use Varies



       PCS use varies among
       delegated authorities due to
       differences in:

       *enforcement and
        environmental priorities

       *user knowledge

       *resource levels











                 Page 10          GAO/IMTEC-92-58BR EPA's Permit Compliance System






Appendix I
Briefing Slides and Explanatory Narrative








While PCs has many useful reporting capabilities, it is not consistently used
by delegated authorities. A major reason for this is that EPA does not
require detailed data on minor facilities to be maintained in the system. In
addition, there are other causes for the inconsistency, including differences
in priorities, and in levels of user knowledge and resources.

Differences in enforcement and environmental priorities among states and
regions allow some authorities to focus on specific geographic areas and
enter detailed data on minor facilities. However, other states only monitor
compliance of major facilities and do not have the time to focus on special
geographic initiatives or enter detailed minor facility data. Many authorities
we spoke with said that it would be beneficial to have data for all facilities
in PCS. They agreed that without all the data, a complete picture of what is
being discharged into our waters cannot be formed.

Differences in PCS use also occur because of differing levels of user
knowledge. PCS is difficult to use primarily because users must know some
500 field name codes and acronyms in order to put data in the system
accurately and to retrieve reports. For example, we observed one query
that required 18 acronyms to identify facilities in the District of Columbia
that had been in noncompliance since October 1991. Many officials noted
that the codes and acronyms in the system are cumbersome and that PCS
and the analytical application developed by Region 2 are not user-friendly.
Although PCS is difficult to use, EPA recommends that users have some
experience with the system prior to attending training. In addition, official
training on the Region 2 application has not been given because it was
regionally developed and is an option for PCS users. One user we spoke
with was unaware that the Region 2 application exists. Another user told us
that PCS documentation is inadequate.

Resource levels also influence PCS use, especially at the state level. States
that designate more resources to PCS activities typically spend more time
using the system and enter detailed data for at least some minor facilities.
States with limited resources generally devoted most of their time
maintaining data for major facilities. Many authorities, including those with
minimal resources, developed applications on personal computers to make
their use of PCS easier and to manipulate PCS data further.









Page 11                      GAOIIMTEC-92-58BR EPA's Permit Compliance System




                Appendix I
                Briefing Slides and Explanatory Narrative






GAO   Optical Character Recognition
        (OCR)


        OCR study completed in 1991
        oonly one system evaluated
        *focus on automating data
         entry
       *no cost/benefit analysis

        EPA is moving away from OCR

        States free to use OCR












                Page 12       GAO/IMTEC-92-58BR EPA's Permit Compliance System






                                      Appendix I
                                      Briefing Slides and Explanatory Narrative








Otical Character                    You asked us to determine what data entry alternatives EPA has considered
           Charactr .for PCS. Optical character recognition (ocR) is one alternative the agency
Recognition                         has studied. OCR allows printed data to be entered into a computer without
                                      manually keying in the data. Text and numeric information on paper is
                                      scanned electronically and converted to an electronic format by a
                                      computer program that interprets each character. Federal agencies that
                                      use OCR include the U.S. Postal Service and the Internal Revenue Service.

                                      While OCR primarily reduces data entry effort, data entry via OCR is not an
                                      error-free process. Read success rates per character can be high for typed
                                      and computer-printed information, but can vary with print quality,
                                      character alignment, and other factors. Read success rates are also
                                      significantly lower for hand-printed information. As a result, any data entry
                                      method involving OCR would still require human operators to review
                                      scanned information and correct errors.

                                      EPA has formally considered OCR as an option for entering DMRs into PCS for
                                      several years. The agency first examined the technology in 1985. However,
                                      the state-of-the-art technology at that time would have increased the size of
                                      the DMR from one to eight pages, so the method was not pursued. More
                                      recently, EPA identified a system produced by Recognition Equipment
                                      Incorporated (REI) that had potential to meet its data entry needs.

                                      EPA contracted with General Sciences Corporation (GSC) to perform a
                                      study of OCR. However, GSC's analysis, completed in 1991, was not
                                      comprehensive. First, the study only examined the REI system, instead of
                                      studying several systems and comparing their costs and capabilities.
                                      Second, the study focused only on automating the PCs data input process
                                      rather than attempting to improve the process by which facilities submit
                                      DMRS. Last, the study did not present any cost/benefit analysis of OCR
                                      comparing it to manual or other data entry methods and EPA cannot show
                                      that any such analysis was performed.

                                      EPA is moving away from OCR as a data entry alternative for PCs. However,
                                      state regulatory agencies are free to implement OCR systems if they choose
                                      to. For example, Ohio's Environmental Protection Agency currently uses
                                      an OCR system to input DMRS into its state permit system. Other authorities
                                      we spoke with are also considering OCR as a DMR data entry option.








                                      Page 13                     GAO/IMTEC-92-58BR EPA's Permit Compliance System




                 Appendix I
                 Briefing Slides and Explanatory Narrative






GAo Electronic Data Interchange
       (EDI)


       EPA is studying EDI

       *1990: Agencywide policy
        statement

        1991:  EDI alternatives
        study for PCS

       *1 992: PCS pilot studies













                 Page 14          GAO/IMTEC-92-58BR EPA's Permit Compliance System






                                      Appendix I
                                      Briefing Slides and Explanatory Narrative








Electronic Data                      In addition to OCR, EPA has considered electronic data interchange (EDI) as
                                      a data entry alternative for PCS. EDI is a means of transferring information
Interchange                          from one computer to another without printing out and rekeying
                                      information. It involves the direct transfer of information between
                                      computers via telecommunication links or magnetic storage media.
                                      Potential benefits of EDI include increased data accuracy, reduced data
                                      entry costs, reduced mailing costs, more rapid availability of data, and
                                      reduced paperwork.

                                      Many businesses in the private sector have implemented EDI as a means of
                                      exchanging information, and the General Services Administration and
                                      Office of Management and Budget already give preferred status to EDI
                                      businesses. While the transition to EDI has not been painless for most
                                      businesses, the benefits are often significant. These businesses'
                                      experiences emphasize the need for proper planning, reasonable goals, and
                                      commitment to change throughout all levels of an organization.

                                      EPA is leaning toward EDI as a method of DMR data entry for PCS and other
                                      programs. In 1990 the agency issued an agencywide policy statement on
                                      standards that EPA programs must follow if they use EDI. In 1991 EPA OWEC
                                      completed a study of EDI as a data entry alternative for PCS. The OWEC
                                      report identified four components necessary in an EDI system: (1) a data
                                      format, (2) a data transmission method, (3) a data certification method,
                                      and (4) a security protocol. The report examined multiple alternatives for
                                      the first three components and provided cost and ease of use data for those
                                      alternatives. The report recommended a base scenario for implementing
                                      EDI, but did not contain a total cost figure for implementing that scenario.
                                      As a result, the specifics of the alternatives proposed by the base scenario
                                      and how they are implemented need further clarification before an
                                      accurate and comprehensive EDI cost figure can be developed. The report
                                      also contained no cost/benefit analysis that examined current data entry
                                      methods and costs.

                                      EPA has used EDI on a limited basis with at least one NPDES facility that
                                      wanted to submit its data electronically. EPA OWEC has also planned at least
                                      three NPDES pilot implementation studies to begin in Spring 1992.
                                      However, these pilot studies have not been designed to provide
                                      cost/benefit data on the use of EDI for PCS.








                                      Page 15                     GAO/IMTEC-92-58BR EPA's Permit Compliance System







                               Appendix I
                               Briefing Slides and Explanatory Narrative















GAO Number of Facilities

          Tracked in PCS


              NPOES Pmnlitted FKaclllts
                                                                         Mhir not Identified Ia In aï¿½ Istm

                                                         11       -    MajonM witM dmetied data In PCS



                                                             A    33#% M- Mlnm it dlaad dta hIn PCS



                                                                         qrstems only



              Mkh' FOaclHtls Only
                                                                     Minors with detailed data In state
                                                                     systems only


                                                13%    <

                                                           50L% c+-  Minors not Identified as In a system
                                            37%




                                                                      Minors with detailed data In PCS























                                Page 16                              GAO/IMTEC-92-58BR EPA's Permit Compliance System






                                       Appendix I
                                       Briefing Slides and Explanatory Narrative








Number of Facilities                   At the time of our review, EPA OWEC officials said they could not provide us
                                       with the number of minor facilities with detailed data in PCS although they
Tracked in PCS                         estimated it was around 25 percent. As a result, we conducted a survey of
                                       officials responsible for Pcs in all 10 EPA regional offices and 27 of the 39
                                       delegated state environmental offices. In providing comments, EPA
                                       informed us that they had recently determined the number of minor
                                       facilities with detailed data in PCS. Both sets of numbers are presented
                                       below (see tables 1.1 & 1.2).

                                       Based on our survey, we estimate that 44 percent of all major and minor
                                       NPDES dischargers have detailed data in Pcs. The survey showed that 37
                                       percent of the minor facilities had detailed data in PCS. The survey also
                                       confirmed that 90 percent of major facilities have detailed data in Pcs. We
                                       assumed that the remaining major facilities are also maintained in the
                                       system, since EPA requires this information.


Table 1.1: Survey Results
                                                                                       Majors     Minors       Total
                                       Total number of NPDES permits as of April 1992  7,139      57,088     64,227
                                       Facilities covered in survey                    6,412      50,925     57,337
                                       Facilities having detailed data in PCS          7,139      21,187     28,326



Table 1.2: EPA Reported Data
                                                                                       Majors     Minors       Total
                                       Facilities having detailed data in PCS          7,139      14,386     21,525





















                                       Page 17                       GAO/IMTEC-92-58BR EPA's Permit Compliance System




                Appendix I
                 Briefing Slides and Explanatory Narrative






GAO Differences in PCS Data



       Major facilities typically have
       more data to enter than minors

       Quality control checks on
       minor data not as rigorous

       Some states enter detailed
       minor data into their own
       systems, but not into PCS












                 Page 18        GAO/IMTEC-92-58BR EPA's Permit Compliance System






                                         Appendix I
                                         Briefing Slides and Explanatory Narrative








Differences lini PCS                   There are several differences between data for major and minor facilities in
                                         Pcs. Major facilities generally have more detailed data to enter than minor
Data                                   facilities. On average, DMRS for major facilities cover more water discharge
                                         points and monitored substances than DMRS for minor facilities. This
                                         requires a greater proportion of resources be devoted to entering
                                         monitoring information for major facilities.

                                         Because EPA requires detailed data on major facilities to be entered into
                                         PCs, these data are more rigorously checked for quality.4 However, the
                                         extent of quality control checks of minor facility data varies among
                                         different authorities. Consequently, the quality of all the minor facility data
                                         currently in PCs is probably not equal to that of major facility data. Some
                                         state agencies do not differentiate between major and minor facilities and
                                         therefore quality check minor facility data to the same extent as major
                                         facility data. Some delegated authorities are in the process of reviewing
                                         their minor facility data.

                                         We found that several state agencies enter detailed major and minor facility
                                         data into their state systems, but only upload their major facility data.
                                         Minor facility data in state systems account for approximately 13 percent
                                         of all minor facilities. Before the minor facility data in state systems can be
                                         submitted to PCs, the data must be in a format compatible with PCs and
                                         should be complete.




















                                         4In this context, quality control checks consist of making sure all necessary fields on DMRs submitted
                                         by facilities have been completed and that the values in the fields fall within logical ranges (e.g., pH
                                        values cannot physically exceed 14, so verification activities would identify a value of 15).


                                        Page 19                       GAO/IMTEC-92-SSBR EPA's Permit Compliance System




                Appendix I
                Briefing Slides and Explanatory Narrative






GAO Estimated Data Entry
       Resource Requirements


       76 FTEs to enter detailed
       data for minor facilities

       109 additional FTEs at an
       estimated cost of $3.3 million
       needed to enter remaining
       facilities not in PCS

       Cautions and assumptions
       should be noted with the
       estimates










                 Page 20         GAO/IMTEC-92-58BR EPA's Permit Compliance System






                                       Appendix I
                                       Briefing Slides and Explanatory Narrative








Estimated Data Entry                  On the basis of our discussions with state and regional officials, we
                                       estimate that it takes approximately 76 full-time equivalents (FTE) to enter
Resource                              detailed data for minor facilities and 54 FrEs to enter detailed data for
Requirements                          major facilities into PCs. We also estimate that 109 additional FTEs at a cost
                                       of $3.3 million would be needed to enter detailed data for the remaining
                                       minor facilities we identified as not in Pcs. (See app. II for the methodology
                                       used to determine these figures.)

                                       We believe our estimate of 109 additional FTEs to enter detailed data for
                                       the remaining minor facilities is on the high side for several reasons:

                                    * our estimate may significantly overemphasize the effort required to enter
                                       minor facility data,
                                    * data for other minor facilities that we did not identify may already be in
                                       PCS,
                                    ï¿½ alternative data entry methods could reduce data entry resource
                                       requirements, and
                                    * the remaining facilities are all minors and should require less total data
                                       entry effort than those in PCs.

                                       Several cautions should be kept in mind when interpreting these estimated
                                       resource requirements. The quality of any estimate is only as good as the
                                       data used in developing the estimate. We were not able to independently
                                       verify the accuracy of the data because it would have required extensive
                                       examination of data at the facility level for thousands of permits. This was
                                       not possible given the time constraints of the job.

                                       These estimates also assume the quality of all minor facility data currently
                                       tracked is acceptable. This may not be the case due to differences in states'
                                       efforts to ensure the quality of minor facility data. If authorities are
                                       required to enter discharge monitoring data of additional minor facilities,
                                       EPA will need to assess the quality of minor facility data already in PCS and
                                       take action to "clean up" any data that are not acceptable or complete.
                                       This may result in a one-time FTE expenditure and additional FTEs on a
                                       regular basis for maintaining the information in the proper form.

                                       In comments provided after our audit work was completed, EPA reported
                                       that 14,386 minor facilities have detailed data in PCS. Using this number
                                       and our data entry resource requirement estimate, we calculated that
                                       approximately 130 additional FTEs would be needed to enter the 42,702
                                       minor facilities that EPA reported are not in PCs.




                                       Page 21                       GAO/IMTEC-92-68BR EPA's Permit Compliance System




               Appendix I
                Briefing Slides and Explanatory Narrative






GAO PCS Operating Costs
      (dollars in thousands)


      Regional data entry and report
      generation:      $974

       Enhancements, operations,
      and maintenance: $842

      Training and headquarters
       user support:    $134

       State data entry
       and other:      $3,940









                Page 22       GAO/IMTEC-92-58BR EPA's Permit Compliance System






                                        Appendix I
                                        Briefing Slides and Explanatory Narrative









PCS  Operating Costs                    We estimate the total operating cost for PCs in fiscal year 1991 to be
                                        approximately $5.9 million. This includes EPA's reported expenditures for
                                        PCs data entry, maintenance, and other items as identified in table 1.3
                                        below. In addition, included as the first line item is the state data entry cost
                                        we estimated. The total cost will be higher, however, because states also
                                        allocate funds for operations and maintenance to support their use of PCs.
                                        We did not obtain estimates for this number.


Table 1.3: PCS Operating Costs (dollars in
thousands)                              Activity                                                           Expenditures
                                        State data entry                                                         $3,270
                                        Data entry including user support by EPA regions and report
                                        generation                                                                  974
                                        PCS system enhancements, operations, and maintenance                       842
                                        Training and headquarters user support                                      134
                                        EDI procedures and pilots                                                   180
                                        Latitude and longitude coordinates                                          150
                                        Data quality procedures                                                     150
                                        Permit-writers' software                                                     80
                                        Quality assurance guide                                                      60
                                        Public access                                                                50
                                        Total                                                                    $5,890


                                        EPA OWEC officials also reported a total of 15.6 FrEs that it provides to
                                        regional offices for data entry, user support, and report generation. These
                                        15.6 FTEs are included in the 130 FTEs that we estimated. Additionally,
                                        OWEC supports the activities in items 3 through 10 above with a total of 5.5
                                        FTEs, which are not a part of our estimated 130 FrEs.



















                                        Page 23                        GAO/IMTEC-92-58BR EPA's Permit Compliance System




                  Appendix I
                  Briefinrg Slides and Explanatory Narrative







GAO Conclusions



       PCS is not user-friendly

       EPA does not know cost
       savings of data entry
       alternatives

       PCS can provide a more
       comprehensive picture













                  Page 24        GAOIIMTEC-92-58BR EPA's Permnit Compliance System






                             Appendix I
                             Briefing Slides and Explanatory Narrative








~Co,\ncolu rsi o~nsPCS has many capabilities to monitor and track compliance of permitted
Conclu.~sionsv    ~facilities. However, these capabilities are not fully utilized because the
                            system is difficult to use and all available data are not included in
                            nationwide compliance reports. As a result, some states have developed
                            their own systems and specialized applications.

                            EPA has assessed OCR and EDI as ways to improve DMR data entry into PCS.
                            However, its analyses did not include cost/benefit studies. It is necessary to
                            compare the costs of the existing method of entering data to the costs of an
                            alternative entry method in order to decide which is the best option.
                            Without knowing how much it currently costs to enter data, EPA cannot
                            determine whether the costs of implementing OCR or EDI would be
                            reasonable.

                            Although EPA does not require detailed permit and discharge data to be in
                            PCS for minor facilities, we found that detailed data for approximately 50
                            percent of all minor facilities are maintained in PCS and state systems.
                            Without discharge data on all NPDES facilities, a clear picture of what is
                            being put into our nation's waters cannot be formed. Approximately twice
                            the number of resources currently expended on PCS data entry would be
                            needed to enter detailed data for all facilities.


























                            Page 25                     GAOIIMTEC-92-SSBR EPA's Permit Compliance System




Appendix II

Methodology for Estimating Data Entry

Resource Requirements





                                       In order to estimate the data entry resources required to enter detailed data
                                       for the remaining minor facilities, we needed to estimate the data entry
                                       resources (represented in s'Es) allocated to the entry of detailed data for
                                       minor facilities. This is straightforward for majors because we had data
                                       from authorities that enter detailed major facility data only. Calculating this
                                       number for minors is difficult because no NPDES authority enters only
                                       minor facility data; all authorities enter either majors only or majors and
                                       some minors. Because detailed data are generally greater for majors than
                                       for minors, we hypothesized that less effort (FTEs) is needed to enter
                                       detailed minor data. We therefore needed to estimate the average effort
                                       devoted to the entry of detailed minor facility data.

                                       We used the following procedure to estimate the average effort used to
                                       enter detailed minor facility data and the total number of additional FrEs
                                       required to enter detailed data for the remaining minor facilities:

                                       (1) We estimated the average number of major facilities that could be
                                       entered by one rFTE. We did this by dividing the total number of major
                                       facilities (530) by the total number of FrEs (3.8) for offices that enter
                                       detailed data for major facilities only.5 This figure produced an estimate of
                                       141.3 major facilities per FTE.

                                       (530 majors)/(3.8 major F'ES) = 141.3 majors/rE

                                       (2) For offices that entered detailed data for both major and minor
                                       facilities, we estimated the amount of their total FrEs expended on entry of
                                       their major facility data. We determined the total number of affected major
                                       facilities (2,813) and divided this number by our estimate of the number of
                                       major facilities that could be entered by one FTE (141.3). This figure
                                       produced an estimate of 19.9 FTrEs.

                                       (2,813 majors)/(141.3 majors/FrE) = 19.9 major FrEs

                                       (3) On the basis of the resources allocated to data entry for major facilities,
                                       we estimated the remaining FrEs attributable to only entry of minor facility
                                       data for those authorities that enter both major and minor facility data. We
                                       subtracted the FTEs for entry of majors (19.9) from the total FTES
                                       expended by the affected authorities (77.6). The result was 57.7 FrEs.


                                       50f the 37 data points we obtained, we excluded 16 from use in estimating additional resource
                                       requirements because the data were not comparable due to special circumstances (i.e. nonmanual data
                                       entry methods, inexperienced data entry personnel, etc.).


                                       Page 26                       GAO/IMTEC-92-58BR EPA's Permit Compliance System






Appendix H
Methodology for Estimating Data Entry
Resource Requirements







(77.6 total FTEs) - (19.9 major FrEs) = 57.7 minor FTrES

(4) We then estimated the average number of minor facilities that could be
entered by one FTE. This was done by dividing the total number of minor
facilities (19,032) by the estimated number of FrTEs allocated to the entry of
minor facility data (57.7). This figure produced an estimate of 329.9 minor
facilities per FrE.

(19,032 minors)/(57.7 minor FrEs) = 329.9 minors/FTE

(5) Using this average number of minor facilities per  rE (329.9) and the
total number of uncovered minors (35,901), we estimated that those
uncovered minor facilities could be entered with approximately 109
additional FTEs. Assuming an average cost of $30,000 per FrE, we
calculated the total cost for the additional FrEs to be approximately $3.3
million.

(35,901 minors)/(329.9 minors/FrE) = 109 FTrEs

(109 FrEs)*($30,000/FiTE) = $3.27 million

(6) On the basis of our interviews with state and regional authorities, we
calculated that they spend a total of approximately 130 FrEs entering
detailed data for major and minor facilities. Using the effort levels from (1)
and (4), we calculated the theoretical number of MrEs for both major and
minor facilities.

(6,412 majors)/(141.3 majors/vFrE) = 45.4 major MTES

(21,187 minors)/(329.9 minors/FrE) = 64.2 minor FrEs

This results in 110 total FrEs which is less than the 130 FrEs reported
because of the excluded data points. Therefore, we calculated the relative
effort of the 110 FrEs and applied it to the 130 FrEs reported. This yielded
76 FrEs required to enter detailed data on minor facilities and 54 FrEs
required to enter detailed data on major facilities.

(45.4 major FTEs/110 FrEs)*(130 FTEs) = 54 FrEs








Page 27                      GAO/IMTEC-92-58BR EPA's Permit Compliance System






Appendix H
Methodology for Estimating Data Entry
Resource Requirements







(64.2 minor FTEs/110 FTEs) * 130 FTEs = 76 FTEs

(7) After EPA provided us with their estimate of the number of minor
facilities currently not in PCs (42,702), we calculated how much effort
would be required to enter the data into PCS. Based on our estimate of
329.9 minors/FTE, we found that approximately 130 FT'Es would be
required to cover this many minor facilities.

(42,702 minors)/(329.9 minors/frE) = 130 minor FIs

We believe our estimate of 109 additional FTES to enter detailed data for
the remaining minor facilities is on the high side for several reasons:

(1) Our estimate may significantly overemphasize the effort required to
enter minor facility data. One state that enters detailed data for all its major
and minor facilities found that the amount of detailed minor facility data
was approximately 20 percent of the detailed data for major facilities. Our
estimates were 141.3 majors/FrE and 329.9 minors/FrE, which indicate that
detailed minor facility data require approximately 42 percent of the effort
that major facilities require. We therefore assume that more effort is
required to enter detailed minor facility data than a certain state authority
assumes.

(2) Other minors that we did not identify may already be in Pcs. States we
did not speak with cover 6,163 minor facilities, so a maximum of 11
percent of all NPDES facilities could have detailed data in pCS in addition to
the 44 percent currently covered.

(3) Alternative data entry methods could reduce data entry resource
requirements. The TrEs reported are based on manual data entry for most
offices. In addition, the projected FrEs required to cover all currently
uncovered facilities are based strictly on the assumption that manual data
entry will be used. As OCR, EDI, and other nonmanual data entry techniques
are used, both the current and projected data entry resource requirements
could decline. For example, Ohio implemented an OCR system and reduced
its data entry requirements from 9 to 4 FTES. There will be an initial
investment cost for converting to OCR or EDI, but if either method requires
less overall effort than the current manual data entry method, that cost
would be recovered over a period of time.

(4) The remaining facilities are all minors and should require less total data
entry effort than those in PCs. Currently, 44 percent (almost half) of all



Page 28                     GAO/IMTEC-92-58BR EPA's Permit Compliance System






Appendix II
Methodology for Estimating Data Entry
Resource Requirements








facilities are maintained with approximately 130 FrEs. A significant portion
of those facilities tracked are major facilities, which require more effort to
maintain. As a result, it is expected that additional FrEs required to cover
the remaining 56 percent of the facilities should not exceed the current
130 FTEs expended on data entry. The 109 additional FTEs calculated meet
this condition.















































Page 29                       GAO/IMTEC-92-58BR EPA's Permit Compliance System




Appendix III
Major Contributors to This Report





        ~~Information      ~      David L. McClure, Project Director
                                Patricia J. Macauley, Project Manager
Management and                     Heather A. Wmand, Computer Scientist
Technology Division,               Scott F. Robohn, Staff Evaluator
Washington, D.C.
































(510812)                           Page 30                     GAO/IMTEC-92-58BR EPA's Permit Compliance System