Environmental Cleanup: DOD's Implementation of the Relative Risk Site
Evaluation Process (Letter Report, 10/07/98, GAO/NSIAD-99-25).
The Defense Department (DOD) uses a relative risk site evaluation
process to allocate about $2 billion annually to clean up contaminated
sites posing the greatest risk to human health and the environment. DOD
estimates that it will spend about $27 billion beginning in fiscal year
1998 to complete the cleanup of contaminated sites. This report
discusses (1) the extent to which DOD has issued uniform relative risk
site evaluation guidance and the application of the guidance by the
defense components and (2) whether the relative risk site evaluation
process provided data that enabled the defense components to categorize
sites and prioritize required work.
--------------------------- Indexing Terms -----------------------------
REPORTNUM: NSIAD-99-25
TITLE: Environmental Cleanup: DOD's Implementation of the Relative
Risk Site Evaluation Process
DATE: 10/07/98
SUBJECT: Environment evaluation
Pollution control
Waste disposal
Defense operations
Prioritizing
Military facilities
Reporting requirements
Environmental policies
Health hazards
******************************************************************
** This file contains an ASCII representation of the text of a **
** GAO report. Delineations within the text indicating chapter **
** titles, headings, and bullets are preserved. Major **
** divisions and subdivisions of the text, such as Chapters, **
** Sections, and Appendixes, are identified by double and **
** single lines. The numbers on the right end of these lines **
** indicate the position of each of the subsections in the **
** document outline. These numbers do NOT correspond with the **
** page numbers of the printed product. **
** **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced. Tables are included, but **
** may not resemble those in the printed version. **
** **
** Please see the PDF (Portable Document Format) file, when **
** available, for a complete electronic file of the printed **
** document's contents. **
** **
** A printed copy of this report may be obtained from the GAO **
** Document Distribution Center. For further details, please **
** send an e-mail message to: **
** **
** **
** **
** with the message 'info' in the body. **
******************************************************************
Cover
================================================================ COVER
Report to the Chairman and Ranking Minority Member, Committee on
Armed Services, U.S. Senate
October 1998
ENVIRONMENTAL CLEANUP - DOD'S
IMPLEMENTATION OF THE RELATIVE
RISK SITE EVALUATION PROCESS
GAO/NSIAD-99-25
Relative Risk Implementation
(709320)
Abbreviations
=============================================================== ABBREV
DOD - Department of Defense
EPA - Environmental Protection Agency
Letter
=============================================================== LETTER
B-279582
October 7, 1998
The Honorable Strom Thurmond
Chairman
The Honorable Carl Levin
Ranking Minority Member
Committee on Armed Services
United States Senate
The Department of Defense (DOD) uses a relative risk site evaluation
process as part of its decision criteria to allocate about $2 billion
annually to clean up contaminated sites that pose the greatest risk
to human health and the environment. DOD estimates that it will
spend about $27 billion to complete cleanup on contaminated sites
from fiscal year 1998 to completion.
The Senate Report on the National Defense Authorization Act for
Fiscal Year 1998 (Report 105-29, June 17, 1997) requires us to review
DOD's relative risk site evaluation process. In response to that
requirement, this report addresses (1) the extent to which DOD has
issued uniform relative risk site evaluation guidance and the
application of the guidance by the defense components and (2) whether
the relative risk site evaluation process provided data that enabled
the defense components to categorize sites and prioritize required
work. This report complements the information previously provided to
you on DOD's relative risk site evaluation process.\1
--------------------
\1 Environmental Protection: Information Used for Defense
Environmental Management (GAO/NSIAD-97-135, June 11, 1997) and
Environmental Cleanup: DOD's Relative Risk Process
(GAO/NSIAD-98-79R, Feb. 26, 1998).
BACKGROUND
------------------------------------------------------------ Letter :1
DOD adopted the relative risk site evaluation process in 1994 to
address inconsistencies in the evaluation methods it used to
prioritize contaminated sites. The process is intended to provide
defense components with a common methodology for assigning high,
medium, and low relative risk categorizations at each potentially
contaminated site on the basis of evaluations of water, soil, and
sediments for their contamination levels; the likelihood of
contaminant migration; and the presence of potential receptors such
as humans, plants, and animals. In addition, DOD's relative risk
site evaluation guidance requires that sites lacking sufficient
information for a relative risk site evaluation be given a "not
evaluated" designation, and provides that certain other sites do not
require evaluation.\2 Not evaluated sites, sites that do not require
evaluation, and sites with risk characterizations are reported in the
Defense Environmental Restoration Program annual reports and budget
justification exhibits provided to Congress.
Environmental remediation includes cleanup and other efforts aimed at
reducing the risk to an acceptable level. The Comprehensive
Environmental Response, Compensation, and Liability Act of 1980, as
amended, serves as the statutory basis for the environmental
remediation of contaminated sites. Under the act, the Environmental
Protection Agency (EPA) ranks sites for inclusion on the National
Priorities List on the basis of public health risks and other
factors.\3
DOD's relative risk site evaluation process provides a tool for
categorizing sites and sequencing priorities for cleanup on the basis
of relative risk. In the relative risk site evaluation process,
relative risk site evaluations are used only to screen and categorize
sites and are not substitutes for baseline risk assessments.\4
DOD has stated that the relative risk site evaluation process
provides a quantifiable basis for justifying requirements and
allocating funds. It has also stated that relative risk is only one
of the priority-setting factors managers consider for programming and
sequencing work at and across defense installations. Also important,
according to DOD, are such things as program and economic evaluations
and baseline risk assessments that managers may use to determine
which sites should be worked on first in light of available
resources.\5
Since DOD's implementation of the process in 1994, Congress has
raised questions about the relative risk site evaluation methodology.
In September 1996, concerned that a high proportion of cleanup sites
was justified as "high relative risk," the House Committee on
National Security requested that we review the relative risk
process.\6 In June 1997, the Senate Committee on Armed Services
questioned the credibility of DOD's risk-based approach and the
degree to which it facilitates the establishment of legitimate
funding priorities. Because of this concern, the Committee directed
DOD to define the elements of a relative risk site evaluation,
develop uniform guidance for evaluations and ranking, and ensure
consistent application of the guidance.
--------------------
\2 The guidance requires sites designated as not evaluated to be
programmed for additional study, a removal action if warranted, or
other appropriate response action, including deferral, before they
are evaluated. DOD does not require relative risk site evaluations
for sites that require only the removal of building demolition/debris
or contain abandoned ordnance, have final cleanup remedies in place,
or are designated as "response complete"--require no further cleanup
action. A risk-based priority system is being developed for
unexploded ordnance removal.
\3 The National Priorities List is the list of the nation's most
heavily contaminated sites.
\4 DOD's 1997 updated Relative Risk Site Evaluation Primer and
subsequent March 1998 guidance amplify this policy. Under the act,
DOD performs comprehensive baseline risk assessments to identify
potential exposure on receptors such as humans, and according to DOD,
these assessments are used to justify decisions to clean up.
\5 DOD's March 1998 "Management Guidance for the Defense
Environmental Restoration Program" details other management factors
that may be considered in sequencing restoration requirements. These
factors include, among other things, legal requirements; actual and
anticipated funding availability; and acceptability of the action to
regulators, tribes, and public stakeholders. DOD's primer describes
the relative risk site evaluation process, and the Quality Assurance
Plan defines objectives for relative risk site evaluation data as
established by the Environmental Security office.
\6 Environmental Protection: Information Used for Defense
Environmental Management (GAO/NSIAD-97-135, June 11, 1997).
RESULTS IN BRIEF
------------------------------------------------------------ Letter :2
DOD's Environmental Security office provided guidance in August 1997
on its relative risk site evaluation process to defense components by
updating its 1994 Relative Risk Site Evaluation Primer and publishing
a Quality Assurance Plan. The principal change in the updated primer
was to establish new, DOD-wide criteria for defining low-risk sites.
Defense components are to provide site evaluation results to DOD
semi-annually. The Environmental Security office issued the Quality
Assurance Plan to help ensure the integrity of data that is reported
to DOD managers and Congress. The plan requires defense components
to ensure that data are credible, auditable, accountable, and
consistent and to follow specific data verification and reporting
procedures. For example, the plan requires defense components to
review the process used to derive relative risk site evaluations for
accuracy and consistency, and report the results to appropriate
organizational elements.
DOD used the relative risk site evaluation process as one of several
key factors for making site funding priority decisions. Our analysis
of site evaluation data being reported by the defense components
showed that over 99 percent of the reported categorizations were
consistent with DOD's criteria. However, while consistent with the
criteria, we did note two categories that DOD identifies as high
relative risk and evaluation not required, account for two-thirds of
the sites and three-fourths of the estimated $15 billion completion
costs at the about 6,000 sites we analyzed. The high relative risk
category contains 1,622 sites with an estimated $9 billion cost to
complete, and the evaluation not required category contains 2,584
sites, with an estimated $2.2 billion cost to complete. Also, sites
in the evaluation not required category may have high to low or no
known levels of contamination, and costly long-term remedial actions
or no anticipated costs. With so many sites in these categories, the
designations are not as helpful as they could be to managers as one
of the key pieces of information they use in making cleanup priority
decisions among competing projects. They are also not as helpful as
they could be in assessing the status of sites not required to be
evaluated. We found that officials at the installations we visited
were differentiating further within the categories to aid in their
decision-making processes.
DOD ACTIONS TO PROVIDE UNIFORM
GUIDANCE AND APPLY THE RELATIVE
RISK SITE EVALUATION PROCESS
------------------------------------------------------------ Letter :3
DOD's Environmental Security office has taken steps to provide
uniform relative risk site evaluation guidance to defense components.
The office updated, in August 1997, its 1994 Relative Risk Site
Evaluation Primer to establish new, agencywide criteria for
categorizing low-risk sites. At that time, the office also published
a new Quality Assurance Plan to help ensure data integrity in the
application of the relative risk site evaluation process.\7 As a
result of the updates to the primer and the establishment of the
Quality Assurance Plan, defense components updated fiscal year 1996
and 1997 relative risk data. The updated data showed changes of
pathway or receptor data, or changes of 20 percent or greater in
reported contamination levels, for about 2,100 of the 6,000 sites
considered in our review. Our analyses of data submitted for DOD's
1997 annual report showed that over 99 percent of the defense
components' relative risk categorizations were consistent with the
Environmental Security office's most recent criteria. DOD corrected
the data for the other 1 percent (92 sites) where we found that
categorizations were not consistent with the reported data.
Officials from the Environmental Security office stated that they
included the revised data in the current fiscal year 1997 annual
report to Congress.\8
About 1,600 of the 6,000 sites changed relative risk rankings from
one year to the next because of changes in contamination data or site
status and/or because of the establishment of a new threshold below
which sites must be ranked as low relative risk. Table 1 shows the
aggregate changes from fiscal year 1996 to 1997 that affected 1,637
of the 6,015 sites that were common to both years.
Table 1
Reported Changes in Site Evaluation
Categories (fiscal years 1996-97)
Sept. Sept.
30, 30, 1996- 1996- 1996-
1996 1997\a 97 97 97
------ ------ ------ ------- -------
Number Number
of of Net Decreas Increas
Category sites sites change es es
---------------------------- ------ ------ ------ ------- -------
High 1,830 1,594 -236 443 207
Medium 626 674 48 218 266
Low 779 645 -134 369 235
Not evaluated 932 553 -379 506 127
Not required 1,848 2,549 701 101 802
======================================================================
Total 6,015 6,015 0\ 1,637 1,637
----------------------------------------------------------------------
\a At the time of our analysis, Environmental Security officials
stated that they considered the data as draft input to the annual
report to Congress on environmental restoration.
Note: This table does not include sites that the Environmental
Security office removed from the fiscal year 1996 data or added to
the fiscal year 1997 data. For the 97 high-cost installations in our
review, DOD provided data on 6,088 sites for fiscal year 1996 and
6,279 sites for fiscal
year 1997; 6,015 sites were common to both years.
--------------------
\7 DOD's primer describes the relative risk site evaluation process,
and the Quality Assurance Plan defines objectives for relative risk
site evaluation data as established by the Environmental Security
office.
\8 In discussions with Environmental Security officials, they stated
that the services and Environmental Security had discovered and
corrected many, if not all, of the errors we had identified during
DOD's quality assurance process.
NEW CRITERIA FOR LOW
RELATIVE RISK
---------------------------------------------------------- Letter :3.1
In August 1997 Environmental Security officials revised the Relative
Risk Site Evaluation Primer to provide more specific guidance for
categorizing sites. The only substantive change to the primer was
the addition of a requirement that media (water, soil, and sediment)
with a numeric value called a "contaminant hazard factor" below 0.005
be assigned to the low relative risk category.\9 The Environmental
Security office made this change to ensure that the defense
components assign to the low relative risk category sites with
reliable analytical data that are within measurement ranges normally
found in noncontaminated areas surrounding a contamination site.\10
Although only five sites actually changed categories from fiscal year
1996 to 1997 as a result of the new criteria, media ranking changes
that occurred from one year to the next show that the site data
appeared to be consistent with DOD's overall criteria for assigning
relative risk categories.\11 For fiscal year 1997, DOD reported that
184 of 6,279 sites had contamination levels meeting the new criteria
as low relative risk. Table 2 shows the five sites that changed
categories.
Table 2
Sites Where the Reported Overall
Relative Risk Category Changed as a
Result of New DOD Criteria in 1997
Fiscal year Fiscal year
1996 1997
relative relative
risk risk
Site category category
------------------------------------------ ------------ ------------
Underground Storage Site 00009, Marine High Low
Corps Air Station, Cherry Point, North
Carolina
Site 000005, Marine Corps Air Station, El Medium Low
Toro, California
Site 000007, Naval Air Station, Moffett High Low
Field, California
Site 00043, Naval Air Station, Patuxent Medium Low
River, Maryland
Site 00048, Naval Coastal Station, Medium Low
Stockton, California
----------------------------------------------------------------------
Note: Our review of the reported data showed that the contamination
levels for each of the five sites either did not change significantly
from one year to the next or did not change at all.
--------------------
\9 The contaminant hazard factor is a ratio that compares
contamination levels with goals that DOD calls comparison values.
\10 Prior to the implementation of the revised primer, factors below
0.005 and within these measurement ranges could be categorized high
or medium. Officials believe that assigning such a limit will help
them to more accurately identify appropriate relative risk
categories.
\11 We analyzed data reported to the Environmental Security office by
the defense components after data were screened by Environmental
Security officials. We verified relative risk site evaluation
categories and supporting data only at the six locations we visited.
In addition, defense components dropped some sites from their
inventory, for example, Anniston Army Depot, one of the site
locations we visited. Based on additional data, managers deleted
this previously high risk site with an estimated cost to complete of
$118.5 million.
RELATIVE RISK
CATEGORIZATIONS CONSISTENT
WITH DOD CRITERIA
---------------------------------------------------------- Letter :3.2
Our review of data submitted for DOD's 1997 annual report showed that
over 99 percent of the defense components' updated relative risk
categorizations were consistent with the Environmental Security
office's most current criteria. Furthermore, before DOD published
its most recent annual report to Congress, DOD corrected data for the
1 percent (92 sites) where we found that categorizations were not
consistent with the reported data.
Specifically, our review of the reported data showed that 28 sites
had contamination levels below DOD's threshold for low relative risk
but were categorized as medium or high relative risk. We brought
these sites to the Environmental Security office's attention during
our review. Officials subsequently provided the final data for the
sites in question, which revised data for 25 of the 28 sites and
adjusted the relative risk evaluation categories for the remaining 3
sites. We also noted that another 67 sites had overall relative risk
categories and/or individual media rankings that did not appear to be
supported by the reported data. Of these, 61 had incorrect overall
relative risk categorizations, and 6 appeared to have correct overall
relative risk categorizations despite errors in supporting media
rankings.\12 Environmental Security officials subsequently provided
us with supplemental spreadsheets showing that the final data had
been corrected by either changing the media or overall relative risk
site evaluation categorizations.
--------------------
\12 Defense components assign a ranking of high, medium or low to
each affected media before assigning an overall relative risk
categorization. In some cases, the supporting media rankings were
inaccurate, but the overall relative risk categorization was correct.
USEFULNESS OF RELATIVE RISK
EVALUATION PROCESS FOR
CATEGORIZING SITES
------------------------------------------------------------ Letter :4
Our review of the reported fiscal year 1996 and 1997 relative risk
evaluation data for over 6,000 sites shows that, with some minor
exceptions, the relative risk site evaluation process enabled the
defense components to categorize sites. However, two categories
account for two-thirds of the sites and three-fourths of the
estimated completion cost: the high relative risk category (1,622
sites with $9 billion cost to complete) and evaluation not required
category (2,584 sites with an estimated $2.2 billion cost to
complete). Having such a large number of sites in these two
categories in comparison with the other categories, raises questions
as to how useful the categorizations are to managers in assessing
relative risks. We observed, however, that officials at the
installations we visited were attempting to further differentiate
within each of the two categories to aid in the decision-making
process.
WIDE RANGE OF
CHARACTERISTICS IN HIGH
RELATIVE RISK CATEGORY
---------------------------------------------------------- Letter :4.1
DOD's Relative Risk Site Evaluation Primer states that for all
environmental media, three factors (contaminant hazard factor,
migration pathway factor, and receptor factor) are used in
determining a site's overall relative risk evaluation categories of
high, medium, or low based on 27 possible different ways these
factors can be related to each other. The high relative risk
categorizations can be given based on a wide variety of
characteristics; 8 out of 27 possible different combinations of
contaminant levels, pathways, and receptors can be categorized as
high, and the level of contamination can range from minimal to
significant.\13 In fiscal year 1997, about 55 percent (1,622) of the
2,964 high, medium, and low relative risk sites fell in the high
relative risk category.\14
These sites have an estimated cleanup completion cost of over $9
billion of the reported $15 billion total to complete cleanup at
sites in all of the categories.
Specific characteristics must be present for a site to be categorized
high relative risk. Table 3 shows the eight combinations of the
three factors that permit categorizing sites as high relative risk.
Table 3
Combinations of Characteristics Required
for the High Relative Risk Category
Contamination\a Migration Pathway Receptor
---------------------- ---------------------- ----------------------
1. Significant Evident Identified
2. Significant Evident Potential
3. Significant Potential Identified
4. Significant Potential Potential
5. Moderate Evident Identified
6. Moderate Evident Potential
7. Moderate Potential Identified
8. Minimal Evident Identified
----------------------------------------------------------------------
\a Our report entitled Environmental Cleanup: DOD's Relative Risk
Process (GAO/NSIAD-98-79R, Feb. 26, 1998) describes DOD's process
for determining contamination levels.
Groupings of fiscal year 1997 high relative risk sites having similar
reported characteristics of contamination, pathway, and receptor
showed little change from our previously reported analyses. The
major difference between our June 1997 analysis of 266 selected high
relative risk sites and our current analysis of 1,622 fiscal year
1997 high relative risk sites was that the percentage of sites in the
significant category fell from 20 percent to 15 percent. Sites in
this category all had reported significant contamination, an evident
pathway, and an identified receptor. In commenting on a draft of
this report, DOD stated that these percentages indicate that the
combination of relative risk and other management factors is working
very well in sequencing requirements because the components are
focusing their efforts on the high relative risk sites, and, within
the high category, they are focusing on the sites with higher levels
of contamination that also have evident pathways and identified
receptors. Figure 1 shows that there has been little change in site
categorizations between fiscal year 1996 and 1997.
Figure 1: DOD Sites With High
Relative Risk
(See figure in printed
edition.)
Source: Our analysis of high relative risk sites based on data
provided by the Environmental Security office.
As shown in figure 1, we also reported in June 1997, that 54 percent
of the high relative risk sites had significant, moderate, or minimal
contamination with an evident pathway and identified receptor. DOD
criteria require the categorization of sites to be evaluated based on
available data. Forty-six percent of the high relative risk sites
were first reported as having contamination levels that fell in EPA's
range of acceptable risk, or had pathways or receptors that were
identified as potential although the available data were not
reported.\15 This latter category increased to 49 percent for fiscal
year 1997.
Finally, the high relative risk category, which is disproportionately
large compared with the medium and low categories, contains
contamination levels ranging from highly contaminated to minimally
contaminated. To make this large category more meaningful,
installation managers have ranked further within the category. They
told us they found the additional differentiation useful in making
management and assessment decisions.
--------------------
\13 DOD defines contaminant hazard factors greater than 100 as
"significant;" 2 to 100 as "moderate;" and less than 2 as "minimal."
Our analysis showed that for the fiscal year 1997 sites categorized
as high relative risk, the contaminant hazard factors for the four
highest groundwater contamination sites were more than 3 million
times greater than values for the four lowest sites.
\14 The 2,964 fiscal year 1997 relative risk site categorizations
were as follows: high = 1,622;
medium = 686; and low = 656.
\15 These include contamination levels with a DOD comparison value
below 1.0. In commenting on a draft of this report, DOD stated some,
but not all, of its comparison values are based on EPA's Preliminary
Remediation Goals. For carcinogens, DOD stated that its comparison
values are 100 times the corresponding remediation goal value.
RANGE OF CHARACTERISTICS IN
EVALUATION NOT REQUIRED
CATEGORY
---------------------------------------------------------- Letter :4.2
DOD's Primer also specifies the circumstances under which site
evaluations are not required and permits a range within the
evaluation not required category as to what can be included. The
category can include sites that range from little or no contamination
to sites with high levels of contamination. The sites may require
costly long-term cleanup actions or less costly monitoring, continued
expenditures under other programs, or no anticipated costs. DOD
officials stated the category is one of the ways DOD defines success
in the restoration program. They stated that they expect this
category to grow as final remedies are put in place and the only
funding requirements left, if any, are for remedial action operations
or long-term monitoring. The evaluation not required category
accounted for 41 percent of all five site evaluation categories for
fiscal year 1997.\16
Continued remediation can be costly, involving millions of dollars
over long periods of time. A recent Army study on the effectiveness
of soil remediation programs and groundwater treatment systems noted
that some of the systems "appear to have an indefinite operational
lifetime with no known or estimated target date for completion of
remediation."\17 Annual costs for the operation and maintenance of
Army remediation systems are approximately $60 million. One
remediation system at the Twin Cities Army Ammunition Plant,
Minnesota, is planned to operate through the year 2080 at a total
estimated cost of over $300 million.
Our analysis of fiscal year 1997 evaluation not required sites showed
that
-- At Bangor Naval Submarine Base, Washington, one of two sites
that fall under the evaluation not required category has
implemented a long-term "pump and treat" remedial action; the
other site contains high levels of contamination requiring
regulators to agree on future actions. The "pump and treat"
remedial action is part of an estimated $26.7 million
remediation effort that is not immediately evident in the
reported data. DOD's annual report to Congress shows one
evaluation not required site, but provides no data on continued
associated estimated costs.
-- At Camp LeJeune, North Carolina, a burn area site was found to
require no further action after a remedial investigation was
completed in 1996. Subsequently, the relative risk ranking was
changed from high to evaluation not required. The site had
previously been listed as high due to the presence of previously
detected contaminants, which were not present during the
remedial investigation. Additionally, within the underground
storage tank program, two sites had been ranked as low and
medium risk, respectively. After corrective action plans were
completed, Camp LeJeune officials discovered that existing
groundwater contamination at both sites contained chlorinated
solvents and should be handled under a separate program.
Therefore, the sites were administratively closed out under the
underground storage tank program (listed as response complete),
the cost to complete reduced to zero, and the relative risk
changed to evaluation not required. These sites were then
transferred to the installation restoration program to address
groundwater contamination with the relative risk listed as high
and medium, respectively. The combined cost to address
groundwater contamination is estimated at $13.3 million.
-- At Norton Air Force Base, California, one site has begun
remedial action that will include about 10 to 15 years of
long-term operations and monitoring at an estimated cost of $17
million. Another site plans to excavate about 5,000 tons of
soil. At the time of this report, officials had not yet
estimated completion costs.
--------------------
\16 The five categories include high, medium, low, not evaluated, and
evaluation not required. For fiscal year 1997, 2,584 sites were
categorized as evaluation not required. The evaluation not required
category also accounted for nearly 78 percent of those sites
categorized as either not evaluated or evaluation not required.
\17 Evaluation of the Effectiveness of Existing Soil Remediation
Programs and Groundwater Treatment Systems in the U.S. Army, U.S.
Army Science Board Infrastructure and Environment Issue Group (Jan.
1998).
REPORTING OF "EVALUATION NOT
REQUIRED" CATEGORY
---------------------------------------------------------- Letter :4.3
DOD's annual environmental restoration report to Congress and
attachments to the fiscal year 1998-99 budget submission include
status information on relative risk sites that have been evaluated.
But, for evaluation not required sites, the range of site
characteristics is not easily distinguishable: the documents do not
disclose associated ongoing or completion cost estimates, or which
sites have ongoing long-term monitoring or operations.\18 For
example, the latest report to Congress for fiscal year 1997
separately identifies 17,124 sites considered to be evaluation not
required, of which over 15,000 are response complete.\19 But the
report also separately identifies 3,177 sites that are in or planned
for long-term operations without indicating how this information
applies to the evaluation not required sites. DOD reported that such
sites cost $172 million in planned execution for fiscal year 1998.
We believe that such an investment warrants identifying which
evaluation not required sites are affected, and at what cost.
Appendix I illustrates the range of site characteristics on
evaluation not required sites for three installations we visited.
--------------------
\18 Also, the annual report differentiates evaluation not required
sites "in progress" from sites with response complete.
\19 As reported in "DOD Summary Status" table B-6, for DOD
installations and formerly used defense sites.
CONCLUSION
------------------------------------------------------------ Letter :5
DOD has taken actions that, if implemented properly, should improve
the accuracy and consistency of information resulting from relative
risk site evaluations. DOD is updating data in accordance with the
Relative Risk Site Evaluation Primer and the Quality Assurance Plan.
However, the current site categorizations are not as helpful as they
could be to managers in making priority decisions among competing
projects, or evident to Congress in the annual reports. The criteria
used for determining whether a site is high relative risk are overly
broad and the evaluation not required category contains sites with a
range of characteristics that isn't easily evident in reviewing the
annual environmental restoration report.
As the process works now, for example, a site with potential risk to
humans can receive the same priority as a site with actual risk to
humans. Similarly, an evaluation not required site with no
contamination or remediation costs is grouped for reporting with
contaminated sites with years of operational costs to remediate.
Given that DOD uses these categories to define success in its
restoration program, it would seem appropriate to separately
identify, for example, the highest relative risk sites in the annual
report, and to identify whether the "evaluation not required" sites
have long-term monitoring or operations, along with their associated
costs.
RECOMMENDATION
------------------------------------------------------------ Letter :6
To assist managers in making priority decisions in the relative risk
site evaluation process, we recommend the Secretary of Defense direct
the Deputy Under Secretary of Defense for Environmental Security
and/or service components to provide more specific categories that
aid in priority setting and in the accurate reporting of the status
of sites.
AGENCY COMMENTS AND OUR
EVALUATION
------------------------------------------------------------ Letter :7
In commenting on a draft of this report, DOD stated that it does not
concur with our recommendation. It stated that the relative risk
tool was never intended to differentiate precisely among sites or
risk levels and that it is used in conjunction with a number of other
factors to determine the priority of a site for funding.
Consequently, it does not believe our recommendation would
necessarily help managers make better decisions. DOD stated that
flexibility in decision making about project sequencing may be lost
if managers are required to address the top category of high relative
risk sites first. Further, it stated that subdividing the evaluation
not required category would also not improve management or oversight
for sites in this group that have final remedies in place or have
completed response actions. DOD stated that restoration activities
at these sites are limited to remedial action operations and
long-term monitoring, which it treats as non-discretionary cost
commitments, not subject to prioritization. DOD stated that
regulatory agencies and the public would be incensed if it delayed
action at remedial sites in the evaluation not required category
based on a presumed priority, rather than fulfill its commitments.
Finally, DOD stated that implementing our recommendation would not
benefit DOD's program but would actually be detrimental because it
would require a reclassification of existing sites causing
restructuring and expansion of programming and budgeting data.
We continue to believe that implementing our recommendation would be
a useful aid in program management and congressional oversight. As
we pointed out in our report, officials at installations we visited
are already differentiating further within the categories to aid them
in their decision-making process. We added information to our report
to make it clear that relative risk is only one of a number of
available factors. However, the remainder of DOD's position does not
accurately characterize our recommendation. The intent of our
recommendation is to provide more specific relative risk categories,
and would not limit flexibility, since neither the current system nor
our recommendation requires sites to be funded strictly in relative
risk order. Also, our recommendation does not suggest the
interruption or delaying of remedial actions. Such actions, if
taken, would be a natural part of the decision-making process.
Finally, we recognize that the recommendation could result in sites
being reclassified, but it would not necessitate delays.
Reclassification is a normal part of the relative risk site
evaluation process and is provided for in DOD program guidance. For
example, as noted in our report, 1,637 of 6,015 sites changed
categories from 1996 to 1997.
DOD also suggested technical comments which we incorporated where
appropriate. DOD's comments and our detailed evaluation are included
in appendix II.
SCOPE AND METHODOLOGY
------------------------------------------------------------ Letter :8
To determine the extent to which DOD has issued uniform relative risk
site evaluation guidance to defense components, we interviewed and
reviewed policy and procedural documents from officials at DOD's
Office of the Deputy Under Secretary of Defense for Environmental
Security; the Army, Navy, Air Force, and Defense Logistics Agency
headquarters; and selected defense component field offices. To
address how defense installations applied the relative risk site
evaluation process, we visited and obtained information from
officials at DOD's Office of the Deputy Under Secretary of Defense
for Environmental Security and selected defense component field
offices. We also requested and analyzed fiscal year 1996-97 data on
over 6,000 sites for 97 high-cost installations identified in our
June 1997 report. The high-cost installations identified in our June
1997 report were reported in DOD's fiscal year 1995 annual report to
Congress. Each installation accounted for more than $20 million of
planned funding during fiscal years 1996-98 or more than $100 million
of projected costs from 1996 to completion.
To determine whether the relative risk site evaluation process
provided data that enabled the defense components to categorize sites
and prioritize work, we analyzed the fiscal year 1996-97 data on the
over 6,000 sites for the 97 high-cost installations identified in our
June 1997 report. DOD extracted the data, which are input from the
defense components, from the environmental Resource Management
Information System database. The data were provided to us
incrementally from November 1997 to January 1998. We relied on the
accuracy of DOD and service data in conducting our analysis and
selectively verified contamination data for specific projects but did
not verify overall database accuracy. We did not assess installation
judgments regarding pathways and receptors of individual sites. We
visited and/or obtained information on the relative risk site
evaluation process at the following military installations, which
included installations in addition to the 97 high-cost installations
just identified:
Army
Anniston Army Depot, Alabama
Navy
Engineering Field Activity, Northwest, Naval Facilities Engineering
Command, Bangor Submarine Base, Washington
U.S. Marine Corps, Camp LeJeune, North Carolina
Air Force
Kirtland Air Force Base, New Mexico
Norton Air Force Base, California
Tinker Air Force Base, Oklahoma
For this report, we analyzed reported data before (fiscal year 1996)
and after (fiscal year 1997) DOD's quality control updates for all of
the over 6,000 sites at the 97 high-cost installations considered in
our June 1997 report. The sites account for about $15 billion of
DOD's estimated total remaining cleanup costs of $27 billion.\20
We conducted our analysis from November 1997 to May 1998 in
accordance with generally accepted government auditing standards.
--------------------
\20 Cost estimates in DOD's database are reported for individual
sites at an installation. Although we could not verify the estimates
for each site, we noted that the total estimates for each
installation generally agreed with the totals reported in DOD's
fiscal year 1996 annual report to Congress.
---------------------------------------------------------- Letter :8.1
We are sending copies of this report to other appropriate
congressional committees; the Secretaries of Defense, the Army, the
Navy, and the Air Force; the Commandant, U.S. Marine Corps; and the
Directors, Office of Management and Budget and Defense Logistics
Agency. We will also make copies available to others on request.
David R. Warren, Director
Defense Management Issues
CHARACTERISTICS OF SITES
DESIGNATED EVALUATION NOT REQUIRED
AT THREE INSTALLATIONS
=========================================================== Appendix I
Relative risk category
------------------------------
Fiscal year Fiscal year
Site 1996 1997 Status
-------------- -------------- -------------- --------------------------------
Bangor 00204 H R Remedial actions implemented; in
long-term operation
Bangor 00025 H R Remedial actions implemented;
long-term monitoring
Norton WT007 R R Final decision document issued;
no further response action
required
Norton S1011 R R Final decision document issued;
no further response action
required
Norton TU015 R R Final decision document issued;
no further response action
required
Norton SS018 R R \Final decision document issued;
no further response action
required
Norton SP009 R R Final closure report issued
Norton WR020 R R Final basewide radionuclide
characterization report issued;
no further actions anticipated
Norton CG097 R R Remedial action implemented; 10-
15 years long-term operations
and monitoring anticipated
Norton DA096 R R Closure report issued; remedial
action complete; long-term
monitoring required
Norton DP003 R R No further response action
decision document signed
Norton DP004 R R No further response action
decision document signed
Norton ID022 R R Final closure report issued; no
removal action required
Norton AOC N R Remedial actions implemented;
closure report in regulatory
review
Norton SA017 N R Remedial actions implemented;
long-term monitoring
Norton DP012 N R Deed restriction placed on the
use of contaminated parcels
Norton LF010 N R Deed restriction placed on the
use of contaminated parcels
Norton SS008 H R Remedial actions implemented;
site closed
Norton SI001 H R Remedial actions implemented;
long-term monitoring
Norton AT005 H R Remedial actions implemented;
additional excavation planned
Norton DP014 H R Final closure report issued;
removal action complete
Norton WT013 H R Final closure report issued;
removal action complete
Norton SA019 L R Remedial actions implemented--
deed restriction
Norton TV021 N R Final closure report prepared;
additional contamination
detected
Norton S1016 N R Remedial actions implemented;
long-term monitoring
LeJeune UST M R Corrective action plans
000047 completed; continuing work under
installation restoration program
as site 00091
LeJeune UST L R Corrective action plans
000023 completed; CERCLA investigation
is underway under a separate
installation restoration
program; site name changed to
site 00073\a
LeJeune 00016 H R Remedial investigation completed
and record of decision signed;
no further action required
--------------------------------------------------------------------------------
Note: Categories are as reported by DOD's Environmental Security
office--H=high relative risk; L=low relative risk; R=not required;
and N=not evaluated.
\a LeJeune site 00073, with an estimated completion cost of $11.3
million, is also reported in DOD's fiscal year 1997 data as a high
relative risk site.
(See figure in printed edition.)Appendix II
COMMENTS FROM THE DEPARTMENT OF
DEFENSE
=========================================================== Appendix I
(See figure in printed edition.)
(See figure in printed edition.)
(See figure in printed edition.)
See comment 8.
See comment 8.
See comment 8.
(See figure in printed edition.)
See comment 9.
See comment 10.
See comment 11.
(See figure in printed edition.)
See comment 8.
See comment 12.
See comment 8.
See comment 8.
(See figure in printed edition.)
See comment 9.
See comment 13.
(See figure in printed edition.)
See comment 8.
See comment 15.
(See figure in printed edition.)
The following are GAO's comments on the Department of Defense's (DOD)
letter dated September 4, 1998.
GAO COMMENTS
1. DOD's high relative risk category is overly large in relation to
the medium and low categories, and thus does not provide three broad
categories. To make this more clear, we added an example in the
report that DOD's highest contaminant hazard factors were more than 3
million times greater than the lowest ones at sites within the high
relative risk category.
2. We agree that relative risk is used in conjunction with other
important factors, but DOD's conclusion does not follow. For those
site rankings where relative risk affected managers' decisions, the
additional category data would help the decision. We added to our
report an example of a previously high relative risk site that
managers deleted from inventory based on additional data.
3. DOD's comment that the two types of sites are non-discretionary
cost commitments is inconsistent with defense components' ongoing
reconsideration of remedial action operations, such as cited in our
report. We agree such sites are not subject to further
prioritization.
4. Our recommendation does not necessitate interruption of remedial
actions. Reclassification is a normal part of the relative risk site
evaluation process and is provided for in DOD's Relative Risk Site
Evaluation Primer. Also, as stated in our report, defense components
are themselves considering where such actions may be appropriate.
5. Although our recommendation could result in reclassified sites,
such actions are a normal part of DOD's process, as evidenced by the
over 1,600 category changes from 1996 to 1997.
6. Our recommendation does not prohibit managers from using other
management considerations in their decision process. It simply
suggests that more specificity within the two categories could be
more helpful to decisionmakers.
7. Both our draft and final reports recognize that relative risk is
one of several factors considered by decisionmakers. Also, as we
stated in addressing DOD comments on our June 1997 report, we
recognize that the relative risk site evaluation process is an
initial screening method that is only one of the factors considered
by decisionmakers. Further, DOD fully concurred with the description
of the relative risk site evaluation process in our report entitled
Environmental Cleanup: DOD's Relative Risk Process
(GAO/NSIAD-98-79R, Feb. 26, 1998). The recommendation, as written,
is sufficiently broad as to allow DOD flexibility in how best to
differentiate within the categories. We do not believe that better
defining and making categories more specific would limit flexibility
because neither the current system nor our recommendation requires
sites to be funded strictly in relative risk order. Thus, the
recommendation also would not delay remedial action, require
programming and budget data restructuring and additional data
collection, upset the existing system for identifying requirements,
or disrupt the existing program.
8. We revised our report to reflect DOD's suggested changes.
9. We addressed DOD's questions. The evaluation not required
category may have high to low or no known levels of contamination,
and costly long-term remedial actions or no anticipated costs.
Officials at installations we visited were differentiating further
within both the high relative risk and evaluation not required
categories to aid in their decision-making processes. We inserted
DOD's statement that it uses the evaluation not required category to
define success in the restoration program.
10. We revised our report to state that the updated data referred to
the about 6,000 sites considered in our current review. This is
simply a statement of fact describing the number of changes and their
extent, and its purpose is evident, as stated.
11. We revised our report to make it more clear that the $2.2
billion refers to sites in the evaluation not required category.
12. The statements simply reflect the results of our analysis. We
added language to reflect DOD's comment that it believes the
combination of relative risk and other management factors is working
well in sequencing requirements.
13. The information in our draft report was based on specific input
from Marine Corps officials. Although the information was accurate
as stated, we included DOD's suggested language.
14. We modified this section of the report to focus on reporting of
the evaluation not required category.
15. We revised our report to include specific reference to DOD' s
positive actions. Although our tests confirmed that components
followed processes for contamination levels, we could not state that
100 percent of all data met requirements.
MAJOR CONTRIBUTORS TO THIS REPORT
========================================================= Appendix III
NATIONAL SECURITY AND
INTERNATIONAL AFFAIRS DIVISION,
WASHINGTON, D.C.
Charles I. Patton, Jr.
Uldis Adamsons
Elizabeth Mead
DALLAS FIELD OFFICE
David Marks
KANSAS CITY FIELD OFFICE
Virgil Schroeder
Ben Douglas
Steve Pruitt
*** End of document. ***