Air Pollution: Estimated Benefits and Costs of the Navajo Generating
Station's Emissions Limit (Letter Report, 01/27/98, GAO/RCED-98-28).

Pursuant to a congressional request, GAO reviewed the Environmental
Protection Agency's (EPA) decision to limit sulfur dioxide emissions
from the Navajo Generating Station, focusing on: (1) the effect on
emissions reductions and the associated costs that resulted from the
negotiated agreement used by EPA in making its decision compared to its
initial proposal; (2) the visibility improvements the agency estimated
would result from the emissions controls and the means by which these
improvements were determined; and (3) how contingent valuation was used
to estimate the monetary value of visibility improvements.

GAO noted that: (1) the negotiated agreement is expected to result in
greater emissions reductions at less cost than EPA had initially
proposed; (2) the agency initially proposed limiting sulfur dioxide
emissions at the Navajo Generating Station by approximately 70 percent
at an annual cost estimated between $91.9 million and $128.3 million;
(3) the negotiated agreement is expected to increase emissions
reductions to approximately 90 percent at an estimated annual cost of
approximately $89.6 million; (4) the lower costs resulted from several
factors, according to the plant operators; (5) according to a project
engineer for the Salt River Project, with its compliance determined on
an annual basis, the plant can operate its emission control equipment
most days at a rate greater than that needed to cut emissions by
approximately 90 percent to make up for those days on which emissions
are not controlled because the equipment is not operating; (6) also,
delaying the initial installation of the emission control equipment by
almost 3 years, from January 1995 to November 1997, allows the project
to be completed in a more cost-effective manner; (7) EPA estimated that
reducing the sulfur dioxide emissions at the Navajo Generating Station
by approximately 90 percent would improve winter seasonal average
visibility at the Grand Canyon approximately 7 percent--from about 124
miles to about 133 miles; (8) most of this improvement was estimated to
result from improvements during certain winter weather conditions; (9)
EPA initially estimated an approximately 14 percent improvement in the
winter seasonal average visibility primarily on the basis of a National
Park Service study of visibility in the vicinity of the Grand Canyon;
(10) EPA revised this estimate to approximately 7 percent after
considering the results of other analyses; (11) however, EPA noted that
its revised estimated may be understated because it did not include
visibility improvements: (a) below the rim of the Grand Canyon; (b) in
seasons other than winter at the Grand Canyon; and (c) year round at
other nearby national parks; (12) both EPA and the Navajo Generating
Station's owners used contingent valuation to estimate the monetary
value of visibility improvements; and (13) although relying on the same
methodology, the studies were different and yielded widely different
results.

--------------------------- Indexing Terms -----------------------------

 REPORTNUM:  RCED-98-28
     TITLE:  Air Pollution: Estimated Benefits and Costs of the Navajo 
             Generating Station's Emissions Limit
      DATE:  01/27/98
   SUBJECT:  Powerplants
             National parks
             Air pollution control
             Environmental law
             Toxic substances
             Projections
             Environmental monitoring
             Cost analysis
             Interagency relations
IDENTIFIER:  Grand Canyon National Park (AZ)
             Navajo Generating Station (AZ)
             Salt River Project (AZ)
             
******************************************************************
** This file contains an ASCII representation of the text of a  **
** GAO report.  Delineations within the text indicating chapter **
** titles, headings, and bullets are preserved.  Major          **
** divisions and subdivisions of the text, such as Chapters,    **
** Sections, and Appendixes, are identified by double and       **
** single lines.  The numbers on the right end of these lines   **
** indicate the position of each of the subsections in the      **
** document outline.  These numbers do NOT correspond with the  **
** page numbers of the printed product.                         **
**                                                              **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced.  Tables are included, but    **
** may not resemble those in the printed version.               **
**                                                              **
** Please see the PDF (Portable Document Format) file, when     **
** available, for a complete electronic file of the printed     **
** document's contents.                                         **
**                                                              **
** A printed copy of this report may be obtained from the GAO   **
** Document Distribution Center.  For further details, please   **
** send an e-mail message to:                                   **
**                                                              **
**                                            **
**                                                              **
** with the message 'info' in the body.                         **
******************************************************************


Cover
================================================================ COVER


Report to the Chairman, Subcommittee on Water and Power Resources,
Committee on Resources, House of Representatives

January 1998

AIR POLLUTION - ESTIMATED BENEFITS
AND COSTS OF THE NAVAJO GENERATING
STATION'S EMISSIONS LIMIT

GAO/RCED-98-28

Navajo Generating Station's Emissions Limit

(141018)


Abbreviations
=============================================================== ABBREV

  BART - best available retrofit technology
  CERCLA - Comprehensive Environmental Response, Compensation and
     Liability Act of 1980
  EPA - Environmental Protection Agency
  GAO - General Accounting Office
  NEPA - National Environmental Policy Act of 1969
  NGS - Navajo Generating Station
  NOAA - National Oceanic and Atmospheric Administration
  NPS - National Park Service
  WHITEX - Winter Haze Intensive Tracer Experiment
  WTP - willingness to pay

Letter
=============================================================== LETTER


B-278419

January 27, 1998

The Honorable John T.  Doolittle
Chairman, Subcommittee on Water
 and Power Resources
Committee on Resources
House of Representatives

Dear Mr.  Chairman: 

In 1989, the Environmental Protection Agency (EPA) made an initial
determination that it was reasonable to expect that sulfur dioxide
emissions from the Navajo Generating Station were contributing to
impaired visibility in the Grand Canyon National Park, most notably
during certain winter weather conditions.  Although emissions
controls for visibility impairment were not required when the
coal-fired power plant became fully operational in 1976, subsequent
amendments to the Clean Air Act require such controls if a source is
found to be causing or contributing to visibility impairment in
certain national parks and wilderness areas, including the Grand
Canyon National Park.  Accordingly, in February 1991, EPA initially
proposed a rule requiring the Navajo Generating Station to reduce its
sulfur dioxide emissions by approximately 70 percent.  Subsequently,
in October 1991, partially on the basis of a negotiated agreement
between the plant's owners and environmental groups, EPA issued a
final rule that required an emissions reduction of approximately 90
percent.  A project to install emission control equipment is under
way, and, according to the plant operators, the project is on
schedule.  The first of three emission control units has been placed
in service.  On October 10, 1997, the plant operators notified EPA
that the first emission control unit was operational and would comply
with the approximately 90 percent emissions reduction beginning
November 19, 1997, as required by the final rule.  The overall
project is scheduled to be completed by 1999. 

Concerned about the benefits and costs of installing sulfur dioxide
controls at the power plant, you asked us to review EPA's decision to
limit emissions.  Following discussions with your office, we agreed
to (1) determine the effect on emissions reductions and the
associated costs that resulted from the negotiated agreement used by
EPA in making its decision compared to its initial proposal, (2)
identify the visibility improvements the agency estimated would
result from the emissions controls and the means by which these
improvements were determined, and (3) determine how contingent
valuation was used to estimate the monetary value of visibility
improvements.  Contingent valuation is a methodology that relies on
surveys to elicit information from consumers to estimate how much
they would be willing to pay for something, including non-use
values.\1 While the contingent valuation methodology is
controversial, it is currently the only known approach for estimating
non-use values.  We are not taking a position on the appropriateness
of contingent valuation.  Appendix I is an overview of the contingent
valuation methodology. 


--------------------
\1 Non-use values are values that people may receive from knowing
that such things as unspoiled natural environments exist, even if
they do not consume or use these goods directly. 


   RESULTS IN BRIEF
------------------------------------------------------------ Letter :1

The negotiated agreement is expected to result in greater emissions
reductions at less cost than EPA had initially proposed.  The agency
initially proposed limiting sulfur dioxide emissions at the Navajo
Generating Station by approximately 70 percent (a reduction of about
50,000 tons of sulfur annually) at an annual cost estimated between
$91.9 million and $128.3 million.  The negotiated agreement is
expected to increase emissions reductions to approximately 90 percent
(about 64,000 tons of sulfur annually) at an estimated annual cost of
approximately $89.6 million.  The lower costs resulted from several
factors, according to the plant operators.  These factors include
measuring the power plant's compliance with emissions reductions
annually rather than monthly--thereby giving it more days over which
it can average the short-term increases in emissions that would occur
when emission control equipment is malfunctioning or being
repaired--and thus eliminating the need for expensive backup emission
control equipment. 

According to a project engineer for the Salt River Project, with its
compliance determined on an annual basis, the plant can operate its
emission control equipment most days at a rate greater than that
needed to cut emissions by approximately 90 percent to make up for
those days on which emissions are not controlled because the
equipment is not operating.  Also, delaying the initial installation
of the emission control equipment by almost 3 years, from January
1995 to November 1997, allows the project to be completed in a more
cost-effective manner.  For example, with more time, the plant
operators were able to identify and select the best technology at the
lowest cost and avoid the higher labor costs associated with an
accelerated construction schedule. 

EPA estimated that reducing the sulfur dioxide emissions at the
Navajo Generating Station by approximately 90 percent would improve
winter seasonal average visibility (expressed in terms of visual
range) at the Grand Canyon approximately 7 percent--from about 124
miles to about 133 miles.  Most of this improvement was estimated to
result from improvements during certain winter weather conditions
(high relative humidity and wind patterns that transport sulfur
dioxide emissions to the Grand Canyon).  It is during these
conditions when the power plant's emissions have the potential to
most severely affect visibility at the Grand Canyon.  These
conditions, and thus the power plant's effect on visibility, are
estimated to be most severe about two to three times each winter,
lasting about 5 to 7 days each time.  EPA initially estimated an
approximately 14 percent improvement in the winter seasonal average
visibility primarily on the basis of a National Park Service study of
visibility in the vicinity of the Grand Canyon.  EPA revised this
estimate to approximately 7 percent after considering the results of
other analyses.  However, EPA noted that its revised estimate may be
understated because it did not include visibility improvements (1)
below the rim of the Grand Canyon, (2) in seasons other than winter
at the Grand Canyon, and (3) year round at other nearby national
parks. 

Both EPA and the Navajo Generating Station's owners used contingent
valuation to estimate the monetary value of visibility improvements. 
Although relying on the same methodology, the studies were different
and yielded widely different results.  EPA's estimates were extracted
from related existing contingent valuation research because, in order
to comply with a court-ordered deadline for completing this
rulemaking, the agency did not have time to conduct original
research.  EPA's estimate of the annual nationwide monetary value of
the visibility improvements ranged from $90 million to $200 million. 
The plant owners, on the other hand, designed a pilot study to
specifically measure the monetary value of the visibility
improvements they expected from an emissions limit at the plant. 
Nonetheless, the owners did not complete their study, in part, due to
time constraints.  Instead, the owners used the pilot study results
to arrive at an estimate of $2.3 million for the annual nationwide
value of visibility improvements.\2 The studies' results were not
used as a basis for EPA's final rule that established an emissions
limit at the plant.  This is because, as a result of the negotiated
agreement, project costs dropped below the $100 million threshold
requiring such an estimate. 


--------------------
\2 Because the plant owners' estimate is based on the number of U.S. 
households in 1995 and EPA's estimate is based on the number of U.S. 
households in the year 2000, these values are not comparable. 


   BACKGROUND
------------------------------------------------------------ Letter :2

The Navajo Generating Station (NGS) is a 2,250-megawatt coal-fired
power plant located near Page, Arizona.  The plant, which became
fully operational in 1976, is located approximately 12 miles from the
northern boundary of the Grand Canyon National Park.  The Salt River
Project Agricultural Improvement and Power District (Salt River
Project) operates the plant and owns 21.7 percent.  Other owners and
their shares are the Department of the Interior's Bureau of
Reclamation, 24.3 percent;\3 Los Angeles Department of Water and
Power, 21.2 percent; Arizona Public Service Company, 14 percent;
Nevada Power Company, 11.3 percent; and Tucson Electric Power
Company, 7.5 percent. 

The 1977 amendments to the Clean Air Act set as a national goal "the
prevention of any future, and the remedying of any existing,
impairment of visibility" in certain parks and wilderness areas where
such impairment results from man-made air pollution.  The amendments
include a requirement that sources with emissions "which may
reasonably be anticipated to cause or contribute to any impairment of
visibility in any such area, shall procure, install, and operate" the
best available retrofit technology.  In determining the emissions
limit that reflects the best available technology, several factors
are to be taken into account, including the costs of compliance, the
energy impacts and impacts besides those on air quality, the
remaining life of the power plant, and the degree of improvement in
visibility that may reasonably be anticipated to result from the use
of the technology. 


--------------------
\3 The Bureau uses its share of power to pump water for the Central
Arizona Project.  Any share of the Bureau's power not used for the
project is marketed by Salt River Project through a contract with the
Department of Energy. 


   NEGOTIATED AGREEMENT PROVIDES
   FOR GREATER EMISSIONS
   REDUCTIONS AT LOWER COST THAN
   INITIALLY PROPOSED
------------------------------------------------------------ Letter :3

EPA's final rule to limit emissions from NGS relied on the details of
a negotiated agreement, between the power plant owners and
environmental groups, which EPA expects to result in greater
emissions reductions at a lower cost than EPA's initial proposal. 
The agreement increased the level of emissions reductions from EPA's
proposed 70 percent to 90 percent, with estimated annual costs
dropping from between $91.9 million and $128.3 million to $89.6
million.\4 The amount of emissions that would be removed annually is
expected to increase from about 50,000 tons of sulfur to about 64,000
tons.  The negotiations included officials representing the owners of
the plant, environmental groups, the state of Arizona, and EPA. 
These officials recommended the negotiated agreement to EPA as an
alternative to the agency's initial proposal to reduce the emissions
from the power plant. 


--------------------
\4 All emissions reductions proposed by EPA and discussed in this
report are approximate. 


      EPA REQUESTED COMMENTS ON
      ALTERNATIVES TO REDUCE
      EMISSIONS
---------------------------------------------------------- Letter :3.1

In February 1991, EPA solicited comments on a proposed rule laying
out a variety of strategies to reduce emissions from the power plant. 
EPA explained that, because of the uncertainty in determining the
improvement in visibility expected as a result of limiting the
emissions, it was considering and sought comments on four options to
limit these emissions--a 50-percent reduction, a 70-percent
reduction, a 90-percent reduction, and allowing the plant owners to
test alternative technologies and select one if it met minimum
emissions reductions at a set cost.  In addition to the four options,
EPA also solicited comments on any other appropriate alternative to
limit sulfur dioxide emissions, such as controls used only on a
seasonal basis.  EPA's proposed 70-percent emissions limit was the
same as the standard the agency used at the time for new facilities. 
EPA estimated that a 70-percent emissions reduction would eliminate
about 50,000 tons of sulfur from the power plant's emissions annually
and that the cost would range from $91.9 million to $128.3 million. 


      NEGOTIATED AGREEMENT COULD
      LEAD TO HIGHER EMISSIONS
      REDUCTIONS AT LESS COST
---------------------------------------------------------- Letter :3.2

Following EPA's initial proposal, representatives of the plant owners
and environmental groups (Grand Canyon Trust and Environmental
Defense Fund) met, at the recommendation of EPA, to discuss the most
cost-effective control option.  This led EPA, in early 1991, to
facilitate discussions between these representatives to find a
mutually acceptable control option.  According to EPA, its
participation included assisting in drafting documents to support a
potential agreement between the parties and providing technical
assistance.  These parties met repeatedly during a 3-month period to
discuss control options and their related costs in an attempt to
clarify all options and their costs.  As a result of these
discussions, the parties reached a negotiated agreement to, among
other things, reduce sulfur dioxide emissions from the power plant by
90 percent.  According to EPA, its final decision, issued in October
1991, substantially adopted the terms of this agreement. 

The agreement specified the time frames in which the emission control
technology should become operational and also the manner in which it
is to be operated.  The agreement specified that the three primary
pieces of equipment ("scrubber" modules) should become operational
over a 3-year period--the first unit by November 1997, the second by
November 1998, and the third by August 1999.  The emissions from all
three units will be subject to a 90-percent emissions reduction that
will be averaged on a 365-day plant operation basis to determine
compliance.\5 The agreement also specified that the maintenance
schedule for the plant would shift so that some planned maintenance
would occur in the winter, thereby shutting down some of the plant's
equipment and further reducing wintertime sulfur dioxide emissions. 

According to Salt River Project officials, two factors account for
the lower expected project costs.  First, the agreement allows the
power plant to determine its compliance with EPA's emissions limit on
an annual rather than a monthly rolling average basis as was
initially proposed.  Determining compliance on an annual basis is a
less stringent requirement (than determining compliance on a monthly
basis) because it gives the plant more days over which it can average
the short-term increases in emissions that would occur when one of
the scrubbers is malfunctioning or being repaired.  As such, the
plant operators can still comply with EPA's emissions limit without
installing the expensive backup equipment they would have to
otherwise operate on days when the primary equipment is not
operating.  According to a project engineer for the Salt River
Project, with its compliance determined on an annual basis, the plant
can operate its emission control equipment most days at a rate
greater than that needed to cut emissions by 90 percent to make up
for those days on which emissions are not controlled because the
equipment is not operating. 

Second, the agreement delays the initial installation of emission
control equipment by almost 3 years, from January 1995 to November
1997, which allows the plant operators to complete the project in a
more cost-effective manner.  According to the plant operators, the
additional time allows them to, among other things, better plan the
engineering.  That is, the operators have had more time to study
emission control technologies and select what they consider to be the
best technology at the lowest cost.  Salt River Project officials
also told us that staging construction over a longer period would
allow them to reduce labor costs as compared to those with an
accelerated construction schedule. 

Despite this almost 3-year delay, EPA concluded that the terms of the
final rule would result in greater visibility improvement than the
proposed rule.  In fact, EPA estimated that the emissions limit in
its final rule would reduce by two-thirds the amount of pollution
that would have been allowed under the proposed rule. 


--------------------
\5 Compliance with the annual rolling average is to be determined on
a daily basis by dividing the total sulfur dioxide emitted by the
total energy of the fuel consumed during the previous 365 days. 


   EPA ESTIMATED AN APPROXIMATELY
   7 PERCENT IMPROVEMENT IN
   WINTERTIME VISIBILITY
------------------------------------------------------------ Letter :4

EPA's estimate of an approximately 7 percent improvement in the
winter seasonal average visibility results primarily from significant
improvements expected to occur during certain winter weather
conditions.  Other less substantial improvements are expected on
other winter days.  EPA initially estimated an approximately 14
percent improvement primarily on the basis of a study by the National
Park Service, although the agency revised its estimate to
approximately 7 percent to reflect the results of other analyses and
studies.  EPA noted that its revised estimate may be understated
because it does not take into account other visibility improvements
(1) below the rim of the Grand Canyon, (2) in seasons other than
winter at the Grand Canyon, and (3) year round at other nearby
national parks.  Appendix II provides additional details on studies
of visibility impairment in and around the Grand Canyon. 


      VISIBILITY IMPROVEMENT
      ESTIMATES WERE BASED ON
      SEVERAL STUDIES
---------------------------------------------------------- Letter :4.1

EPA's initial estimate of an approximately 14 percent visibility
improvement relied primarily on data from a Park Service study--the
National Park Service Report on the Winter Haze Intensive Tracer
Experiment (WHITEX)--of visibility impairment in the vicinity of the
Grand Canyon.\6 The study was designed to evaluate a variety of
modeling approaches to attribute visibility impairment from a single
source--NGS.  Specifically, various models were to be evaluated for
their ability to link NGS' emissions to winter visibility impairment
at the Grand Canyon and other nearby national parks.  In conducting
this study, researchers released a traceable chemical from NGS'
smokestack and tracked its movement to monitoring stations in the
region, including at the Grand Canyon.  The study concluded that NGS
contributes approximately 40 percent on average to wintertime
visibility impairment in the canyon and approximately 60 to 70
percent during the winter weather conditions when NGS has the most
severe effect. 

After considering information received following its proposed rule,
EPA revised its estimate of the winter seasonal average visibility
improvement to approximately 7 percent.  This estimate translates
into an increase in the average visual range from about 124 miles to
about 133 miles.  In revising its estimate, EPA relied on the WHITEX
study, air monitoring information, and a visibility study conducted
by the plant owners.\7

EPA estimated that the largest improvements from reducing emissions
from NGS would occur during certain winter weather conditions.  These
conditions are, according to EPA officials, (1) high relative
humidity, which facilitates the conversion of the plant's gaseous
sulfur dioxide emissions to visibility-impairing sulfate particles,
and (2) wind patterns that transport the emissions to the Grand
Canyon.  EPA estimated that these conditions occur between 10 and 15
times per winter, lasting from 3 to 5 days each occurrence. 

However, a Park Service official who was a principal investigator on
the WHITEX study told us that the effect of emissions from NGS on
impaired visibility at the Grand Canyon during these episodes can be
mitigated by local weather patterns.  The official explained that,
due in part to local weather conditions, the most severe effects
occur approximately two to three times per winter, lasting from 5 to
7 days each time.  This official explained that visibility can be
impaired during these winter weather conditions because of both
naturally occurring impairment--mist, fog, clouds--and man-made
sources, primarily NGS.  However, the official noted that
photographic and air monitoring data show that the impairment from
man-made sources can continue for several days after the naturally
occurring conditions have dissipated.  In addition, the evidence also
indicates that impairment from man-made sources is perceptible even
on some days that include natural impairment. 

In addition to improvements during certain winter weather conditions,
EPA also estimated visibility improvements on other winter days. 
These estimated improvements were measured in terms of "changes in
contrast," which, like visual range, is another method of measuring
visibility improvements.  EPA defined "contrast" as the percentage
difference between the brightness of a scenic element and its
background.  Using this method, EPA estimated that reducing NGS'
sulfur dioxide emissions by 90 percent could result in at least a
"perceptible" change in visibility conditions (defined as a 4-percent
change in contrast) on approximately 100 days during the winter.\8
EPA later dropped these estimates due to an error in the
calculations.  The plant owners attempted to correct this error and
estimated 54 days of at least a perceptible change.  Later, using the
results of their own visibility study, the owners reduced this
estimate to 6 days. 

EPA also relied on other studies and analyses in calculating the
degree of visibility improvement that could result from reducing NGS'
sulfur dioxide emissions.  These studies included a review of the
WHITEX study by a committee established by the National Academy of
Sciences' National Research Council and a separate visibility study
conducted by the plant owners.  After reviewing the techniques and
data used in the WHITEX study, the committee concluded that, on some
days during the study period, NGS contributes significantly to
visibility impairment in the Grand Canyon.  However, the committee
also concluded that the WHITEX study was not sufficient to make a
quantitative determination of the exact fraction of visibility
impairment at the Grand Canyon that is attributable to NGS.  The
power plant owners' study found a lesser impact on visibility in the
canyon.  The study estimated that the average wintertime visual range
would improve by no more than 2 percent as a result of reducing NGS
sulfur dioxide emissions by 90 percent.  In reviewing this
information, EPA concluded that there was reasonable agreement
between the plant owners' study and the WHITEX study.  EPA noted that
the major difference is that the WHITEX study led to the conclusion
that peak impairment conditions occur more frequently and that
nonpeak impairment conditions are greater than zero more often than
found during the plant owners' study. 


--------------------
\6 National Park Service Report on the Winter Haze Intensive Tracer
Experiment (Dec.  4, 1989). 

\7 L.  Willard Richards, Charles L.  Blanchard, Donald L. 
Blumenthal, Navajo Generating Station Visibility Study--Executive
Summary--Draft Number 2, (Apr.  16, 1991).  Prepared for Prem
Bhardwaja, Salt River Project, Phoenix, Ariz. 

\8 EPA defined the winter period as the period between November 1 and
March 31. 


      EPA BELIEVES VISIBILITY
      BENEFITS MAY BE MORE THAN
      THE 7 PERCENT ESTIMATED
---------------------------------------------------------- Letter :4.2

EPA identified additional benefits from reducing sulfur dioxide
emissions by 90 percent that suggest there may be more than a
7-percent improvement in the winter seasonal average visibility. 
These benefits include a greater visibility improvement that would
occur below the rim of the Grand Canyon and improvements during
seasons other than winter at the Grand Canyon and year round at other
nearby national parks.  First, EPA's estimated 7-percent improvement
in the winter seasonal average visibility did not reflect the more
pronounced improvement expected below the rim of the canyon because
the air below the rim may be more affected by NGS' emissions.  The
National Research Council committee's review of the WHITEX study
noted that meteorological evidence, still photographs, and time-lapse
video suggested that sulfur concentrations (indicative of plant
emissions) in the canyon might have been considerably greater than
those that were observed at the monitoring station used during the
WHITEX study--a monitoring station located at the rim of the canyon. 
The Park Service subsequently established an air monitoring station
within the canyon, and, from its results, EPA found that visibility
impairment was worse in the canyon than was measured at the rim of
the canyon.  EPA said that it did not quantify the additional
visibility improvement expected below the rim of the canyon due to
the limited amount of data available and a limited understanding of
the air movements below the rim. 

Second, EPA's estimated 7-percent improvement in the winter seasonal
average visibility did not reflect benefits in seasons other than
winter at the Grand Canyon and throughout the year at other nearby
national parks.  EPA explained that, on the basis of information
received during its public comment period, emissions from NGS may
significantly impair visibility year round at the Grand Canyon as
well as at other national parks in the region.\9 For example, a study
prepared by the Grand Canyon Trust, which modeled emissions from NGS
over a 5-year period, indicated visibility impairment at the Grand
Canyon in seasons other than winter.  Furthermore, the study
suggested that the emissions could impair visibility in surrounding
national parks between 60 and 80 percent of the time year round.  EPA
said that the emissions controls, required by the final rule, would
significantly reduce if not eliminate NGS' contribution to visibility
impairment in nearby national parks. 


--------------------
\9 NGS is near several other national parks located on the Colorado
Plateau--Arches, Bryce Canyon, Canyonlands, Capitol Reef, Mesa Verde,
and Zion. 


   EPA AND THE OWNERS USED
   CONTINGENT VALUATION TO
   ESTIMATE THE MONETARY VALUE OF
   VISIBILITY BENEFITS
------------------------------------------------------------ Letter :5

Both EPA and the plant owners estimated, using contingent valuation,
the monetary value of visibility improvements from reducing sulfur
dioxide emissions from NGS.  EPA estimated annual nationwide values
ranging from $90 million to $200 million.  The plant owners estimated
a nationwide value of $2.3 million.  Although relying on the same
methodology, the studies were technically different.  EPA's estimate
was extracted from existing research because EPA was under a
court-ordered deadline to complete the rulemaking.  Therefore, it did
not have time to conduct original research to estimate the monetary
value of visibility improvements at the Grand Canyon National Park. 
Unlike EPA in its reliance on existing research, the owners
specifically designed contingent valuation research to estimate the
visibility improvements they expected from emissions controls at the
plant.  Nonetheless, the owners did not complete their study for
several reasons, including time constraints.  Instead, they used the
pilot study results to estimate an annual nationwide value of the
visibility improvements they expected to occur.  Neither studies'
results were used as a basis for EPA's final rule that established an
emissions limit because, as a result of the negotiated agreement,
project costs dropped below the $100 million threshold requiring such
an estimate. 


      EPA ESTIMATED THE MONETARY
      VALUE OF VISIBILITY BENEFITS
      BUT DID NOT USE THE
      ESTIMATES
---------------------------------------------------------- Letter :5.1

EPA set out to estimate the monetary value of visibility improvements
to comply with the terms of Executive Order 12291.  This order
provided that, to the extent permitted by law, agencies should not
take regulatory action unless the potential benefits to society
outweighed the potential costs to society.  The order required
agencies, including EPA, to prepare a regulatory impact analysis that
included a cost-benefit analysis.  Agencies were to do this for
proposed rules that, among other things, were likely to result in an
annual effect on the economy of at least $100 million.\10 In such
cases, an agency's analysis was required to describe the
benefits--expressed in monetary terms, if possible--as well as
potential costs.  If the analysis did not show that benefits exceeded
costs, the agency was to explain any legal reasons why the regulation
should still be promulgated. 

When EPA first proposed the rule requiring emissions controls at NGS,
it believed that the cost-benefit analysis was required, as the
annual cost was thought likely to exceed $100 million (estimates
ranged from $91.9 million to $128.3 million).  By the time EPA issued
its final rule, however, the estimated annual cost--as a result of
the negotiated agreement--had decreased to $89.6 million. 
Accordingly, the Office of Management and Budget exempted EPA from
the requirements for a regulatory impact analysis, including a
cost-benefit analysis. 

When EPA began the cost-benefit analysis, it was faced with
court-ordered deadlines to complete this rulemaking effort.  As a
result of the deadlines, EPA effectively had less than 6 months to
complete its analysis and did not have time to conduct original
research to estimate the monetary value of limiting the plant's
emissions.\11 Instead, EPA estimated the value of limiting these
emissions by extrapolating from the results of earlier contingent
valuation research that sought to value the benefit of reducing air
pollution at national parks across the country, including those in
the Southwest.\12 EPA in its proposed rule, estimated that the
monetary value of visibility improvements would range from $1.30 to
$2.50 annually per U.S.  household.\13 Later, to reflect the revised
estimate of visibility improvement from approximately 14 percent to
approximately 7 percent, EPA decreased its annual household value to
$0.75 to $1.75.  EPA estimated the monetary value, nationwide, would
range from $90 million to $200 million in the year 2000.\14

The owners also used contingent valuation to estimate the monetary
value of visibility improvements in response to EPA's use of the
existing study and monetary value estimate.  Unlike EPA in its
reliance on existing related research, the owners specifically
designed their study to value visibility improvements they expected
from emissions controls at the plant.  Nonetheless, the owners did
not complete their research because they did not see value in doing
so and because of time and resource constraints.  Therefore, the
owners' estimated value of expected visibility improvements was based
on the results of a pilot test of a proposed survey instrument.  The
owners' study estimated the national value of visibility benefits to
be $2.3 million.\15 This equates to about $0.023 per U.S.  household. 
Appendix III discusses similarities in the two contingent valuation
studies and their specific technical differences. 


--------------------
\10 Executive Order 12291 was subsequently replaced by Executive
Order 12866, which similarly requires agencies to assess benefits and
costs for regulatory actions that may, among other things, have an
annual effect on the economy of $100 million or more. 

\11 EPA was under court order to, by August 31, 1989, determine
whether a specific pollution source caused or contributed to the
visibility impairment at the Grand Canyon National Park, and, if so,
issue a finding to that effect.  Following such a finding, EPA was to
conduct a best available retrofit technology (BART) analysis on the
identified source, and if the analysis indicated emissions controls
would improve visibility at the Grand Canyon National Park, EPA was
to propose regulations by February 1, 1990 (less than 6 months
later), requiring their installation and use in order to achieve the
emissions limit representing BART.  While on January 9, 1990, the
court extended the deadline for this proposed rule to February 1991,
EPA did not actually have more time to study the issue, since it had
by that time nearly completed its study. 

\12 EPA used two studies for its estimates:  Chestnut and Rowe of
RCG/Hagler, Bailly, Inc., "Preservation Values for Visibility
Protection at the National Parks" (1990) and Schulze, Brookshire,
Walther, and Kelley, "The Benefits of Preserving Visibility in the
National Parklands of the Southwest," Methods Development for
Environmental Control Benefit Assessment, EPA, Vol.  VIII (1981). 

\13 These values are expressed in 1988 dollars. 

\14 These values are expressed in 1992 dollars. 

\15 This value is for the 100 million U.S.  households expected in
1995. 


   AGENCY COMMENTS AND OUR
   EVALUATION
------------------------------------------------------------ Letter :6

We provided a draft of this report to the Department of the Interior
and to the Environmental Protection Agency for their review and
comment.  In written comments, Interior officials said that they
found the report to be generally accurate and a fairly balanced
summary of certain technical aspects of EPA's decision to require
emissions reductions at NGS.  (See app.  IV.)

We received comments from directors of two EPA offices:  the Director
of the Office of Policy Analysis and Review, representing the Acting
Assistant Administrator of the Office of Air and Radiation, and the
Director of the Office of Economy and Environment, representing the
Assistant Administrator of the Office of Policy, Planning, and
Evaluation.  EPA's Office of Air and Radiation said that the report
was generally accurate and complete.  EPA's Office of Policy,
Planning, and Evaluation raised concerns about our discussion of
contingent valuation methodology (see app.  I) and our comparison of
the contingent valuation studies conducted by EPA and the plant
owners (see earlier in this letter and app.  III).  Both offices also
suggested technical clarifications, which we incorporated as
appropriate. 

Office of Policy, Planning, and Evaluation officials said that
appendix I of our report gives undue attention to the guidelines of a
blue-ribbon advisory panel convened by the National Oceanic and
Atmospheric Administration, which they believe implies that the
recommendations have some relation to EPA's use of contingent
valuation.  The officials also said that our report does not give a
balanced view of contingent valuation and places too much emphasis on
arguments critical of contingent valuation.  The officials suggested
that we include a reference to a specific article by a prominent
researcher in support of contingent valuation, a reference to
comments EPA has made on using contingent valuation to assess natural
resource damages, and arguments to counter the advisory panel's
guidelines regarding surveys and formats used in eliciting
information from survey respondents. 

As we state in our report, we are not taking a position on the
appropriateness of contingent valuation.  Appendix I provides a brief
overview of the contingent valuation method, including public policy
uses, historical development, characteristics of a contingent
valuation study, criticisms, and some further issues.  In this
context, we summarize the advisory panel's guidelines because we
believe that the panel's deliberations represent valuable critical
and impartial thinking related to contingent valuation.  The appendix
does not evaluate the merits of the advisory panel's guidelines or of
various arguments for or against the use of contingent valuation
methodology.  However, to make our presentation more complete, we
made minor modifications to the text, added a reference to the
article recommended by EPA, and expanded our discussion of
alternative survey modes.  We did not add the other information
suggested by EPA because it is beyond the scope of this appendix. 

Office of Policy, Planning, and Evaluation officials also said that
appendix III of our report does not suitably explain the reasons for
differences in EPA's and the owners' contingent valuation studies and
the appropriate interpretations of these differences.  Without such
explanation, the officials believe that a reader may erroneously
conclude that there is something wrong with the reliability of the
method.  EPA suggested that we not directly compare the studies
because neither was pursued to the point where any useful comparisons
could be made, and that we emphasize what EPA believes to be the more
important problems with the owners' study, such as incomplete
documentation, questionable statistical techniques, and a sample size
that was too small. 

As we note in our report, appendix III describes the similarities in
the two contingent valuation studies and specific technical
differences between them--in their purpose, design, and
implementation--which led to their different estimates of nationwide
values.  Because neither study used sampling strategies that would
allow nationwide projections, we question the certainty of both
studies' estimates of nationwide values.  Some of the differences in
these studies added uncertainty to their estimates of nationwide
values.  Wherever information was available, we point out the reasons
for these differences and the impact they had on both studies'
results; however, in some instances, neither study had sufficient
information, and further testing would be needed to determine the
effects of each difference on the estimates.  It was not our intent
to complete or refine either study to provide a valid nationwide
projection, but merely to point out how each study was conducted and
why they produced different results.  Nevertheless, to clarify that
the studies were done separately, we made minor revisions to the text
of this letter. 


   SCOPE AND METHODOLOGY
------------------------------------------------------------ Letter :7

To obtain information for this report, we reviewed EPA's documents on
its NGS regulatory action.  The information included numerous
analyses of the plant's effects on visibility at the Grand Canyon and
the visibility improvements that might be expected from the addition
of emissions controls.  The information also included analyses on the
economic costs and benefits of emissions controls.  We supplemented
this information through discussions with officials of various
federal agencies:  EPA, the Department of the Interior and its Bureau
of Reclamation and National Park Service, the National Oceanic and
Atmospheric Administration, and the Department of Energy's Western
Area Power Administration.  We reviewed and compared two contingent
valuation studies that estimated the monetary value of expected
visibility improvements from emissions controls.  One of the studies
was conducted by RCG/Hagler, Bailly, Inc., and was the basis for
EPA's estimates.  The other was conducted for the NGS owners by
Decision Focus, Incorporated.  We also interviewed officials of the
Salt River Project; Decision Focus, Incorporated; RCG/Hagler, Bailly,
Inc.; the Navajo Nation; the Environmental Defense Fund; the Grand
Canyon Trust; the Grand Canyon Visibility Transport Commission; Air
Resource Specialists, Inc.; Northern Arizona University; and others. 
To describe the contingent valuation methodology, we searched and
reviewed economic literature. 

We conducted our review from January through December 1997 in
accordance with generally accepted government auditing standards. 
While we did not independently verify or test the reliability of data
provided by the agencies or the plant owners, EPA used this
information in reaching its regulatory decision. 


---------------------------------------------------------- Letter :7.1

As arranged with your office, unless you publicly announce its
contents earlier, we plan no further distribution of this report
until 30 days after the date of this letter.  At that time, we will
send copies to the Ranking Minority Member of the Committee; the
Administrator, EPA, the Secretary of the Interior, and other
interested parties.  We will also make copies available to others on
request.  If you or your staff have any questions, please call me at
(202) 512-3841.  Major contributors to this report are listed in
appendix V. 

Sincerely yours,

Barry T.  Hill
Associate Director, Energy, Resources,
 and Science Issues


CONTINGENT VALUATION METHODOLOGY
=========================================================== Appendix I

The contingent valuation method uses surveys to ask respondents for
information that can be used to provide estimates of how much
they--and often, by extension, society--are willing to pay for a
certain program or policy, such as those designed to improve the
quality of some environmental or natural resource amenity. 
Proponents of contingent valuation methodology believe that it is a
valuable technique for making inferences about these values,
particularly in cases in which consumer behavior is not (easily)
observed.  However, the use of the contingent valuation method has
been the subject of controversy, particularly in applications
involving non-use values.\1

This appendix provides a brief overview of the contingent valuation
method.  The first section describes the public policy uses of the
method and aspects of its historical development.  The second section
describes characteristics of a contingent valuation study and
presents the suggestions intended to improve contingent valuation
practice made by a blue-ribbon panel of social scientists.  The third
section discusses some of the criticisms that have been leveled at
the contingent valuation method.  The final section discusses some
further issues related to the use of contingent valuation, including
some aspects related to its application to regulatory proceedings. 


--------------------
\1 Non-use values can be thought of as those values that people may
receive from the knowledge that such things as, for example, rare
plants, animals, and unspoiled natural environments exist, even if
people do not "consume" or use these goods directly. 


   THE CONTINGENT VALUATION METHOD
   AND ITS DEVELOPMENT
--------------------------------------------------------- Appendix I:1

Contingent valuation studies use surveys to elicit information about
how much people would be willing to pay for particular goods or
services.  These values can be important in estimating the benefits
applicable to a wide variety of public policy contexts, including
those that require regulatory or environmental impact analyses. 
While in many instances, economic benefits can be estimated using
information on market prices and quantities--because under certain
conditions price and quantity data can be used to estimate underlying
values held by consumers--in other cases, often involving natural
resources or environmental goods, complete market information may not
be available.  This could be because markets do not exist at all, as
in the case of public goods,\2 or because consumers combine their
time with purchases in markets for complementary goods needed to
undertake a recreational experience, for example.  If the values
people do have for these goods are not considered in policy
decisions, then less desirable resource management outcomes may
occur. 

As a general proposition, asking people a question about how much
they value a particular item seems a direct way of getting estimates
of their value for it.  However, economists have generally been
skeptical of this approach and have historically viewed market-based
methods, or so-called "revealed preference" methods in which actual
spending decisions can be observed, as inherently superior to "stated
preference" methods.  Nevertheless, there are many instances in which
no behavioral patterns exist through which consumers reveal the
values they hold.  In such instances, the contingent valuation method
can be thought of as a valuation exercise in which a "contingent," or
hypothetical, market is described for the purpose of replicating the
consumer choice framework that is used to generate values for
traditional market goods.  That is, the approach attempts to create a
market-based choice context for goods without (complete) markets,
such as public or quasi-public goods, so that through their choices
people will reveal their preferences much as they do when making
actual spending decisions. 

Contingent valuation practice developed using theory and practice
from different disciplines, especially economics and survey research. 
A prominent resource economist, Ciriacy-Wantrup, is generally
credited with the suggestion of asking people directly for the values
they placed on natural resource programs with public good aspects.\3
The first practitioner of what is now known as contingent valuation
was Robert K.  Davis who used questionnaires as one way to estimate
values people placed on recreational experiences in Maine.\4

The theory and practice of contingent valuation continued to develop
in the 1960s and 1970s, and most of the first applications were to
resource and environmental issues.\5

During this period, many contingent valuation studies also examined
underlying research issues.  Some of this research worked toward
grounding contingent valuation within the economic theory of consumer
behavior.  For example, economic theory includes many well understood
theoretical relationships involving a consumer's utility,\6 income,
expenditures, and the conditions under which the concept of
willingness to pay is an appropriate measure of underlying value. 
Also, advances in cognitive psychology contributed to understanding
the possible biases in a respondent's answers that may result from
such things as the choice of wording or order of questions. 
Furthermore, researchers gained practical experience in designing,
implementing, and analyzing contingent valuation studies.  By the
1990s, researchers had performed hundreds of such studies. 

The federal government sponsored many of these studies, as various
federal agencies performed and funded contingent valuation studies
and general research on contingent valuation.  These included the
U.S.  Army Corps of Engineers, the Department of the Interior, and
the Environmental Protection Agency (EPA).  EPA in particular was
interested in the analytical potential of contingent valuation in a
variety of environmental regulatory contexts in light of Executive
Order 12291 (and its successor), which required executive branch
agencies to more systematically examine the costs and benefits of
certain of their proposed regulations.  In EPA's case, this involved
the use of contingent valuation to estimate the benefits associated
with various pollution control regulations.\7

Although contingent valuation is a methodology that can be used for
different purposes, it has become inextricably linked with the
measurement of non-use values.\8 Interest in non-use values has been
heightened in part because of the possibility that they may be
considered in resource damage assessment contexts.  The federal
government in its role as trustee may include non-use values when
calculating damages to be recovered through litigation.  The
Comprehensive Environmental Response, Compensation and Liability Act
of 1980 (CERCLA), or Superfund, provided government officials the
right to sue on behalf of the public for resource damages resulting
from release of hazardous materials.\9 The Congress directed the
President, who delegated the responsibility to the Department of the
Interior, to develop regulations applicable to resource damage
assessment.  After a number of groups challenged the regulations, a
federal appeals court upheld Interior's adoption of contingent
valuation methodology for assessing damages to natural resources and
directed Interior to revise its rule to avoid limiting the role of
non-use values or "non-consumptive" values in the calculation of
damages.\10 The grounding of the Exxon Valdez led to the passage of
the Oil Pollution Act of 1990, which required the Department of
Commerce, acting through the National Oceanic and Atmospheric
Administration (NOAA), to develop regulations governing damage
assessment.  The Exxon Company USA could be subject to liability
under the provisions of the Oil Pollution Act and sponsored research
concerning contingent valuation, much of which is critical of the
ability of contingent valuation to measure non-use values
accurately.\11


--------------------
\2 Two characteristics of a "pure" public good are that (1) one
person's consumption of the good does not reduce the amount available
for others to consume and (2) an individual cannot be excluded from
its consumption.  Private sector provision of goods with these
characteristics is not generally profitable, and markets tend to
underprovide such goods. 

\3 S.  V.  Ciriacy-Wantrup, "Capital Returns from Soil Conservation
Practices," Journal of Farm Economics, Nov.  1947, pp.  1181-96. 

\4 Robert K.  Davis, The Value of Outdoor Recreation:  An Economic
Study of the Maine Woods, Doctoral dissertation in economics, Harvard
University, 1963. 

\5 In 1989, Robert Cameron Mitchell and Richard T.  Carson published
a book surveying the theory, practice, and development of contingent
valuation:  Using Surveys to Value Public Goods:  The Contingent
Valuation Method. 

\6 Economists use the term utility to represent the level of
well-being or satisfaction that an individual receives from consuming
various quantities of goods and services. 

\7 EPA's guidance for conducting economic analysis provided for in
Executive Order 12291 stated that the contingent valuation method was
one of four basic methods for estimating environmental effects and
that its use would most likely involve nonmarket goods such as
improvements in aesthetics and the preservation of wildlife and
wilderness areas.  Guidelines for Performing Regulatory Impact
Analyses (Dec.  1983). 

\8 The non-use value concept is generally attributed to economist
John Krutilla.  See "Conservation Reconsidered," American Economic
Review, Vol.  57 (Sept.  1967), pp.  777-86.  Non-use values are also
referred to as passive use or existence values. 

\9 Under CERCLA, the federal government is also liable to natural
resource trustees for monetary damages associated with its release of
hazardous materials.  We reported in August 1996 that the Department
of Energy's potential liability for natural resource damages could
vary from $2.3 billion to $20.5 billion.  Natural Resource Damages at
DOE (GAO/RCED-96-206R, Aug.  16, 1996). 

\10 Ohio v.  United States Department of the Interior, 880 F.2d 432
(D.C.  Cir.  1989). 

\11 Much of this research is published in Jerry A.  Hausman, ed.,
Contingent Valuation:  A Critical Assessment (Amsterdam: 
North-Holland Press, 1993). 


   KEY CHARACTERISTICS OF
   CONTINGENT VALUATION STUDIES
--------------------------------------------------------- Appendix I:2

In an overview of contingent valuation practice, a leading resource
economist stated that while there is no "standard approach,"
contingent valuation studies typically include three general
features.\12

First, a contingent valuation study contains descriptions of the
policy or program at issue and the likely environmental effects so
that respondents can understand the good they are valuing.  Second, a
contingent valuation study contains a framework or mechanism for
eliciting willingness to pay.  Several mechanisms have been used in
contingent valuation studies, such as open-ended questions (How much
would you be willing to pay?), payment cards (Select an amount from a
list of options.), and referendum formats (Would you vote for the
described proposal if your taxes increase by $10?).  Third, a
contingent valuation study may gather information on socioeconomic
variables and attitudes about the environment.  This information can
be used to estimate willingness-to-pay functions using econometric
techniques. 

Researchers have developed many methods to implement contingent
valuation studies within this broad framework.  Additionally, within
the context of the method's development, there have been analytical
debates over the merits of particular aspects of contingent valuation
practice.  The Exxon-sponsored research represented a change in the
discussion of contingent valuation issues in that much of this
research was carried out by economists and others who were not
primarily specialists in natural resource and environmental issues. 
These researchers raised some new issues and provided new emphasis on
other issues on which there had been ongoing analytical debate. 

As part of the process by which it developed its regulations related
to oil spill damages, NOAA convened a blue-ribbon advisory panel to
address a variety of issues, including the fundamental question of
whether the contingent valuation method was capable of providing
reliable estimates of non-use values for use in resource damage
assessments.\13

The panel's report stated that contingent valuation "can produce
estimates reliable enough to be the starting point of a judicial
process of damage assessment, including lost passive-use (non-use)
values." Although NOAA was concerned with the use of contingent
valuation in the damage assessment context, the NOAA guidelines have
applicability to the contingent valuation method more generally.  We
refer to them because we believe that the NOAA panel's deliberations
represent valuable critical and impartial thinking related to
improving the use of contingent valuation.  The panel listed some
guidelines for producing credible studies and noted some strong
concerns about the results of some contingent valuation studies that
it reviewed.  Although its conclusion gave credence to the views of
those who favor the use of the contingent valuation method, adherence
to the panel's suggestions would likely require changes in contingent
valuation practice in that none of the studies the panel reviewed had
been carried out to its suggested standards. 

The panel's report listed a number of suggestions for producing
high-quality contingent valuation studies.  Some of these suggestions
pertained to the importance of the underlying survey research in
contingent valuation studies, in which the survey instruments often
have to provide a substantial amount of background material in a
manner that is accessible to the respondents.  The panel suggested
(1) using probability sampling and appropriate statistical sampling
procedures, (2) subjecting the survey instruments to pretesting, and
(3) taking steps to reduce nonresponse rates.  Additionally, the
panel suggested that contingent valuation studies disclose
information on the sample selection process and provide information
on survey instruments and responses.  The panel stated a strong
preference for the use of in-person surveys as superior to telephone
or mail surveys.  The panel's report stated that it is "unlikely that
reliable estimates of values could be elicited with mail surveys."
The panel also suggested that it was desirable to pretest any
photographs that would be used to convey information to respondents. 

In terms of the elicitation format, the panel suggested that the
referendum format, as opposed to open-ended elicitation, was
desirable.  In its basic form, a referendum format contingent
valuation study describes a proposal to provide a specific
improvement in an environmental good, and the survey respondents are
asked if they would support this proposal as if it were a referendum
item to be voted on.  As part of the proposal, a "payment vehicle" is
described, such as a tax increase or a utility bill increase, and
each respondent is given a specific per person (or per household)
dollar amount that this proposal will cost.  The voting question is a
dichotomous choice ("yes" or "no"), and, in conjunction with other
information gathered in the survey, such as environmental attitudes,
income level, etc., econometric techniques appropriate to dichotomous
choice situations can be used to determine a measure of willingness
to pay for the described proposal on the basis of the observed
pattern of yes or no votes.\14 Supporters of the referendum model
argue that it creates a contingent market mechanism with which
consumers are familiar.  First, consumers are familiar with "posted
price" market choice contexts.\15 Second, the referendum format
itself is familiar to people as a method of expressing political
preferences. 

The panel was concerned that steps be taken so that results of
contingent valuation studies conform to common notions of economic
rationality.  The NOAA panel endorsed the use of follow-up questions
asking respondents the reasons that they voted the way they did as
well as questions designed to test how well the respondent understood
the program or policy at hand.  The panel also suggested that survey
respondents be provided with a reminder that paying for the non-use
good at issue would result in a smaller budget to spend on other
goods and services and that they be told of any available
substitutes. 

One aspect of rationality is that, generally speaking, people are
willing to pay more for greater amounts of a good.  In its
deliberations, the panel had concerns about evidence presented in one
contingent valuation study that estimated willingness to pay "for the
cleanup of all lakes in Ontario was only slightly more than
willingness to pay for cleaning up lakes in just one region" and in
another study that estimated "willingness to pay to take measures to
prevent 2,000 migratory birds (not endangered species) from dying in
oil-filled ponds was as great as that for preventing 20,000 or
200,000 birds from dying."\16

The panel suggested that a contingent valuation study demonstrate its
sensitivity to these so-called "scope effects."


--------------------
\12 Paul R.  Portney, "The Contingent Valuation Debate:  Why
Economists Should Care," Journal of Economic Perspectives, Vol.  8,
No.  4 (Fall 1994), pp.  3-17. 

\13 The panel was composed of Kenneth Arrow and Robert Solow
(cochairs), Edward Leamer, Roy Radner, Howard Schuman, and Paul
Portney.  Schuman is a prominent survey researcher, and the others
are economists.  Arrow and Solow are Nobel laureates.  The NOAA
panel's report can be found at 58 Fed.  Reg.  4601 (Jan.  15, 1993). 

\14 The development of the referendum format contingent valuation
study is attributed to Richard Bishop and Thomas Heberlein,
"Measuring Values of Extra-Market Goods:  Are Indirect Measures
Biased?" American Journal of Agricultural Economics, Vol.  61 (Dec. 
1979), pp.  926-30.  Hanemann introduced a formula for retrieving an
estimate of willingness to pay from the estimated parameters of a
dichotomous choice regression.  See W.  Michael Hanemann, "Welfare
Evaluations in Contingent Valuation Experiments with Discrete
Responses," American Journal of Agricultural Economics, Vol.  66
(Aug.  1984), pp 332-41. 

\15 In most consumer transactions in developed market economies, a
price is posted and consumers decide whether or not to purchase at
that price. 

\16 The study cited was "Measuring Natural Resource Damages with
Contingent Valuation:  Test of Validity and Reliability," by William
H.  Desvousges, F.  Reed Johnson, Richard W.  Dunford, Sara P. 
Hudson, K.  Nicole Wilson, and Kevin J.  Boyle, published in Hausman,
ed., Contingent Valuation. 


   CRITICISMS OF CONTINGENT
   VALUATION METHOD HAVE BEEN
   RAISED
--------------------------------------------------------- Appendix I:3

Some economists and other analysts have voiced criticisms of
contingent valuation methods.\17 An overarching concern among some
observers is that contingent valuation does not adequately capture
true estimates of willingness to pay.  One component of this
criticism is that respondents make choices but that these choices do
not require real economic commitments.  Also, particularly with
respect to non-use values, critics argue that it can be difficult for
respondents to comprehend a particular environmental or resource
valuation issue, or to distinguish what researchers envision as a
well-defined specific issue from a more general "warm glow" effect. 
Furthermore, some critics argue that the statistical estimation
process by which willingness-to-pay estimates are produced from
survey responses can be imprecise.  At the same time, proponents of
contingent valuation have made arguments that respond to many of
these criticisms. 

One criticism of the contingent valuation method is that contingent
markets do not create choice contexts with binding budget constraints
and the financial consequences associated with "real" choice
contexts.  In general, the issue is that by actually spending a
certain amount of money, an individual or household can no longer
spend that money on something else.  Thus, the goods and services
that are purchased presumably represent the true preferences of the
individual or household.  In contrast, responding yes to a contingent
valuation question does not financially bind the respondent in the
same way.  Proponents of the contingent valuation method have been
also concerned with this issue and suggest that appropriate steps in
survey design serve to reduce the problem.\18

Others maintain that the existence of opportunities for strategic
misrepresentation, among other problems, reduces the usefulness of
the contingent valuation method.\19

Another criticism that has been leveled at contingent valuation is
that, particularly for applications involving non-use values, it is
difficult to create a choice context in which the respondent can be
sufficiently informed to provide a reliable response.  Although the
goal of contingent valuation is to construct frameworks capable of
eliciting values that conform to principles of economic rationality,
some argue that this task is too ambitious.  For instance, one
analyst states that individuals may be

     ".  .  .  wired differently than the economic model of fully
     formed, stable, rational preferences requires.  While the
     consumer's wiring may produce patterns of market behavior that
     will often be approximated well by the economist's model, when
     we approach the consumer from a different angle, asking direct
     and unusual questions about values, we find alarming variations
     from the standard economist's story.  All these consumers, so
     normal and rational on the outside, are revealed to be shells
     filled with vast rule-books of heuristics written by natural
     selection.  Throw these people a curve ball, in the form of a
     valuation question that fails to fit a standard heuristic for
     market response, and the essential mindlessness of the organism
     is revealed."\20

Critics have also argued that estimates produced by contingent
valuation studies may not be limited to values of the specific
environmental amenity under consideration but may also incorporate a
variety of broader values.  The NOAA panel recognized the concern
that contingent valuation estimates may contain a "warm glow"
component associated with supporting worthy causes. 

One additional criticism is that resulting estimates of willingness
to pay can be particularly sensitive to the statistical methods used. 
One analyst examined a variety of statistical issues in contingent
valuation estimation and concluded that the estimates were sensitive
to context effects, including anchoring effects,\21 as well as the
choice as to how statistical outliers were handled.\22

Proponents of contingent valuation have responded to many of the
arguments developed by critics.  In particular, a prominent
contingent valuation researcher has written an overview article that
provides many general arguments in favor of contingent valuation, as
well as a point-by-point discussion of several specific issues raised
by critics of contingent valuation.\23


--------------------
\17 See Hausman, ed., Contingent Valuation and Peter A.  Diamond and
Jerry A.  Hausman, "Contingent Valuation:  Is Some Number Better than
No Number?" Journal of Economic Perspectives, Vol.  8, No.  4 (Fall
1994), pp.  45-64. 

\18 For example, one chapter in Using Surveys to Value Public Goods: 
The Contingent Valuation Method, by Mitchell and Carson, is devoted
to this topic. 

\19 For instance, one study concludes that "there simply exists no
basis for non-speculative, dogmatic statements regarding free-riding
behavior in the [contingent valuation methodology] one way or
another." This study summarized studies that examined the extent to
which respondents actually pay amounts they report in contingent
valuation studies:  "A few studies show that contingent valuation
values may be 'close' to values that reflect real economic
commitments.  However, a number of other studies show that contingent
valuation values overstated real economic commitments, and that these
overstatements can be quite large." Ronald G.  Cummings and Glenn W. 
Harrison, "Was the Ohio Court Well Informed in its Assessment of the
Accuracy of the Contingent Valuation Method?" Natural Resources
Journal, Vol.  34, No.  1 (Winter 1994), pp.  1-36. 

\20 Daniel McFadden, "Why is Natural Resource Damage Assessment So
Hard?" Hibbard Lecture, Department of Agricultural and Resource
Economics, University of Wisconsin, Apr.  12, 1996.  Quoted material
on p.  7. 

\21 Contingent valuation researchers use the term anchoring effects
to describe a process in which respondents, who may be uncertain
about the values they hold, base their estimates on an initial value
that may be found in material provided to them.  This material is
extraneous to providing information about value.  It likely biases
the response toward the cue contained in the extraneous material. 

\22 Daniel McFadden, "Contingent Valuation and Social Choice,"
American Journal of Agricultural Economics, Vol.  76(4), (Nov.  1994)
pp.  689-708. 

\23 W.  Michael Hanemann, "Valuing the Environment through Contingent
Valuation," Journal of Economic Perspectives, Vol.  8, No.  4 (Fall
1994), pp.  19-43. 


   FURTHER CONSIDERATIONS
--------------------------------------------------------- Appendix I:4

Many observers believe that the use of contingent valuation seems
likely to continue to grow.\24 Some aspects of the contingent
valuation method that are not entirely analytical may also influence
the future path of its use in regulatory and damage assessment
proceedings.  One aspect concerns potential problems with
incorporating evolving scientific understanding of the specific
environmental issues crucial to a given policy evaluation into survey
instruments that take time to develop, implement, and analyze. 
Another aspect involves consideration of the geographic extent of the
affected population.  A further issue concerns the "calibration" of
willingness-to-pay estimates for use in regulatory or damage
assessment proceedings.  Additionally, some practitioners of
contingent valuation are concerned that some of the specific
recommendations of the NOAA panel may inappropriately preclude other
analytical alternatives that may prove to be superior or more
cost-effective. 

Federal regulatory actions often trigger specific requirements and
may involve deadlines.  For instance, the National Environmental
Policy Act (NEPA) of 1969 requires federal agencies to prepare an
environmental impact statement if a proposed federal action is likely
to significantly affect environmental quality.  Although neither NEPA
nor its implementing regulations require non-use values to be
considered, non-use values have been considered in NEPA
proceedings.\25 A contingent valuation study requires an accurate
description of the likely change in environmental amenity, which in
turn requires careful consideration of the underlying environmental
impacts, perhaps including anthropological, atmospheric, biological,
and physical components.  In some contexts, much of the underlying
scientific information may have to be developed during the
environmental impact statement process.  Because there are many steps
required to develop, implement, and analyze survey instruments, there
is a chance that the willingness-to-pay estimates will be produced on
the basis of descriptions of expected environmental impacts that do
not accurately reflect later scientific understanding, or that
regulatory decisionmaking time frames are lengthened as that
information is incorporated.\26 In other regulatory contexts, such as
the one involving the Navajo Generating Station (NGS), court-imposed
deadlines may influence not only a decision to undertake a contingent
valuation study, but decisions as to how underlying scientific
understanding is incorporated into the survey research process.\27 If
a particular policy action is controversial or disputed, the accuracy
of the underlying description of environmental impacts is likely to
be challenged as leading to inaccurate calculations of willingness to
pay for those improvements. 

Much of the analytical discussion focuses on estimates of per person
or per household willingness to pay, and how sensitive or robust such
estimates may be to particular choices in underlying description or
analytical technique.  However, for use in benefit-cost analysis or
in estimating damage assessments, the issue of how many people are
affected--for instance, how many people are assumed to have non-use
values--is important in calculating gross benefit numbers.  For
contingent valuation estimates of recreation values, samples of
recreationists offer a fairly straightforward way of defining the
relevant population.\28 For non-use values, the choice of the
relevant population may not be so clear.  For resources of national
significance, researchers may reasonably consider that the national
population is the relevant population and may design a study on the
basis of that premise.  In other cases, the answer is less clear.  In
any event, it is possible to generate a large benefit number when
even fairly small estimates of willingness to pay are multiplied by
100 million, approximately the number of households in the country. 

Some observers have argued that contingent valuation estimates of
willingness to pay need to be adjusted, or calibrated, because of the
inherent limitations.  In its deliberations, the NOAA panel reported
that it was "persuaded that hypothetical markets tend to overstate
willingness to pay for private as well as public goods" and that the
same bias would be likely to occur in contingent valuation studies. 
In its proposed rule, the Department of Commerce (NOAA) recommended a
50-percent calibration factor to adjust for biases of unknown
magnitude but of an upward direction.\29 Although a comparison of
contingent valuation estimates with other estimates is not possible
for non-use values, some researchers have more recently compared
contingent valuation estimates with "revealed preference" estimates
in a number of studies for which both kinds of estimates were
produced.  The researchers examined a variety of recreation studies
and also cases in which amenities might be capitalized into an asset
price, such as a price premium a house with a beautiful view might
command over a similar house without the view.  The authors located
83 studies that provided 616 comparisons of contingent valuation to
revealed preference estimates.  The authors reported that contingent
valuation estimates were "smaller, but not grossly smaller, than
their [revealed preference] counterparts." Although some contingent
valuation estimates were larger than their counterparts, the authors
concluded that suggestions for a routine downward adjustment of
contingent valuation estimates appear unwarranted.\30

Some advocates for the use of the contingent valuation method have
voiced concern over some of the NOAA panel's suggestions.  In
particular, the panel's strong preference for in-person surveys over
mail surveys has been criticized by proponents of mail surveys, as
has the panel's preference for the referendum format.  The panel's
preference for in-person surveys had much to do with the fact that
sampling frames available for mailing provide incomplete coverage for
the national population.  The panel also was concerned that targeted
respondents can review the subject of the questionnaire before
deciding to respond, so those most interested in the subject may
choose to respond.  Proponents of mail surveys counter that other
survey methods, such as in-person interviews, also have their
drawbacks, such as problems caused by the presence of an interviewer,
which may bias responses, or pressures on respondents to answer
quickly while the interviewer is present.  They also add that mail
surveys of large samples offer significant cost savings over
in-person interviews. 

The panel's preference for the referendum format was based on a
number of factors, including the fact that people are "rarely asked
or required in the course of their everyday lives to place a dollar
value on a particular public good." Even though open-ended
elicitation is not familiar, some researchers point to results from
experimental economics indicating that posted price choice contexts
perform poorly relative to open-ended contexts in "early rounds" of
bidding situations in which respondents are not experienced.  Given
that respondents are not likely to be well informed in many
contingent valuation contexts (at least for non-use goods), these
researchers argue that the experimental results that people overpay
in early rounds suggests that a "one-round" referendum may lead to an
overstated willingness to pay.\31 Other research suggests that the
specific price that a referendum survey respondent is confronted
with--the bid price--may lead to anchoring effects, so the resulting
willingness-to-pay estimates may be too high.  In contrast to the
typical practice in which bid prices are distributed randomly to
respondents, this research suggests that some initial investigation
incorporating open-ended valuations could be useful in avoiding the
assignment of high bid prices to respondents with low values (and
vice versa).\32


--------------------
\24 For instance, Paul Portney stated that "Both regulatory agencies
and governmental offices responsible for natural resource damage
assessment are making increasing use of it in their work." Portney,
"The Contingent Valuation Debate," p.  16. 

\25 There are at least two examples, both from the Department of the
Interior:  The Reintroduction of Gray Wolves to Yellowstone National
Park and Central Idaho, Final Environmental Impact Statement, U.S. 
Fish and Wildlife Service (May 1994) and Operation of Glen Canyon
Dam, Final Environmental Impact Statement, Bureau of Reclamation
(Mar.  1995). 

\26 For a further discussion, see David A.  Harpman, Michael P. 
Welsh, and Richard C.  Bishop, "Nonuse Economic Value:  Emerging
Policy Analysis Tool," Rivers, (Apr.  16, 1991) Vol.  4 (1993), pp. 
280-91. 

\27 As discussed in this report, EPA decided not to undertake a
contingent valuation study of the specific issue of wintertime
visibility improvements in the Grand Canyon due to reduced emissions
from NGS, but decided rather to make use of results produced for a
wider study of visibility issues. 

\28 Although a sample based only on visitors to the site of interest
is likely to overrepresent the values held by more frequent
recreators. 

\29 59 Fed.  Reg.  1062 (Jan.  7, 1994). 

\30 Richard T.  Carson, Nicholas E.  Flores, Kerry M.  Martin and
Jennifer L.  Wright, "Contingent Valuation and Revealed Preference
Methodologies:  Comparing the Estimates for Quasi-Public Goods."
University of California, San Diego, Department of Economics,
Discussion Paper 94-07 (May 1994). 

\31 William Schulze, Gary McClelland, Donald Waldman, and Jeffrey
Lazo, "Sources of Bias in Contingent Valuation," in David J. 
Bjornstad and James R.  Kahn, eds., The Contingent Valuation of
Environmental Resources (Cheltenham, U.K., 1996). 

\32 Kevin J.  Boyle, F.  Reed Johnson, and Daniel W.  McCollum,
"Anchoring and Adjustment in Single-Bounded, Contingent-Valuation
Questions," forthcoming in the American Journal of Agricultural
Economics (Dec.  1997). 


STUDIES OF VISIBILITY IMPAIRMENT
CAUSES IN AND AROUND GRAND CANYON
NATIONAL PARK
========================================================== Appendix II

The largest benefit EPA expected to occur at the Grand Canyon
National Park as a result of reducing emissions from NGS was an
improvement in visibility during certain winter weather conditions. 
These conditions are expected to occur approximately 10 to 21 days
each winter.  EPA initially estimated that reducing the sulfur
dioxide emissions by 90 percent would improve the winter seasonal
average visibility by approximately 14 percent.  These estimated
improvements were based in part on a study of visibility impairment
in the vicinity of the Grand Canyon by the National Park Service
(NPS).  EPA revised its estimate to approximately 7 percent after
considering information from other studies that suggested that NGS
has a lesser effect.  EPA noted that its revised estimate may be
understated because of other unquantified visibility improvements. 


   EPA INITIALLY ESTIMATED AN
   APPROXIMATELY 14 PERCENT
   IMPROVEMENT IN VISIBILITY
-------------------------------------------------------- Appendix II:1

The Clean Air Act requires that, upon a finding that it is reasonable
to anticipate that an emissions source may be causing or contributing
to the impairment of visibility in certain national parks or
wilderness areas, the relevant state or EPA is required to determine
an emissions limit for the source that reflects the best available
retrofit technology (BART).\1 In determining an emissions limit, EPA
is required to take into consideration, among other things, the costs
of reducing emissions and the degree of improvement in visibility
that may reasonably be anticipated to result from the use of such
technology.\2 In September 1989, EPA proposed to attribute visibility
impairment in the Grand Canyon to emissions from NGS and, as a
result, was required to carry out a technology assessment of NGS. 

EPA's determination of an emissions limit relied on data from an NPS
study (the National Park Service Report on the Winter Haze Intensive
Tracer Experiment [WHITEX]) of visibility impairment in the vicinity
of the Grand Canyon.\3 WHITEX was designed to evaluate the ability of
using a variety of modeling approaches to attribute visibility
impairment from a single source, NGS.  Specifically, various models
were to be evaluated in their ability to link NGS' emissions to
visibility impairment at the Grand Canyon and other nearby national
parks.\4 According to WHITEX, wintertime meteorological conditions in
the area are characterized by several periods of stagnation in which
air pollutants can be trapped by a persistent thermal inversion,
resulting in a distinct visible surface haze layer.  Although several
earlier investigations have been conducted to determine the origins
of the haze, WHITEX was a more comprehensive effort to address
persistent questions about the nature and sources of the winter haze
conditions. 

According to EPA and NPS officials, the emissions from NGS can have
the largest effect during certain weather conditions that include (1)
high relative humidity, which facilitates the conversion of the
plant's gaseous sulfur dioxide emissions to visibility-impairing
sulfate particles, and (2) wind patterns that transport the emissions
to the Grand Canyon.  EPA estimated that these conditions occur
between 10 and 15 times per winter, lasting from 3 to 5 days each
occurrence.  However, NPS officials explained that the effect of
emissions from NGS on visibility impairment at the Grand Canyon
during these conditions can be mitigated by local weather patterns. 
According to these officials, due in part to local weather
conditions, the most severe effects occur approximately two to three
times per winter, lasting from 5 to 7 days each time.  These
officials explained that visibility can be impaired during these
winter weather conditions because of naturally occurring
impairment--mist, fog, clouds--and because of man-made sources,
primarily NGS.  However, the officials noted that photographic and
air monitoring data show that visibility impairment from man-made
sources can continue for several days after the naturally occurring
conditions have dissipated.  In addition, the evidence also indicates
that impairment from man-made sources is perceptible even on some
days that include natural impairment. 

WHITEX, carried out during January and February of 1987, relied on
injecting a unique chemical into NGS' smokestack and tracking this
chemical to air monitoring stations that were placed around the
region, including at the Grand Canyon.  The study concluded that NGS
was the single largest contributor to visibility impairment in the
Grand Canyon during the days for which air monitoring data were
available.  WHITEX's results indicated that, for days on which air
monitoring data were available, NGS contributed approximately 40
percent on the average to wintertime visibility impairment and
approximately 60 percent to 70 percent during the worst visibility
impairment conditions. 

EPA's estimates of the degree of visibility improvement used WHITEX
data to establish the relationship between NGS' emissions and
visibility impairment in the canyon.\5 EPA explained that, because of
the complex terrain in and around the Grand Canyon, the WHITEX data
provided a more reliable estimate than the models often used to
estimate improvements in visibility.  Specifically, EPA used the
ratio of sulfur dioxide emissions at NGS to sulfate particles in the
Grand Canyon attributable to NGS.  Using this ratio, EPA applied a
"linear rollback" model, which used regression analysis techniques to
estimate the level of visibility impairment that would result from a
given level of NGS' sulfur dioxide emissions.  The model's formula
contained terms that attempted to account for, among other things,
the percentage of sulfate that contributes to overall visibility
impairment, the percentage of NGS' contribution to total sulfates,
and the removal rate of the control technology. 

EPA's analysis of NGS' emissions reductions and the resulting
visibility improvements was complicated in several ways.  For
example, EPA had to determine whether to account for the possibility
that a linear relationship may not exist between NGS' sulfur dioxide
emissions and the resulting visibility impairment in the Grand
Canyon.  In other words, would a reduction in NGS' sulfur dioxide
emissions result in a proportional or less-than-proportional
reduction in visibility impairment in the canyon attributable to NGS? 
EPA explained that WHITEX showed that the conversion of sulfur
dioxide to visibility-impairing sulfate particles is greater in a
moisture-rich environment (e.g., clouds or fog) and the lack of such
an environment tends to limit such conversion.  However, EPA also
explained that other studies showed that moisture-rich environments
may also inhibit the conversion of sulfur dioxide to
visibility-impairing sulfate particles because the compounds with
which the sulfur dioxide might react (typically hydrogen peroxide and
ozone) may combine first with other compounds and lessen the
conversion of sulfur dioxide to the visibility-impairing sulfate
particles.  EPA determined that this issue was insignificant because
adequate quantities of compounds, such as hydrogen peroxide, would
likely exist during the winter and other studies of trends in various
parts of the country did not indicate any significant nonlinearity. 
However, EPA did modify its model to address another complication. 
This complication stemmed from the possibility that reducing sulfur
dioxide emissions could increase the amount of other
visibility-impairing compounds to be formed.  EPA was concerned that,
if sulfur dioxide was reduced, ammonia that would have combined with
sulfur dioxide to form visibility-impairing sulfate would instead
combine with nitrogen oxides, forming ammonium nitrate.  EPA modified
its model to account for this potential "nonlinear" complication. 

EPA assessed a variety of different scenarios to determine the
potential visibility improvements.  First, the model assessed the
potential visibility improvements (in terms of visual
range--expressed in kilometers) under average conditions found during
the WHITEX study period that could result from emission control rates
of 70, 80, and 90 percent at NGS.  Second, the model assessed the
potential visibility improvements under the worst-case conditions
found during the WHITEX study period that could result from removal
rates of 70, 80, and 90 percent.  EPA assessed scenarios that assumed
a linear relationship between sulfur dioxide emissions and visibility
impairment and other scenarios that assumed a nonlinear relationship. 

Model results showed a dramatic increase in estimated visibility
improvements during the worst-case conditions compared to
improvements during the average conditions.  For example, the results
showed visibility improvements under average conditions that ranged
from approximately 11 percent for a 70-percent level of emissions
reduction to approximately 14 percent for a 90-percent level of
emissions reduction.  These figures compare to the model's estimates
of visibility improvements under worst-case conditions that ranged
from approximately 60 percent for a 70-percent level of emissions
reduction to approximately 94 percent for a 90-percent level of
emissions reduction. 

Because modeling average conditions does not necessarily represent
actual conditions on a given day, EPA also examined potential
visibility improvements using actual data collected during the WHITEX
study.  In cases where total visibility impairment data were not
available (because of weather conditions during the study
period--i.e., during periods of cloud cover), EPA reconstructed data
that were measured during the study period.  This analysis found
visibility improvements that ranged, on average, from approximately
23 percent to 43 percent, depending on the level of emissions
reduction.  This level of visibility improvement was approximately 2
to 3 times higher than the estimated improvement found using average
visibility conditions and approximately half the values found under
the average worst-case conditions. 


--------------------
\1 The act requires EPA to promulgate plans to protect visibility in
cases where a state fails to do so to EPA's satisfaction. 

\2 EPA is also required to consider the energy and non-air-quality
environmental impacts of compliance, any existing pollution control
technology in use at the source, and the remaining useful life of the
source. 

\3 National Park Service Report on the Winter Haze Intensive Tracer
Experiment (Dec.  4, 1989).  The study was part of a larger
cooperative effort by electric utilities, including the Salt River
Project--the operators and part owners of NGS--and federal agencies
to study visibility issues using continual visibility and aerosol
measurements and in-depth intensive studies such as WHITEX. 

\4 According to EPA, NGS was chosen as the test candidate because (1)
it had documented a buildup of haze layers around the facility, (2)
it was the largest uncontrolled source of sulfur dioxide emissions in
the Southwest, (3) it was located near the Grand Canyon and several
other national parks, and (4) it was an isolated source. 

\5 William Barnhard, E.H.  Pechan and Associates, Inc., Best
Available Retrofit (BART) Analysis for the Navajo Generating Station
in Page, AZ, (Durham, N.C., Jan.  31, 1990). 


   EPA ESTIMATED IMPROVEMENTS ON
   OTHER WINTER DAYS
-------------------------------------------------------- Appendix II:2

In addition to improvements during certain winter weather conditions,
EPA also estimated visibility improvements on other winter days. 
These estimated improvements were reported in terms of "changes in
contrast," which, like visual range, is another method of measuring
visibility.  EPA defined "contrast" as the percentage difference
between the brightness of a scenic element and its background.  With
this method, EPA estimated that, using information developed in the
WHITEX study and extrapolating it to the winter period (and applying
a nonlinearity factor), reducing the emissions from NGS by 90 percent
would have the following effects on visibility:  (1) at least a
"perceptible" change in visibility conditions (defined by EPA as a
4-percent change in contrast) approximately 100 days of the total
winter days, (2) a "quite noticeable" change in visibility conditions
(10-percent change in contrast) approximately 58 days of the total
winter days, and (3) a "very apparent" change in visibility
conditions (20-percent change in contrast) approximately 21 days of
the total winter days.\6

Although these estimates illustrate the varying effect of NGS on
visibility during the winter, EPA eventually dropped the estimates. 
EPA explained that its calculations were in error because it did not
take into account natural atmospheric scattering of light.  Similar
calculations were made by the plant owners, who attempted to correct
for EPA's error, and also showed that differing levels of
improvements can be expected during the winter months.  The plant
owners' estimates showed that 54 days, rather than EPA's estimate of
100 days, would have at least a perceptible change.  Using the
results of their own visibility study, the plant owners argued that
reducing the emissions by 90 percent would result in (1)
approximately 4 days during the winter of a perceptible improvement
in visibility, (2) approximately 2 days during the winter of a quite
noticeable improvement in visibility, and (3) 0 days during the
winter of a very apparent improvement in visibility. 


--------------------
\6 EPA defined the winter period as the period between November 1 and
March 31 (151 days in nonleap years). 


   NATIONAL ACADEMY OF SCIENCES
   REVIEWED WHITEX STUDY
-------------------------------------------------------- Appendix II:3

The National Academy of Sciences' National Research Council
established a committee to evaluate the WHITEX study.\7 The Council
noted that one of the study's greatest weaknesses was that no
measurements of visibility impairment were made below the rim of the
Grand Canyon, within the canyon itself.  The Council noted that
meteorological evidence, still photographs, and time-lapse video
suggest that sulfur concentrations in the canyon might have been
considerably greater than was observed at the monitoring station used
during the study and located at the rim of the canyon.  On the basis
of the data presented in the WHITEX study, the Council concluded
that, on some days during the study period, NGS contributed
significantly to haze in the Grand Canyon.  However, the review also
concluded that the study was not sufficient to ascertain the
quantitative contribution by NGS to haze at any given time.\8 The
authors of the WHITEX study agreed with the Council that two of the
quantitative techniques used in the study could not be used alone or
to exactly apportion NGS sulfur dioxide emissions to visibility
impairment at the canyon.  Rather, the authors explained that they
used these quantitative analytical techniques in conjunction with
qualitative techniques to make reasonable estimates of NGS' effect. 


--------------------
\7 The Council established the Committee on Haze in National Parks
and Wilderness Areas to address issues related to visibility
degradation in these protected regions.  As part of its charge, the
committee was asked to evaluate the WHITEX study.  The committee's
work was sponsored by EPA, the Department of the Interior, the
Department of Energy, and the Salt River Project. 

\8 Haze in the Grand Canyon:  An Evaluation of the Winter Haze
Intensive Tracer Experiment, National Academy of Sciences (Oct. 
1990). 


   PLANT OWNERS CONDUCTED SEPARATE
   VISIBILITY STUDY
-------------------------------------------------------- Appendix II:4

Concerned with what they believed to be shortcomings of the WHITEX
study, the plant owners conducted their own visibility study.  This
study was similar to the WHITEX study in its use of a unique tracer
through NGS' smokestack and air quality monitoring stations around
the Grand Canyon.\9 The owners' study found that NGS' sulfur dioxide
emissions contributed less to visibility impairment at the canyon
than the WHITEX study concluded.  The owners' study estimated that a
90-percent reduction in NGS' sulfur dioxide emissions would not
improve the average visual range in winter by more than 2 percent. 
In reviewing the owners' study, EPA concluded that there was
reasonable agreement between the findings of this study and the
findings of the WHITEX study with respect to NGS' peak contribution
to sulfate and visibility impairment at the Grand Canyon.  According
to EPA, the major difference was that the WHITEX study led to a
conclusion that the peak visibility impairment conditions occur more
frequently and that the nonpeak visibility impairment conditions are
greater than zero more often than found by the plant owners' study. 


--------------------
\9 L.  Willard Richards, Charles L.  Blanchard, Donald L. 
Blumenthal, Navajo Generating Station Visibility Study--Executive
Summary--Draft Number 2, (Apr.  16, 1991).  Prepared for Prem
Bhardwaja, Salt River Project, Phoenix, Ariz. 


   IN-CANYON VISIBILITY IMPAIRMENT
   WAS ESTIMATED BUT NOT
   QUANTIFIED
-------------------------------------------------------- Appendix II:5

NPS established an air monitoring station within the canyon that
addressed one of the shortcomings of its study that was noted by the
National Research Council--that impairment measurements within the
canyon were not made.  Preliminary results from the monitoring
station showed that visibility impairment in the canyon was worse
than the impairment measured at the monitoring station used during
the WHITEX study, which was located at the rim of the canyon.  Data
from the new monitoring site below the rim of the canyon confirmed
that air transport and conversion processes below the rim of the
canyon are sometimes decoupled from those processes above the rim. 
EPA also explained that photographic data taken during the WHITEX
study indicated that airflow below the rim of the canyon could result
in higher visibility impairment due to the trapping of pollution. 
EPA said that it did not quantify the expected visibility
improvements below the rim of the canyon due to the limited amount of
data available and a limited understanding of the air transport
mechanisms below the rim of the canyon. 


   EPA REVISED ITS ESTIMATE TO AN
   APPROXIMATELY 7 PERCENT
   IMPROVEMENT IN VISIBILITY
-------------------------------------------------------- Appendix II:6

Following a public comment period, EPA revised its estimate that
reducing NGS' sulfur dioxide emissions by 90 percent could improve
the winter seasonal average visibility above the rim of the canyon
from its initial estimate of approximately 14 percent to
approximately 7 percent.  In revising its estimate, EPA relied on the
two visibility studies and other air monitoring information.  EPA
noted that it still believed that the primary improvement in
visibility would stem from reductions of emissions from NGS during
winter weather conditions.  However, EPA also noted that other
visibility improvements will occur, including improvements below the
rim of the canyon, during seasons other than winter at the canyon,
and at other national parks in the area.  Therefore, EPA noted that
its estimate of approximately 7 percent is likely an underestimate. 

EPA explained several factors that tend to make the estimate an
understatement.  First, EPA's estimate did not include the more
pronounced improvement that would be realized in the canyon, below
the rim.  EPA noted that, a comparative analysis, prepared by Air
Resource Specialists, Inc., of 3 years (1988 to 1991) of sulfate
levels from above-rim and in-canyon air monitoring stations, showed
in-canyon visibility impairment up to 10 times greater than that
measured on the rim and concluded that high sulfate conditions below
the rim typically last from 3 to 5 days longer than do those observed
at the rim.\10 In addition, the study concluded that there is a high
degree of confidence that NGS is responsible for at least 90 percent
of the visibility impairment at the Grand Canyon during these
periods. 

Second, EPA noted that the principal improvement will likely be
during certain wintertime weather conditions.  EPA's approximately 7
percent estimate reflects an average over the 5-month period from
November through March.  Since the winter weather conditions during
which NGS can have its largest effect occur intermittently throughout
the 5-month period, EPA expects that the visibility improvement
during these winter weather conditions is substantially greater than
7 percent. 

Third, EPA did not estimate the expected visibility benefits to be
realized during nonwinter seasons at the Grand Canyon or at other
surrounding national parks.  NGS is located near several national
parks located on the Colorado Plateau--which in addition to the Grand
Canyon include Arches, Bryce Canyon, Canyonlands, Capitol Reef, Mesa
Verde, and Zion.  Two studies were submitted to EPA that estimated
NGS' year-round impacts on these other parks.  One study, by the Air
Resource Specialists on in-canyon visibility impairment, estimated
the year-round effects of NGS.\11 The study estimated the visibility
effects of NGS' emissions for every hour from December 1985 to
November 1990 and concluded that NGS' emissions were present at the
Grand Canyon (1) an average of 35 percent of the time in the winter
and (2) near or above an average of 20 percent of the time 8 months
of the year.  The study concluded that when NGS' emissions are not in
the Grand Canyon, they are most likely affecting another national
park in the area.  The study estimated that NGS' emissions are
present in these other parks on average at least 50 percent of each
month throughout the year.  The other study, prepared by Latimer and
Associates, analyzed the impact of NGS' emissions on impairment
during all seasons in these national parks (including the Grand
Canyon) for the same 5-year period.\12 The study concluded that haze
impacts generally are highest in the Grand Canyon in the winter and
calculated that perceptible sulfate haze impacts due to NGS'
emissions occured in all other parks and in each season during the
5-year period modeled.  The study concluded that since NGS is
surrounded by national parks, the likelihood is high that at least
one park is impacted at any given time. 


--------------------
\10 Air Resource Specialists, Inc., Technical Information Regarding
Visibility Degradation Caused by the Navajo Generating Station (NGS)
in the Grand Canyon and Other National Parks and Wilderness Areas
Within 500 Kilometers of NGS.  (Fort Collins, Colo., Apr.  18, 1991). 

\11 Air Resource Specialists, Inc., Technical Information. 

\12 Douglas Latimer, Latimer and Associates, Haze Impacts on the
Golden Circle of National Parks of Sulfur Dioxide Emissions From the
Navajo Generating Station:  Haze Puff Model Calculations for
1986-1990.  (Boulder, Colo., Apr.  18, 1991). 


CONTINGENT VALUATION STUDIES USED
TO VALUE VISIBILITY IMPROVEMENTS
AT THE GRAND CANYON NATIONAL PARK
========================================================= Appendix III

This appendix discusses two contingent valuation studies related to
EPA's 1991 regulatory action that established an emissions limit for
NGS.  One study was performed for EPA, the other for the plant
owners.  Both studies set out to value changes in visibility at the
Grand Canyon, had survey instruments that were carefully designed by
their researchers, and showed that people were willing to pay some
amount to improve visibility at the Grand Canyon.  The studies were
different, however, in the specifics of what they were to value and
how they went about doing so.  Some of these differences added
uncertainty to the studies' results.  It is less clear how the other
differences affected results, and testing would be needed to
determine the effects due solely to each of these differences. 


   WHY EPA AND THE PLANT OWNERS
   ESTIMATED A MONETARY VALUE OF
   VISIBILITY IMPROVEMENTS
------------------------------------------------------- Appendix III:1

EPA set out to estimate the monetary value of visibility improvements
in order to comply with Executive Order 12291.  The order provided
that, to the extent permitted by law, agencies should not take
regulatory action unless the potential benefits to society outweighed
potential costs to society.  The order required agencies, including
EPA, to prepare a regulatory impact analysis that included a
cost-benefit analysis.  Agencies were to do this for proposed rules
that, among other things, were likely to result in an annual effect
on the economy of at least $100 million.\1 In such cases, an agency's
analysis was required to describe the benefits--expressed in monetary
terms, if possible--as well as the potential costs.  If the analysis
did not show that benefits exceeded costs, the agency was to explain
any legal reason why the regulation should still be promulgated. 

Cost-benefit analyses were expected to conform to guidelines
developed by the Office of Management and Budget and EPA.  The
guidelines allowed EPA considerable flexibility in estimating its
benefits.  They stated, among other things, that the scope and
precision of analysis should depend on the specific requirements of
authorizing legislation, the quality of underlying data, the
scientific understanding of the problems to be addressed through the
regulation, and resource constraints at EPA. 

EPA, according to officials, was faced with such resource
constraints.  It had, in effect, a court-ordered deadline for
completing its estimate of the monetary value of visibility
improvements expected from limiting the plant's emissions.  On the
basis of a 1982 lawsuit filed by environmental groups and a
subsequent settlement agreement and revisions to the settlement
agreement between EPA and these groups, EPA was under court order at
this time to determine whether a specific pollution source caused or
contributed to the visibility impairment at the Grand Canyon and, if
so, issue a finding to that effect by August 31, 1989.  In addition,
following any finding, EPA was to conduct a best available retrofit
technology (BART) analysis on the identified source.  And, if the
analysis indicated emissions controls would improve visibility at the
Grand Canyon National Park, EPA was to propose regulations requiring
their installation and use in order to achieve the emissions limit
representing BART.  Under the court order, EPA was to complete its
technology analysis by February 1, 1990.  This was less than 6 months
from August 31, 1989, when EPA was required to issue its finding as
to whether NGS was a source of impairment.\2

EPA concluded that it did not have time to complete original research
to estimate a monetary value of the specific visibility improvements
expected from emissions controls at the plant.  The agency chose,
instead, to extract the monetary value from the results of existing
contingent valuation research related to visibility changes at the
Grand Canyon.  EPA's decision to estimate benefits based on
contingent valuation was, according to a former EPA official who was
the project economist for this rulemaking, partially an attempt to
foster a wider review of the use of the contingent valuation
methodology so that, if accepted, it could be used on other
environmental policy issues and regulatory decisions. 

The existing study, "Preservation Values for Visibility Protection at
the National Parks," was partially funded by EPA through a
cooperative agreement with the University of Colorado Center for
Economic Analysis and performed by the research firm of RCG/Hagler,
Bailly, Inc.\3 The existing study was designed to advance the state
of the art in estimation of use and non-use values because existing
methods were considered to be quite limited when the need for such
values was increasing for reasons including Executive Order 12291
requirements.  The researchers were Lauraine G.  Chestnut and Robert
D.  Rowe (EPA researchers).\4 EPA selected this study, from among
others, because it included many recent methodological developments
intended to respond to earlier criticisms of the contingent valuation
methodology for valuing visibility conditions.  EPA also selected
this study because its estimates of the monetary value of visibility
improvements were conservative when compared with another earlier
study's. 

The plant owners also used contingent valuation to estimate a
monetary value for the visibility benefits expected from limiting the
plant's emissions.  In response to EPA's use of the existing study
and monetary value estimate, the owners decided to conduct their own
study and contracted with a research firm, Decision Focus,
Incorporated, to do so. 


--------------------
\1 Executive Order 12291 was subsequently replaced by Executive Order
12866.  Executive Order 12866 similarly requires agencies to assess
benefits and costs for regulatory actions that may, among other
things, have an annual effect on the economy of $100 million or more. 

\2 On January 9, 1990, the court extended the deadline for the
proposed rule on whether or not to require the BART emissions limit
to February 1, 1991, and the final rule to October 1991, or 6 months
after the close of the 60-day comment period for the proposed rule. 

\3 EPA used a second study to estimate the monetary value of
visibility improvements to persons who directly use the Grand Canyon
National Park:  Schulze, Brookshire, Walther, and Kelley, "The
Benefits of Preserving Visibility in the National Parklands of the
Southwest," Methods Development for Environmental Control Benefit
Assessment, EPA, Vol.  VIII (1981).  Because the values in this study
represent only 1 percent of EPA's total estimated monetary value of
visibility benefits, we do not discuss this research as part of this
appendix. 

\4 While not associated with the original study, a third RCG/Hagler,
Bailly researcher, Marina Skumanich, was involved with the
extrapolation. 


   THE STUDIES VALUED VISIBILITY
   IMPROVEMENTS
------------------------------------------------------- Appendix III:2

Both the study EPA used and the owners' study set out to value
visibility improvements at the Grand Canyon National Park.  The
studies, however, valued different degrees of visibility improvement. 
That is, the study on which EPA based its estimated monetary value
was intended to value a much broader visibility issue than the
wintertime visibility improvements expected from emissions controls
at the plant.  The owners' study, on the other hand, set out to value
wintertime visibility improvements the owners expected would result
from controlling their plant's emissions. 

EPA expected, as stated in its proposed rule for a 90-percent
emissions limit, that there would be an approximately 14 percent
improvement in the winter seasonal average visibility at the Grand
Canyon.  The improvement was expected to occur over a 30-year period,
which was EPA's estimate for the remaining useful life of the plant. 
However, the study on which EPA based its estimate valued changes in
annual average visibility that would last forever at several
individual national parks, including the Grand Canyon.  The
photographs used in the study, which survey respondents were asked to
value, were labeled summer days.  And the broader study valued
different visual range improvements than EPA expected would occur
from limiting emissions:  a 61-percent improvement in visual range; a
29-percent improvement in visual range; and a 26-percent degradation
in visual range. 

The owners' contingent valuation study was specifically designed to
measure the wintertime visibility improvements expected from
emissions controls at the plant.  This study asked respondents to
value five different scenarios of visibility improvements. 
Interviewers described, for respondents, the expected visibility
improvements of each scenario and showed them photographs that
illustrated the improvements. 

The five scenarios, shown in table III.1, were chosen by Decision
Focus to relate to potential regulatory actions by EPA.  The last
three represent the different types of winter improvements Decision
Focus hypothesized would result from emissions controls at the plant. 



                              Table III.1
                
                Visibility Improvement Scenarios Valued
                          by the Owners' Study

Scenario                Expected visibility improvements
----------------------  ----------------------------------------------
Summer and winter       Large, visually apparent improvements during
                        both the summer and winter in both clear and
                        other weather conditions

Summer only             Visually apparent improvements only during the
                        summer

Winter only             Visually apparent improvements only during the
                        winter

20 winter-weather days  Visually apparent improvements only during 20
                        days of layered haze that occur during some
                        weather events

10 winter-weather days  Visually apparent improvements only during 10
                        days of layered haze that occur during some
                        weather events
----------------------------------------------------------------------
Source:  Balson, Hausman, and Hulse, Decision Focus, Incorporated,
"Navajo Generating Station (NGS) BART Analysis" 1991. 


   SURVEY INSTRUMENT DEVELOPMENT
------------------------------------------------------- Appendix III:3

We found that the researchers for both the study EPA used and the
owners' study made very thorough efforts in developing their survey
instruments.  They followed accepted survey research standards to
ensure the validity of their survey instruments.  As a result of
these efforts, the survey instruments should have measured the
visibility concepts the researchers intended them to measure and with
wording that the researchers found to be most effective for their
studies' purposes.  And, as one would expect because the studies were
intended to measure different visibility improvements, the survey
instruments provided respondents different information about what
they were to value and used different photographs to demonstrate
visibility improvements. 

Careful survey design, in our view, is critical to averting problems
with bias or comprehension.  It is needed because people, on whom
these researchers relied to value visibility changes, are complex and
their reactions to specific words or concepts are not always
predictable.  If the right questions are not asked or if questions
are not asked in the right way, researchers are less likely to obtain
high-quality results.  Asking the right questions in the right way is
both science and art.  It is a science because it is guided by
empirical evidence and uses many scientific principles developed from
various fields of applied psychology, sociology, cognitive research,
and evaluation research.  It is an art because it requires
anticipating the respondents' interactions with the survey
instrument. 

Both research groups pretested their survey instruments to avoid bias
and comprehension problems.  Pretesting involves administering survey
questions to people who represent the population to be surveyed. 
This can be discussions with focus groups or in-person or telephone
interviews and is intended to identify problems that researchers can
correct before administering their survey instrument to a larger
group.  The researchers for the study EPA used held two rounds of
pilot tests, each involving about 10 respondents.  Then, after
revising the survey instrument, the researchers had it peer reviewed
by sociologists familiar with issues concerning national park
visitors and survey design issues, economists familiar with
contingent valuation, and an atmospheric scientist familiar with
visibility.  They then hired professional interviewers to conduct a
final pretest with 20 respondents.  The owners' researchers held two
rounds of focus groups to explore basic assumptions about visibility
improvements, each followed by a round of telephone interviews. 
Then, after analyzing the information gathered, they conducted two
more rounds of focus groups and preliminary in-person interviews. 
Drawing on the information gathered, the researchers then developed a
survey instrument that they revised following two more rounds of
focus groups, two rounds of test interviews, and finally a pretest
with 22 respondents that was conducted by professional interviewers. 


      CONTEXTUAL INFORMATION AND
      THE USE OF PHOTOGRAPHS
----------------------------------------------------- Appendix III:3.1

For contingent valuation surveys to elicit useful information about
respondents' willingness to pay for specific environmental
improvements, we believe researchers must ensure that the respondents
understand exactly what they are being asked to value.  In deciding
what kind and how much information to provide respondents,
researchers must weigh providing enough and properly ordered
information with the possibility of overloading respondents or being
criticized for trying to lead respondents. 

The survey instruments, for both the study EPA used and the owners'
study, provided respondents different background information and used
different photographs to depict changes in visibility.  The obvious
reason for these differences, in our view, is that EPA used existing
research designed to value different visibility changes than those
expected because of emissions controls at the plant.  Therefore,
agency researchers could not have been expected to describe the
environmental improvement expected from emissions controls at the
plant.  Another important reason is that contextual information and
photographs are matters of researcher choice and are an area where
contingent valuation may be more art than science. 


         CONTEXTUAL INFORMATION
--------------------------------------------------- Appendix III:3.1.1

The nature of the information the research groups provided
respondents was different.  For example, in the owners' study, before
respondents were asked to value five levels of visibility
improvements, they were given specific background information.  Among
other things, the respondents were told that

  -- on high visibility days, one can see more than 100 miles at the
     Grand Canyon;

  -- the rural southwest has some of the clearest air in the country;

  -- the actual amount of pollution at the Grand Canyon National Park
     is very low compared with the amount in cities;

  -- most visitors come in the summer period; and

  -- if any of the programs to improve visibility that the
     respondents were asked to value were implemented, certain older
     power plants, already meeting all current state and national air
     pollution standards, would have to install and maintain new
     equipment to remove pollutants. 

These background statements, in our view, might have caused
respondents to minimize their concern over visibility problems at the
Grand Canyon and accordingly to assign lower willingness-to-pay
values for visibility improvements.  On the other hand, these
statements could be exactly what the respondents needed in order to
understand what they were to value. 

Another example of contextual information is from the study EPA used. 
In that case, prior to asking respondents to value visibility
changes, the researchers first introduced respondents to several
nonvisibility effects of air pollution at national parks.  Before
asking the respondents to value visibility changes, the researchers
first asked them to prioritize nonvisibility effects that were
happening or could happen in national parks due to people's
activities outside park boundaries, for example, injury to vegetation
and historic structures from air pollution.  Then, later in the
questionnaire, following the valuation questions, respondents were
asked to separate, from their willingness-to-pay values, any amount
they had included for nonvisibility improvements. 

A possible effect of introducing these additional results of air
pollution before the valuation questions, in our view, might have
been that respondents assigned higher willingness-to-pay values than
they might otherwise have.  And subsequent efforts to separate out
any inflated amounts might not have been successful.  On the other
hand, introducing nonvisibility issues might have been a critical
step in ensuring that respondents valued only visibility
improvements, that by identifying these other, nonvisibility,
effects, respondents might have been better able to value only
visibility changes. 

Another aspect of contextual information is its level of detail.  The
study EPA used and the owners' study were very different in terms of
their level of detail.  The owners' researchers, knowing they were to
value visibility improvements from emissions controls at the plant,
were able to give respondents very specific descriptions of
visibility conditions and ask very specific questions about the
visibility improvements.  In contrast, EPA's researchers provided
respondents more general descriptions of visibility conditions and
possible events that might change them. 

This difference in detail, we believe, could cause substantial
variation in the values the studies' respective respondents placed on
visibility improvements, depending upon the cognitive patterns of the
respondents.  A test, administering the two survey instruments to
randomly selected samples from the same population, would be needed
to determine the effects due solely to the level of detail in the
survey instruments. 


         THE USE OF PHOTOGRAPHS
--------------------------------------------------- Appendix III:3.1.2

The researchers for both the study EPA used and the owners' study
used photographs to illustrate the visibility improvements
respondents were to value.  The characteristics of the photographs
they chose (e.g., size or season) were different.  The effect, if
any, these differences had on the values respondents assigned to
visibility improvements is not known.  Testing would be needed to
determine the effects due solely to the differences in photographs. 

The selection of photographs, according to a former official who was
EPA's project economist for this rulemaking, demonstrates a challenge
in accurately depicting what it is respondents are to value.  At a
minimum, according to this official, problems stemming from the
selection of photographs can increase the uncertainty of the results
and provide another avenue for criticism of results.  At worst,
problems can yield biased results with an unknown direction of bias. 

Both research groups said they selected their photographs to minimize
bias.  EPA's researchers used NPS photographs that represented four
visibility conditions on summertime days:  15 percent of the
summertime days (the best conditions), 20 percent, 40 percent, and 25
percent (the poorest conditions).\5 The photographs were taken at the
same time each day and selected to minimize variations in extraneous
factors such as clouds and snow.  While the owners' researchers also
used NPS pictures and selected them with the assistance of a leading
visibility scientist from NPS, the pictures they chose were very
different.  The owners' researchers selected photographs representing
different seasons and weather categories, for example, summer clear
skies, winter clear skies, winter overcast, and winter layered haze
events.  These photographs were also selected to represent issues,
including the change in the Grand Canyon's appearance from season to
season (primarily due to the change in the sun's angle), summer
afternoon thunderstorms' tendency to frequently obscure views, and
the impact the time of day has on the appearance of vistas. 

In addition to different weather patterns, the researchers used
different numbers, sizes, and qualities of photographs.  The EPA
researchers used four photographs printed to be mailed to
respondents--each picture was 3 by 5 inches.  The owners' researchers
used 12 pictures, measuring 8 by 12 inches, mounted on display boards
to be shown to respondents during in-person interviews.  We believe
these differences could cause substantial variation in the values
respondents assigned for visibility improvements.  A test showing the
different photographs to two randomly selected samples, from the same
population, would be needed to determine the effects due solely to
the differences in photographs. 


--------------------
\5 While presented as being typical of summertime conditions,
according to EPA, the photographs were taken on four different days
in 1985:  January 17, April 2, June 8, and July 13. 


   SURVEY ADMINISTRATION
------------------------------------------------------- Appendix III:4

EPA's and the owners' researchers administered their survey
instruments in different ways.  EPA's researchers used mail
questionnaires to contact the 710 respondents in its study,\6

while the owners' researchers contacted 202 respondents in-person. 
There are trade-offs when choosing between these two survey
techniques.  Both techniques have their strengths and weaknesses. 

In-person interviews, in our view, have both strengths and
weaknesses.  Strengths of in-person interviews include researchers'
being able to control the amount of information respondents have
available when answering specific valuation questions and
interviewers' ensuring that survey questions are asked in the exact
order the researchers intended.  Because in-person interviews are
conducted in one setting, respondents are less likely to be
interrupted by outside events, for example, personal or family
illness, that might change their perspective when answering
questions.  Furthermore, these interviews are also generally more
successful with respondents whose reading levels are low in
comparison to the complexity of the questions.  The weaknesses of the
method include higher costs because interviewers not only must be
trained, but they must also travel to and from interviews--some of
which may not be successful.  In addition, interviewers, by their
presence, may affect how respondents answer questions.  For example,
respondents may provide an answer they believe the interviewer wants
or give any answer just to further the interview process. 

Strengths of mail questionnaires include being substantially cheaper
than in-person interviews.  Being less expensive, mail questionnaires
can be sent to larger samples than may be possible with in-person
interviews and, as such, may be more appropriate for research issues
requiring nationwide results, such as the valuation of visibility
improvements at the Grand Canyon National Park.  Mail questionnaires
allow respondents time to carefully consider each question and their
response.  Weaknesses of the method include the possibility of
respondents' having more information than researchers intend them to
have when they answer a specific question (because they can skip back
and forth between questions or read ahead).  Also, when mail
questionnaires are used, there is no one who can assess for
researchers whether respondents understand the questions or what it
is they are to value.  While some in the research community tend to
prefer in-person interviews for contingent valuation surveys, mail
questionnaires have not been proven to be a less valid technique for
collecting data. 


--------------------
\6 In its original study, EPA's researchers mailed questionnaires to
3,345 households and received 1,647 mail responses.  There were 710
responses relevant to the issue at hand. 


   WILLINGNESS-TO-PAY RESULTS AND
   HOW THEY WERE CALCULATED
------------------------------------------------------- Appendix III:5

While both studies showed respondents were willing to pay some amount
for visibility improvements at the Grand Canyon, the researchers used
different techniques to calculate their willingness-to-pay values. 
EPA, in extrapolating from the results of the earlier contingent
valuation study, made various assumptions and judgments about how to
translate values elicited for larger benefits to values for the
narrower, specific benefits of this case.  And the plant's owners, in
calculating results from their pilot study, used a data-trimming
technique that removed a fixed amount of data from the calculations. 
These techniques added uncertainty and possible bias to the study
results.  Additional uncertainty may have also been added through
adjustments both groups of researchers made in response to the final
results of visibility studies that showed less visibility improvement
than the studies valued.  Any uncertainty in these results was
magnified when both groups of researchers projected their
willingness-to-pay values to the nation without conducting national
samples.  EPA then projected its nationwide results to the number of
years it expected the regulation to be in effect. 


      EPA'S EXTRAPOLATION PROCESS
      AND RESULTS
----------------------------------------------------- Appendix III:5.1

EPA's goal was to identify that portion of the broader study's
willingness-to-pay values relevant to the expected visibility
improvements from emissions controls at the plant.  To accomplish
this, EPA's researchers did the following: 

  -- created a database of the results from the original research
     that were pertinent to visibility at the Grand Canyon.  The
     original research used six surveys, three of which contained
     questions about visibility at the Grand Canyon National Park. 
     The results of the three survey instruments were what
     researchers combined for the database. 

  -- determined, for the database, the relationship between
     visibility improvements and the willingness to pay for the
     improvements.  The original study made three willingness-to-pay
     estimates--one each for a 61-percent visibility improvement, a
     29-percent visibility improvement, and a 26-percent degradation
     of conditions.  The researchers calculated a mean
     willingness-to-pay value for each of these levels of
     improvement. 

  -- determined for each level of improvement, using regression
     analysis, the relationship between the individual
     willingness-to-pay values and other factors such as respondents'
     age, household income, gender, and history of visiting national
     parks, which were important in determining the willingness to
     pay. 

  -- using this empirical relationship, predicted willingness-to-pay
     values for the approximately 14 percent visual range improvement
     that EPA initially expected would occur from the addition of
     emissions controls at the plant. 

  -- using both sensitivity analyses and comparisons to related past
     contingent valuation studies, tested the validity of their
     predicted willingness-to-pay values.\7

EPA's researchers' estimate of the monetary value of visibility
improvements, expected from a 90-percent emissions reduction, ranged
from $1.30 to $2.50\8 per year per U.S.  household.  EPA recognized
that extrapolation, by definition, added considerable uncertainty to
the resulting values.  Nevertheless, EPA believed that the results
were sufficient to serve as indicators of the direction (i.e.,
negative or positive) and the order of magnitude (i.e., whether the
values were in millions, tens of millions, or hundreds of millions)
of the values. 


--------------------
\7 For their sensitivity analysis, EPA's researchers varied critical
assumptions, including visibility improvements, on-site use and
preservation value functional forms, population, growth in the number
of park visitors, and the remaining useful life of the plant.  The
researchers also used results of other visibility studies to test the
validity of their willingness-to-pay predictions.  The researchers
considered the results of other contingent valuation studies that
discussed whether individuals might be more or less concerned about
visibility impairment (event days) and the severity of the events. 
From this research, the researchers concluded that converting from
estimates of event day impacts to estimates of associated changes in
annual average conditions was likely to cause a downward bias
(understate the results), if any bias. 

\8 Values are expressed in 1988 dollars. 


      OWNERS' DATA-TRIMMING
      PRACTICE AND RESULTS
----------------------------------------------------- Appendix III:5.2

The owners' researchers calculated the mean willingness-to-pay values
for households for each of the five visibility programs included in
their study.  They calculated these values using a data-trimming
procedure that involved removing a fixed amount of data (first the
highest and lowest 5 percent of the willingness-to-pay values and
then the highest and lowest 10 percent of the willingness-to-pay
values) from both ends of the data distribution curve and then
calculating "trimmed means." According to the researchers, they used
trimmed means because the ordinary means of the untrimmed data were
grossly distorted by a very small number of outliers.  According to
the researchers, the ordinary mean is the correct statistic under
traditional welfare economic theory, if one is willing to ignore
distribution consequences, that is, accept a program in which--in the
worse case--all the benefits are accrued by one individual. 
Trimming, according to the researchers, is an alternative that avoids
this extreme case and results in willingness-to-pay value being based
on a more central part of the distribution.  EPA's project economist
for this rulemaking told us that data trimming in this case was
problematic because the distribution of the study's results was
highly skewed, with 90 percent of the willingness-to-pay values being
$0. 

Data trimming eliminated some results from respondents who said that
they would pay a large amount for visibility improvements and some
results from respondents who said that they would pay nothing for
visibility improvements.  This practice greatly affected the owners'
results.  For example, for visibility improvements on 20 winter days,
the untrimmed mean of the willingness-to-pay distribution was $2.38. 
This is compared with $0.50 for the 5-percent trimmed mean and $0.02
for the 10-percent trimmed mean.  Table III.2 shows these results for
each of the five visibility programs the owners' research examined. 



                              Table III.2
                
                  Comparison of the Willingness-to-Pay
                  Values Estimated for Five Levels of
                 Visibility Improvements Studied by the
                          Owners' Researchers

                        Summer                            20        10
Estimated WTP\a            and    Summer    Winter    winter    winter
values                  winter      only      only      days      days
--------------------  --------  --------  --------  --------  --------
Mean WTP                $27.78    $15.71     $6.34     $2.38     $2.28
5% trimmed mean WTP     $20.20    $10.51     $2.92     $0.50     $0.46
10% trimmed mean WTP    $16.15     $8.11     $1.25     $0.02     $0.00
Median WTP              $10.00     $0.00     $0.00     $0.00     $0.00
----------------------------------------------------------------------
\a Willingness to pay (WTP) is the dollar amount respondents were
willing to pay for the visibility improvements.  We are not certain
in what year's dollars these values are expressed. 

Source:  Balson, Hausman, and Hulse, Decision Focus, Incorporated,
"Navajo Generating Station (NGS) BART Analysis" (1991). 


      ADDITIONAL UNCERTAINTY FROM
      ADJUSTMENTS
----------------------------------------------------- Appendix III:5.3

Both EPA's and the owners' researchers made additional adjustments to
their willingness-to-pay results to reflect the final results of
visibility studies--results that were not available at the time they
began their studies.  These adjustments may have added additional
uncertainty to the final results. 

EPA, as previously discussed, was operating under a court-ordered
deadline and began its analysis using the best available, rather than
final, estimates of expected visibility improvements.  So when the
final results became available and were significantly less than the
preliminary results its researchers had used to estimate the value of
visibility improvements (a winter seasonal average visibility
improvement of approximately 7 percent instead of the approximately
14 percent used in the analysis), EPA scaled its willingness-to-pay
estimates downward.  And while the EPA researchers (at EPA's request)
had designed the computational formulas so that results could be
revised when final visibility improvement estimates became available,
they also recognized the possibility of adding uncertainty to the
results.  The researchers said that uncertainties could be added to
results if willingness-to-pay values drop off dramatically at some
point when visibility changes from summer to winter or from most days
to some days.  However, they also said that while there was no
evidence of such a dramatic drop in mean values, available evidence
on the question is quite limited.  The revised willingness-to-pay
estimates ranged from $0.75 to $1.75 annually per household.\9

The owners' researchers, faced with the same time constraints, also
were required to begin their research while awaiting final estimates
of the visibility improvements expected from emissions controls. 
Then, after their visibility research efforts were completed, the
owner-funded visibility study concluded that visibility improvements
from emissions controls would be less than any of the scenarios
valued.  The study concluded that with a 90-percent emissions
reduction, visibility would improve (at least perceptibly) on 6 days. 
Therefore, the researchers extrapolated their monetary value from the
values they had estimated for higher degrees of visibility
improvement.  The researchers' final report did not explicitly state
a willingness-to-pay value per U.S.  household.\10 Rather, the report
indicated a total public value of $2.3 million--this equates to
approximately $0.023 per household.\11 A senior associate at Decision
Focus, Incorporated, was unable to provide specific details on how
the final calculations were made because of the time that had passed
since the study was completed. 


--------------------
\9 Values are for 2000 expressed in 1992 dollars. 

\10 Balson, Hausman, and Hulse, Decision Focus, Incorporated: 
"Navajo Generating Station (NGS) BART Analysis" (1991). 

\11 The total public value is based on 100 million U.S.  households,
which the researchers expected in 1995.  We are not certain in what
year's dollars the amount is expressed. 


      UNCERTAINTY IS MAGNIFIED
      WHEN RESULTS ARE PROJECTED
----------------------------------------------------- Appendix III:5.4

Any uncertainty in willingness-to-pay values is magnified, in our
view, when the results are used to project a nationwide value and
applied to the entire period of time to be affected by the regulatory
action.  While neither group of researchers had a nationwide sample,
both projected their results to the nation as a whole.  EPA
additionally projected its results to the entire time period the
regulation would be in effect. 


         SAMPLING STRATEGY
--------------------------------------------------- Appendix III:5.4.1

Neither EPA's nor the owners' researchers had sampling strategies
that allow nationwide results.  Faced with resource constraints, EPA
sacrificed its ability to obtain nationally representative values by
choosing to extrapolate from existing research.  And while the
plant's owners planned to have nationwide results, time and resource
constraints contributed to their not completing the planned study. 

The contingent valuation study from which EPA extrapolated surveyed a
sample drawn from residents of five states:  Arizona, California,
Missouri, New York, and Virginia.  These states were selected so that
there would be variation in the distances between respondents'
residences and the national parks studied.  In addition, Arizona,
California, and Virginia were selected because they were states with
national parks being studied.  For each of the selected states, a
survey instrument was mailed to respondents whose names were selected
from national databases drawn from drivers' licenses, car and voter
registrations, and other sources.  Although different surveys were
sent to the different states, with different questions about the
different national parks, EPA extrapolated from the 710 responses
that pertained to the Grand Canyon National Park. 

The EPA researchers agreed that their sample did not technically
allow a reliable assessment of the U.S.  population.  However, they
believed that they had compensated for the partial sample by
adjustments they made to the willingness-to-pay estimates.  These
adjustments were intended to account for socioeconomic differences
(e.g., household income, age, sex, and distance of residence from the
parks in question) between their sample and the U.S.  population. 
Nonetheless, the researchers also said that they were unable to
calculate the expected level of error in the sample. 

The plant owners' sample was drawn from households in two
counties--San Diego and St.  Louis.  This sample, however, was for a
pilot study that tested a survey instrument.  With appropriate
revisions, the survey was to be administered to a much larger sample
from which national results could be drawn.  The owners' researchers
selected the two counties to provide different settings.  Then, to
select households for interview, the researchers first randomly
selected blocks within the counties.  And second, beginning with a
random predesignated starting point and proceeding in a predetermined
manner, they went from household to household until they had
conducted five interviews with heads of households that met
established age and gender quotas.  The researchers set these quotas
to ensure that men and young people were interviewed since women and
older people are easier for researchers to locate.  In total, 202
persons were interviewed. 

The owners' researchers recognized that their sample was not a
national sample and planned to conduct a national sample.  However,
according to a senior associate at Decision Focus, by the time they
completed the pilot, there was not enough time remaining to complete
and summarize a national sample so it could be used in EPA's
decision-making.  In addition to time and resource constraints,
according to an official of the Salt River Project--part owner and
the operator of the plant--the owners did not authorize the remaining
research because they did not see value in doing so. 


         SURVEY RESULTS WERE
         PROJECTED
--------------------------------------------------- Appendix III:5.4.2

While both EPA and the owners projected their per U.S.  household
monetary value to the nation as a whole, EPA also projected these
results to the number of years the regulation would be in effect (the
number of years the plant was expected to operate).  EPA estimated
that the monetary value of the visibility benefits would range from
$90 million to $200 million in 2000 (measured in 1992 dollars).  EPA
estimated the present value of the monetary benefit stream, as of
January 1992 (expressed in 1992 dollars and discounted using a
10-percent real rate), at $523 million to $970 million. 




(See figure in printed edition.)Appendix IV
COMMENTS FROM THE DEPARTMENT OF
THE INTERIOR
========================================================= Appendix III


MAJOR CONTRIBUTORS TO THIS REPORT
=========================================================== Appendix V

Jonathan T.  Bachman
Stephen M.  Brown
Daniel J.  Feehan
Sue E.  Naiberk
Cheryl L.  Pilatzke
Cynthia S.  Rasmussen
Victor S.  Rezendes
Pamela K.  Tumler


*** End of document. ***