Regulatory Reform: Agencies Could Improve Development, Documentation, and
Clarity of Regulatory Economic Analyses (Chapter Report, 05/26/98,
GAO/RCED-98-142).

Pursuant to a congressional request, GAO reviewed the extent to which
federal agencies are incorporating the best practices set forth in the
Office of Management and Budget's (OMB) guidance for preparing economic
analyses in accordance with Executive Order 12866 and the Unfunded
Mandates Reform Act (UMRA) of 1995.

GAO noted that: (1) some of the 20 economic analyses that it reviewed
did not incorporate the best practices set forth in OMB's guidance; (2)
for example, 5 of the 20 analyses did not discuss alternatives to the
proposed regulatory action, 6 did not assign dollar values to benefits,
and 1 did not assign dollar values to costs--all of which are practices
recommended by the guidance; (2) OMB's guidance gives agencies the
flexibility to decide how thorough their economic analyses should be;
(3) at the same time, the guidance stresses the importance of fully
disclosing the reasons for omissions, gaps, or other limitations; (4)
although GAO found many instances in which best practices were not
followed in the analyses, the reason for not following was disclosed in
only one instance; (5) in addition, eight of the economic analyses did
not include an executive summary that could help Congress,
decisionmakers, the public, and other users quickly identify key
information addressed in the analyses; (6) finally, only 1 of the 20
analyses received an independent peer review; (7) because Executive
Order 12866 and UMRA establish nearly identical requirements for
economic analyses and because agencies typically use the same analyses
to comply with both when UMRA is applicable, GAO's findings reflect the
extent to which the nine analyses called for under UMRA satisfy the
act's as well as the executive order's requirements for economic
analyses; (8) according to agency officials, economic analyses play a
valuable role in regulatory decisionmaking; (9) twelve of the 20
analyses were used to help identify the most cost-effective of several
similar alternatives or to cost-effectively implement health-based
regulations; (10) seven other analyses were used to define a
regulation's scope and implementation date, document and defend
regulatory decisions, or reduce a health risk at a feasible cost; and
(11) one analysis played almost no role in decisionmaking because,
according to agency officials, the authorizing statute was so
prescriptive that the agency was left with virtually no discretion in
developing the implementing regulation.

--------------------------- Indexing Terms -----------------------------

 REPORTNUM:  RCED-98-142
     TITLE:  Regulatory Reform: Agencies Could Improve Development, 
             Documentation, and Clarity of Regulatory Economic
             Analyses
      DATE:  05/26/98
   SUBJECT:  Proposed legislation
             Executive orders
             Economic analysis
             Cost effectiveness analysis
             Regulatory agencies
             Agency proceedings
             Policy evaluation
             Reporting requirements

             
******************************************************************
** This file contains an ASCII representation of the text of a  **
** GAO report.  Delineations within the text indicating chapter **
** titles, headings, and bullets are preserved.  Major          **
** divisions and subdivisions of the text, such as Chapters,    **
** Sections, and Appendixes, are identified by double and       **
** single lines.  The numbers on the right end of these lines   **
** indicate the position of each of the subsections in the      **
** document outline.  These numbers do NOT correspond with the  **
** page numbers of the printed product.                         **
**                                                              **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced.  Tables are included, but    **
** may not resemble those in the printed version.               **
**                                                              **
** Please see the PDF (Portable Document Format) file, when     **
** available, for a complete electronic file of the printed     **
** document's contents.                                         **
**                                                              **
** A printed copy of this report may be obtained from the GAO   **
** Document Distribution Center.  For further details, please   **
** send an e-mail message to:                                   **
**                                                              **
**                                            **
**                                                              **
** with the message 'info' in the body.                         **
******************************************************************


Cover
================================================================ COVER


Report to the Committee on Governmental Affairs, U.S.  Senate

May 1998

REGULATORY REFORM - AGENCIES COULD
IMPROVE DEVELOPMENT,
DOCUMENTATION, AND CLARITY OF
REGULATORY ECONOMIC ANALYSES

GAO/RCED-98-142

Regulatory Reform

(160418)


Abbreviations
=============================================================== ABBREV

  DOT - Department of Transportation
  EPA - Environmental Protection Agency
  FDA - Food and Drug Administration
  OMB - Office of Management and Budget
  OSHA - Occupational Safety and Health Administration
  UMRA - Unfunded Mandates Reform Act
  USDA - U.S.  Department of Agriculture

Letter
=============================================================== LETTER


B-279614

May 26, 1998

The Honorable Fred Thompson
Chairman
The Honorable John Glenn
Ranking Minority Member
Committee on Governmental Affairs
United States Senate

This report responds to your request that we describe the extent to
which federal agencies are incorporating the best practices set forth
in the Office of Management and Budget's (OMB) guidance for preparing
economic analyses in accordance with Executive Order 12866 and the
Unfunded Mandates Reform Act of 1995.  The report discusses the
development, documentation, and use of economic analyses in agencies'
regulatory decision-making and contains recommendations to the
Director, OMB.  The recommendations are designed to enhance the
quality and credibility of agencies' economic analyses. 

We are sending copies of this report to other appropriate
congressional committees; the Director, OMB; the Secretaries of
Agriculture and Transportation; the Administrators of the
Environmental Protection Agency and the Occupational Safety and
Health Administration; and the Commissioner of the Food and Drug
Administration.  Copies are available to others upon request. 

If you or your staff have any questions, please call me at (202)
512-6111.  Major contributors to this report are listed in appendix
I. 

Peter F.  Guerrero
Director, Environmental
 Protection Issues


EXECUTIVE SUMMARY
============================================================ Chapter 0


   PURPOSE
---------------------------------------------------------- Chapter 0:1

The last 20 years have seen enormous growth in the number and scope
of federal regulations.  According to the Office of Management and
Budget (OMB), although these regulations have improved public health
and safety and environmental quality, their costs are high.  In 1996,
OMB estimated the costs of federal regulations at $200 billion
annually and the benefits at $300 billion.  To control the costs of
regulation, the administration has issued executive orders, including
Executive Order 12866, and the Congress has enacted laws, including
the Unfunded Mandates Reform Act of 1995 (UMRA).  These orders and
laws require federal agencies to prepare and use economic
analyses--also known as regulatory impact analyses--to assess the
benefits and costs of proposed actions before promulgating
regulations.  These analyses are intended to inform and improve the
regulatory process by identifying the likely costs and benefits of
feasible alternatives.  An interagency group convened by OMB has
developed guidance for implementing Executive Order 12866 and UMRA. 
This guidance sets forth best practices for preparing economic
analyses. 

To assist the Senate Committee on Governmental Affairs in carrying
out its regulatory oversight responsibilities, the Chairman and
Ranking Minority Member asked GAO to describe (1) the extent to which
federal agencies' economic analyses incorporate the best practices
set forth in OMB's guidance and (2) the agencies' use of these
analyses in regulatory decision-making. 


   BACKGROUND
---------------------------------------------------------- Chapter 0:2

In 1993, President Clinton issued Executive Order 12866, the most
recent of several executive orders requiring federal agencies to
conduct economic analyses when developing regulations.  Under the
order, an agency must conduct an economic analysis of a planned
regulation and alternatives to it for an economically significant
rule--one that may have an annual effect on the economy of $100
million or more.  In 1995, the Congress enacted UMRA, which imposes a
statutory requirement on federal agencies to conduct benefit-cost
analyses of planned regulations.  UMRA's scope differs slightly from
the scope of the executive order.  Specifically, the act requires
analyses for proposed or final rules that may result in the
expenditure of $100 million or more in any one year, either by state,
local, and tribal governments in the aggregate or by the private
sector alone.  Most recently, Senators Thompson and Levin introduced
a bill (S.  981) that would, among other things, require executive
summaries and peer reviews for economic analyses.  In the past, GAO
has recommended executive summaries for economic analyses to enhance
their clarity, and peer reviews to enhance their quality and
credibility. 

In 1996, OMB issued a document describing best practices for
preparing economic analyses under Executive Order 12866 and UMRA. 
These best practices include considering the most important
alternative approaches to the problem, analyzing the benefits and
costs of these alternatives, and fully disclosing information about
the analysis, including the underlying uncertainties and assumptions. 

GAO included in this review all economically significant proposed and
final rules issued between July 1996 and March 1997 that addressed
environmental, health, and safety matters.  As a result, GAO reviewed
the economic analyses used in promulgating 20 regulations by five
agencies--the Departments of Agriculture and Transportation, the
Environmental Protection Agency (EPA), the Food and Drug
Administration (FDA) within the Department of Health and Human
Services, and the Occupational Safety and Health Administration
(OSHA) within the Department of Labor.  Nine of these regulations
involved potential expenditures large enough to bring the regulations
within the scope of UMRA. 


   RESULTS IN BRIEF
---------------------------------------------------------- Chapter 0:3

Some of the 20 economic analyses that GAO reviewed did not
incorporate the best practices set forth in OMB's guidance.  For
example, 5 of the 20 analyses did not discuss alternatives to the
proposed regulatory action, 6 did not assign dollar values to
benefits, and 1 did not assign dollar values to costs--all of which
are practices recommended by the guidance.  OMB's guidance gives
agencies the flexibility to decide how thorough their economic
analyses should be.  At the same time, the guidance stresses the
importance of fully disclosing the reasons for omissions, gaps, or
other limitations.  Although GAO found many instances in which best
practices were not followed in the analyses, the reason for not
following was disclosed in only one instance.  In addition, eight of
the economic analyses did not include an executive summary that could
help the Congress, decisionmakers, the public, and other users
quickly identify key information addressed in the analyses.  Finally,
only 1 of the 20 analyses received an independent peer review. 
Because Executive Order 12866 and UMRA establish nearly identical
requirements for economic analyses and because agencies typically use
the same analyses to comply with both when UMRA is applicable, GAO's
findings reflect the extent to which the nine analyses called for
under UMRA satisfy the act's as well as the executive order's
requirements for economic analyses. 

According to agency officials, economic analyses play a valuable role
in regulatory decision-making.  Twelve of the 20 analyses were used
to help identify the most cost-effective of several similar
alternatives or to cost-effectively implement health-based
regulations.  Seven other analyses were used to define a regulation's
scope and implementation date, document and defend regulatory
decisions, or reduce a health risk at a feasible cost.  One analysis
played almost no role in decision-making because, according to agency
officials, the authorizing statute was so prescriptive that the
agency was left with virtually no discretion in developing the
implementing regulation. 


   PRINCIPAL FINDINGS
---------------------------------------------------------- Chapter 0:4


      SOME ECONOMIC ANALYSES
      LACKED FULL DISCLOSURE
-------------------------------------------------------- Chapter 0:4.1

For 15 of the 20 regulations that GAO reviewed, the agencies included
at least one alternative to the proposed action, but in some
instances, the discussion of the alternative was limited.  For the
five remaining regulations, no evidence was available to show that
the agencies had considered alternatives.  Agency officials stated
that for these five analyses, the agencies either had considered
alternatives but had not included them in the analyses or had not
considered alternatives at all.  Agency officials' reasons for not
addressing or considering alternatives included the specificity of
the authorizing legislation or the need to issue regulations quickly. 
Although OMB's guidance states that these can be legitimate reasons
for agencies to limit the consideration of alternatives, the guidance
also states that even when such limitations apply, agencies should
provide some analysis of alternatives to provide decisionmakers with
information for judging the consequences of statutory constraints. 

Nineteen of the economic analyses assigned dollar values to some
costs, and 14 assigned dollar values to some benefits.  Similarly, 15
of the analyses discussed the uncertainties associated with the
estimates of benefits and costs, but none of the remaining 5 analyses
explained why they did not discuss the uncertainty associated with
the estimated benefits and costs. 

The clarity of the 20 analyses varied, making it difficult at times
to determine where or whether elements of OMB's guidance were
discussed.  Eight of the analyses did not include an executive
summary.  GAO has previously recommended that EPA's analyses, and S. 
981 would require that all agencies' analyses, contain an executive
summary that clearly describes the results of the economic analysis
and the key points of the analysis.  Only one of the 20 analyses
underwent an independent peer review.  GAO has previously stated that
EPA should use peer review to help ensure the quality and credibility
of an analysis.  While a similar requirement for peer review for all
agencies would entail some costs, as OMB has observed, peer review by
independent experts--either internal or external to the agency--could
be tailored to reflect the importance, sensitivity, and
innovativeness of the analysis and of the associated regulatory
decision. 


      AGENCIES OFTEN USED ECONOMIC
      ANALYSES TO IDENTIFY
      COST-EFFECTIVE APPROACHES
-------------------------------------------------------- Chapter 0:4.2

According to agency officials, the analyses were most frequently used
to identify the most cost-effective approach within a fairly narrow
range of options.  For example, EPA used its economic analysis for a
rule on marine engine emissions to examine the costs of different
emission levels and to select the most cost-effective level.  Four
other analyses were used primarily to help agencies better define a
rule's coverage or to determine when to implement a rule.  For
example, EPA's economic analysis for a proposed rule on procedures
for testing emissions from motor vehicles incorporated data provided
by the automobile industry and led to revisions that gave the
industry additional time to implement the final rule.  Two analyses
were used principally to help agencies document or justify decisions
that they had already made.  According to agency officials, specific
statutory requirements limited their discretion in making regulatory
decisions and were a primary reason why economic analyses played a
limited role in regulatory decision-making.  For example, the Clean
Air Act of 1990 directed EPA to review and revise its regulations on
motor vehicle testing to better reflect actual driving conditions. 


   RECOMMENDATIONS
---------------------------------------------------------- Chapter 0:5

To strengthen the clarity and credibility of the economic analyses
required for regulatory decision-making, GAO recommends that the
Director, Office of Management and Budget, amend the Office's
guidance to include additional elements, two of which are proposed in
S.  981.  Specifically, GAO recommends that the guidance be amended
to provide that economic analyses should

  -- address all of the best practices identified in OMB's guidance
     or state the agency's reasons for not addressing them;

  -- contain an executive summary that briefly and concisely (1)
     identifies all benefits and costs--both those that can be
     described quantitatively and those that can be described
     qualitatively; (2) describes the range of uncertainties
     associated with the benefits and costs; and (3) compares the
     reasonable alternatives considered by the agency; and

  -- undergo an appropriate level of internal or external peer review
     by independent experts and state the agency's basis for
     selecting that level. 


   AGENCY COMMENTS
---------------------------------------------------------- Chapter 0:6

GAO provided a draft of this report to the Office of Management and
Budget; the Departments of Agriculture and Transportation, EPA, FDA,
and OSHA.  GAO received comments from all of these agencies except
OSHA, which informed GAO that it had no comments on the draft.  The
agencies generally agreed with the information presented in the
report and concurred with GAO's recommendations calling for economic
analyses to address OMB's best practices and to include an executive
summary.  Although the agencies agreed with GAO that peer review can
be beneficial, they suggested that GAO clarify and expand its
discussion and recommendation on this issue to more clearly
acknowledge that agencies should have discretion in selecting an
appropriate level of peer review.  FDA urged GAO to delete this
recommendation, maintaining that such a requirement would likely make
it impossible for the agency to meet other statutory
responsibilities.  GAO has revised the discussion and recommendation
on peer review to clarify that agencies should have such discretion
but should also state their basis for selecting a given level of peer
review.  The agencies offered several technical and/or clarifying
comments, which GAO incorporated throughout the report as
appropriate. 


INTRODUCTION
============================================================ Chapter 1

Each year, federal agencies establish or revise rules and regulations
designed to promote, among other purposes, public health and safety
and environmental quality.  According to the Office of Management and
Budget (OMB), these regulations produce great benefits but also
impose great costs.  In 1997, OMB estimated annual benefits of about
$300 billion and annual costs of about $200 billion for federal
regulations in effect at that time.  Because of the magnitude of
these estimated values, as well as the effect of the rules on
individuals, firms, industries, and government agencies, the
executive branch and the Congress require federal agencies to prepare
and use economic analyses--also called regulatory impact analyses--in
their regulatory decision-making process.  These analyses are
intended to inform and improve the regulatory process by estimating
the likely benefits and costs of feasible alternatives and
identifying the alternative that has the greatest net benefits
(benefits minus costs).  Although the weight that the analyses should
receive in the decision-making process is the subject of some
disagreement, the analyses themselves are generally recognized as an
important and useful tool. 


   EXECUTIVE BRANCH'S EFFORTS TO
   IMPROVE THE REGULATORY PROCESS
---------------------------------------------------------- Chapter 1:1

Since 1971, a series of executive orders and directives by OMB have
required federal agencies to consider the benefits and costs
associated with individual regulations.  In February 1981, President
Reagan issued Executive Order 12291, which required federal agencies
to prepare economic analyses identifying the benefits, costs, and
alternatives for all proposed and final major rules that the agencies
issued.  A major rule was defined as any regulation that was likely
to result in (1) an annual effect on the national economy of $100
million or more; (2) a major increase in costs or prices for
consumers, industries, governments, or geographic regions; or (3)
significant adverse effects on competition, employment or
investments, productivity, innovation, or the international
competitive position of U.S.  firms.  In September 1993, President
Clinton issued Executive Order 12866, replacing Executive Order 12291
and directing federal agencies to assess benefits, costs, and
alternatives for all economically significant regulatory actions. 
Under the order, an economically significant regulatory action is one
that is likely to result in a regulation that may have an annual
effect on the economy of $100 million or more or adversely affect in
a material way the economy, a sector of the economy, productivity,
competition, jobs, the environment, public health or safety, or
state, local, or tribal governments or communities. 

Both executive orders designated OMB as the reviewer of proposed
regulations and of the economic analyses supporting them.  OMB
developed guidance for implementing both orders.  Shortly after
President Clinton issued Executive Order 12866, OMB convened an
interagency group to review the state of the art for economic
analyses.  The group was co-chaired by a Member of the Council of
Economic Advisers and included representatives of all major
regulatory agencies.  Over 2 years, the group compiled best practices
for preparing economic analyses, which OMB published in January 1996
as guidance for implementing the executive order. 

OMB's guidance emphasizes that an economic analysis should provide
information to allow decisionmakers to determine that

  -- there is adequate information indicating the need for and
     consequences of the proposed action;

  -- the potential benefits to society justify the potential costs,
     recognizing that not all benefits and costs can be described in
     monetary or even quantitative terms, unless otherwise prohibited
     by statute;

  -- the proposed action will maximize net benefits to society,
     unless otherwise prohibited by statute;

  -- when a statute requires a specific regulatory approach, the
     proposed action will be the most cost-effective; and

  -- the agency's decision is based on the best reasonably obtainable
     scientific, technical, economic, and other information. 

The Environmental Protection Agency (EPA) and the Department of
Transportation (DOT) developed additional guidance to address unique
issues their agencies may face in preparing their economic
assessments. 


   CONGRESSIONAL EFFORTS TO
   IMPROVE THE REGULATORY PROCESS
---------------------------------------------------------- Chapter 1:2

Since the late 1970s, the Congress has taken a number of steps to
improve the regulatory process and control the costs of regulation. 
For example, the Congress has enacted several statutes to reduce the
costs and burdens of federal regulations, including the Paperwork
Reduction Act of 1980, the Regulatory Flexibility Act, the Small
Business Regulatory Enforcement Fairness Act of 1996, and the
Unfunded Mandates Reform Act of 1995 (UMRA). 

UMRA requires agencies to prepare benefit-cost and other
analyses--unless prohibited by law--for any regulations imposing
mandates likely to result in expenditures of $100 million or more in
any one year either by state, local, and tribal governments in the
aggregate or by the private sector alone.  Although UMRA's scope and
requirements differ from Executive Order 12866's, both authorities'
provisions on economic analysis are very similar.  Accordingly, OMB's
guidance for implementing the executive order states that "the
economic analysis that the agency prepares should also satisfy the
requirements of the Unfunded Mandates Reform Act."

The Congress has also considered--but not enacted--other initiatives
to reform the regulatory process.  Some of the more comprehensive
initiatives proposed to establish regulatory budgets; create
deadlines for phasing out regulations, programs, and agencies; revise
and expand the judicial review of regulatory actions; and require the
federal government to reimburse state and local governments for the
costs they incur in complying with federal regulations. 

Currently, the Congress is considering S.  981, the Regulatory
Improvement Act of 1998.  Intended to improve the quality of
regulatory decision-making, the bill would, among other things,
codify many of the requirements of Executive Order 12866 and
establish a requirement for independent peer reviews (critical
evaluations of technical work products by independent experts) of
economic analyses.  To make the regulatory process clearer, or more
"transparent," to the public, the bill would require agencies to
prepare executive summaries for their economic analyses that would
succinctly present, among other things, (1) the benefits and costs
expected to result from the rule; (2) the benefits and costs of
reasonable alternatives considered by the agency; and (3) the key
assumptions and scientific or economic information upon which the
agency relied. 


   RELATED GAO REVIEWS
---------------------------------------------------------- Chapter 1:3

GAO has issued a number of reports on economic analyses, peer review,
and unfunded mandates.  In 1984, we issued a report on EPA's use of
economic analyses.\1 To help agency decisionmakers, we recommended
that economic analyses include executive summaries that identify (1)
all benefits and costs--that is, both those that can be described
quantitatively and those that can be described qualitatively; (2) the
range of uncertainties associated with the benefits and costs; and
(3) a comparison of all feasible alternatives.  In April 1997, we
revisited this issue and made a similar set of recommendations to EPA
to help agency decisionmakers and the Congress better understand the
implications of proposed regulatory actions.\2 In September 1997, we
issued a report on the economic analyses prepared by the Consumer
Product Safety Commission in which we recommended that the Commission
develop procedures to ensure that its analyses are comprehensive and
reported in sufficient detail.\3

In 1996, we issued a report on EPA's implementation of peer review,\4
in which we recommended wider, more consistent implementation of the
agency's policy on peer review to enhance the quality and credibility
of the agency's decision-making.  In response to questions raised at
a March 1997 hearing on this issue, we said that, given the
uncertainties associated with predicting the future economic effects
of various regulatory alternatives, peer review would help to provide
the rigorous independent review of economic analyses needed to
enhance the quality, credibility, and acceptability of both the
economic analyses and the associated regulatory decisions. 

In 1998, we issued a report on the Unfunded Mandates Reform Act of
1995.\5 That report concluded that UMRA has had little effect on
agencies' rulemaking actions because the act's requirements (1) do
not apply to many large rulemaking actions; (2) allow agencies not to
take certain actions if the agencies determine that the actions are
duplicative or infeasible; and (3) direct agencies to take actions
that they are already required to take. 

Most recently, we provided testimony on S.  981.\6 In that testimony,
we concluded that the passage of S.  981 would provide a statutory
foundation for such principles as openness, accountability, and sound
science in rulemaking.  We cautioned, however, that our reviews of
current regulatory requirements suggest that even if S.  981 becomes
a law, the Congress will need to carefully oversee its implementation
to ensure that the principles embodied in the bill are faithfully
implemented. 


--------------------
\1 Cost-Benefit Analysis Can Be Useful in Assessing Environmental
Regulations, Despite Limitations (GAO/RCED-84-62, Apr.  6, 1984). 

\2 Air Pollution:  Information Contained in EPA's Regulatory Impact
Analyses Can Be Made Clearer (GAO/RCED-97-38, Apr.14, 1997). 

\3 Consumer Product Safety Commission:  Better Data Needed to Help
Identify and Analyze Potential Hazards (GAO/HEHS-97-147, Sept.  29,
1997). 

\4 Peer Review:  EPA's Implementation Remains Uneven
(GAO/RCED-96-236, Sept.  24, 1996). 

\5 Unfunded Mandates:  Reform Act Has Had Little Effect on Agencies'
Rulemaking Actions (GAO/GGD-98-30, Feb.  4, 1998). 

\6 Regulatory Reform:  Comments on S.  981--The Regulatory
Improvement Act of 1998 (GAO/T-GGD/RCED-98-95, Feb.  24, 1998). 


   OBJECTIVES, SCOPE, AND
   METHODOLOGY
---------------------------------------------------------- Chapter 1:4

To assist the Senate Committee on Governmental Affairs in carrying
out its regulatory oversight responsibilities, the Chairman and the
Ranking Minority Member asked GAO to describe (1) the extent to which
federal agencies' economic analyses incorporate the best practices
set forth in OMB's guidance and (2) the agencies' use of these
analyses in regulatory decision-making. 

To describe the extent to which federal agencies' economic analyses
incorporate the best practices set forth in OMB's guidance, we
reviewed all analyses prepared for "economically significant"\7
proposed and final rules issued between July 1996 and March 1997 that
addressed environmental, health, and safety matters.  Using these
selection criteria, we identified 20 proposed and final rules
promulgated by five agencies.  Nine of these rules were expected to
impose mandates likely to result in expenditures of $100 million or
more annually either by state, local, and tribal governments in the
aggregate or by the private sector; therefore, the agencies also used
these analyses to satisfy UMRA's requirements for economic analyses. 
Table 1.1 presents the rules, by agency, together with their dates of
publication in the Federal Register and the stages in rulemaking when
the economic analyses were published. 



                                        Table 1.1
                         
                         Economically Significant Rules Involving
                         Environmental, Health, or Safety Issues
                           Promulgated Between July 1, 1996 and
                                      March 30, 1997

                                               Date published in
Department or agency                           the
and office            Title of rule            Federal Register      Rulemaking stage
--------------------  -----------------------  --------------------  --------------------
Department of Agriculture
-----------------------------------------------------------------------------------------
Farm Service Agency   Conservation Reserve     Sept. 23, 1996        Proposed
                      Program--Long-Term       Feb. 19, 1997         Final
                      Policy

Natural Resources     Environmental Quality    Oct. 11, 1996         Proposed
Conservation Service  Incentives Program       May 22, 1997          Final

Animal and Plant      Karnal Bunt Disease:     Aug. 2, 1996          Proposed
Health Inspection     Domestic Plant-Related   Oct. 4, 1996          Final
Service               Quarantine

Food Safety and       Pathogen Reduction:      July 25, 1996         Final
Inspection Service    Hazard Analysis and
                      Critical Control Point
                      (HACCP) Systems\a


Department of Health and Human Services
-----------------------------------------------------------------------------------------
Food and Drug         Food Labeling:           Aug. 7, 1996          Final
Administration        Nutrition Labeling,
                      Small Business
                      Exemption

                      Medical Devices:         Oct. 7, 1996          Final
                      Current Good
                      Manufacturing Practice
                      (CGMP)

                      Regulations Restricting  Aug. 28, 1996         Final
                      the Sale and
                      Distribution of
                      Cigarettes and
                      Smokeless Tobacco to
                      Protect Children and
                      Adolescents\a

                      Substances Prohibited    Jan. 3, 1997          Proposed
                      From Use in Animal Food  June 5, 1997          Final
                      or Feed; Animal
                      Proteins Prohibited in
                      Ruminant Feed


Department of Labor
-----------------------------------------------------------------------------------------
Occupational Safety   Occupational Exposure    Jan. 10, 1997         Final
and Health            to Methylene Chloride\a
Administration


Environmental Protection Agency
-----------------------------------------------------------------------------------------
Solid Waste and       Financial Assurance      Nov. 27, 1996         Final
Emergency Response    Mechanisms for Local
                      Government Owners and
                      Operators of Municipal
                      Solid Waste Landfill
                      Facilities

Air and Radiation     Regulation of Fuels and  July 5, 1996          Final
                      Fuel Additives;
                      Certification Standards
                      for Deposit Control
                      Gasoline Additives\a

                      Acid Rain Programs       Dec. 19, 1996         Final
                      Nitrogen Oxides
                      Emission Reduction\
                      Program\\a

                      Motor Vehicle Emissions  Oct. 22, 1996         Final
                      Federal Test Procedure
                      Revisions\a

                      National Ambient Air     Dec. 13, 1996         Proposed
                      Quality Standards for    July 18, 1997         Final
                      Ozone\b

                      National Ambient Air     Dec. 13, 1996         Proposed
                      Quality Standards for    July 18, 1997         Final
                      Particulate Matter\b

                      Emission Standards for   Feb. 11, 1997         Proposed
                      Locomotives and
                      Locomotive Engines\a

                      Air Pollution Control;   Oct. 4, 1996          Final
                      Gasoline Spark-
                      Ignition Marine
                      Engines: New Nonroad
                      Compression-Ignition
                      and Spark-Ignition
                      Engines, Exemptions\a

Pollution Prevention  Lead: Requirements for   Aug. 29, 1996         Final
and Toxics            Lead-Based Paint
                      Activities in Target
                      Housing and Child-
                      Occupied Facilities


Department of Transportation
-----------------------------------------------------------------------------------------
National Highway      Federal Motor Vehicle    Feb. 20, 1997         Proposed
Traffic Safety        Safety Standards; Child
Administration        Restraint Systems;
                      Tether Anchorages for
                      Child Restraint
                      Systems; Child
                      Restraint Anchorage
                      System\a

                      Federal Motor Vehicle    Jan. 6, 1997          Proposed
                      Safety Standards:        Mar. 19, 1997         Final
                      Occupant Crash
                      Protection (Air Bag
                      Depowering)
-----------------------------------------------------------------------------------------
\a Rule also triggers UMRA's requirement for economic analysis.  \b
EPA maintains that it was not required to prepare economic analyses
under UMRA for these rules even though they come within UMRA's scope
because (1) UMRA requires the preparation of economic analyses for
covered rules unless otherwise prohibited by law; (2) the Clean Air
Act prohibits EPA from considering costs in setting these
health-based standards; and (3) the Conference Report for UMRA states
that if the agency is prohibited by law from considering the estimate
or analysis, it need not prepare one under UMRA. 

We reviewed the analyses to describe the extent to which they
incorporated the best practices recommended by OMB's guidance. 
Specifically, we examined the analyses' treatment of alternatives,
benefits and costs, uncertainty, and assumptions, as well as of the
requirement for full disclosure.  We did not, however, verify the
accuracy of the data used in the analyses.  Although OMB's guidance
did not discuss the use of executive summaries or peer review, we
also determined whether the analyses contained executive summaries or
underwent peer review.  We verified our findings through interviews
with agency officials who were responsible for preparing the
analyses. 

To describe how the agencies used the economic analyses in regulatory
decision-making, we interviewed agency officials with decision-making
responsibility for the 20 rules to obtain more detailed explanations
of how the analyses were used.  Because our scope involved rules that
had already progressed to the proposed or final rulemaking stages, we
were unlikely to address situations in which an economic analysis
resulted in a determination not to regulate or significantly alter
the regulation under consideration.  To account for this limitation,
we asked agency officials if they were aware of other regulatory
actions outside our scope in which an analysis played an important
role in withdrawing or significantly altering a regulatory
initiative. 

We conducted this review between April 1997 and April 1998 in
accordance with generally accepted government auditing standards. 


--------------------
\7 Under Executive Order 12866, an economically significant
regulatory action is a substantive action by an agency that is likely
to result in a regulation that may have an annual effect on the
economy of $100 million or more or adversely affect in a material way
the economy; a sector of the economy; productivity; competition;
jobs; the environment; public health or safety; or state, local, or
tribal governments or communities. 


ECONOMIC ANALYSES INCORPORATED
BEST PRACTICES TO VARYING DEGREES,
AND SOME LACKED FULL DISCLOSURE
============================================================ Chapter 2

OMB's guidance sets forth best practices for federal agencies to
consider in preparing economic analyses.  Although incorporating
these best practices can provide valuable information, the guidance
recognizes that economic analyses cannot be written according to a
formula.  Accordingly, it gives agencies the flexibility to use their
professional judgment in deciding how thorough their analyses should
be.  At the same time, the guidance stresses the importance of full
disclosure.  Therefore, in this review of the extent to which 20
economic analyses incorporated OMB's best practices, we focused not
only on which best practices were included but also on whether and
how clearly the agencies' methods were explained.\8

Some of the 20 economic analyses that GAO reviewed did not
incorporate the best practices set forth in OMB's guidance.  For
example, the 20 economic analyses varied in the number and range of
alternatives considered; the degree to which benefits and costs were
described--in monetary, quantitative, or qualitative terms--for the
proposed action and alternatives; the degree to which assumptions and
key variables were explained; and the ways in which uncertainty was
accounted for in the analyses' conclusions.  In some instances, the
analyses provided only a limited discussion of alternatives or other
best practices.  Additionally, when the analyses omitted or only
partially incorporated OMB's best practices, they typically did not
explain the reasons for these omissions.  This lack of explanation is
not consistent with the principle of full disclosure.  Furthermore,
in some instances, the lack of full disclosure obscured the
thoroughness of an agency's efforts and/or the constraints on the
agency's time or resources.  In these instances, full disclosure
would have enhanced the reader's understanding and the credibility of
the analyses. 

The clarity of the 20 analyses varied, making it difficult for
readers to determine whether or where OMB's best practices were
considered.  Some of the analyses contained executive summaries,
while others relied on the preambles to the proposed and final rules,
published in the Federal Register, to summarize their results.  GAO
has recommended, and S.  981 would require, the inclusion of an
executive summary in an economic analysis to clarify an agency's
approach and emphasize the key points of the analysis.  Only one of
the analyses underwent an independent peer review.  GAO has
recommended, and S.  981 would require, the use of peer review to
help ensure both the quality and the credibility of an analysis. 


--------------------
\8 As mentioned in ch.  1, OMB's guidance applies to economic
analyses prepared in response to the requirements of UMRA as well as
of Executive Order 12866.  Because agencies rarely prepare separate
analyses when UMRA is applicable (only one of the nine regulations we
selected that came within the scope of UMRA had a separate analysis),
our findings reflect the extent to which the nine analyses called for
under UMRA satisfy the act's as well as the executive order's
requirements for economic analyses. 


   ANALYSES VARIED IN
   INCORPORATING BEST PRACTICES
   AND DID NOT ALWAYS PROVIDE
   REASONS FOR OMISSIONS
---------------------------------------------------------- Chapter 2:1

OMB's guidance describes in detail how economic analyses should
consider alternatives, benefits, costs, assumptions, uncertainty, and
other factors.  This guidance is consistent with standard economic
principles, and incorporating its recommended practices into economic
analyses could provide valuable information on the benefits and costs
of regulatory alternatives.  Nonetheless, the guidance also notes
that the amount of analysis required depends on the "importance and
complexity" of the regulatory issue, as well as on the time available
for analysis.  In some instances, the need to respond to an emergency
or meet a statutory deadline may limit an analysis.  The guidance
also identifies the "nature of the statutory language and the extent
of statutory discretion" as important in determining how much
analysis is needed.  In particular, the guidance maintains that "a
less detailed or intensive analysis of the entire range of regulatory
options is needed when regulatory options are limited by statute."
For example, the statute directing the Food and Drug Administration
(FDA) to exempt small businesses from certain food labeling
requirements was so prescriptive that agency officials described the
implementing regulations as little more than a photocopy of the law. 
Nevertheless, the guidance also states that even when such
limitations apply, agencies should provide some analysis of
alternatives to provide decisionmakers with information for judging
the consequences of statutory constraints.  Finally, the guidance
recognizes that practical considerations, such as constraints on
resources, may limit the scope of an analysis. 

OMB's guidance allows agencies to exercise their professional
judgment in deciding how thorough their analyses should be.  At the
same time, it stresses the importance of full disclosure in
presenting the analyses.  Furthermore, when agencies depart from the
best practices, the guidance directs them to explain why they have
chosen to do so. 

The 20 economic analyses that we reviewed varied in the extent to
which they considered alternatives, described benefits and costs,
explained key variables, and accounted for uncertainty.  Although
this variation reflects the flexibility inherent in OMB's guidance,
the frequent absence of an agency's rationale for omitting or paying
limited attention to certain best practices was not consistent with
OMB's guidance. 


      ANALYSES CONSIDERED
      ALTERNATIVES TO VARYING
      DEGREES
-------------------------------------------------------- Chapter 2:1.1

According to OMB, a key goal of an economic analysis in rulemaking is
to determine what degree of regulation is needed to maximize net
benefits.  An economic analysis cannot determine whether net benefits
are maximized unless it considers the most important regulatory
alternatives or, in the words of the Executive Order, "potentially
effective and reasonably feasible alternatives." Therefore, a
complete analysis considers a range of alternatives, measures the
benefits and costs of each, and determines which one achieves the
greatest net benefits. 

In 15 of the 20 analyses that we reviewed, the agencies included at
least one alternative to the proposed action, but in some instances,
discussion of the alternative was limited.  The five other analyses
did not indicate why alternatives were not discussed.  Agency
officials told us that, in preparing two analyses, they considered
alternatives but did not discuss them in the analyses.  In preparing
the three remaining analyses, agency officials told us they did not
consider alternatives to the proposed actions either because the
authorizing statute (1) specified the regulatory approach to take or
(2) did not provide enough time to consider regulatory alternatives. 
Figure 2.1 summarizes our findings. 

   Figure 2.1:  Economic Analyses'
   Consideration of Alternatives

   (See figure in printed
   edition.)

Agency officials provided us with reasons for not discussing or
considering alternatives in the analyses.  These reasons--including
the specificity of, or the time constraints imposed by, the
authorizing statute--are among those that OMB's guidance cites as
legitimate constraints on an agency's consideration of alternatives. 
Although the guidance states that these can be legitimate reasons
limiting the consideration of alternatives, it also states that even
when such limitations apply, agencies should provide some analysis of
alternatives to provide decisionmakers with information for judging
the consequences of statutory constraints.  In addition, we noticed
that agencies did not always document in their analyses why they did
not discuss or consider alternatives in the analyses.  For example,
for one analysis, EPA initially considered two alternatives for
implementing a regulation on certification standards for detergents
added to gasoline to reduce emissions.  One alternative specified the
steps manufacturers should take to comply with the regulation; the
other established performance-based standards and allowed the
manufacturers to decide how they would achieve the standards. 
Because Executive Order 12866 and OMB's guidance favor
performance-based regulations over command-and-control regulations,
EPA dismissed the command-and-control alternative before preparing
the analysis and discussed only the performance-based alternative in
the analysis.  FDA's regulation exempting small businesses from
certain food-labeling requirements also included no alternatives and
provided no explanation for this departure from OMB's guidance.  FDA
officials told us, however, that the legislation setting forth the
exemptions was so specific that no alternative to the proposed action
was feasible. 

The 15 analyses that included at least one alternative also varied in
the attention given to the alternative or alternatives that were
considered and rejected.  For example, the analysis for the
regulation on adolescents' use of tobacco examined six regulatory
alternatives but contained only a few paragraphs on the five that
were ultimately rejected.  According to the responsible officials,
FDA gathered and reviewed data for all six alternatives, and experts
evaluated each one before FDA proposed an action.  The final economic
analysis did not reflect the thoroughness of FDA's review.  A more
thorough discussion of the alternatives would have enabled the reader
to better understand why the agency chose the proposed action. 


      ANALYSES VARIED IN THEIR
      TREATMENT OF BENEFITS AND
      COSTS
-------------------------------------------------------- Chapter 2:1.2

According to OMB, an economic analysis should measure the benefits
and costs of the proposed action and of the alternatives in
comparable terms to ensure an accurate determination of net benefits. 
The benefits and costs should be measured against a baseline,
preferably in numerical terms.  A baseline generally describes the
condition that is expected to exist without the regulation and
provides a standard for measuring the incremental benefits and costs
of each alternative.  When possible, dollar values should be assigned
to benefits and costs to enhance the consideration of regulatory
alternatives that may produce equal or greater benefits at lower
costs.  However, if dollar values cannot be assigned, the benefits
and costs should be expressed in consistent quantitative or
qualitative terms.  Although completeness is desirable, OMB's
guidance recognizes that accurate data may not always be available
for estimating benefits and costs and that agencies may not have the
resources or the time to estimate values for every alternative. 


         ALL 20 ANALYSES INCLUDED
         BASELINE INFORMATION
------------------------------------------------------ Chapter 2:1.2.1

In the 20 economic analyses that we reviewed, the baseline was either
explicitly identified or was implicit within the context of the
analysis.  In these later analyses, the use of a baseline was more
difficult to discern but was evident after some review.  For example,
the analysis for the U.S.  Department of Agriculture's (USDA) rule on
mandatory controls to reduce foodborne illness from meat and poultry
did not explicitly identify a baseline.  However, our review of the
analysis indicated that costs were indeed measured relative to a
baseline because they reflected the costs of the manufacturing
controls that would be put in place after the regulation became
operative. 

FDA's regulation to restrict adolescents' use of tobacco describes
the baseline quantitatively in terms of the number of adolescents
who, in the absence of additional regulation, would be likely to
start smoking each year--estimated to be 1 million under the age of
18.  Although the analysis does not assign a dollar value to the
costs of the baseline, it does quantify the effects of cutting the
number of underage smokers in half, calculating how many fewer adults
would smoke, how many deaths would be avoided, and how many
life-years would be saved.  The analysis then assigns dollar values
to these benefits and concludes that the total monetary value of a
50-percent reduction in adolescents' use of tobacco would be between
$28 billion and $43 billion at a 3-percent discount rate or between
$9 billion and $10 billion at a 7-percent discount rate. 


         ANALYSES ESTIMATED SOME
         BENEFITS AND COSTS
------------------------------------------------------ Chapter 2:1.2.2

All 20 economic analyses that we reviewed estimated benefits in some
terms--whether monetary, quantitative or qualitative.  Fourteen\9 of
the analyses assigned dollar values to some benefits.  Seven of these
analyses assigned dollar values to benefits for both the proposed
action and at least one alternative, while the other seven assigned
dollar values only for the proposed action.  The analyses that did
not assign dollar values to benefits did not document their reasons
for omitting this element of OMB's guidance.  Furthermore, only six
analyses specifically identified net benefits (benefits remaining
after costs have been accounted for)--a key element in OMB's
guidance.  Executive Order 12866 emphasizes that agencies should
select approaches that maximize net benefits (including potential
economic, environmental, public health and safety, and other
advantages; distributive impacts; and equity), unless a statute
requires another regulatory approach.\10

Figure 2.2 shows the extent to which the 20 analyses assigned dollar
values to benefits. 

   Figure 2.2:  Economic Analyses'
   Assignment of Dollar Values to
   Benefits

   (See figure in printed
   edition.)

Agencies assigned dollar values to different types of benefits,
including health benefits and costs saved.  For example, EPA's
analyses for regulations on ozone and particulate matter assigned
dollar values to health and other benefits gained through reductions
in exposure to these two substances.  These benefits included
life-years saved and increases in crop yields.  EPA's analysis for a
regulation on landfills assigned dollar values to the cost savings
achieved by using two new, less expensive methods of providing
financial assurance.  EPA estimated these savings by subtracting the
costs of using the new methods from the costs of using the current
method and determining the dollar savings.  Finally, FDA's analysis
for a regulation on ensuring disease-free animal feed assigned dollar
values to the costs avoided by not having to destroy cattle. 

Agencies' analyses described benefits in quantitative or qualitative
terms, sometimes in combination with dollar values.  For example,
EPA's analysis for a rule on gas certification standards assigned
dollar values to fuel consumption benefits, quantified emission
reduction benefits, and qualitatively described improvements in
maintenance.  Four other EPA analyses--those for regulations on
federal engine-testing procedures, locomotives, acid rain and
nitrogen oxides, and marine engines--also quantified emission
reduction benefits.  The analysis for the rule on marine engine
emissions qualitatively described other improvements in air quality. 
Other benefits that were described in quantitative or qualitative
terms included reductions in fatalities due to accidents, deaths
avoided through reductions in exposure to cancer-causing agents,
reductions in injuries and impairments, and improvements in health. 

Nineteen of the 20 analyses that we reviewed assigned dollar values
to some costs.  However, nine of the analyses estimated dollar values
only for the proposed action.  Four of these nine analyses discussed
at least one other alternative but did not assign dollar values to
them, while the other five did not discuss any alternatives to the
proposed action.  Figure 2.3 shows how the 20 analyses assigned
dollar values to costs. 

   > Figure 2.3:  Economic
   Analyses' Assignment of Dollar
   Values to Costs

   (See figure in printed
   edition.)


--------------------
\9 For 1 of these 14 analyses--on lead-based paint abatement
activities in certain housing and child-occupied facilities--no data
were available to estimate the incremental benefits of the training
required for certification and to compare these benefits with the
incremental costs of the rule.  Consequently, EPA decided to estimate
the total benefits of lead paint abatement work and compare these
benefits with the incremental costs of the rule in a break-even type
of analysis, since these figures were not appropriate for a net
benefit analysis. 

\10 Distributive impacts (or equity) indicate how the benefits and
costs of a proposed regulatory action are distributed across
individual members or groups or classes in society.  While
recognizing that distributive impacts and equity are important
considerations in making decisions, economists sometimes treat them
separately from net benefits. 


      ANALYSES DIFFERED IN
      TREATMENT OF ASSUMPTIONS AND
      UNCERTAINTY
-------------------------------------------------------- Chapter 2:1.3

To determine the present value of future benefits and costs, analysts
apply a discount rate.  When attempting to estimate the dollar value
of benefits for regulations anticipated to extend or save lives, they
may use the value of a "statistical life."\11 And to help quantify
the effect of uncertainty on benefit and cost estimates, they may use
sensitivity or other types of analyses.\12 Although OMB's guidance
provides agencies with flexibility in selecting assumptions and
treating uncertainty, the guidance stresses that agencies should
explicitly identify the assumptions underlying their economic
analyses and the uncertainty associated with the resulting estimates. 
The economic analyses we reviewed often were not explicit on these
matters. 


--------------------
\11 A "statistical life" is the product of (1) one minus the
estimated probability of death, given no remediation of the problem
that the regulation is supposed to correct, and (2) the size of the
affected population. 

\12 A sensitivity analysis assigns a variety of numerical values to
key parameters, such as the discount rate, to see how sensitive the
benefit and cost estimates are to these different values. 


         KEY VARIABLES DIFFERED,
         AND REASONS FOR
         DIFFERENCES WERE NOT
         STATED
------------------------------------------------------ Chapter 2:1.3.1

Many economic analyses rely on assumed values of key variables, such
as the discount rate and the value of a statistical life, to estimate
the benefits and costs of regulations.  In economic analyses, the
discount rate is the interest rate used to determine the present
value of future benefits and costs.  The statistical value placed on
a human life greatly affects estimates of benefits gained through
improvements in safety, reductions in exposure to harmful substances,
and other types of health benefits.  For analyses that do not
estimate values over time, a discount rate is not relevant. 
Similarly, for analyses that do not consider the impact of regulatory
alternatives on human health or safety, the statistical value of a
human life is not relevant. 

Of the 20 analyses that we reviewed, 15 used one or more discount
rates, which ranged from 2.1 percent to 10 percent.  While OMB
recommends a 7-percent discount rate (adjusted for inflation) for
economic analyses, the guidance allows agencies to use different
rates if justified.  The majority of the 15 analyses that used a
discount rate followed OMB's recommendation.  The five analyses that
did not use a discount rate did not explain why they did not do so. 
A discount rate was not used because (1) benefits and costs were
estimated over only 1 year or (2) dollar values were not assigned to
either benefits or costs. 

For 6 of the 20 analyses, a reduction in the risk of mortality was a
benefit associated with the rule, and a dollar value was, therefore,
assigned to a statistical human life for the purpose of calculating
benefits.  The value of this statistical life varied in the six
analyses, ranging from $1.6 million to $5.5 million, as indicated in
table 2.1.  OMB's guidance does not prescribe any particular value
for agencies to use and allows for a variety of approaches to
estimate the benefits of a reduction in the risk of mortality,
including both explicit and implicit valuation methods.  In each of
the six analyses, the agency fully explained the basis for the
assigned value.  For the analysis for the lead paint rule, EPA
estimated the mean value of a statistical life from 26 selected
studies. 



                               Table 2.1
                
                  Dollar Value Assigned to Human Life

                         (Dollars in millions)

Analysis for rule                                       Assigned value
--------------------------------------------------  ------------------
Pathogen Reduction; Hazard Analysis and Critical                  $1.6
 Control Point (HACCP) Systems
Regulations Restricting the Sale and Distribution                 $2.5
 of Cigarettes and Smokeless Tobacco to Protect
 Children and Adolescents
National Ambient Air Quality Standards for Ozone                  $4.8
National Ambient Air Quality Standards for                        $4.8
 Particulate Matter
Medical Devices: Current Good Manufacturing                     $5.0\a
 Practice
Lead; Requirements for Lead-Based Paint Activities                $5.5
 in Target Housing and Child-Occupied Facilities
----------------------------------------------------------------------
\a The economic analysis for this regulation did not assign a
specific value to human life.  However, the preamble to the rule
published in the Federal Register estimates this value at $5.0
million. 

Of the 14 analyses that did not assign a dollar value to human life,
11 did not identify a reduction in the risk of mortality as a
benefit; therefore, a value for life was not applicable.  The three
other analyses that did have an impact on the risk of mortality were
prepared by the National Highway Traffic Safety Administration and
the Occupational Safety and Health Administration (OSHA).  According
to agency officials, the agency does not assign an explicit dollar
value to human life or suffering in its analyses because it believes
that such a value conveys a false sense of precision and is morally
objectionable.  Instead, the agencies prefer to describe benefits
quantitatively in terms of fewer deaths, injuries, or illnesses. 


         MAJORITY OF ANALYSES
         ACKNOWLEDGED SOME
         UNCERTAINTY
------------------------------------------------------ Chapter 2:1.3.2

Uncertainty may arise from lack of data, variability in populations
or natural conditions, limitations in fundamental scientific
knowledge (both social and natural) that result in lack of knowledge
about key relationships, or the fundamental unpredictability of
certain phenomena.  While recognizing that the effects of regulatory
actions are often uncertain, OMB's guidance observes that the
probability of their occurrence can, in some instances, be predicted
through the use of appropriate statistical techniques.  In other
instances, when different assumptions are plausible, sensitivity
analyses can be used to test the impact of the differences. 

For 15 of the 20 regulations, the economic analyses or other related
documents acknowledged the uncertainty associated with estimates of
benefits and/or costs.  Seven of the 15 economic analyses used
sensitivity analysis to evaluate the impact of different assumptions
on the estimates, and eight of the analyses discussed uncertainties
either qualitatively or in terms of ranges of estimates.  The five
analyses that did not discuss uncertainties did not document the
agencies' reasons for not doing so. 

FDA's economic analysis for the regulation to restrict adolescents'
use of tobacco illustrates the role of sensitivity analysis in
regulatory decision-making.  For this analysis, FDA assigned dollar
values to the health benefits that it estimated would result from
reducing, by varying percentages, the number of adolescents who
currently use tobacco, assuming a 3-percent discount rate.  It
estimated that a 50-percent reduction in the number of adolescent
smokers would produce annual benefits of $28.1 billion to $43.2
billion, while a 5-percent reduction would produce annual benefits of
$2.8 billion to $4.3 billion.  Under either scenario, the estimated
annual benefits would vastly outweigh the estimated annual costs of
complying with the regulation--$149 million to $185 million. 
Although FDA did not identify a single-value "best estimate" for
anticipated net benefits, it did provide a best estimate for
reductions in tobacco use from a range of possibilities.  Three other
analyses also identified some types of best estimates from the range
of estimates presented. 


      ANALYSES DID NOT PROVIDE A
      RATIONALE FOR OMITTING BEST
      PRACTICES
-------------------------------------------------------- Chapter 2:1.4

While the 20 analyses that we reviewed generally incorporated
elements of OMB's guidance to some degree, they seldom accounted for
omissions, even when these omissions were consistent with the
flexibility inherent in the guidance.  As table 2.2 indicates, we
found 36 instances in which best practices were not included in the
analyses.  Although agency officials told us that specific best
practices were not relevant in 16 of these instances, these reasons
were not provided in the economic analyses themselves.  Overall, in
only 1 of these 36 instances did the analysis fully disclose why the
practice was omitted. 



                               Table 2.2
                
                   Extent to Which Economic Analyses
                 Provided Reasons for Not Incorporating
                       Elements of OMB's Guidance

                                                       Provided reason
                                           Did not             for not
                                       incorporate       incorporating
Recommended element                        element             element
------------------------------  ------------------  ------------------
Discuss at least one                             5                   0
 alternative
Assign dollar values to some                     6                   0
 benefits
Assign dollar values to some                     1                   1
 costs
Acknowledge uncertainties                        5                   0
Assign a value to                             14\a                   0
 human life
Use a discount rate                            5\b                   0
----------------------------------------------------------------------
\a Element was not relevant for 11 of these analyses, and agency's
policy prohibited assigning a value to human life for other 3
analyses. 

\b Element was not relevant for these analyses. 

Source:  GAO's analysis of 20 economic analyses. 


   GUIDANCE COULD DO MORE TO
   ENSURE FULL DISCLOSURE, AND
   PEER REVIEW COULD STRENGTHEN
   ANALYSES' CREDIBILITY
---------------------------------------------------------- Chapter 2:2

The clarity of the 20 analyses that we reviewed varied, making it
difficult for the reader to determine whether or where particular
elements of OMB's guidance were incorporated.  While about half of
the analyses included some form of summary, the other half used the
preambles to the rules to summarize key information.  Because only
one of the analyses was submitted for an independent peer review,
most of the analyses did not benefit from the enhanced credibility
that such a review could have conferred. 


      EXECUTIVE SUMMARIES
      FREQUENTLY NOT PROVIDED
-------------------------------------------------------- Chapter 2:2.1

Twelve of the 20 analyses contained an executive summary that clearly
and concisely summarized the reports' major findings and eight did
not.  In general, when agencies did not provide an executive summary,
they relied on the preamble to the final or proposed rule, published
in the Federal Register, to summarize the results of their work.  In
terms of full disclosure, the preambles were subject to the same
limitations as the analyses. 

As we have noted in prior reviews of EPA's economic analyses,\13 the
lack of a summary in an economic analysis restricts the ability of
the Congress, the public, and at times the decisionmakers to quickly
identify key issues and to be fully informed.  Accordingly, we
recommended to EPA that its economic analyses should, to the extent
possible, include executive summaries that identify (1) all benefits
and costs--even those that cannot be quantified; (2) the range of
uncertainties associated with the benefits and costs; and (3) a
comparison of feasible alternatives.  S.  981 would require agencies
to include an executive summary in the economic analyses.  The
summary would include, among other things, (1) the benefits and costs
expected to result from the rule, (2) the benefits and costs of
reasonable alternatives considered by the agency, and (3) the key
assumptions and scientific or economic information on which the
agency relied. 


--------------------
\13 Cost-Benefit Analysis Can Be Useful in Assessing Environmental
Regulations, Despite Limitations (GAO/RCED-84-62, Apr.  6, 1984) and
Air Pollution:  Information Contained in EPA's Regulatory Impact
Analyses Can Be Made Clearer (GAO/RCED-97-38, Apr.  14, 1997). 


      ANALYSES DID NOT UNDERGO
      PEER REVIEW
-------------------------------------------------------- Chapter 2:2.2

Only 1 of the 20 analyses that we reviewed was submitted for peer
review--independent experts' critical evaluation of scientific or
technical work products.  While OMB does not require agencies to
submit their analyses for external peer review, the Administrator of
OMB's Office of Information and Regulatory Affairs testified in
September 1997\14 that the administration supports peer review but
recognizes that it is not cost-free, in terms of an agency's
resources or time.  Of the five agencies whose analyses we reviewed,
only EPA has a formal peer review policy in place. 

GAO is on record in support of peer review for important economic
analyses.  At a March 1997 hearing on peer review at EPA, we said
that "given the uncertainties associated with predicting the future
economic impacts of various regulatory alternatives, the rigorous,
independent review of economic analyses should help enhance the
products'--and the associated agency decisions'--quality,
credibility, and acceptability."

EPA's peer review policy, issued in 1994, applies to major scientific
or technical work products that may affect policy or regulatory
decisions.  Each office is to develop procedures for implementing the
policy that include preparing a list of products for peer review
during the upcoming year and documenting the status of products
previously nominated.  The policy is somewhat flexible, noting that
statutory and court-ordered deadlines, resource limitations, and
other constraints may limit or even preclude the use of peer review. 
Accordingly, the policy calls for different levels of peer review,
depending upon these constraints, as well as the products'--and
associated decisions'--complexity and sensitivity.  Factors to take
into account in making decisions about peer review include whether or
not the product

  -- establishes a significant precedent, model or methodology;

  -- addresses significant controversial issues;

  -- focuses on significant emerging issues;

  -- has significant cross-agency/interagency implications;

  -- involves a significant investment of the agency's resources;

  -- considers an innovative approach for a previously defined
     problem/process/methodology; or

  -- satisfies a statutory or legal mandate for peer review. 

Under the policy, soliciting stakeholders' involvement or public
comment is not a substitute for peer review, which is intended to
solicit the independent, objective views of experts.  While these
experts may be internal or external to the agency, EPA's revised
guidance on peer review\15 states that external peer reviewers are
generally preferred.  Regardless of their relationship to the agency,
the reviewers should be unbiased (i.e.  have not contributed to the
product's development or have a material stake in the outcome of the
review) and have appropriate expertise.  The guidance also notes that
in some circumstances, peer review may not be needed or may not be
possible.  For example, products that are primarily based on work
that was previously peer reviewed can generally forgo additional peer
review.  According to the guidance, "in a few instances, statutory
and court ordered deadlines and other time constraints may limit or
preclude peer review." However, the guidance emphasizes that agency
officials should "make every attempt possible to assure that peer
review of major work products occurs taking into account these
deadlines." The guidance also provides discretion in determining the
timing and frequency of peer review, noting that different products
warrant differing timing and frequency.  A common approach is to have
a single peer review when the final draft product becomes available. 
The guidance also states that the final product should incorporate
the peer reviewers' comments or state why these comments are not
incorporated. 

EPA acknowledges that its implementation of the policy has been
uneven, and it has taken steps to better ensure that the policy is
understood, used, and considered more seriously.  In response to
recommendations we made in 1996, EPA has agreed to adopt steps to
ensure that all major products are considered for peer review and to
identify individual products that are not selected for review.\16 EPA
officials told us that they were considering peer reviews for some
economic analyses in the future. 

Officials at the agencies we visited acknowledged that peer review
could improve the quality and credibility of economic analyses.  For
example, USDA officials told us that the results of peer reviews
provide useful, ongoing guidance for economic analyses prepared for
similar types of proposals.  However, a number of officials
incorrectly identified the process of seeking public comment through
the publication of proposed or final rules in the Federal Register as
a form of external peer review.  Other officials maintained that
submitting many of the analyses we reviewed for peer review would
have delayed their publication and increased their costs but might
not have added value.  A common theme among the agencies was that
statutory directives, time constraints, and limited resources
precluded them from submitting their economic analyses to external
experts for peer review.  Some officials also believed that they
might have difficulty finding independent reviewers with the
necessary expertise. 

According to a panel of leading economists, peer review should be
used for economic analyses supporting regulations with a potentially
large impact on the economy.  The panelists recommended that the
reviewers be selected on the basis of their expertise and reputation. 
The panel also recommended that agencies use a standard format to
present their results, including a summary highlighting key results
and uncertainties.\17 A recent report by the
Presidential/Congressional Commission on Risk Assessment and Risk
Management also supported the use of peer review for key economic
documents.\18 In a recent article co-authored by EPA's Associate
Assistant Administrator for Policy, Planning, and Evaluation, the
authors stressed the importance of conducting economic analyses in a
more open manner, involving outside experts and stakeholders.  They
also suggested that despite time constraints, such outside
involvement could occur more often if economic analyses were
initiated at the beginning of the rulemaking process.\19


--------------------
\14 Statement of Sally Katzen, Administrator, Office of Information
and Regulatory Affairs, OMB, before the Senate Committee on
Governmental Affairs (Sept.  12, 1997). 

\15 Science Policy Council Handbook:  Peer Review (EPA 100-B-98-001,
Jan.  1998). 

\16 Peer Review:  EPA's Implementation Remains Uneven (RCED-96-236,
Sept.  1996). 

\17 Arrow, Cropper, et al., Benefit-Cost Analysis in Environmental,
Heath, and Safety Regulation:  A Statement of Principles (1996). 

\18 Risk Assessment and Risk Management in Regulatory
Decision-Making, The Presidential/Congressional Commission on Risk
Assessment and Risk Management (1997). 

\19 "Economic Analysis:  Benefits, Costs, Implications," Economic
Analyses at EPA:  Assessing Regulatory Impact (1997). 


   CONCLUSIONS
---------------------------------------------------------- Chapter 2:3

Agencies' economic analyses sometimes omitted best practices
recommended by OMB's guidance.  While agencies have taken advantage
of the flexibility that OMB's guidance gives them to use their
professional judgment in deciding how thorough their analyses should
be, they often have not documented the reasons why they omitted best
practices recommended by the guidance--even when their reasons are
among those that OMB has identified as legitimate for limiting an
analysis.  Full disclosure would be consistent with the guidance and
would provide decisionmakers with information for judging the
consequences of statutory constraints.  Thus, full disclosure could
generally enhance the credibility of the analyses.  Similarly,
including executive summaries with the analyses would help to
highlight and succinctly present the key points supporting the
agency's regulatory decision.  Although independent reviews by
internal or external experts may not be warranted for all economic
analyses, such reviews could enhance both the quality and the
credibility of the analyses. 


   RECOMMENDATIONS
---------------------------------------------------------- Chapter 2:4

To facilitate full disclosure and add credibility to the economic
analyses required for regulatory decision-making, we recommend that
the Director, Office of Management and Budget, amend the Office's
guidance to include additional elements, the latter two of which are
reflected in S.  981.  Specifically, we recommend that the guidance
be amended to provide that economic analyses should

  -- address all of the best practices identified in OMB's guidance
     or state the agency's reasons for not addressing them;

  -- contain an executive summary that briefly and concisely (1)
     identifies all benefits and costs--both those that can be
     described quantitatively and those that can be described
     qualitatively; (2) describes the range of uncertainties
     associated with the benefits and costs; and (3) compares the
     reasonable alternatives considered by the agency; and

  -- undergo an appropriate level of internal or external peer review
     by independent experts and state the agency's basis for
     selecting that level. 


   AGENCY COMMENTS
---------------------------------------------------------- Chapter 2:5

We provided a draft of this report for comment to OMB and the five
agencies that prepared the economic analyses we reviewed:  USDA, FDA,
EPA, DOT, and OSHA.  We received comments from all of the agencies
except OSHA, which informed us that it had no comments on the draft
report.  Most of the comments we received involved editorial or
technical clarification issues, which we incorporated throughout the
report as appropriate. 

The agencies agreed with our findings and recommendations concerning
the need for economic analyses to address OMB's best practices and
include executive summaries.  However, all of the agencies raised
issues related to our recommendation on peer review.  While USDA
agreed with us that peer review is generally appropriate and useful,
the Department maintained that using peer reviewers from within the
agency is frequently more timely and cost-effective.  Accordingly,
USDA asked us to clarify what constitutes "an appropriate level of
peer review." Similar requests for clarification were raised by OMB
and DOT.  FDA urged us to delete this recommendation altogether,
maintaining that a requirement for peer review by experts external to
the agency would have minimal benefits and the resource burden would
likely preclude the agency from meeting its statutory requirements. 

We acknowledge that peer review imposes some time and resource
burdens on agencies and that different types of economic analyses
warrant different levels of peer review.  We believe that EPA's peer
review policy addresses this issue well, providing for either
internal or external peer review.  However, the policy also
emphasizes that, as a general rule, the more important, novel, or
sensitive the document and the associated regulatory action, the more
rigorous the peer review should be.  The policy also emphasizes that
whether the review is conducted within or outside the agency, two
basic requirements must be met:  The reviewers must be unbiased, and
they must have appropriate expertise.  We have clarified the report's
discussion and recommendation on peer review to clarify that agencies
should be allowed discretion in the level of peer review selected for
individual analyses but should also state the basis for selecting
that level. 


AGENCIES OFTEN USED ECONOMIC
ANALYSES TO IDENTIFY
COST-EFFECTIVE APPROACHES
============================================================ Chapter 3

According to OMB's guidance, economic analyses should play an
important role in agencies' regulatory decision-making.  Agency
officials said that they generally used the analyses in their
decision-making, most frequently to help identify the most
cost-effective alternative that would fulfill an authorizing
statute's mandate.  Because our scope involved rules that had already
progressed to the proposed or final rulemaking stages, it was
unlikely that any of the 20 analyses we reviewed resulted in the
reversal of an agency's decision to regulate or led to major
revisions to the proposed action.  Nonetheless, agency officials told
us analyses conducted early in the rulemaking stages sometimes lead
to significant changes in agencies' decisions.  Agency officials
responsible for making regulatory decisions stated that their
decisions to regulate frequently respond to specific statutory
mandates or perceived emergencies. 


   OMB'S GUIDANCE URGES AGENCIES
   TO USE ECONOMIC ANALYSES IN
   DECISION-MAKING
---------------------------------------------------------- Chapter 3:1

OMB's guidance encourages the use of economic analyses in developing
regulations, stressing that "good data and good analysis are critical
to inform sound decision-making." However, the guidance recognizes
that the same factors that may limit the thoroughness of the analyses
may also restrict their use.  For example, the need to respond to an
emergency, meet a statutory deadline, or comply with specific
language in an authorizing statute may limit the use of an analysis. 
According to the guidance, the most critical of these factors is the
extent to which the statute affords discretion in selecting
regulatory alternatives.  But even when the statute limits an
agency's discretion, OMB's guidance urges the agency to "provide some
analysis of other regulatory options .  .  .  in order to provide
decisionmakers with information for judging the consequences of the
statutory constraints."


   MOST ANALYSES WERE USED TO
   IDENTIFY THE MOST
   COST-EFFECTIVE APPROACH
---------------------------------------------------------- Chapter 3:2

According to agency officials, nearly all of the economic analyses we
reviewed played some role in regulatory decision-making.  However,
this role was most often limited to identifying and selecting the
most cost-effective approach within a predetermined regulatory
approach.  The analyses rarely led decisionmakers to select a
significantly different alternative or fundamentally revise the
regulatory proposal under consideration.  Table 3.1 summarizes agency
officials' views on the primary uses of the 20 economic analyses. 



                               Table 3.1
                
                    Officials' Views on How Economic
                    Analyses Were Used in Regulatory
                            Decision-Making

Use of analysis                                     Number of analyses
--------------------------------------------------  ------------------
Identify the most cost-effective approach                           10
Implement health-based regulations cost-                             2
 effectively
Define regulation's coverage                                         3
Define regulation's implementation date                              1
Defend/document a regulatory decision                                2
Reduce health risks at feasible cost                                 1
Play no role in the regulatory decision                              1
----------------------------------------------------------------------
Note:  Because some of the analyses fall into more than one category,
we categorized them according to their primary use, as defined by
agency officials. 

Source:  GAO's analysis. 

The following examples show how agencies have used economic analyses
in their regulatory decision-making: 

  -- Identify the most cost-effective approach:  The economic
     analysis for EPA's proposed rule on emission standards for
     marine engines estimated manufacturers' compliance costs for
     different emission standards.  According to EPA officials who
     prepared the analyses and were involved in the decision-making
     process, the analysis clearly identified the point at which
     greater reductions in emissions would come at a dramatically
     higher cost to industry.  The EPA decisionmaker for this rule
     recalled asking her staff why EPA could not set the standards
     more stringently and being told that the analysis had
     demonstrated that the proposed standard was the most
     cost-effective of several alternatives considered. 

  -- Implement health-based regulations cost-effectively:  In some
     instances, according to EPA and the courts, regulatory decisions
     are to be based on health rather than cost or other
     considerations.  In setting primary air quality standards for
     ozone and particulate matter, EPA maintained that its first
     responsibility under the law was "to select standards that
     protect public health" with "an adequate margin of safety."
     According to EPA's and the courts' interpretation of the Clean
     Air Act, the setting of these standards is a health-based
     decision that specifically is not to be based on cost or other
     economic considerations.  Nevertheless, the agency maintains
     that economic analyses could help inform decisionmakers on ways
     to implement these health-based standards cost-effectively.  In
     addition, according to EPA, the analyses can inform the public
     about the potential costs and benefits of implementing the
     regulations. 

  -- Define a regulation's coverage:  The Federal Agriculture
     Improvement and Reform Act of 1996 authorized the Secretary of
     Agriculture to combine into one program the functions of several
     conservation programs that the act rescinded.  According to USDA
     officials involved in the decision-making process, the economic
     analysis prepared for the implementing rule played "a tremendous
     role" in defining the "livestock operations" that are covered by
     the rule.  Because the definition of the rule's coverage was
     politically contentious, the analysis also provided the agency
     with a basis for defending its decision. 

  -- Reduce health risks at feasible cost:  According to the preamble
     to OSHA's rule on methylene chloride, the agency determined, on
     the basis of new animal and human data, that current standards
     place employees at "a significant risk of material impairment of
     health." The preamble also states that OSHA's standards must be
     "highly protective" as long as they are technologically and
     economically feasible.  The preamble then concludes, on the
     basis of OSHA's economic analyses, that "the rule is the most
     cost-effective alternative for implementation of OSHA's
     statutory objective of reducing significant risk to the extent
     feasible."

  -- Define a regulation's implementation date:  EPA's economic
     analysis for a proposed rule on procedures for testing emissions
     from motor vehicles incorporated data provided by the automobile
     industry and led to revisions that gave the industry additional
     time to implement the final rule.  After EPA published an
     initial cost analysis as part of a proposed rule, the industry
     questioned the validity of EPA's data and provided more current
     data.  EPA then adjusted its cost calculations, dropped one
     component of its proposal, and extended the deadline for
     implementing the final rule. 

  -- Defend a regulatory decision:  According to FDA officials, the
     economic analysis for a rule on manufacturing medical devices
     provided the agency with a credible rebuttal to manufacturers'
     complaints that compliance costs would be excessive.  USDA
     officials also told us that they sometimes use their analyses to
     defend controversial regulatory decisions. 

  -- Play no role in the decision-making process:  The economic
     analysis supporting FDA's final rule exempting small businesses
     from food-labeling requirements played virtually no role in the
     decision-making process.  Because the authorizing
     legislation--the 1993 amendments to the Federal Food, Drug, and
     Cosmetic Act--was so specific about who would be eligible for
     the exemption, the analysis was not really necessary, FDA
     officials said. 


   STATUTES LIMITED THE USE OF
   ECONOMIC ANALYSES
---------------------------------------------------------- Chapter 3:3

According to agency officials, economic analyses are generally used
for the purposes summarized in table 3.1 and are less frequently used
for deciding whether or not to regulate or for identifying
significantly different regulatory approaches.  Agency officials told
us that statutory mandates frequently limited their discretion in
deciding whether to regulate and/or in selecting alternative
regulatory approaches.  In one instance, the statute was so specific
that officials described the rule as not much more than a photocopy
of the law.  In addition, officials cited instances in which the
agency believed that it had little discretion or time to react to an
emergency situation.  The following are some of the other instances
cited by agency officials in which the agency issued regulations in
response to statutory directives or emergencies: 

  -- In the Clean Air Act Amendments of 1990, the Congress directed
     EPA, within 18 months, to review and revise as necessary its
     regulations on testing motor vehicles and engines to ensure that
     the tests reflect actual, current driving conditions, including
     conditions related to fuel, temperature, acceleration, and
     altitude.  Because the agency concluded that the current test
     procedures had shortcomings in representing, among other things,
     aggressive driving, rapid speed fluctuations, and the use of air
     conditioning, EPA decided new regulations were warranted. 

  -- The National Highway Traffic Safety Administration Authorization
     Act of 1991, among other things, directed the Secretary of
     Transportation to determine whether additional regulations were
     needed to ensure the safety of child seats used in motor
     vehicles.  In studying this issue, DOT concluded that because so
     many different types of seat belts were in use, the child
     restraints were difficult to attach correctly to improve safety. 
     Accordingly, the Department proposed a regulation requiring the
     use of a specific attachment system.  The proposed rule noted
     that there were data gaps in the economic analyses and stated
     that if new information became available, DOT would consider
     other possible alternatives. 

  -- USDA issued emergency quarantine regulations after the Karnal
     Bunt disease was detected in Arizona and California.  The
     regulations were issued about 6 months before the economic
     analysis was completed and published.  Karnal Bunt is a serious
     fungal disease that can affect both the yield and quality of
     wheat.  Although it does not present a risk to human or animal
     health, it makes wheat taste like fish and can dramatically
     affect wheat sales at home and abroad.  Many countries prohibit
     the import of wheat from countries where Karnal Bunt is known to
     exist.  Although the economic analysis played no role in the
     initial quarantine, USDA officials told us that it was useful in
     later decisions about the number and location of acres subject
     to the quarantine. 


MAJOR CONTRIBUTORS TO THIS REPORT
=========================================================== Appendix I

RESOURCES, COMMUNITY, AND ECONOMIC
DEVELOPMENT DIVISION

Luther L.  Atkins, Assistant Director
Greg A.  Kosarin, Evaluator-in-Charge
Jay R.  Cherlow, Assistant Director for Economic Analysis
Timothy J.  Guinane, Senior Economist
Stephen M.  Brown, Senior Economist

OFFICE OF CHIEF ECONOMIST

Harold J.  Brumm, Senior Economist

OFFICE OF GENERAL COUNSEL

Karen Keegan, Senior Attorney

BOSTON FIELD OFFICE

Maureen T.  Driscoll, Senior Evaluator
Jerry Laudermilk, Senior Evaluator


*** End of document. ***