[Analytical Perspectives]
[Ranking Regulatory Investments in Public Health]
[24. Ranking Regulatory Investments in Public Health]
[From the U.S. Government Publishing Office, www.gpo.gov]
[[Page 417]]
========================================================================
RANKING REGULATORY INVESTMENTS IN PUBLIC HEALTH
========================================================================
[[Page 419]]
24. RANKING REGULATORY INVESTMENTS IN PUBLIC HEALTH \1\
---------------------------------------------------------------------------
\1\This chapter is prepared pursuant to Section 624 of the Treasury
and General Government Appropriations Act, 2001, also known as the
``Regulatory Right to Know Act,'' Public Law 106-554 (Dec. 21, 2000).
---------------------------------------------------------------------------
An essential role of government is to protect citizens from risks to
human health, safety and the environment. Since the nation does not
possess enough resources to eliminate all risks, an important
performance goal for government is to deploy risk-management resources
in a way that achieves the greatest public health improvement for the
resources available--that is the most ``cost-effective'' allocation of
risk-management resources. In this chapter, we demonstrate how cost-
effectiveness ratios can be used to compare the payoffs from different
regulatory investments in public health. We also discuss the promise and
limitations of the use of cost-effectiveness analysis to inform
decisions at regulatory agencies.
Using Cost-Effectiveness Ratios to Construct League Tables
A widely used tool for ranking purposes is the ``league table,'' first
used by the English to rank their soccer teams by point standings and
later to rank their schools by student achievement scores. More
recently, league tables have been used to rank programs, technologies,
regulations and therapies aimed at saving lives and improving public
health. There is a significant academic literature on the use of league
tables in public health that began in the 1960s and continues to grow.
OMB believes that government and the public can benefit from the
insights generated by league tables.
The OMB first published a league table with the Budget in 1992. In
this table, 50 risk-reducing regulations were ranked using cost per life
saved as the measure of investment performance. The information in that
table was based on analyses by Federal agencies and others in the 1970s
and 1980s. The monetary resources required to save one ``statistical''
life ranged from several hundred thousand dollars to billions of
dollars.
In Table 24-1 below, OMB presents a league table of 10 risk-reducing
regulations based on information developed by three Federal agencies
(DOT, OSHA, and EPA) in the 1995 to 2000 period. Our purpose in
presenting this table is to illustrate how cost-effectiveness analysis
of public health has changed over the last decade and what technical and
policy issues are raised by presentation of league tables. \2\
---------------------------------------------------------------------------
\2\ The technical details that support the information presented in
Table 24-1, including ratios based on a ``lives saved'' metric, can be
found at www.whitehouse.gov/omb under regulatory policy or upon request.
---------------------------------------------------------------------------
Table 24-1. COST PER LIFE-YEAR SAVED FOR TEN SELECTED REGULATIONS
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Cost per life-year saved
Regulation Health or Safety Net Costs ($2001) Life-years saved ($2001)
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Petroleum Refining NESHAP (EPA)................................. Health <0 <10 per year <0
Powered Industrial Truck Operating Training (OSHA).............. Safety <0 146 per year <0
Head Impact Protection (DOT).................................... Safety $390 to $516 million per year 8,360 to 10,007 per year $50,00 to $53,000
Reflective Devices for Heavy Trucks (DOT)....................... Safety $65 million (PV) 946 (PV) $69,000
Child Restraints (DOT).......................................... Safety $54 to $112 million per year 370 to 515 per year $105,000 to $331,000
Rail Roadway Workers (DOT) a.................................... Safety $227 million (PV) 434 (PV) $523,000
Interim Enhanced Surface Water Treatment (EPA) b................ Health <0 to $95 million per year 140 to 640 per year <0 to $679,000
NOx SIP Call (EPA) c............................................ Health $1265 million in 2007 1590 to 3390 per year $373,000 to $714,000
Methylene Chloride (OSHA) d..................................... Health $112 million per year 96 per year $1.16 million
Stage I Disinfection By-Products (EPA) e........................ Health <0 to $764 million per year 0 to 5130 per year <0 to infinite
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Note: Net costs were calculated by subtracting from compliance costs an estimate of any non-fatality benefits such as a reduction in injuries or morbidity. PV = Present Value.
a The estimate does not include possible increased capacity of rail lines and improved worker morale.
b The estimate doe not include reduced risks from the pathogens (in addition to cryptosporidiosis) and avoided costs of averting behavior from a well-publicized outbreak.
c The estimate does not include a variety of potential benefit categories including possible reductions in ozone-related mortality, acute and chronic respiratory damage, nitrogen deposition in
estuarine and coastal waters, damage to ecosystems and vegetation.
d The estimate does not include a variety of potential adverse health effects including: cancers resulting from dermal contact, central nervous system effects, and eye, nose, etc. irritation.
e The estimate does not include possible reductions in colon and rectal cancer and possible reductions in adverse reproductive and developmental effects.
[[Page 420]]
These ten rules, which are a non-random sample of risk-related
rulemakings, were selected because the regulatory analyses provided
sufficient information to prepare a cost-effectiveness ratio. Many
agency rules, even those with a primary purpose of protecting public
health, do not provide adequate information to construct a cost-
effectiveness ratio. The estimates presented in the table are based on
data and estimates provided by the agencies. Where the agencies did not
provide estimates of life-years saved, we calculated life-years using
standard assumptions about age and life expectancies. Each of the ten
rules was reviewed by OMB under Executive Order 12866; five address
health issues and five address safety issues.
Interestingly, the tendency for safety rules to be more cost-effective
than health rules (see Table 24-1) is consistent with the insights from
the early league tables published more than a decade ago. The table also
illustrates a finding not evident from the earlier league tables. The
range of cost-effectiveness estimates for specific rules varies
substantially. For example, the cost per life-year saved for EPA's
disinfection by-products rule ranges from less than zero to infinite.
The table suggests that we need to do a better job at both refining
estimates of the cost-effectiveness of regulatory proposals and setting
priorities for the use of the nation's limited resources to protect
citizens from health, safety, and environmental risks.\3\
---------------------------------------------------------------------------
\3\ OMB set forth its program to improve regulatory outcomes in Making
Sense of Regulation: 2001 Report to Congress on the Costs and Benefits
of Regulations and Unfunded Mandates on State, Local, and Tribal
Entities (OMB 2001) available on our website at www.whitehouse.gov/obm/
inforeg/costbenefitreport.pdf or upon request.
---------------------------------------------------------------------------
Which Rules Should Be Compared?
In constructing a league table, many issues arise about which rules to
include. League tables are most useful if based on information about
potential or proposed rules, since the decision makers can consider
reallocating resources to those rulemaking opportunities that rank the
highest in cost-effectiveness. The challenge is ensuring that league
tables are generated early enough in the decision making process to
inform regulatory priorities.
When league tables include only recently adopted (final) rules, the
utility for policy makers is reduced. Once the agency has adopted a
rule, it is difficult to reverse course based on a ranking reported in a
league table. Moreover, it may be infeasible for an agency to adopt
``more'' of a final rule that ranks highly in a league table.
Nonetheless, league tables of adopted rules can provide insight into
their relative payoffs, which can provide a rough perspective to
evaluate future rules.
An intra-agency league table compares only those rules within the
jurisdiction of a particular agency. This type of table is appropriate
in certain budgetary contexts where only matters in the jurisdiction of
a specific agency are subject to comparison, ranking, and decision
making. An inter-agency league table, such as Table 24-1, is more useful
for synoptic purposes or for decision making by governmental entities
with inter-agency responsibility (e.g., appropriations committees and
OMB).
Identifying a Performance Measure
Early league tables in the public health field used the number of
lives saved (premature deaths averted) as the metric of effectiveness.
This metric has been criticized on the grounds that lives are never
really saved, only extended. The expected number of life-years saved was
developed as an alternative and continues to be used in the academic
literature. ``Life-years'' gives relatively more credit to rules that
reduce mortality rates early in the lifespan and less weight to rules
that reduce mortality rates late in the lifespan. Although it is
sometimes argued that ``life-years'' discriminates against the elderly,
there are strong arguments that ``life-years'' is a better measure than
``lives'' of the effectiveness of regulatory alternatives.
Which Costs Should be Counted?
If one were only concerned about impacts on the Federal budget, then
the only regulatory costs that would be counted would be those incurred
(or saved) by a Federal agency. To reflect the full effect of a
regulation, all costs to society--whether Federal, State, or private
costs--should be counted when cost-effectiveness ratios are computed.
This ``societal perspective'' on cost estimation is already embraced in
OMB guidance and is widely practiced by Federal agencies and academic
analysts.
Rulemakings may also yield cost savings (e.g., energy savings
associated with using new technologies). It is generally accepted that
the numerator in the cost-effectiveness ratio presented in a league
table should be based on net costs, defined as the total cost incurred
in meeting the requirements minus any cost savings. Similarly, the
denominator of the ratio should reflect net life-years saved if the rule
has both beneficial and adverse impacts on public health, such as the
side effects of a vaccine.
Should Future Costs and Health Effectiveness be Discounted to Their
Present Value?
Analysts generally agree that future costs and health effects should
be discounted at the same rate, but there is a range of opinion about
the appropriate rate of discount (e.g., 3 to 7 percent). If future
health savings were discounted at a lower rate than future costs, then
it can be shown that it always makes sense to delay adoption of a cost-
effective rule. We have generally used 7 percent in our calculations,
but following EPA's practice we have used a 5 percent discount rate in
calculating life-years for EPA rules.
Limitations of League Tables
Generally, league tables are most helpful for comparing a set of
government actions with the same primary outcome (e.g., a reduction in
premature mortality risk or acres of wetlands saved). Where an action
yields a variety of beneficial outcomes, the comparison be
[[Page 421]]
comes more problematic because these multiple benefits all need to be
considered. Where the agency analysis provides a monetary estimate for
these other benefit categories, we have subtracted the value of these
benefits from the aggregate cost estimate to yield a net cost estimate.
In some cases, the resulting net cost estimate for the rule is
negative--that is, the other (non-mortality) benefits exceed the cost of
the rule. Where the agency analysis fails to provide estimates for key
benefit categories, the cost-effectiveness ratio may be overstated
substantially, and thus, the regulatory action may be a more attractive
candidate than suggested by the league table. For rules that have
significant ecological as well as public health benefits, it is not
clear how to construct a league table. Ecological benefits deserve
serious consideration, but it is infeasible to express them in the same
units as public health benefits. Finally, in some cases, the mortality
reduction benefits may be largely ancillary to the overall effect of the
rule, and the opportunity for realizing cost-effective improvements in
risk reduction may be limited because the risk reduction gains are
relatively small.
One of the most common ancillary benefits of lifesaving rules is a
reduction in morbidity--i.e., the number of cases of nonfatal illness or
injury. To account for such benefits, OMB is considering the use of new
effectiveness measures that combine information on mortality and
morbidity. Two such measures are already in widespread use in the
academic literature. The ``quality-adjusted life-year'' (QALY) measure
rates each year of life on a 0 to 1.0 scale based on an expert panel or
patient assessment of the quality of life associated with different
health states. The QALY measure is widely used in the medical literature
in both the USA and Europe and has recently been recommended for use by
an expert panel assembled by the U.S. Department of Health and Human
Services. A close cousin to the QALY, the disability-adjusted life-year
(DALY) measure, is widely used in the developing world and has been
promoted by the World Health Organization and the World Bank. While the
QALY measure values equally all healthy years of life, the DALY measure
gives the greatest weight to healthy life-years in the prime of life,
since these years--whether through work or child rearing--make a major
contribution to societal production.
Strictly speaking, ranking regulatory investments based on cost-
effectiveness ratios focuses on economic efficiency. Decision makers may
desire (or be required) to consider other values as well (e.g., various
notions of fairness and equity). There is no accepted approach to
incorporating equity considerations into a league table. However, there
is broad consensus that a qualitative description of equity and fairness
concerns should be presented to regulators in a rulemaking process and
such considerations are clearly authorized for consideration under E.O.
12866.
Taking Steps Toward Cost-Effectiveness in the Regulatory Process
OMB is in the process of taking modest steps toward greater use of
cost-effectiveness and league tables in decision making. First, OMB has
issued government-wide guidelines on information quality that will
promote greater transparency and consistency in agency analyses of
health and safety risks. The development of league tables as an
analytical construct depends on achieving a degree of analytical
consistency across agency evaluation of health and safety risks. Second,
OMB has committed to update periodically its guidelines for regulatory
analysis, which are used when OMB reviews agency rulemakings. OMB
intends to use guideline revision as a vehicle to consider the analytic
measures of effectiveness and performance used by agencies and the
informational burdens associated with moving toward greater analytic
consistency in agency practices. This multi-year process will involve
analysts from multiple agencies and will include opportunities for
public comment and peer review.
While this approach has been more fully developed in the public health
and medical literature, it can be applied to other types of programs.
One of the key challenges in extending this analysis into other areas,
of course, is developing a suitable measure of the effectiveness of
disparate programs directed toward enhancing other aspects of the
nation's welfare (e.g., recreational opportunities). OMB encourages
agencies to develop objective measures of program effectiveness that can
facilitate cost-effectiveness analysis.