Consumer Product Safety Commission: Better Data Needed to Help Identify
and Analyze Potential Hazards (Letter Report, 09/29/97, GAO/HEHS-97-147).
Pursuant to a congressional request, GAO reviewed the Consumer Product
Safety Commission's (CPSC) project selection, use of cost-benefit
analysis and risk assessment, and information release procedures,
focusing on: (1) the criteria CPSC uses to select projects and the
information it relies upon in making these choices; (2) the information
CPSC draws on to perform risk assessment and cost-benefit analyses and
CPSC's methodology for conducting cost-benefit analyses; and (3) CPSC's
procedures for releasing manufacturer-specific information to the public
and whether evidence exists that CPSC violated its statutory
requirements concerning the release of such information.
GAO noted that: (1) although CPSC has established criteria to help
select new projects, with the agency's current data these criteria can
be measured only imprecisely, if at all; (2) although CPSC has described
itself as "data driven," its information on product-related injuries and
deaths is often sketchy; (3) this makes it more difficult not only for
agency management to monitor current projects but also for staff and
commissioners to assess and prioritize the need for new projects in
different hazard areas; (4) CPSC has insufficient data on both internal
agency efforts and external product hazards to assess the impact and
cost of each project; (5) to help evaluate alternative methods of
addressing potential hazards, CPSC may perform a risk assessment to
estimate the likelihood of injury associated with a hazard or conduct a
cost-benefit analysis to assess the potential effects of a proposed
regulation; (6) although CPSC does not complete either a risk assessment
or cost-benefit analysis for every project, the agency conducts these
analyses more often than it is required to by law; (7) nevertheless,
CPSC's data are often insufficient to support a thorough application of
these analytical techniques; (8) to evaluate relative risks, it is
usually necessary to have information on how many consumers use the
product--information that CPSC frequently does not have; (9) risk
assessment of consumer products requires measurement of the number of
harmful incidents; (10) CPSC's imprecise and incomplete death and injury
data make risk assessment and cost-benefit analysis at best less
reliable and at worst impossible to do; (11) the cost-benefit analyses
conducted by CPSC between 1990 and 1996 were often not comprehensive;
(12) CPSC has established procedures to implement statutory requirements
concerning the release of manufacturer-specific information; (13) when
releasing information to the public that identifies a specific
manufacturer, CPSC is required to verify the information and allow the
manufacturer an opportunity to comment; (14) evidence from the industry
and from legal cases suggests that CPSC has met its statutory
requirements in this area; (15) individuals within CPSC, as well as some
industry representatives and consumer groups, expressed dissatisfaction
with the requirements of this law; and (16) some of these individuals
have proposed statutory changes that range from reducing to expanding
the current requirements.
--------------------------- Indexing Terms -----------------------------
REPORTNUM: HEHS-97-147
TITLE: Consumer Product Safety Commission: Better Data Needed to
Help Identify and Analyze Potential Hazards
DATE: 09/29/97
SUBJECT: Consumer protection
Statistical data
Data collection
Data integrity
Statistical methods
Cost effectiveness analysis
Safety regulation
Reporting requirements
Product safety
Management information systems
IDENTIFIER: CPSC National Electronic Injury Surveillance System
******************************************************************
** This file contains an ASCII representation of the text of a **
** GAO report. Delineations within the text indicating chapter **
** titles, headings, and bullets are preserved. Major **
** divisions and subdivisions of the text, such as Chapters, **
** Sections, and Appendixes, are identified by double and **
** single lines. The numbers on the right end of these lines **
** indicate the position of each of the subsections in the **
** document outline. These numbers do NOT correspond with the **
** page numbers of the printed product. **
** **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced. Tables are included, but **
** may not resemble those in the printed version. **
** **
** Please see the PDF (Portable Document Format) file, when **
** available, for a complete electronic file of the printed **
** document's contents. **
** **
** A printed copy of this report may be obtained from the GAO **
** Document Distribution Center. For further details, please **
** send an e-mail message to: **
** **
** **
** **
** with the message 'info' in the body. **
******************************************************************
Cover
================================================================ COVER
Report to the Chairmen, Committee on Commerce, Science, and
Transportation, U.S. Senate, and Committee on Commerce, House of
Representatives
September 1997
CONSUMER PRODUCT SAFETY COMMISSION
- BETTER DATA NEEDED TO HELP
IDENTIFY AND ANALYZE POTENTIAL
HAZARDS
GAO/HEHS-97-147
Consumer Product Safety Commission
(205333)
Abbreviations
=============================================================== ABBREV
AAPCC - American Association of Poison Control Centers
ANPR - Advance Notice of Proposed Rulemaking
ATV - all-terrain vehicle
CO - carbon monoxide
CPSC - Consumer Product Safety Commission
FOIA - Freedom of Information Act
HMO - health maintenance organization
IPII - Injury or Potential Injury Incident
LP - liquified petroleum
MECAP - Medical Examiners' and Coroners' Alert Program
MIS - management information system
NEISS - National Electronic Injury Surveillance System
NFIRS - National Fire Reporting System
NFPA - National Fire Protection Association
OMB - Office of Management and Budget
Letter
=============================================================== LETTER
B-274042
September 29, 1997
The Honorable John McCain
Chairman, Committee on Commerce,
Science, and Transportation
United States Senate
The Honorable Thomas J. Bliley, Jr.
Chairman, Committee on Commerce
House of Representatives
Every year, children are hurt while using playground equipment; users
or spectators are injured when fireworks explode unexpectedly; and
homeowners are cut while operating chain saws and lawn mowers.
Created to protect consumers from "unreasonable risk of injury," the
U.S. Consumer Product Safety Commission (CPSC) oversees these and
about 15,000 other consumer products ranging from kitchen appliances
and children's toys to hot tubs and garage door openers. With a
budget of about $42.5 million, CPSC carries out its mission by (1)
enforcing federal consumer product regulations (by recalling products
from store shelves, for example) and (2) conducting projects to
address products with potential hazards not covered by existing
regulations.\1
These projects may result in CPSC issuing new regulations concerning
specific products, assisting in the development of voluntary industry
standards, or providing information to consumers about how to use the
products safely.
Contending that the agency is ineffectively allocating its resources,
CPSC's critics have voiced dissatisfaction with the selection of
certain agency projects and have questioned the validity of CPSC's
risk assessment and cost-benefit analyses supporting those projects.
In addition, congressional and interest group critics have questioned
the agency's procedures for ensuring the accuracy of
manufacturer-specific information before releasing it to the public,
contending that such releases can mar the reputation of responsible
corporate citizens.
In light of these concerns, you asked us to review CPSC's project
selection, use of cost-benefit analysis and risk assessment, and
information release procedures. Specifically, this report (1)
identifies the criteria CPSC uses to select projects and reviews the
information it relies upon in making these choices, (2) assesses the
information CPSC draws on to perform risk assessment and cost-benefit
analyses and evaluates the agency's methodology for conducting
cost-benefit analyses, and (3) describes CPSC's procedures for
releasing manufacturer-specific information to the public and reports
whether evidence exists that CPSC violated its statutory requirements
concerning the release of such information.
To address these objectives, we reviewed internal CPSC documents,
relevant legislation and regulations, and the literature on
cost-benefit analysis and on consumer product safety issues. We
interviewed CPSC commissioners and staff, four former commissioners,
consumer advocates, industry representatives, and outside experts to
obtain their perspectives on CPSC's work. We identified CPSC
projects by compiling from various agency documents a list of 115
potential product hazards examined by the agency from January 1990 to
September 30, 1996, and we reviewed available agency documentation on
each of these projects. We examined the agency's internal databases
to obtain project information and to assess the agency's information
on product hazards. In reviewing CPSC's cost-benefit analyses, we
consulted with experts to develop objective criteria that reflected
elements commonly used in the evaluation of cost-benefit analyses.
These evaluation questions were designed to elicit whether CPSC
conducted comprehensive analyses and reported them in sufficient
detail. However, they do not make up a complete measure of the
quality of a cost-benefit analysis; for example, our evaluation
assessed whether these elements were included but not how well they
were measured or incorporated. We reviewed CPSC's internal
procedures concerning information clearance and release, and we
examined relevant legal cases in this area. For more detailed
information on our scope and methodology, see appendix I.
--------------------
\1 Projects vary widely in scope, and CPSC has no standard definition
of what constitutes a project. For our review, we defined a
"project" as work CPSC conducted on any specific consumer product
that was associated with a potential hazard or hazards not covered by
existing regulation.
RESULTS IN BRIEF
------------------------------------------------------------ Letter :1
Although CPSC has established criteria to help select new projects,
with the agency's current data these criteria can be measured only
imprecisely, if at all. The criteria for selecting projects include
the number of product-related injuries, chronic illnesses, and
deaths. However, although CPSC has described itself as "data
driven," its information on product-related injuries and deaths is
often sketchy. For example, because the agency's measure of injuries
generally includes only hospital emergency room reports, CPSC has an
incomplete picture of injuries. Similarly, CPSC's data on
product-related deaths understate the total number of deaths and are
generally available to the agency only with a 2-year time lag. In
addition, CPSC does not maintain a comprehensive list or description
of either its past or ongoing projects. This makes it more difficult
not only for agency management to monitor current projects but also
for staff and commissioners to assess and prioritize the need for new
projects in different hazard areas. As a result, CPSC has
insufficient data on both internal agency efforts and external
product hazards to assess the impact and cost of each project, either
when it is selected or after it has been implemented.
To help evaluate alternative methods of addressing potential hazards,
CPSC may perform a risk assessment to estimate the likelihood of
injury associated with a hazard or conduct a cost-benefit analysis to
assess the potential effects of a proposed regulation. Although CPSC
does not complete either a risk assessment or cost-benefit analysis
for every project, the agency conducts these analyses more often than
it is required to by law. Nevertheless, CPSC's data are often
insufficient to support a thorough application of these analytical
techniques--a problem that frequently arises in doing both risk
assessment and cost-benefit analysis. To evaluate relative risks, it
is usually necessary to have information on how many consumers use
the product--information that CPSC frequently does not have. In
addition, risk assessment in a consumer product context requires
measurement of the number of harmful incidents. CPSC's imprecise and
incomplete death and injury data make risk assessment and
cost-benefit analysis at best less reliable and at worst impossible
to do. Furthermore, the cost-benefit analyses conducted by CPSC
between 1990 and 1996 were often not comprehensive, and the reports
on these analyses were not sufficiently detailed. For example,
experts generally agree that sensitivity analysis--a technique that
enables the reader to determine which assumptions, data limitations,
or parameters are most important to the conclusions--should be
incorporated in cost-benefit analyses. Most of CPSC's cost-benefit
analyses did not include such information. We are making
recommendations to the Chairman of CPSC to improve the agency's
project selection and cost-benefit analyses.
CPSC has established procedures to implement statutory requirements
concerning the release of manufacturer-specific information. When
releasing information to the public that identifies a specific
manufacturer--for example, in a safety alert or recall notice--CPSC
is required to verify the information and allow the manufacturer an
opportunity to comment. Evidence from the industry and from legal
cases suggests that CPSC has met its statutory requirements in this
area. Individuals within CPSC, as well as some industry
representatives and consumer groups, expressed dissatisfaction with
the requirements of this law. Some of these individuals have
proposed statutory changes that range from reducing to expanding the
current requirements.
BACKGROUND
------------------------------------------------------------ Letter :2
CPSC was created in 1972 under the Consumer Product Safety Act (P.L.
92-573) to regulate consumer products that pose an unreasonable risk
of injury, to assist consumers in using products safely, and to
promote research and investigation into product-related deaths,
injuries, and illnesses. CPSC currently has three commissioners, who
are responsible for establishing agency policy.\2
One of these commissioners is designated the chairman; the chairman
directs all the executive and administrative functions of the agency.
The Consumer Product Safety Act consolidated federal safety
regulatory activity relating to consumer products within CPSC. As a
result, in addition to its responsibilities for protecting against
product hazards in general, CPSC also administers four laws that
authorize various performance standards for specific consumer
products. These laws are the Flammable Fabrics Act (June 3, 1953,
c.164), which authorizes flammability standards for clothing,
upholstery, and other fabrics; the Federal Hazardous Substances Act
(P.L. 86-613), which authorizes the regulation of substances that
are toxic, corrosive, combustible, or otherwise hazardous; the Poison
Prevention Packaging Act of 1970 (P.L. 91-601), which authorizes
requirements for child-resistant packaging for certain drugs and
other household substances; and the Refrigerator Safety Act of 1956
(Aug. 2, 1956, c.890), which establishes safety standards for
household refrigerators.
In fiscal year 1997, CPSC carries out this broad mission with a
budget of about $42.5 million and a full-time-equivalent staff of
480. As figure 1 shows, after adjusting for inflation, the agency's
budget has decreased by about 60 percent since 1974. Similarly,
CPSC's current staffing level represents 43 percent fewer positions
as compared with the agency's 1974 staff.
Figure 1: CPSC Funding, in
Inflation-Adjusted Dollars,
1974-96
(See figure in printed
edition.)
Notes: Budget figures were obtained from CPSC. The 1993 budget
figure excludes a special appropriation for office space relocation.
Budget figures were adjusted for inflation using the Gross Domestic
Product deflator for federal nondefense spending, with 1992 as the
base year.
CPSC uses a number of regulatory and nonregulatory tools to reduce
injuries and deaths associated with consumer products. Under several
of the laws that it administers, CPSC has the authority to issue
regulations that establish performance or labeling standards for
consumer products. For example, in 1993, CPSC issued regulations
under the Consumer Product Safety Act requiring disposable cigarette
lighters to be child-resistant. If CPSC determines that there is no
feasible standard that would sufficiently address the danger, CPSC
may issue regulations to ban the manufacture and distribution of the
product. In addition, under the Consumer Product Safety Act, if a
product violates a safety regulation or presents a "substantial
hazard," CPSC may order a product recall, in which the item is
removed from store shelves and consumers are alerted to return the
item for repair, replacement, or refund.\3 CPSC can also impose civil
penalties for violations of federal safety standards.
Although CPSC has these broad regulatory powers, much of the agency's
efforts are carried out using nonregulatory methods. In addition to
federally mandated product safety standards, many consumer products
are covered by voluntary standards. These voluntary standards, which
are often established by private standard-setting groups, do not have
the force of law. However, many voluntary standards are widely
accepted by industry.\4 The 1981 amendments to the Consumer Product
Safety Act require CPSC to defer to a voluntary standard--rather than
issue a mandatory regulation--if CPSC determines that the voluntary
standard adequately addresses the hazard and that there is likely to
be substantial compliance with the voluntary standard. As a result,
voluntary standards development is an important tool in CPSC's
hazard-reduction efforts. For example, in 1996 CPSC helped a private
group develop a voluntary standard to address the risk of children
getting their heads stuck between the slats of toddler beds, and in
1995 CPSC assisted a standard-setting group in upgrading safety
standards to prevent fires associated with Christmas tree lights.
CPSC also addresses product hazards by providing information to
consumers on safety practices that can help prevent product-related
accidents. For example, to encourage consumers to use electricity
safely--and particularly to promote the use of ground fault circuit
interrupters--CPSC conducted a far-reaching publicity campaign that
included radio public service announcements, messages printed on
carryout bags for hardware stores, a joint press conference with
industry representatives, presentations on television's Home Shopping
Network, and promotional letters to real estate and home inspection
associations. In addition to its own active efforts to disseminate
information, CPSC provides considerable amounts of information in
response to requests from the public. Like other federal agencies,
CPSC must comply with the Freedom of Information Act (FOIA) when
responding to requests from the public for information. A notable
feature of FOIA is its presumption in favor of disclosure: any
person has the right to inspect and copy any government records
unless the documents requested fall within one of the exemptions to
the act (for example, disclosure of trade secrets). FOIA requests
may come to CPSC from regulated industries, the press, consumer
groups, or individuals. During calendar year 1995, CPSC responded to
16,424 formal requests made under FOIA.
CPSC's resource base and extensive jurisdiction require the agency to
select among potential product hazards. New initiatives may come to
CPSC in several ways. First, any person may file a petition asking
CPSC to issue, amend, or revoke a regulation. Petitions, which can
be as simple as a letter or as formal and detailed as a legal brief,
have come to CPSC from doctors and nurses, consumers and advocacy
groups, and industry representatives. CPSC may grant or deny a
petition either in full or in part. Even when CPSC denies a petition
and declines to issue a regulation, it may still begin a project to
address the hazard by promoting a voluntary standard or conducting a
consumer education campaign. For example, a project on heat tapes
(heated wraps for exposed pipes) originated with a petition from a
concerned consumer. CPSC denied the petition for a mandatory
standard, but conducted a research study and review of the existing
voluntary standard for heat tapes.
Second, CPSC receives some product hazard projects from the Congress.
The Congress may require CPSC to study a wide-ranging product area.
For example, the Consumer Product Safety Improvement Act of 1990
resulted in a large body of work on products affecting indoor air
quality, including wood stoves, kerosene heaters, and carpets. The
Congress may also direct CPSC to impose a specific regulation, such
as when it directed CPSC to require additional labeling on toys
intended for children aged 3 to 6 warning parents of possible choking
hazards when the toy is used by children under age 3.\5
Finally, CPSC commissioners and agency staff may initiate projects or
suggest areas to address. CPSC gathers death and injury information
to help identify potential product hazards and also obtains input
from the public. The agency maintains a toll-free hot line and an
Internet site that the public can use to notify agency staff of a
possible product hazard. In addition, CPSC holds public meetings to
get input on possible hazards to address and on which hazards should
receive priority. For example, CPSC increased its efforts to remove
drawstrings from children's clothing after receiving a letter from a
woman whose daughter was strangled when her jacket string caught on a
playground slide.
The selection of projects to address new product hazards takes place
at different levels of the agency throughout the year. For a
petition, the commissioners decide whether the product hazard
warrants further agency involvement, and the commissioners vote on
whether to grant or deny the petition. If a project is believed to
have a high potential for regulatory action or involve a substantial
amount of agency resources, the commissioners decide whether to
pursue it. Projects of this caliber are often noted in the agency's
annual budget and operating plan, which must also be approved by
commissioner vote.\6 Staff request a decision on such a project by
preparing a briefing package about the product hazard for the
commissioners, who vote to begin the regulatory process, take some
other action, or terminate the project. Agency staff generally may
decide to initiate projects that are unlikely to result in
regulation, and no briefing package is sent to the commissioners for
a vote.\7
(Of the 115 CPSC projects we identified, 80 (70 percent) were
detailed in briefing packages.)
The scope of the agency's projects varies greatly, and CPSC has no
standard definition of what constitutes a project. A project might
cover general product areas, such as fire hazards, or address only a
specific product, like cigarette lighters. A project might require
undertaking an extensive research study or providing technical
assistance to a group that is developing a voluntary standard.
--------------------
\2 The Consumer Product Safety Act provides for the appointment of
five commissioners by the President of the United States for
staggered 7-year terms. However, since 1986, no more than three
commissioners have served at one time.
\3 In practice, CPSC rarely uses its regulatory power to order a
recall, but works cooperatively with manufacturers to carry out
recalls.
\4 Voluntary standards may benefit manufacturers by giving consumers
added confidence in a product, providing some degree of protection
from product liability, and allowing manufacturers to benefit from
the safety expertise developed by voluntary standards groups. In
addition, although federal law does not compel manufacturers to
comply with voluntary standards, state or local regulations may
incorporate some voluntary standards regarding consumer products, and
some retailers prefer to carry only those goods that comply with the
applicable voluntary standards.
\5 This mandate was imposed in the Child Safety Protection Act (P.L.
103-267, June 16, 1994).
\6 Commission votes are also taken in several other instances, such
as initiating a regulation, the use of the agency logo by outside
groups, and the application of civil penalties.
\7 For more information about the agency's organization and
structure, see app. II.
CPSC HAS LIMITED INFORMATION
AVAILABLE TO ASSIST IN PROJECT
SELECTION
------------------------------------------------------------ Letter :3
The bulk of CPSC's workload is made up of projects selected by the
agency rather than by the Congress. CPSC has established criteria to
help in project selection, such as the numbers of deaths and injuries
associated with a product. However, CPSC is unable to accurately
measure these criteria because its data on potential hazards are
incomplete. In addition, CPSC does not maintain systematic
information on past and ongoing projects, which makes it difficult to
assess and prioritize the need for new projects in different hazard
areas. The lack of comprehensive data on individual product hazards
and on agency initiatives raises questions about CPSC's ability to
evaluate its own effectiveness--which it is now required to do under
the Government Performance and Results Act of 1993 (the Results Act).
AGENCY HAS CONSIDERABLE
DISCRETION IN PROJECT
SELECTION
---------------------------------------------------------- Letter :3.1
CPSC has wide latitude over which potential product hazards it
targets for regulatory and nonregulatory action. Although it has
little or no discretion over projects mandated by the Congress, CPSC
can choose to accept or reject suggestions that are submitted by
petition or proposed by the agency staff. As shown in figure 2, 59
percent of CPSC projects were initiated by CPSC, 30 percent
originated from a petition, and about 11 percent resulted from
congressional mandates. Of the 115 projects the agency worked on
from January 1, 1990, to September 30, 1996, 97 (about 90 percent)
were chosen by the agency.\8 Data were unavailable to assess the
extent to which staff suggestions for projects were accepted or
rejected. Of the petitions filed with CPSC between January 1, 1990,
and September 30, 1996, 60 percent resulted in projects (32 percent
by granting the petition in whole or in part, and 27 percent by
denying the petition to establish a mandatory regulation but creating
a nonregulatory project). In 27 percent of cases, CPSC decided that
no action was needed or that existing actions or standards were
sufficient to address the issue raised by the petition. In the
remaining cases, a decision is still pending or the petition was
withdrawn before a decision was rendered.
Figure 2: Origin of CPSC
Projects, January 1,
1990-September 30, 1996
(See figure in printed
edition.)
--------------------
\8 For a listing of these 115 projects, see app. III.
CPSC PROJECT SELECTION
CRITERIA OPEN TO DIFFERENCES
IN EMPHASIS AND
INTERPRETATION
---------------------------------------------------------- Letter :3.2
CPSC has established criteria for setting agency priorities and
selecting potential hazards to address. These criteria, which are
incorporated in agency regulations, include the following:
-- the frequency of injuries and deaths resulting from the hazard;
-- the severity of the injuries resulting from the hazard;
-- addressability--that is, the extent to which the hazard is
likely to be reduced through CPSC action--agency regulations
note that the cause of the hazard should be analyzed to help
determine the extent to which injuries can reasonably be
expected to be reduced or eliminated through CPSC action;
-- the number of chronic illnesses and future injuries predicted to
result from the hazard;
-- preliminary estimates of the costs and benefits to society
resulting from CPSC action;
-- unforeseen nature of the risk--that is, the degree to which
consumers are aware of the hazard and its consequences;
-- vulnerability of the population at risk--whether some
individuals (such as children) may be less able to recognize or
escape from potential hazards and therefore may require a
relatively higher degree of protection;
-- probability of exposure to the product hazard--that is, how many
consumers are exposed to the potential hazard, or how likely a
typical consumer is to be exposed to the hazard; and
-- other--additional criteria to be considered at the discretion of
CPSC.
VULNERABLE POPULATIONS,
NUMBERS OF DEATHS AND
INJURIES, AND CAUSALITY
EMPHASIZED OVER OTHER
SELECTION CRITERIA
-------------------------------------------------------- Letter :3.2.1
CPSC's regulations allow for considerable freedom in applying these
criteria; commissioners and staff can base their project selections
on what they perceive as the most important factors. For example,
the regulations do not specify whether any criterion should be given
more weight than the others, nor must all criteria be applied to
every potential project. Indeed, our interviews with present and
former commissioners and our review of CPSC briefing packages
revealed a pattern in which three criteria--the numbers of deaths and
injuries, the causality of injuries, and the vulnerability of the
population at risk--were more strongly emphasized than the others.
In addition, each of the commissioners we interviewed identified some
criteria as being more important than others for project selection.
For example, one commissioner indicated that the number of deaths and
injuries was most important, while another commissioner included
awareness of the hazard in a list of several criteria she believed
were most important. However, there was considerable agreement among
the commissioners about the importance of several criteria. The
commissioners cited two criteria--vulnerability of population and
number of deaths and injuries--as especially important for project
selection. In addition, several--but not all--commissioners
emphasized causality of injuries. None of the other criteria was
emphasized by more than one or two commissioners.
Because the commissioners use their judgment in applying these
criteria, there is no systematic checklist or scoring system that
would enable us to determine which factors were considered most
important for a particular product. However, information related to
some or all of these criteria is sometimes contained in briefing
packages and other documents. Our review of CPSC project
documentation showed that information on vulnerable populations and
the numbers of deaths and injuries associated with the product was
likely to be compiled at some time during the project, but
information associated with other criteria was less likely to be
documented. For example, of the 115 projects we reviewed, death and
injury information was available in 97 cases. However, only 26 cases
included information on exposure to the hazard, a less-emphasized
criterion.
Although data were insufficient to compare the universe of possible
projects with the ones selected by CPSC, the characteristics of CPSC
projects appear generally consistent with the stronger emphasis on
death and injury data and on vulnerable populations expressed by the
current and former commissioners. For example, while 76 of the 115
projects we examined were directed at least partially at a vulnerable
population group, only 13 projects were associated with a chronic
illness. However, although the number of deaths and injuries
associated with product hazards was almost always available in
project documentation, there was no pattern of only those projects
with high numbers of injuries or deaths being selected. Of the 97
projects that had death and injury statistics, 19 showed fewer than
50 injuries and/or deaths associated with the product. The estimated
number of annual injuries associated with product hazards ranged from
1 to 162,100 (for baseball injuries), and the estimated number of
deaths associated with product hazards ranged from zero to a high of
3,600 annually (for smoke detectors). This wide range is consistent
with CPSC staff's statement that there is no threshold for the number
of deaths and injuries that would require acceptance or rejection of
a project.
COMMISSIONERS INTERPRET
SOME SELECTION CRITERIA
DIFFERENTLY
-------------------------------------------------------- Letter :3.2.2
Although the commissioners and former commissioners we interviewed
generally agreed on the criteria they emphasized for project
selection, they expressed very different views on how some of these
criteria should be interpreted. For example, several commissioners
viewed vulnerable populations as focusing on children, while others
highlighted additional segments of the population that they
considered vulnerable. One commissioner also listed low-income and
poorly educated consumers as vulnerable populations, and another
expressed concern that the elderly were especially vulnerable to
injury from product hazards. Project documentation focused on
children more frequently than on other population segments thought to
be at special risk. Many projects we examined contained no
information in the documentation that indicated a particular
population was being considered vulnerable. However, of the 76
projects for which information was available on special populations,
69 (91 percent) mentioned children.\9
Industry observers, consumer advocates, current and former
commissioners, and others expressed widely diverging views on how to
apply causality of injuries in selecting projects. All seven
commissioners we interviewed mentioned this criterion, and several
stressed causal factors. A major issue surrounding the application
of causality is determining the appropriate level of protection the
agency should provide when a product hazard results, at least in
part, from consumer behavior. For example, a consumer advocate
stated that regulatory action may be necessary whatever the cause of
the incident if children who were incapable of protecting themselves
get hurt. Similarly, another individual told us that CPSC should
deal with potential hazards on the basis of the behavior that
actually took place, not the behavior that might be expected or
considered reasonable. However, other individuals asserted that CPSC
should address only those hazards that result from products that are
defective--that is, products that create a hazard even when used as
intended by the manufacturer. Some industry representatives stated
that it was inappropriate for CPSC to take action concerning a
product if the product was "misused" by the consumer.
Complicating this debate is the difficulty of defining misuse of the
product or negligence of the consumer. For example, the appropriate
degree of parental supervision is frequently an issue with children's
products. One of the agency's more controversial projects
illustrates this point. CPSC staff conducted a project to
investigate the deaths of children using baby bath seats or rings.
In these incidents, infants slipped out of the seat and drowned in
the bathtub when the parent or caregiver stepped out of the room and
left the child unsupervised, despite warning signs on the seats not
to leave children unattended. The Commission disagreed on the proper
course of action, largely because of differing views on causality.
In 1994, the staff recommended that the Commission issue an Advance
Notice of Proposed Rulemaking (ANPR), the first step in the
regulatory process. The staff argued (and one commissioner agreed)
that some parents will leave a young child alone in the bathtub
regardless of a warning not to. However, in voting against issuing
an ANPR, the other two commissioners stated that they believed
regulation was not appropriate because the lack of supervision, not
the product, caused the tragedies.
CPSC staff have also encountered other instances in which the
behavior of consumers might be viewed as inappropriate. For example,
the role of alcohol and drug use in accidents can also raise
questions about the appropriate level of regulatory protection.\10 In
addition, a 1991 CPSC study found that at least 33 percent of bicycle
accidents involved behaviors such as performing stunts and going too
fast. Similarly, a 1991 CPSC study of fires associated with heat
tapes found that at least 38 percent of the heat tapes had been
installed improperly. In each of these cases, no regulatory action
was taken; in the case of bicycles, the staff did recommend
increasing efforts to encourage consumers to take safety precautions
such as using lights at night and wearing helmets.
--------------------
\9 Many of these projects involved children's products, for which
only children would be affected, such as cribs or baby walkers.
Other projects dealt with products the general public would be
exposed to but for which children would be more likely to receive
injury, including mouthwashes with high concentrations of alcohol and
automatic garage doors that could crush a child when they close.
\10 Alcohol and drug use contributed to an estimated 1.35 million
injury-related visits to hospital emergency departments in 1992. See
Cheryl R. Nelson and Barbara J. Stussman, "Alcohol- and
Drug-Related Visits to Hospital Emergency Departments: 1992 National
Hospital Ambulatory Medical Care Survey," Advance Data, No. 251
(Aug. 10, 1994).
DATA SYSTEMS PROVIDE
INSUFFICIENT INFORMATION TO
MEASURE SELECTION CRITERIA,
MONITOR ALL PROJECTS, OR
EVALUATE RESULTS
---------------------------------------------------------- Letter :3.3
CPSC uses data from internal management systems and from external
sources to assist in project selection. CPSC collects information on
product-related deaths and injuries to provide information for
project selection as well as to perform risk assessments and
cost-benefit analyses. Furthermore, the agency maintains a
computerized management information system (MIS) that contains
information on some of its major activities and is used by the agency
to develop its annual budget. Both these internal and external data
are of limited value. The inadequacy of the information raises
questions about CPSC's ability to make informed project selection
decisions so that agency resources are being spent efficiently.
SIGNIFICANT GAPS EXIST IN
CPSC'S DATA ON
PRODUCT-RELATED INJURIES
AND DEATHS
-------------------------------------------------------- Letter :3.3.1
CPSC has developed a patchwork of independent data systems to provide
information on deaths and injuries associated with consumer products.
To obtain estimates of the number of injuries associated with
specific consumer products, CPSC relies on its National Electronic
Injury Surveillance System (NEISS). NEISS gathers information from
the emergency room records of a nationally representative sample of
101 hospitals. CPSC also obtains information on fatalities by
purchasing a selected group of death certificates from the states.
It supplements this information with anecdotal reports from
individual consumers and with data from private organizations such as
fire-prevention groups and poison control centers. Because neither
NEISS nor death certificate data provide detailed information on
hazard patterns or injury causes, CPSC investigates selected
incidents to obtain more detailed information. In addition, CPSC
sometimes uses mathematical modeling techniques or conducts special
surveys to obtain information on product exposure. (For more
information on CPSC's data sources, see app. V.)
CPSC's data give the agency only limited assistance in applying its
project selection criteria. (These criteria, the measures used for
each, and major data limitations are given in table 1.) CPSC's injury
and death data allow the agency to piece together at best an
incomplete view of the incidents that result from consumer product
hazards. Product-related injuries may be treated in a variety of
ways--in an emergency room, in a physician's office, or through an
outpatient clinic, for example. As figure 3 illustrates, CPSC
obtains systematic surveillance information only on deaths and on
injuries treated in the emergency room; injuries treated in other
settings (such as physicians' offices) are not represented in CPSC's
surveillance data.
Table 1
CPSC Regulatory Priority-Setting
Criteria and Systematic Information
Available
Criterion Measure used by CPSC Major limitations
-------------- -------------------- --------------------
Number of Number of reported Incomplete because
deaths deaths where not all certificates
consumer product are gathered and not
involvement can be all product-related
inferred. incidents are coded.
Number of Estimated number of Generally omits
injuries injuries involving injuries treated in
consumer products other settings.
treated in emergency
rooms.
Severity of Estimated percentage Not representative
injuries of emergency-room- of the severity of
treated injuries all injuries treated
requiring in all settings.
hospitalization.
Chronic Some limited Little systematic
illnesses information from information.
emergency room
diagnosis
information.
Predicted Prediction based on Of questionable
future NEISS data. validity because of
injuries changes in medical
care over time.
Vulnerable Percentage of Incomplete.
populations emergency-room- Information
treated injuries or available only on
deaths involving age, not on other
children or the vulnerable
elderly. populations, such as
persons with
disabilities.
Exposure Various measures, Exposure surveys are
including sales, time consuming and
estimated products expensive; not done
in use, and for all projects;
population. done only after
project is well
under way.
Addressability Judgment based on Often impossible to
/causation cause of incident, make an informed
hazard pattern, and judgment until
information on project is well
product design; under way;
investigations of investigations are
selected incidents time consuming and
may be conducted to expensive.
obtain this
information.
Preliminary Results of analysis. Quality data are
cost-benefit frequently not
analysis available; limited
accuracy in early
stages of project.
----------------------------------------------------------
Figure 3: Types of Death and
Injury Data Covered by CPSC's
Systematic Surveillance
Information
(See figure in printed
edition.)
Notes: Death data are considered partially included because CPSC
obtains death certificates for selected causes of death. For the
other sources of treatment, CPSC may obtain some anecdotal
information from consumer reports or other sources (see app. V).
The actual numbers of injuries treated in each setting is unknown.
A "near miss" refers to an incident in which a product-related injury
nearly occurred but was narrowly averted. In its regulations that
address priority-setting, CPSC states that such incidents can be as
important as actual injuries in identifying potential hazards.
CPSC staff identified the lack of data on injuries treated in
physicians' offices and other settings as a key concern. Because
CPSC's data sets reveal only a portion of the injury picture, the
agency underestimates the total numbers of deaths and injuries
associated with any given consumer product. The extent of this
undercount is unknown. For example, researchers report widely
varying estimates of the percentage of injuries that are treated in
emergency rooms as opposed to other medical settings. A 1991 study
by researchers at RAND found that approximately 65 percent of
injuries were treated in the emergency room.\11 However, recent data
indicate that the number of injury-related visits to physicians'
offices alone was more than double the number of injury-related
visits to the emergency room.\12 CPSC's estimates of product-related
deaths are also undercounted, for two reasons. First, for budgetary
reasons, the agency purchases only a subset of the total number of
death certificates from states. Second, CPSC death counts include
only those cases in which product involvement can be inferred from
the information on the death certificate, and in some cases,
product-related information is not recorded.
Even if a reliable figure was available to determine the exact
percentage of product-related injuries that were treated in emergency
rooms, this percentage would not necessarily apply to any specific
type of product-related injury. For example, even if it was
established that 40 percent of all product-related injuries were
treated in emergency rooms, the percentage of bunk bed injuries
treated in emergency rooms might be much larger or smaller. The
setting in which injuries are treated depends on a wide array of
factors that vary among individuals, across geographic regions, and
among different types of injuries. Research indicates that African
Americans are more likely to use the emergency room than Caucasians
are.\13 Access to the emergency room or to a physician also depends
on the type of medical insurance a person has. For example, health
maintenance organizations (HMO) often place restrictions on
reimbursement for emergency room care, and HMO membership as a
percentage of the total population varies widely from state to
state.\14 In addition, injuries that occur at night, when most
physicians' offices are closed, may be more likely to be treated in
the emergency room. As a result, it is unlikely that CPSC could
approximate the number of injuries associated with a specific product
by using data that apply to all consumer products as a group.
The incompleteness of CPSC's injury information also hampers its
ability to reliably discern long-term trends in injuries, which is
not only a criterion for project selection but also an important
factor for evaluating the success of CPSC's injury-reduction efforts
and determining the need for follow-up actions. The relative sizes
of the pieces of the injury puzzle in figure 3 are unknown but appear
to change over time. For example, hospitalizations decreased by 5
percent on a per capita basis between 1982 and 1994, while between
1983 and 1993, hospital outpatient clinics saw a 53-percent increase
in visits on a per capita basis.\15 As a result, it is impossible to
determine whether any change in the number of emergency room visits
represents a true change in injuries or a shift to other medical
settings.\16
According to CPSC staff, identifying chronic illnesses associated
with consumer products is nearly impossible. CPSC staff stated that
little is known about many chronic illness hazards that may be
associated with potentially dangerous substances, and even less
information is available about which consumer products may contain
these substances. Chronic illnesses are especially likely to be
underestimated in CPSC's NEISS data because they are underrepresented
among emergency room visits and because product involvement is more
difficult to ascertain. Similarly, consumer product involvement is
very seldom recorded on death certificates in cases of chronic
illnesses.
CPSC's surveillance data also give an incomplete picture of the
severity of incidents. Although the data capture many relatively
severe injuries�that is, those that result in death or require
treatment in an emergency medical facility--data are missing for
individuals who are admitted to the hospital through their physician
rather than through the emergency room. Potentially less severe
cases�for example, those treated in physicians' offices, walk-in
medical centers, or hospital outpatient clinics�are not represented
at all in CPSC's systematic surveillance data systems. Consequently,
CPSC has no data on some consumer product problems that may result in
numerous but potentially less severe injuries.
Sketchy information about accident victims also limits CPSC's ability
to assess which consumer product hazards have a disproportionate
impact on vulnerable populations. NEISS and death certificates
provide only the age of the victim; no systematic or comprehensive
information is available to determine whether a given hazard has a
special impact on other vulnerable populations, such as persons with
disabilities. A former commissioner told us that the lack of other
demographic information such as race, income, and disability status
made it difficult for her to know which subpopulations were
predominantly affected by a particular hazard. Another commissioner
echoed this concern, and said that such information would be useful
in targeting public information campaigns on certain hazards to those
groups that needed the information most.
CPSC staff identified the need for additional exposure data as a
major concern. However, they also told us that obtaining information
on exposure to products and establishing causation require special
efforts that can be time consuming and costly. Although CPSC's
priority-setting criteria include exposure to the hazard, exposure
data are generally not included in CPSC's ongoing data collection
efforts. As a result, exposure is assessed either not at all or
further along in the project, precluding the use of exposure as an
effective criterion for project selection. Similarly, CPSC's
emergency room and death certificate data provide little information
on the circumstances surrounding the incident. As a result, CPSC
staff perform follow-up investigations of selected incidents to
obtain additional detail. These investigations may include detailed
interviews with victims and/or witnesses, police or fire reports,
photographs of the product and/or accident site, laboratory testing
of the product, or recreations of the incident. As with exposure
data, these investigations are not conducted for every project and
are completed only after a project is well under way. Thus,
assessment of causation at the project selection stage is unavoidably
speculative.
--------------------
\11 Deborah R. Hensler, and others, Compensation for Accidental
Injuries in the United States (Santa Monica, Calif.: RAND, 1991).
\12 See Susan M. Schappert, "National Ambulatory Medical Care
Survey: 1992 Summary,"Advance Data No. 253 (Aug. 18, 1994), and
Linda F. McCaig, "National Hospital Ambulatory Medical Care Survey:
1992 Emergency Department Summary," Advance Data, No. 245 (Mar. 2,
1994).
\13 See Linda F. McCaig, "National Hospital Ambulatory Medical Care
Survey: 1992 Emergency Department Summary," p. 1.
\14 1996 figures from InterStudy show that while only 1.2 percent of
individuals in Mississippi belonged to HMOs, 44.8 percent of
individuals in Oregon were HMO members. See The InterStudy
Competitive Edge: HMO Industry Report 6.2, InterStudy Publications
(1996), p. 28.
\15 These figures were derived from data provided by the American
Hospital Association.
\16 Precise estimates on the impact of changing treatment patterns
are rare, and the impact may vary among local areas. However, one
research team found that because of the growth in alternative
facilities in their local area, emergency rooms handled about half of
outpatient injury cases--down from a previous estimate of 75 percent.
See Julian A. Waller, Joan M. Skelly, and John H. Davis, "Treated
Injuries in Northern Vermont," Accident Analysis and Prevention, 27
(6) (1995), pp. 819-28.
AGENCY PROJECT MANAGEMENT
INFORMATION INCOMPLETE OR
UNAVAILABLE
-------------------------------------------------------- Letter :3.3.2
CPSC conducts a number of projects annually, but staff were unable to
provide a comprehensive list of projects the agency had worked on in
the 6-year period we examined. CPSC was also unable to verify the
completeness of the project list we compiled from agency documents
and interviews with staff. According to CPSC staff, internal
management systems do not contain this information and such a list
could be compiled only by relying on institutional memories of staff
members who had been with the agency long enough to know which
products the agency had addressed. Without systematic and
comprehensive information on its past efforts, CPSC cannot assess
whether some hazard areas have been overrepresented and whether
agency resources might be more efficiently employed.
CPSC also lacks information on the characteristics of, resources used
on, or outcomes of individual projects. CPSC's MIS tracks contract
dollars and staff time by accounting codes that cover some specific
projects and general categories, such as compliance work, which are
composed of numerous activities. According to agency officials,
CPSC's MIS generally cannot provide descriptive information on
individual projects, such as when a project was started or concluded;
the number of staff days used; what aspect of the product was
addressed; whether the project originated from a petition,
congressional mandate or other source; or what action was taken to
address the hazard (mandatory standard, voluntary standard, or public
information campaign, for example). In addition, CPSC staff told us
that two separate projects involving the same or similar products at
different times may be assigned the same MIS code. As a result, even
if a project appears to be tracked in the MIS, reliable inferences
cannot be drawn from MIS data.
CPSC HAS LIMITED ABILITY
TO EVALUATE AGENCY IMPACT
-------------------------------------------------------- Letter :3.3.3
CPSC's limited data on deaths and injuries, combined with its lack of
information on projects, reduce the agency's ability to evaluate the
impact of its work, a process it is now required to undertake under
recently passed legislation. The Results Act requires every federal
agency to evaluate the effectiveness of its efforts starting in
fiscal year 1999. The Results Act is aimed at increasing the
investment return of tax dollars by improving agencies' performance.
Under the Results Act, an agency is to set mission-related goals and
measure progress toward these goals to evaluate agency impact. CPSC
has preliminarily identified results-oriented goals in four areas:
(1) reducing head injuries to children, (2) reducing deaths from
fires, (3) reducing deaths from carbon monoxide poisoning, and (4)
reducing deaths from electrocutions. However, the limitations in
CPSC's injury and death data raise a question about how well CPSC
will be able to evaluate the effectiveness of agency actions in these
and other areas.
CPSC USES COST-BENEFIT ANALYSIS
AND RISK ASSESSMENT, BUT
IMPROVEMENTS ARE NEEDED IN DATA
AND METHODOLOGY
------------------------------------------------------------ Letter :4
CPSC uses two analytical tools--risk assessment and cost-benefit
analysis--to assist in making decisions on regulatory and
nonregulatory methods to address potential hazards. Risk assessment
involves estimating the likelihood of an adverse event (such as
injury or death). For example, CPSC estimated that the risk of death
from an accident involving an all-terrain vehicle (ATV) was about 1
death for every 10,000 ATVs in use in 1994. Cost-benefit analysis
details and compares the expected effects of a proposed regulation or
policy, including both the positive results (benefits) and the
negative consequences (costs). Although cost-benefit analysis may
not be applicable to every decision and may not be the only factor
appropriately considered in a decision, it can be a useful
decision-making tool. The Congress requires CPSC to perform
cost-benefit analysis before issuing certain regulations, and CPSC
has conducted cost-benefit analysis for these regulations and in
other situations in which it was not required. Although perfectly
complete and accurate data are rarely available for any analysis,
CPSC's data are frequently inadequate to support detailed, thorough,
and careful risk assessment and cost-benefit analysis. In addition,
CPSC's cost-benefit analyses are frequently not comprehensive, and
the reports on these analyses are not sufficiently detailed.
Improvements in the agency's methodology and in the quality of the
underlying data are necessary to ensure the clarity and accuracy of
CPSC's risk assessments and cost-benefit analyses.
CPSC PERFORMS COST-BENEFIT
ANALYSIS MORE OFTEN THAN
REQUIRED BY LAW
---------------------------------------------------------- Letter :4.1
Cost-benefit analysis can help decisionmakers by organizing and
aggregating all the relevant information to clarify the nature of the
trade-offs involved in a decision. Although cost-benefit analysis
may not be appropriately used as the sole criterion for making a
decision, a well-constructed cost-benefit analysis can highlight
crucial factors, expose possible biases, and facilitate informed
decisions even when it is impossible to measure all the potential
effects of a specific regulatory proposal. The Congress has required
CPSC to perform and publish a cost-benefit analysis when issuing a
regulation (such as a mandatory standard or product ban) under the
Consumer Product Safety Act. In addition, CPSC is also required to
conduct cost-benefit analyses before issuing regulations under the
authority of portions of the Federal Hazardous Substances Act
(specified labeling provisions are exempt from this requirement) and
the Flammable Fabrics Act.\17
Because most of the agency's projects do not involve mandatory
regulation, relatively few CPSC projects conducted between January 1,
1990, and September 30, 1996, were subject to these requirements. We
identified 8 cost-benefit analyses that CPSC performed in accordance
with these requirements, and an additional 21 analyses it conducted
in situations in which it was not required. For example, CPSC
performed cost-benefit analyses in eight instances in which it was
considering issuing requirements for child-resistant packaging under
the Poison Prevention Packaging Act, which does not require
cost-benefit analysis.\18 CPSC frequently conducts cost-benefit
analysis with respect to regulatory procedures, whether or not it is
required to do so. However, a complete cost-benefit analysis is done
less frequently for voluntary standards projects or information and
education efforts, although some economic information may be
generated to assist such projects. In addition to the complete
cost-benefit analyses, we identified an additional 23 cases in which
some information was provided on some economic benefits or costs.
Before issuing a mandatory regulation, CPSC is required to consider
the degree and nature of the risk of injury the regulation is
designed to eliminate or reduce. However, CPSC usually does not
conduct a formal numerical risk assessment before issuing a
regulation, and the law does not require it. We found 24 risk
assessments conducted by CPSC between January 1, 1990, and September
30, 1996; only 4 of these were associated with regulatory action.
--------------------
\17 As an independent agency, CPSC is not required to comply with
Executive Order 12866, which requires federal agencies to perform a
cost-benefit analysis and submit it to the Office of Management and
Budget (OMB) for review whenever the agency issues a regulation that
is economically significant.
\18 For child-resistant packaging regulations, CPSC is legally
required to consider only whether requiring child-resistant packaging
is "technically feasible, practicable, and appropriate."
AGENCY DATA ARE GENERALLY
INSUFFICIENT TO SUPPORT
THOROUGH AND DETAILED
ANALYSIS
---------------------------------------------------------- Letter :4.2
Both risk assessment and cost-benefit analysis require extensive
data. Risk assessment requires information both on the adverse event
and on exposure to the precipitating circumstances. For example,
when CPSC performed a risk assessment on floor furnaces, the agency
estimated the number of previous injuries associated with floor
furnaces and the number of floor furnaces in use.\19 Similarly, when
CPSC performed a risk assessment to examine the risk of contracting
cancer from dioxin traces in common paper products, the agency used
information from laboratory studies on dioxin's link to cancer and
also incorporated data on exposure to paper products.\20 Because
cost-benefit analysis includes a comprehensive delineation of the
expected effects of a given proposal, a careful and thorough
cost-benefit analysis will also be very data intensive and rich with
detail.
CPSC's data systems are frequently unable to adequately meet the
extensive demands for information posed by risk assessment and
cost-benefit analysis. As a result, the agency's estimates of risks,
costs, and benefits are less accurate because they reflect the
substantial limitations of the underlying data. Available
information does not permit us to determine the potential impact of
better data on the results of CPSC's cost-benefit analyses and risk
assessments. Some of these data weaknesses tend to make product
hazard risks seem larger, and other problems tend to make the same
risks appear smaller. For example, because CPSC's data undercount
the deaths and injuries associated with particular consumer products,
estimates of risk--and the potential benefits of reducing that
risk--appear smaller. However, CPSC's surveillance data provide
information only on whether a product was involved in an accident,
not on whether the product caused the accident. At least at the
initial stages of a project, this can make the risks assessed by
CPSC--and the benefits of reducing those risks--appear larger.
For risk assessment, CPSC must also obtain information on exposure to
the hazard. For example, to assess the risk associated with aluminum
ladders, CPSC obtained estimates of the number of ladders available
for use and on the number of times each year the ladder was used.
Obtaining exposure data presents special challenges for CPSC.
Because the product definition that relates to a particular hazard is
often relatively narrow, existing data sources frequently offer
insufficient detail. For example, CPSC was unable to use Census
sources to determine the number of saunas in the United States
because saunas were included in a broader classification of products
when government data were collected. In addition, CPSC staff told us
that it is often difficult to find accurate information on the number
of products that are in households and available for use.
CPSC sometimes responds to these challenges by using mathematical
modeling techniques or easier-to-obtain proxy measures (such as
population) to estimate product exposure. In addition, for a few
large-scale projects, CPSC has incurred the substantial expenses
necessary to conduct its own detailed exposure survey. For example,
CPSC conducted a survey of households that asked detailed questions
on matches and disposable cigarette lighters--the number purchased,
where they are generally kept, how they are used, and other details.
However, for the majority of the projects we reviewed, CPSC did not
gather any data on exposure. Of the 80 projects we reviewed for
which briefing packages were prepared, only 26 included information
on exposure to the hazard, and CPSC's risk assessments were confined
to 24 cases between 1990 and 1996--approximately 21 percent of all
projects conducted over that time period.
--------------------
\19 CPSC found that the risk of injury associated with floor furnaces
was small in terms of the number of incidents but relatively higher
when considered in relation to the number of products in use.
\20 CPSC's study found that the risk from this type of dioxin
exposure was negligible--at most, 5 cases per billion people
exposed--and as a result no action was taken.
CPSC COST-BENEFIT ANALYSES
ARE OFTEN NOT COMPREHENSIVE
AND NOT REPORTED IN
SUFFICIENT DETAIL
---------------------------------------------------------- Letter :4.3
The methodology used to conduct a cost-benefit analysis will
frequently depend on the circumstances and the context of the
analysis. For this reason, there is no complete set of standards for
evaluating the quality of an individual cost-benefit analysis.
However, the professional literature offers some guidance for
analysts, and certain specific elements are frequently mentioned as
essential for cost-benefit analysis. For example, because
cost-benefit analysis is meant to be a complete delineation of the
expected effects of a proposed action, all potential impacts (even
those that cannot be quantified) should be discussed. To ensure that
the reader is able to make an informed judgment, it is important to
be explicit about the underlying data, methodology, and assumptions.
Accordingly, the literature suggests that all methodological choices
and assumptions should be detailed, all limitations pertaining to the
data should be revealed, and measures of uncertainty should be
provided to allow the reader to take into account the precision of
the underlying data. Similarly, the literature calls for sensitivity
analysis, which enables the reader to determine which assumptions,
values, and parameters of the cost-benefit analysis are most
important to the conclusions.
On the basis of our review of the cost-benefit literature, we
developed a list of the elements that are frequently used in
evaluating cost-benefit analyses. This list, and a description of
all the factors we examined, are in appendix IV.\21
Although we compared each of these elements with each of CPSC's
analyses, not all elements were applicable to each case. For
example, in some cases, the circumstances indicated by a given
element--such as reliance on statistical data--were not found, and
those cases were treated as not applicable to that element. In
addition, for some elements it was not always possible to determine
whether CPSC's analysis was consistent with the element. For these
reasons, and to emphasize those areas that we viewed as most
critical, we reported only the evaluation results that relate to key
elements, applied to the majority of CPSC's analyses, and for which a
determination was possible in all or nearly all cases.
Our review of all the cost-benefit analyses that CPSC conducted
between January 1, 1990, and October 31, 1996, showed that for
many--but not all--elements, CPSC's analyses were not comprehensive
and not reported in sufficient detail (see table 2). For example,
CPSC provided descriptive information on proposals and also provided
information on a variety of reasonable alternatives in almost 100
percent of cases. However, CPSC analyses generally did not provide
measures of uncertainty for the underlying data. Estimates derived
from samples are subject to sampling error, which can be especially
large when the estimates are projected from relatively fewer cases.
In only 17 percent of its analyses did CPSC provide any statistical
information on the precision of the underlying estimates. Similarly,
when estimates are based on a relatively small sample size,
projections are generally not considered reliable. CPSC analysts
cautioned the reader against making conclusions based on small sample
data only 45 percent of the time. In addition, some of CPSC's data
sets have a known upward or downward bias because of the way the data
were constructed. For example, CPSC's estimates of deaths based on
CPSC's death certificate database will be understated, and when
estimates of incidents are based only on investigated or reported
cases (such as cases reported to CPSC's hot line), two potential
biases are likely to be introduced into the analysis: (1) the
estimates are likely to be biased downward by nonreporting and (2)
the incidents reported tend to be the more severe ones. In only 53
percent of applicable cases did CPSC's analysis inform the reader of
known limitations inherent in the data being used for cost-benefit
analysis.
Table 2
Evaluation of CPSC Analyses Shows
Inconsistencies With Several Evaluation
Elements
Percentage of
CPSC's
analyses that
were
consistent
with this
Evaluation element element
------------------------------------------ --------------
Provided descriptive information about a 98
well-defined proposal
Addressed multiple alternatives 95
Reported measures of precision for 17
underlying data
Cautioned reader about making inferences 45
from data with a small sample size
Reported known biases in underlying data 53
Provided any sensitivity analysis 26
information
Included all important categories of 54
benefits and costs
Considered "risk-risk" trade-offs\a 49
----------------------------------------------------------
\a A "risk-risk" trade-off refers to an action to decrease a hazard's
risk that unintentionally increases that or another risk.
We identified several other areas in which CPSC analyses could
benefit from improvement. For example, researchers agree that
sensitivity analysis--a technique that enables the reader to
determine which assumptions, data limitations, or parameters are most
important to the conclusions--should be incorporated in cost-benefit
analyses. CPSC usually did not provide sensitivity analysis
information. For example, agency briefing packages did not include
any information on how CPSC's injury cost estimates were derived or
what factors were the largest components of injury costs. CPSC
applies a statistical model to injury estimates to derive a figure
for injury cost. The model that computed injury cost estimates
accounts for a number of components, including medical costs, forgone
wages, and pain and suffering. With only one exception, CPSC
briefing packages provided only the total cost, without any
information on the derivation of those costs or the individual
components. In addition, CPSC provided only an average injury cost,
not a range of injury cost estimates. For situations in which
injuries differ in severity, or for projects in which severity is
probably overstated or understated in the data, the reader would find
such information useful.
Forty-six percent of CPSC analyses did not consider the full range of
costs and benefits likely to result from regulation. For example,
CPSC analysts frequently omitted mentioning intangible costs and/or
benefits (costs or benefits that are difficult to quantify, such as
loss of consumer enjoyment) or potential indirect effects (such as
changes in the prices of related goods). In addition, CPSC
frequently excluded risk-risk considerations from its evaluation of
the costs and benefits of potential actions. Sometimes actions taken
to reduce one risk can have the unintended effect of increasing that
or another risk. Individuals may take more or fewer precautions in
response to a change in a product's safety features, and this
behavior can result in an increase in the risk the intervention was
designed to mitigate. For example, in establishing a standard for
child-resistant packaging that was also "senior-friendly," CPSC
considered that because child-resistant medicine bottles can be
difficult to open, a grandparent may leave the cap off the bottle,
creating an even greater risk than would be the case with the
original cap. Although CPSC considered such factors in some cases,
only 49 percent of its analyses reflected potential risk-risk
trade-offs.
CPSC has not established internal procedures that require analysts to
conduct comprehensive analyses and report them in sufficient detail.
For example, according to CPSC staff, the agency has little written
guidance about what factors should be included in cost-benefit
analyses, what methodology should be used to incorporate these
factors, and how the results should be presented. Staff also told us
that CPSC analyses are not generally subject to external peer review.
Such reviews can serve as an important mechanism for enhancing the
quality and credibility of the analyses that are used to help make
key agency decisions.
--------------------
\21 These elements are also similar to OMB's guidance for preparing
economic analyses of significant regulatory actions.
CPSC HAS ESTABLISHED PROCEDURES
FOR COMPLYING WITH STATUTORY
REQUIREMENTS FOR RELEASING
MANUFACTURER-SPECIFIC
INFORMATION
------------------------------------------------------------ Letter :5
To help minimize the possibility that a product might be unfairly
disparaged, in section 6(b) of the Consumer Product Safety Act the
Congress imposed restrictions on the disclosure of
manufacturer-specific information by CPSC.\22
Before CPSC can disclose any information that identifies a
manufacturer,\23 the agency must
-- take "reasonable steps" to verify the accuracy of the
information and to ensure that disclosure is fair,
-- notify the manufacturer that the information is subject to
release, and
-- provide the manufacturer with an opportunity to comment on the
information.
If the manufacturer requests that its comments be included in CPSC's
disclosure of the information, CPSC can release the information only
if it is accompanied by the manufacturer's comments. If the
manufacturer objects to the release even if its comments are
included, it can challenge CPSC in U.S. district court to block
disclosure. These restrictions on the release of information apply
not only to information the agency issues on its own--such as a press
release--but also to information disclosed in response to a request
under FOIA. In addition, section 6(b) also requires CPSC to
establish procedures to ensure that releases of information that
reflect on the safety of a consumer product or class of products are
accurate and not misleading, regardless of whether the information
disclosed identifies a specific manufacturer.
CPSC has established procedures to implement these requirements,
including requiring technical staff to "sign off" on information
releases and notifying manufacturers. Evidence from several
sources--industry sources, published legal decisions, and agency
retractions--suggests that CPSC has complied with its statutory
requirements. CPSC staff and commissioners, industry
representatives, and consumer advocates expressed a wide variety of
opinions on the effectiveness of these requirements, and some
individuals favored specific changes.
--------------------
\22 An exception to these restrictions is given if CPSC has declared
that the product is an "imminent hazard" under section 12 of the
Consumer Product Safety Act.
\23 These restrictions also apply even if the manufacturer is not
named but the information would allow the reader to readily identify
the manufacturer from the context. For example, if there is only one
manufacturer of a product that is identified in the information, the
information may be subject to restriction even if the manufacturer's
name is not given.
CPSC HAS ESTABLISHED
PROCEDURES FOR VERIFYING AND
CLEARING INFORMATION FOR
RELEASE
---------------------------------------------------------- Letter :5.1
Part of CPSC's mission is to provide the public with information to
help individuals use consumer products safely. CPSC disseminates
information through its own initiatives and also in response to
requests from the public. For example, CPSC informs both consumers
and businesses about product hazards through product recall notices,
provision of information at trade shows and special events, and a
telephone hot line and an Internet site. In addition, CPSC responds
to thousands of telephone and written requests for information each
year.
CPSC's mission and its responsibility under FOIA require the agency
to disseminate a great deal of information. However, because much of
this information is about specific products or manufacturers, CPSC's
information disclosure is often restricted under section 6(b). In
its regulation implementing section 6(b), CPSC established several
measures designed to ensure compliance with the statutory
requirements. These measures include obtaining written verification
from consumers of the information they report to the agency,
notifying manufacturers by certified mail when manufacturer-specific
information is requested, and giving manufacturers the option of
having their comments published along with any information being
disclosed.
CPSC's procedures outline several steps to verify all information
before it is released. For example, CPSC checks each report the
agency receives from consumers about incidents involving potentially
hazardous products to ensure that CPSC's records accurately reflect
the consumer's version of the incident. Agency procedures require
staff to send a written description of each incident back to the
person who reported it with a request that he or she review it and
state if any information needs to be corrected or supplemented.\24
The commission staff review each of these incident reports for
discrepancies or any obvious inaccuracies. Once they have been
checked and confirmed with the consumer, incident reports are made
available to the public upon request. If the reports contain
information that would identify a specific manufacturer, they are
subject to 6(b) requirements regarding disclosure.
CPSC also investigates events surrounding selected product-related
injuries or incidents. Investigation reports provide details about
incident sequence, human behavior factors, and product involvement.
The reports generally contain the consumer's version of what happened
and the observations of witnesses, fire and police investigators, and
others. Investigations may also include follow-up inspections at
retail stores or service centers. However, neither investigations
nor incident reports include the manufacturer's view of the incident.
Its point of view may be expressed in comments it submits before the
report is released. Like incident reports, investigation files are
available to the public upon request and are subject to 6(b)
requirements.
CPSC has issued clearance procedures to cover situations in which
commissioners or staff initiate public disclosures--for example, when
the Commission publishes the results of agency research. These
procedures are intended to verify any information--oral or
written--released by the Commission, regardless of whether the
information identifies a manufacturer. Under CPSC's guidelines, each
assistant or associate executive director whose area of
responsibility is involved must review the information and indicate
approval for the release in writing.\25 After all other review has
been completed, the Office of the General Counsel must also review
and approve the release. Press releases with respect to product
recalls are written and issued jointly by CPSC and the affected
manufacturer. In addition, CPSC's clearance procedures for press
releases state that final clearance must be obtained from the Office
of the Chairman of the Commission.\26 In addition, CPSC staff told us
that the current chairman's policy of coordinating media inquiries
through the Office of Public Affairs is intended to ensure that
information provided is in compliance with section 6(b).
CPSC has also established procedures to implement the notification
and comment provisions of section 6(b). Before CPSC releases
information in response to an FOIA request, an information specialist
determines whether a manufacturer could be readily identified. CPSC
staff said that agency policy is to clearly and narrowly identify
hazardous products (including by manufacturer) whenever possible, in
order to prevent the person receiving the information from confusing
safe products with unsafe products of the same type. However, if an
information request is broad, like "all bicycle accidents," names of
manufacturers are removed before the information is released,
according to CPSC staff. If the requested information could identify
a manufacturer, then staff review the information for appropriate
exemptions (such as trade secrets), and delete portions as
appropriate. The manufacturer is given 20 calendar days in which to
review and comment on a summary of the information CPSC plans to
release.\27 Because CPSC often receives multiple requests for the
same information, the agency informs manufacturers that it will not
send them copies of subsequent requests for the same information
unless specifically requested to do so. However, according to CPSC
staff, more than 80 percent of the manufacturers that submit 6(b)
comments routinely request such notification. In calendar year 1993
(the most recent year for which data were available), CPSC sent out
487 notices to manufacturers and received 154 responses (32 percent).
Twenty-five manufacturers (5 percent) contested the accuracy of the
information or claimed that the proposed disclosure would be unfair.
If a manufacturer fails to comment, the information can be released
30 days from the date CPSC notified the manufacturer. After taking
the manufacturer's comments into account, CPSC may decide to disclose
incident information despite the firm's objection if, for example,
the comments lack specific information to support a claim of
inaccuracy or a request for confidentiality. If CPSC chooses to
disclose information over a manufacturer's objection, it must release
the manufacturer's comments along with the other information, unless
the manufacturer requests otherwise. In addition, if CPSC decides to
release information and the manufacturer objects, the manufacturer
has 10 working days to go to court and seek to enjoin CPSC from
disclosing the information. Manufacturers have sued CPSC to prohibit
disclosure of records only 11 times since the agency was founded, and
CPSC was prohibited from releasing the information in 2 of these
cases.\28
--------------------
\24 The cover letter that accompanies the incident report also asks
recipients if they would like to have their names withheld if
information about the incident is made public.
\25 For a description of CPSC's directorates, see app. II.
\26 An exception is made for those hazards considered by the
Compliance staff as most serious, for which clearance must be
obtained from a majority of the commissioners.
\27 Since small companies may lack the technical ability to comment,
CPSC staff told us that they will sometimes help small businesses to
formulate their comments.
\28 Currently, Daisy Manufacturing, Inc. v. CPSC is on appeal,
after a judge ruled in CPSC's favor in U.S. district court.
EVIDENCE SUGGESTS THAT CPSC
COMPLIES WITH LEGAL
REQUIREMENTS REGARDING THE
RELEASE OF INFORMATION
---------------------------------------------------------- Letter :5.2
Information from three sources of evidence--industry, published legal
cases, and data on retractions--suggests that CPSC complies with its
statutory requirements concerning information release. Industry
sources, even those otherwise critical of the agency, told us that
CPSC generally does a good job of keeping proprietary information
confidential as required by law. Our review of published legal
decisions found no rulings that CPSC violated its statutory
requirements concerning the release of information. Retractions by
CPSC are also rare. If CPSC finds that it has disclosed inaccurate
or misleading information that reflects adversely on any consumer
product or class of consumer products or on any manufacturer, it must
publish a retraction. Any retraction must be made in the same manner
in which CPSC released the original information. According to CPSC,
it has published only three such retractions. Two of these
retractions, in 1984 and 1994, were made in response to requests from
firms. A third retraction, in 1990, was issued after CPSC discovered
that a report in its public reading room had mistakenly included
inaccurate information.
INDUSTRY OBSERVERS, CONSUMER
GROUPS, AND OTHERS SUGGESTED
CHANGES TO 6(B) REQUIREMENTS
---------------------------------------------------------- Letter :5.3
Industry observers, CPSC staff, and consumer groups expressed a wide
range of opinions on the effectiveness of section 6(b). In response
to our inquiries, some CPSC commissioners and former commissioners
said that these restrictions serve a useful purpose and should not be
changed. However, CPSC's current chairman, industry and advocacy
group representatives, and others expressed dissatisfaction with
6(b), and some of these suggested possible changes. Although these
individuals raised issues about the extent of the protection afforded
to manufacturers and the resources necessary to ensure compliance, we
did not assess whether the specific suggestions were necessary or
feasible.
CPSC's chairman, other CPSC officials, former commissioners, and the
representative of a consumer advocacy group stated that compliance
with 6(b) is costly for CPSC and delays the agency in getting
information out to the public. Although CPSC has not estimated the
cost of complying with 6(b), agency staff told us that it takes much
more staff time to respond to FOIA requests that come under 6(b) than
it does to respond to FOIA requests that do not involve company
names. To reduce the burden of complying with these requirements,
CPSC staff have suggested that the notification requirement that
gives manufacturers 20 days in which to comment should apply only the
first time an item is released. Some have suggested that instead of
requiring CPSC to verify information from consumer complaints, the
agency should be allowed to issue such information with an explicit
disclaimer that CPSC has not taken a position on the validity of the
consumer's report.
Instead of reducing CPSC's verification requirements, some industry
representatives suggested expanding them. These manufacturers stated
that before CPSC releases incident information, it should
substantiate the information rather than relying on a consumer's
testimony. Industry representatives stated--and CPSC staff
confirmed--that many of the requests for CPSC information come from
attorneys for plaintiffs in product liability suits. As a result,
some industry representatives expressed concern that unsubstantiated
consumer complaints could be used against them in product liability
litigation. They suggested that 6(b) should require CPSC to
substantiate all incident reports by investigating them before they
can be disclosed, instead of merely checking with the consumer as it
does now. However, CPSC officials told us that investigations--which
are time consuming and costly--can be conducted only on a small
proportion of specially selected cases because of limited resources.
Industry representatives also said that the current restrictions do
not provide sufficient protection when information is released on
product groups instead of on the products of an individual
manufacturer. Several industry representatives expressed concern
that producers of safer products may be unfairly maligned when CPSC
releases information about a group of products, only some of which
may be associated with a safety problem. According to some of these
industry representatives, CPSC should extend protection to product
groups similar to the safeguards manufacturers receive under 6(b).
Retailers' representatives also suggested specific changes to CPSC's
information release requirements. They said that retailers do not
receive timely notice of recalls because CPSC has interpreted the law
to prohibit advance notification of retailers. Consequently, the
retailers said that they sometimes receive notice of recalls at the
same time as their customers and have no time to prepare. For
example, when consumers come in with recalled products, the retailer
may not yet know whether the manufacturer has agreed to replace the
product, refund the purchase price, or provide some other remedy.
Retailers' representatives suggested amending 6(b) to give 5 business
days' advance notice to retailers before the public announcement of a
recall. CPSC officials said that typically manufacturers are and
should be the ones to contact the retailers and make all arrangements
for a recall. Although they disagreed on the need for a statutory
change, both CPSC staff and a major retailers' association said that
they were trying to work out a more satisfactory arrangement.
CONCLUSIONS
------------------------------------------------------------ Letter :6
CPSC's current data provide insufficient information to monitor
ongoing projects and to determine whether potential projects adhere
to the agency's selection criteria. Moreover, inadequate agency data
often prevent CPSC from conducting risk assessments on projects,
potentially limiting the agency's ability to target resources to the
hazards presenting the greatest risks. The lack of sufficient data,
combined with methodological problems, also makes CPSC's cost-benefit
analyses less useful than they could be. With more detailed
information on both internal resources and external product hazards,
CPSC would be better able to assure the Congress and taxpayers that
its resources are expended wisely.
We identified several key areas where CPSC management could improve
its collection and analysis of external data. For example, CPSC
would be better able to make informed decisions on potential agency
projects if it had additional statistically reliable and timely data
in several areas, including (1) injuries treated outside of hospital
emergency rooms; (2) exposure to consumer products and
product-related hazards; (3) chronic illnesses related to consumer
products; and (4) hazards that disproportionately affect certain
vulnerable populations, such as low-income individuals and consumers
with disabilities.
In addition, project selection and implementation could be improved
if commissioners and staff had tracking information on CPSC projects,
such as starting and ending dates, project origin, project costs
(including staff days and contract costs), and agency actions taken
to address the potential hazard. Such information could assist the
commissioners in monitoring ongoing projects, targeting new efforts
on the basis of previous work, and assessing the allocation of
resources across current projects and among major hazard areas.
CPSC could also benefit from an improved methodology for cost-benefit
analysis. A stronger methodological base for CPSC's cost-benefit
analyses, including more complete documentation, would promote sound
regulatory decision-making and could improve the quality of the input
and comment CPSC receives during the regulatory process.
Without these improved data, CPSC will remain unable to accurately
apply measurable criteria in choosing projects or to rigorously
assess relative risks among potential hazards. Some of this
necessary information�the need for more representative and complete
injury and exposure data, for example�may require a significant
investment of resources, so CPSC may need to prioritize these data
needs. In doing so, is important for CPSC to draw on the insight of
individuals outside the agency to ensure that all available
alternatives for obtaining these data are explored. Some of the
other information CPSC needs, however, could be compiled internally
at relatively little additional cost and effort. For example, more
detailed information on individual projects could be collected within
the agency. Similarly, the methodological problems we identified in
CPSC's cost-benefit analysis could be remedied without additional
data.
RECOMMENDATIONS
------------------------------------------------------------ Letter :7
We recommend that the Chairman of CPSC take the following actions:
-- Improve the quality of CPSC's injury, death, and exposure data
by consulting with experts both within and outside the agency to
(1) prioritize CPSC's needs for additional statistically valid
surveillance data on injuries and deaths related to consumer
products and on exposure to consumer products and
product-related hazards, (2) investigate the feasibility and
cost of alternative means of obtaining these data, and (3)
design data systems to collect and analyze this information.
-- Direct agency staff to develop and implement a project
management tracking system to compile information on current
agency projects. For each project, such a system should
include, at a minimum, a description of the hazard addressed,
start and end dates, project origin, and major agency action
resulting from it.
-- Direct agency staff to develop and implement procedures to
ensure that all cost-benefit analyses performed on behalf of
CPSC are comprehensive and reported in sufficient detail,
including providing measures of precision for underlying data,
incorporating information on all important costs and benefits,
and performing sensitivity analysis.
AGENCY COMMENTS
------------------------------------------------------------ Letter :8
We received two separate sets of comments from CPSC's
commissioners--one from Chairman Brown and Commissioner Moore and one
from Commissioner Gall. CPSC staff also submitted some technical
comments, which we incorporated in the report as appropriate.
Chairman Brown and Commissioner Moore stated that they are
considering our recommendations and that in some respects our
recommendations parallel efforts already under way at the Commission.
However, they disagreed with some of the specific findings of our
report. They stated that CPSC's actions are based on solid injury
and death estimates and that CPSC (1) employs sound economic analyses
that are appropriate for the circumstances, (2) tracks projects to
monitor the progress of its work, and (3) has been successful in
dramatically reducing the threat to consumers from unsafe products.
We concluded that CPSC's death and injury data are generally
insufficient to support the agency's project selection process for
two reasons: (1) CPSC has little or no data on several project
selection criteria and (2) CPSC's data on its other project selection
criteria exhibit significant gaps. Similarly, our finding that
CPSC's cost-benefit analyses were not comprehensive and not reported
in sufficient detail supports our conclusion that these analyses were
less useful than they could have been in the agency's decision-making
process. Because CPSC's current management information system
operates at a very high level of generality and (according to CPSC
staff) does not produce consistent, reliable information, we
recommend that the agency implement an improved tracking system that
would provide enough information to monitor the projects selected and
the resources spent for each hazard. We did not review whether
CPSC's actions had been successful in reducing the number of injuries
and deaths associated with consumer products. The full text of
Chairman Brown and Commissioner Moore's detailed comments and our
response are in appendix VI.
Commissioner Gall stated in her comments that she is looking forward
to implementing many of the reforms we recommended. She also stated
that she agrees with our conclusion that CPSC could improve the way
it gathers information, including reassessing the need for injury
data gathered outside of hospital emergency rooms. Commissioner Gall
also agreed that CPSC needs an improved system to track ongoing
activities or projects and that sensitivity analysis, measures of
uncertainty, and risk-risk analysis should be incorporated into CPSC
analyses. However, she also commented that additional GAO analysis
of the implications of inadequate data could have been helpful.
Unfortunately, available information does not permit us to determine
the impact of better-quality data on the decisions made by CPSC. In
addition, Commissioner Gall said that she believes the Compliance
function of CPSC also warrants further review. Such a review is
outside the scope of this report. Commissioner Gall's detailed
comments and our response are in appendix VII.
---------------------------------------------------------- Letter :8.1
As agreed with your office, unless you publicly announce its contents
earlier, we plan no further distribution of this report until 30 days
after its issue date. At that time, we will send copies to the
commissioners of the Consumer Product Safety Commission and make
copies available to others upon request.
If you or your staff have any questions about this report, please
contact me at (202) 512-7014. Major contributors are listed in
appendix VIII.
Carlotta C. Joyner
Director, Education
and Employment Issues
SCOPE AND METHODOLOGY
=========================================================== Appendix I
This report provides information on project selection, analytical
research, and information release procedures at the Consumer Product
Safety Commission (CPSC), which we gathered from a variety of sources
inside and outside the agency. For example, we interviewed current
CPSC staff, including analysts and line managers, to obtain
information about CPSC's data and procedures. We also interviewed
the three current CPSC commissioners and four former commissioners.
In addition, we interviewed representatives of manufacturers,
retailers, trade associations, consumer groups, and academic and
other experts to obtain their perspectives on CPSC's activities. We
reviewed the legislation governing CPSC's activities and the
associated regulations. We also reviewed statements by public and
private organizations and the academic research literature dealing
with consumer protection issues and the technical literature on
cost-benefit analysis and risk assessment.
We gathered extensive information from CPSC documents. We reviewed
every agency budget request, operating plan, mid-year review, and
regulatory agenda from 1990 to 1996. We reviewed all CPSC's annual
reports from fiscal years 1986 to 1996. We also obtained all CPSC
Federal Register notices from January 1993 to May 1996 and selected
Federal Register notices from January 1990 to December 1992. We
reviewed all CPSC press releases from September 1992 to February
1996. We also reviewed 644 CPSC briefing packages prepared from
January 1, 1990, to September 30, 1996, including all briefing
packages that concerned a specific, identifiable potential product
hazard under consideration for regulatory, voluntary, or
information-gathering activities. We excluded matters exclusively
related to compliance with and enforcement of existing standards,
civil penalties, and internal management issues as well as other
items not related to a specific potential product hazard. In
addition, we reviewed other agency documents, including
information-clearing procedures, documentation of data systems,
consumer education materials, internal memorandums and
correspondence, and selected documents downloaded from CPSC's
Internet site.
In describing CPSC's project selection, we drew on our interviews
with commissioners, former commissioners, and agency staff, and we
reviewed the project selection criteria in CPSC's regulations.
Although we described the criteria used by the agency, we did not
make judgments about the merit of individual criteria or about
whether any criterion was appropriately applied in any specific case.
We compiled a list of agency projects on the basis of the documents
we reviewed. We included not only major regulatory efforts but also
smaller-scale projects. We gathered information on the
characteristics of these projects; for example, we obtained
information on how the projects originated, what action CPSC took,
and (for most) how many deaths and injuries were associated with the
hazard. CPSC officials examined our list and provided some
information that was not readily available in agency documents.
However, because CPSC's internal data management system does not
track agency projects, neither we nor CPSC were able to verify the
accuracy and comprehensiveness of this information. We also reviewed
agency documentation and the technical literature and interviewed
outside experts to obtain information on CPSC's data-gathering
systems.
To examine CPSC's cost-benefit analysis, we reviewed the technical
literature and developed a set of objective evaluation questions to
elicit descriptions of and to evaluate the analytical work. These
evaluation questions were designed to indicate only whether CPSC's
cost-benefit analyses were consistent with elements that are commonly
used to evaluate whether cost-benefit analyses are comprehensive and
reported in sufficient detail. Our review did not assess whether
CPSC's analyses were the best that could be done on any particular
topic. We reviewed these evaluation questions with two leading
experts in the field of risk assessment and cost-benefit analysis:
Professor John Mendeloff of the University of Pittsburgh and
Professor John Graham of Harvard University. Then we examined all
CPSC's cost-benefit analyses from January 1990 through September 1996
to see how they measured up against these evaluation questions. In
making these assessments, we reviewed all the information available
to the commissioners in the written record; that is, we did not
confine our analysis to the portion of the briefing packages that
dealt explicitly with cost-benefit analysis or risk assessment, and
we examined all the briefing packages that pertained to that
particular project. Although we evaluated CPSC's cost-benefit
analyses using a wide-ranging set of evaluation questions, we
reported results of our analysis only for those questions where: (1)
the question was applicable to the majority of the CPSC analyses we
reviewed; and (2) we were able to make a determination of whether the
analyses were consistent with the evaluation question in applicable
cases.
To review CPSC's information release process, we reviewed internal
agency procedures, discussed the information release requirements
with agency officials and industry representatives, and reviewed the
relevant legal cases. We provided information on CPSC's information
release procedures but did not audit whether CPSC complied with these
procedures in releasing information. Instead, we relied on other
sources--industry sources, published legal decisions, and
retractions--to assess whether this readily available evidence
suggested that CPSC had violated its statutory requirements in
releasing manufacturer-specific information.
Our review was conducted between August 1996 and May 1997 in
accordance with generally accepted government auditing standards.
AGENCY ORGANIZATION AND STRUCTURE
========================================================== Appendix II
The Consumer Product Safety Act (P.L. 92-573) provides for the
appointment of five commissioners by the President of the United
States for staggered 7-year terms. Not more than three of the
commissioners may be affiliated with the same political party. The
President appoints the commissioners, subject to Senate confirmation.
However, the President cannot directly overrule an agency decision or
fire a commissioner for making an unpopular decision. Since 1986,
there have been no more than three commissioners at one time. In
fulfilling the provisions of the government in the Sunshine Act (P.L.
94-409),\29 in general, commissioners must open to the public any
meeting held for the purpose of disposing of CPSC business--that is,
if any two of the three commissioners want to discuss agency
business, generally public notice must first be given. On a daily
basis, communication among CPSC offices takes place at the staff
level.
The chairman, the principal executive officer of the Commission,
directs all executive and administrative functions of the agency.
The chairman oversees the appointment and supervision of all agency
personnel except those employed in the immediate offices of other
commissioners. CPSC annually elects a vice chairman to act in the
absence or disability of the chairman or in case of a vacancy in the
office of chairman.
As figure II.1 shows, six offices report directly to the chairman:
-- the Office of the Secretary manages the agency's records,
publishes CPSC's public meetings calendar, and administers the
requirements of the Freedom of Information Act;
-- the Office of Congressional Relations serves as CPSC's liaison
with the Congress;
-- the Office of General Counsel provides advice and counsel to the
agency on legal matters;
-- the Office of the Inspector General is an independent office
that undertakes activities to prevent and detect waste, fraud,
and abuse; and
-- the Office of Equal Opportunity and Minority Enterprise monitors
compliance with equal opportunity employment laws.
The agency's executive director, who is appointed by and reports
directly to the chairman, is responsible for overseeing the
day-to-day management of the agency. The executive director's office
also houses the agency's small business ombudsman, who acts as a
liaison to small businesses, providing technical assistance
concerning CPSC programs and regulations, among other things.
The executive director has line authority over nine operating
directorates:
-- The Office of Compliance is responsible for compliance with and
administrative enforcement of CPSC regulations, including
product standards. The office initiates investigations on
safety hazards of products already in the consumer marketplace
or being offered for import. It negotiates and subsequently
monitors corrective action plans designed to give public notice
of hazards and recall defective or noncomplying products.
-- The Office of Hazard Identification and Reduction manages many
of CPSC's activities that involve identifying, examining, and
remedying new product hazards. The office's responsibilities
include collecting and analyzing data to identify hazards and
hazard patterns, carrying out CPSC's regulatory and voluntary
standards development projects, and coordinating international
activities related to consumer product safety. Serving in
various groups under the director of hazard identification and
reduction are the epidemiologists and statisticians, who provide
information on particular products; engineers and human factors
specialists, who help develop design remedies for product
hazards; and economists, who provide market information and
perform cost-benefit analysis on commission projects.
-- The Office of Information and Public Affairs is CPSC's
touchstone with consumers and the media. It prepares and
publishes brochures, booklets, fact sheets, and safety alerts
providing safety information to consumers on products used in
the home environment. The Office of Information and Public
Affairs also handles requests from the media for information and
access to CPSC staff and prepares press releases announcing CPSC
actions or decisions.
-- The Office of Information Services manages the agency's
toll-free hot line, Internet, and fax-on-demand services.
Within the Office of Information Services, CPSC's Information
Clearinghouse provides summary information on product-related
injuries in response to requests from the public.\30
-- The Office of the Budget is responsible for overseeing the
development of CPSC's budget.
-- The Office of Human Resources Management provides support to the
agency in recruitment and placement, position classification,
training, and other personnel areas.
-- The Office of Planning and Evaluation assists with long-term
planning efforts and manages and ensures agency compliance with
paperwork reduction regulations. In addition, the Office of
Planning and Evaluation is currently preparing for the
implementation of the Government Performance and Results Act
(the Results Act) and reviewing the effectiveness of the
agency's outreach efforts.
-- The Directorate for Field Operations coordinates the activities
of CPSC's 128 field staff located in 38 cities across the
country. CPSC field staff carry out a wide range of agency
activities, including conducting investigations of injury
incidents; acting as liaisons with state and local
organizations; working with the local press to support consumer
education campaigns; and inspecting manufacturers, importers,
distributors, and retailers to ensure compliance with safety
regulations and standards. To complement the efforts of the
field staff at the local level, CPSC contracts some product
safety work to state and local entities; commissions state and
local officials to function as officials of CPSC for the purpose
of conducting investigations, inspections, recalls, and sample
collections; and transmits information on CPSC programs and
activities to states.
-- The Directorate for Administration is responsible for executing
general administrative policies.
Figure II.1: CPSC Organization
Chart
(See figure in printed
edition.)
--------------------
\29 The purpose of the Sunshine Act is to provide the public with
information regarding the decision-making processes of the federal
government while protecting both the rights of individuals and the
ability of the government to carry out its responsibilities.
\30 If such a request cannot be filled without identifying the
manufacturer, however, the request is sent to the freedom of
information officer for processing.
CPSC PROJECTS, 1990-96
========================================================= Appendix III
CPSC has addressed many different product hazards. We identified 115
projects the agency worked on between January 1, 1990, and September
30, 1996. Because CPSC does not maintain a list of projects it has
worked on or any project characteristics, such as project origin or
result, we used various agency documents, such as budget requests,
annual operating plans, regulatory agendas, and project briefing
documents, to compile our list.
We attempted to determine how each project came to the attention of
the agency, for example, through a petition, congressional direction,
or internal initiation. This was not always clear. For example, in
three cases, a petition was submitted and subsequently denied, but
the issue was later mandated by the Congress as a project the agency
must address. For example, a petition concerning the safety of
bicycle helmets submitted in 1990 was denied. In 1994, the Congress
passed the Children's Bicycle Helmet Safety Act of 1994, which
requires CPSC to establish a mandatory standard. We counted such
projects as originating through the Congress. In other cases, we
categorized projects on the basis of information in CPSC documents.
We also attempted to identify the most significant action that
resulted from the agency's efforts. Some projects were ongoing for
several years and involved more than one agency action, such as
information campaigns conducted in conjunction with voluntary
standards efforts. For example, agency work on carbon monoxide (CO)
detectors resulted in a voluntary standard in 1992, a "CO Awareness
Week" information campaign in 1994, and public hearings on the issue
in 1996. In cases such as these, we attempted to identify the
activity with the greatest long-lasting impact; usually this was a
voluntary or mandatory standard. Also, projects classified as
"voluntary standard activity" include those for which a new voluntary
standard was created, an existing voluntary standard was revised, or
staff are currently participating in voluntary standard activities.
In addition, we identified whether a risk assessment or cost-benefit
analysis was completed for a given project, using CPSC briefing
packages. We defined a formal risk assessment to include only those
cases for which a numerical estimate of unit risk was calculated; we
defined a complete cost-benefit analysis to include those projects
for which both economic costs and benefits were explicitly compared,
even if quantitative estimates were not made for all economic factors
considered. Some risk-related or economic information was provided
for many of the projects for which no formal, complete risk
assessment or cost-benefit analysis was performed.
Table III.1
CPSC Projects, 1990-96
Origin Agency action Analytical efforts
---------------------------------------- ------------------------------------------------------------- --------------------------
Cost-
Voluntary Information benefit Risk
Initiated by Initiated by Mandatory standard and analysis assessment
Product the Congress Petition agency Ban standard activity\a education Other\b done done
---------------- ------------ ------------ ------------ -------- ------------ ------------ ------------ --------- ------------ ------------
Fire/gas codes and standards
-----------------------------------------------------------------------------------------------------------------------------------------------------
Camping heaters x x
Carbon monoxide x x
detectors
Children's x x x
sleep-wear
flammability
Cigarette x x x x
lighters
Cigarette safety x x
Elderly x x
sleepwear
flammability
Fire safety x x
devices
Fireworks, fuse- x\c x x
burn time
Fireworks, large x x x x
and small
reloadable
shell
Fireworks, x x x
multiple tube
mine and shell
Flexible gas x x
connectors
Floor furnaces x x x
Fuel gas x x
detectors
Gas control x x
valves,
automatic
Gas furnaces x x
Gas grills, 20- x x
lb. systems
Gas systems, x x
over-
pressurization
Heaters, x x
unvented gas
space heaters
Liquified x x
petroleum (LP)
gas odorant
fade
Mattress/ x x
bedding fires
Multipurpose x x\d x
lighters
Range/oven fires x x
Smoke detectors x x
Upholstered x x\e
furniture
Water heaters x x x
Sports and Recreation
-----------------------------------------------------------------------------------------------------------------------------------------------------
All-terrain x x\f x
vehicles (ATV)
Baseball x x
injuries\g
Baseball bases x x
Baseball, chest x x
protectors for
young batters
Baseball, face- x x x x
guards on
helmets
Baseball, safety x x
baseballs
(soft)
Bicycles x x x
Bicycle x x
reflectors
Bicycle/ x\h x x
recreation
helmets
In-line skates x x
Model rocket x x
motors
Pools, barriers x x
for swimming
pools and spas
Pools, diving x x
injuries
Pools, swimming x x
pool covers
Saunas x x
Soccer injuries x x
Soccer goals x x
Spas and hot x x x
tubs
Electrical/power codes and standards
-----------------------------------------------------------------------------------------------------------------------------------------------------
Garage door x x x\i
openers
Go-carts x x
Hair dryers x x
Heaters, x x
portable
electric
Heat tapes x x x
Holiday lights x x
Home electrical x x
system fires
Ladders, x x x x
portable
aluminum
Lamps, portable x x
Microwave ovens x x
Mowers, riding x x x x
Mowers, walk- x x
behind
Receptacle x x
outlets
Shock protection x x x
devices (GFCI)
Thermoplastics x x
in electrical
products
Children's products
-----------------------------------------------------------------------------------------------------------------------------------------------------
Baby bath seats x x
Baby walkers x x\d x x
Bean bag chairs x x
Bunk beds x x
Child restraints x x x x
on grocery
carts
Choking hazards- x x x
-balloons,
balls, marbles,
small figures,
pom-poms
Clacker balls x x x
Crib corner post x x x
extensions
Cribs, non- x x
full-size
Crib toys x x
Drawstrings on x x
children's
clothing
Electronic video x x x
games
5-gallon buckets x x x
Furniture x x
tipover
High chairs x x
Infant cushions x x x
Infant x x
suffocation
Lead in paint x x
Playground x x x x
equipment, home
Playground x x x\j
equipment,
public
Playground x x
equipment, soft
Playground x x
surfacing
Play yards x x
Strollers x x
Toddler beds x x
Window blind x x
cords
Window guards x x
Chemical/poison prevention
-----------------------------------------------------------------------------------------------------------------------------------------------------
Acetone in 1- x\k x
gallon
containers
Architectural x x x
glazing
Art materials x x
Charcoal x x x x
labeling
Child- x x x
resistant,
adult-friendly
packaging of
drugs
Child-resistant x x
packaging of
certain dietary
products with
iron powders
Child-resistant x x x
packaging of
dibucaine
Child-resistant x x x
packaging of
ibuprofen
Child-resistant x x x
packaging of
drugs with
loperamide
Child-resistant x\l x x
packaging of
glue removers
with
acetonitrile
Child-resistant x\l x x
packaging of
hair wave
neutralizers
Child-resistant x x x
packaging of
lidocaine
Child-resistant x x x
packaging of
mouthwashes
with ethanol
Child-resistant x x
packaging of
naproxen
Child-resistant x\m x
packaging of
spot remover
with naphtha
Coal-and x x x
woodburning
stove emissions
Dichlorobenzene x x x
Dioxin in paper x x x
products
Exemption to x x
child-
resistant
packaging of
mouthwashes
with ethanol
Indoor air x x
quality
Indoor air x x x
quality--
carpet
emissions
Indoor air x x
quality--
formaldehyde in
pressed wood
products
Indoor air x x
quality--
glycol ethers
Indoor air x x
quality--
heating,
ventilation,
and air-
conditioning
systems
Indoor air x x
quality--
kerosene
heaters
Indoor air x x
quality--
portable room
humidifiers
Lead poisoning x x
abatement
Methylene x x x
chloride in
paint strippers
Sulfuric acid x x
drain cleaners
Other
-----------------------------------------------------------------------------------------------------------------------------------------------------
Waterbeds x x x
=====================================================================================================================================================
Total 115 13 34 68 2 21 55 5 32 29 24
-----------------------------------------------------------------------------------------------------------------------------------------------------
\a "Voluntary standard activity" includes efforts in which agency
staff participated in the development of a new voluntary standard or
revision of an existing voluntary standard. This participation may
have included regularly attending meetings of a standards development
group; taking an active part in discussions; and/or providing
research or other support, such as information and education programs
or administrative assistance.
\b "Other" includes projects that were completed with no agency
action, terminated, currently in process or pending, or completed
with a study or report.
\c Two petitions were submitted for fuse-burn time. The first,
submitted in 1991, was denied. The second, submitted in 1996,
resulted in a change to an existing mandatory standard.
\d CPSC published an Advance Notice of Proposed Rulemaking (ANPR).
This is the first step toward enacting a mandatory standard.
\e CPSC published an ANPR in 1994, which is the first step toward
enacting a mandatory standard. The ANPR also solicited offers to
develop or modify a voluntary standard. During fiscal year 1996,
CPSC worked with an industry group on voluntary standards
development.
\f CPSC has worked with states to develop state ATV legislation.
\g The baseball injuries project started in the 1980s and resulted in
a special report on baseball injuries and protective equipment. The
projects--baseball bases, chest protectors, faceguards on helmets,
and safety baseballs--all grew out of this umbrella project and, as
such, are counted separately.
\h A petition was also submitted in 1990. The Congress mandated this
project in the Children's Bicycle Helmet Safety Act of 1994. Project
origin is counted as congressionally initiated.
\i The cost-benefit analysis for garage door openers applied to the
labeling and record-keeping requirements only, not to the provisions
of the safety standard.
\j The risk assessment for playground equipment examined the hazard
associated with exposure to arsenic from wooden equipment. The risk
assessment concluded that the risk was small. Most of CPSC's efforts
concerning playground equipment are related to injuries from falls or
impacts of equipment and are not related to chemical exposure.
\k Petition, submitted in 1995, was withdrawn.
\l One petition was submitted in 1988 by the American Association of
Poison Control Centers (AAPCC) requesting CPSC to require
child-resistant packaging both for household glue removers containing
acetonitrile and home cold wave permanent neutralizers containing
sodium bromate or potassium bromate.
\m Petition, submitted in 1991, was withdrawn.
A REVIEW OF THE LITERATURE ON
COST-BENEFIT ANALYSIS
========================================================== Appendix IV
Cost-benefit analysis can be described as an analytical technique
that details the expected positive and negative effects (expected
benefits and costs) of a given policy proposal. To construct this
framework, researchers approach a policy question needing
considerable information about the potential remedy. Because
cost-benefit analysis requires an inclusive approach to evaluating a
proposal's impact, the proposal itself must be well defined, and some
information must be known about its impact. Once the expected
benefits and costs of the proposal are thoroughly identified and
delineated, the next step is to place a value on each benefit or
cost. Frequently, these expected costs and benefits are expressed in
numerical or monetary terms to facilitate a comparison of the
aggregate costs and benefits. If all relevant factors can be
translated into monetary terms, the decision rule suggested by this
proceeding is to accept the proposal if its aggregate benefits exceed
its aggregate costs.\31
Although at first the concept behind cost-benefit analysis seems
relatively straightforward, the application of cost-benefit analysis
is not. The practical difficulties associated with measuring
effects, quantifying results, and accounting for uncertainty (to name
only a few issues) can create a gap between the way cost-benefit
analysis is described in theory and the way it is implemented in
practice. For this reason, experts generally agree that analysts
should be comprehensive in including all important factors and be
explicit in their description of the underlying data, methodology,
and assumptions.
This appendix describes many of the major methodological issues that
often arise in cost-benefit analysis and outlines specific elements
that are frequently used to evaluate cost-benefit analyses. The
methodology used to conduct a cost-benefit analysis frequently varies
depending on the circumstances and the context of the analysis. For
this reason, there is no complete set of standards for evaluating the
quality of an individual cost-benefit analysis. However, the
professional literature offers some guidance for analysts, and
certain specific elements are frequently used to determine whether a
given analysis meets a minimum threshold of comprehensiveness and
openness. These elements are necessary, but not sufficient, for a
quality analysis.
From the extensive cost-benefit literature, we developed objective
evaluation questions to evaluate cost-benefit analyses performed by
CPSC. These evaluation questions are summarized in table IV.1.
Although these evaluation questions are not a comprehensive measure
of the quality of an analysis, they were designed to reflect whether
an analysis is comprehensive and reported in sufficient detail. We
applied these questions to each of CPSC's analyses, but not all
evaluation questions were applicable to each case. In addition, for
some questions it was not always possible to determine whether CPSC's
analysis complied with the particular element reflected in the
question. For these reasons, and to emphasize those areas that we
considered as most critical, we reported only the evaluation results
that related to key elements and that applied to the majority of
CPSC's analyses; these eight are shown in bold in table IV.1.
Table IV.1
Frequently Used Elements of Evaluation
of Cost-Benefit Analysis
Number Evaluation question
-------- ------------------------------------------------
Evaluating potential courses of action
----------------------------------------------------------
1 Was descriptive information given about well-
defined alternative courses of action?
2 Was more than one alternative course of action
considered in the analysis?
3 Was evidence given to support the degree to
which the proposal was assumed to mitigate the
problem?
Considering all important factors for analysis
----------------------------------------------------------
4 Were all important categories of potential costs
and benefits included in the analysis? (For
example, did the analysis include [where
applicable] intangible benefits and costs,
health and safety benefits, costs of compliance,
upfront costs, price changes, and changes in
consumer surplus?)
5 Were the effects of intangible costs and
benefits detailed explicitly, even if they could
not be quantified? (For example, did the
analysis include [where applicable] consumer
utility?)
6 Did the analysis show how the existence of
intangible costs or benefits could affect the
conclusions?
7a Were possible indirect effects discussed? (For
example, did the analysis include [where
applicable] price changes of related goods or
likely changes in market concentration?)
7b If indirect effects were measured
quantitatively, were the calculations and
assumptions behind this measurement discussed in
detail?
7c If indirect effects were measured qualitatively,
did the analysis show how the existence of these
factors could affect the conclusions?
Considering interdependent factors
----------------------------------------------------------
8 Were possible issues of standing identified?
9 Was an incremental (marginal) analysis of costs
and benefits performed?
10 Were major policy interdependencies identified?
(For example, did the analysis discuss the
implications if another agency also had
jurisdiction over the product, or if the
proposal could create a conflict with existing
government policy?)
Considering distribution issues
----------------------------------------------------------
11 Did the analysis make clear the distribution of
gains and losses? (For example, who would pay
the costs and who would get the benefits of the
proposal?)
12a Was the impact of distribution of gains and
losses assessed? (For example, would the
proposal be likely to have a greater adverse
impact on small businesses, potentially leading
to increased market concentration?)
12b Were distributional weights employed in the
analysis?
12c If weights were used, was information provided
about the basis for those weights?
13a Is the standard assumption that relative market
prices are insensitive to the policy change
likely to hold in this case--that is, are there
likely to be few or no macroeconomic effects?
13b If no, was this impact discussed or considered
in the analysis?
Considering risk-risk factors and potential offsetting
behavi
----------------------------------------------------------
14 Were risk-risk or offsetting behavior concerns
identified and considered in the analysis?
Using the willingness-to-pay approach to value risk
reduction
----------------------------------------------------------
15 Were willingness-to-pay measures used to value
reductions in the risk of death or injury?
16 Is the numerical value of a statistical life
used in this analysis consistent with the
literature?
17 Is the numerical value of a reduction in the
chance of injury consistent with the literature?
Discounting future costs and benefits
----------------------------------------------------------
18a Are costs and benefits that occur at different
points discounted for differences in timing?
18b Does the discount rate or rates used include the
suggested rate by the Office of Management and
Budget?
18c Does the discount rate or rates used include the
Treasury bill rate for the time horizon of the
analysis?
18d Does the discount rate or rates used include a
lower "social" discount rate?
Considering data issues
----------------------------------------------------------
19a Where applicable, were the implications of
unverified or unverifiable data provided by an
interested party identified and discussed?
19b Where applicable, were the implications of
uncertainty surrounding a dose-response model
identified and discussed?
19c Where applicable, were the implications of
uncertainty surrounding cross-species
extrapolation identified and discussed?
19d Where applicable, were the implications of the
use of data relying on investigated or reported
cases identified and discussed?
19e Where applicable, were the implications of
survival bias in the underlying data identified
and discussed?
19f Where applicable, were the implications of small
sample sizes identified and discussed?
19g Where applicable, were the implications of other
known biases in the underlying data identified
and discussed?
20 If the underlying data were derived from a
statistical sample, were appropriate measures of
precision provided?
Performing sensitivity analysis
----------------------------------------------------------
21a Was sensitivity analysis performed on any
parameter in the analysis?
21b Was sensitivity analysis performed on the value
of a statistical life?
21c Was sensitivity analysis performed on the value
of injury reduction?
21d Was sensitivity analysis performed on the
discount rate?
21e Was sensitivity analysis performed on the
precision of the underlying data?
21f Was sensitivity analysis performed on other
important parameters?
----------------------------------------------------------
Note: We reported results for questions that are in bold. We viewed
these measures as especially important, they applied to the majority
of CPSC analyses, and a determination could be made for these
questions in each case.
--------------------
\31 Variations of cost-benefit analysis include cost-effectiveness
analysis, which relates the costs and outcomes of alternative methods
to achieve the same desired goal. For example, one
cost-effectiveness analysis compared the cost per case detected for
different methods of treating the same medical condition. Because
only cost-benefit analysis is relevant to CPSC's activities, related
techniques are not considered here. For more information on these
and other related analytical tools, see Michael F. Drummond, Greg L.
Stoddart, and George W. Torrance, Methods for the Economic
Evaluation of Health Care Programmes (New York: Oxford University
Press, 1996).
BACKGROUND
-------------------------------------------------------- Appendix IV:1
As an organizing framework, cost-benefit analysis can help a
decisionmaker to organize and aggregate all the relevant information
in a way that can clarify the nature of the trade-offs involved in a
decision. At the same time, by providing a framework to convert
dissimilar effects to a common measurement, cost-benefit analysis can
allow this information to be weighed and aggregated to help make a
decision. A well-constructed cost-benefit analysis can highlight
crucial factors, expose possible biases, and perhaps expand the
openness of the decision-making process by clarifying the factors on
which the decision was based--whether these factors are purely
economic criteria or include other social factors. A key advantage
of carefully built cost-benefit analysis is that it promotes explicit
rather than implicit decision-making, even when it is impossible to
monetize or even quantify all the potential effects of a given
regulatory proposal.
Despite the value of this analytical tool, in some situations using
cost-benefit analysis as the sole basis for decision-making may be
inappropriate. For example, alternatives that would eliminate basic
human rights or dramatically increase income inequality may be viewed
as morally unacceptable. Similarly, when uncertainty about possible
effects is so pervasive that attempting to identify potential costs
and benefits would amount to nothing more than uninformed
speculation, cost-benefit analysis probably has little to offer.\32
In addition, for minor decisions it may not be necessary to employ a
rigorous, time- and resource-consuming decision-making process, and a
less detailed cost-benefit analysis--or none at all--may be more
appropriate. Virtually all observers agree that the appropriate role
for cost-benefit analysis--sole decision-making rule, input into
decision-making, or not done at all--will depend on the context in
which the particular decision is being considered.
To some individuals, application of an abstract analytical technique
such as cost-benefit analysis is especially unpalatable in certain
situations. For example, the idea of applying a value to �saving
life� may be distasteful, because we regard our own lives as
�priceless� or of infinite value. However, although our lives may be
priceless, avoiding risks often means forgoing time, convenience,
enjoyment, or other opportunities--all of which do have a price.
Thus, policy interventions that can affect life expectancy pose an
unavoidable problem--to refuse to consider the value of potentially
lifesaving interventions is to implicitly value them at zero, and to
consider any potentially lifesaving activity as infinitely valuable
implies that individuals would never take any action that involves
risk (driving a car, for example). The literature on cost-benefit
analysis makes a key distinction in this area. For ethical reasons,
most practitioners consider it inappropriate to use cost-benefit
analysis to evaluate alternatives that would (with certainty) affect
the life expectancy of a given, known individual. But government
policy does not usually involve making such decisions. Instead,
policy questions typically center on actions that may bring about
small changes in the statistical life expectancy of anonymous members
of a large group. For example, when CPSC considered imposing a
mandatory regulation on large, multiple-tube fireworks, the agency
estimated that such a standard could reduce the number of individuals
that die in related accidents by one anonymous consumer over a 3-year
period. On a daily basis, each of us makes such trade-offs between
perfect safety and other things we value, such as the convenience of
driving a car or the excitement of skiing down a mountain. Such
assessments, which place a value on reductions in risk, are viewed as
appropriate, whereas valuing a given person's life may be viewed as
less reasonable.
Similarly, a decision to surrender basic rights--such as liberty--may
be unacceptable on moral grounds, and so cost-benefit analysis might
not be applicable in a situation involving such rights. In addition,
some individuals object to cost-benefit analysis in circumstances in
which other species may be affected adversely. For example, one
individual objected to a proposal that would allow increased
pollution in a particular river because the proposal would adversely
affect the fish and other species in the river. Cost-benefit
analysis would probably be unable to address this objection because
there is no accepted method to place a value on the losses suffered
by the fish.
--------------------
\32 However, one should use caution in assuming that this situation
applies. Sometimes the conclusions of a cost-benefit analysis can be
quite robust even in the face of substantial uncertainty. For
example, a recent analysis of the benefits and costs of reducing
emissions of acrylonitrile concluded that imposing strict standards
on acrylonitrile emissions fails reasonable cost-benefit tests,
despite as much as a threefold factor of uncertainty in acrylonitrile
emissions, additional uncertainty in the amount of exposure to
acrylonitrile, and a probable but unmeasurable upward bias in the
estimate of cancer risk. (See John A. Haigh, David Harrison, Jr.,
and Albert L. Nichols, "Benefit-Cost Analysis of Environmental
Regulation: Case Studies of Hazardous Air Pollution," ch. 1 in The
Moral Dimensions of Public Policy Choice: Beyond the Market
Paradigm, eds., John Martin Gilroy and Maurice Wade (Pittsburgh, Pa.:
University of Pittsburgh Press, 1992), pp. 15-57.
CONCEPTUAL, ANALYTICAL, AND
TECHNICAL ISSUES ARISE IN
APPLICATION OF COST-BENEFIT
ANALYSIS
-------------------------------------------------------- Appendix IV:2
Over the past 3 decades, a substantial economic literature has
explored the conceptual, analytical, and technical issues posed by
the application of cost-benefit analysis. In this literature, some
elements that cover a wide range of research methods are generally
accepted. As one experienced researcher has said, �Good studies
follow procedures that are in accord with economic theory for
estimating benefits and costs, provide a clear statement of all
assumptions, point out uncertainties where they exist, and suggest
realistic margins of error.�\33 Even when undertaken by careful and
competent researchers, cost-benefit analysis can sometimes be
difficult to interpret, especially when uncertainty is substantial
and information is incomplete.
--------------------
\33 See Eban Goodstein, "Benefit-Cost Analysis at the EPA," The
Journal of Socio-Economics, 24 (2) (1995), pp. 375-89.
CONCEPTUAL ISSUES IN
COST-BENEFIT ANALYSIS
------------------------------------------------------ Appendix IV:2.1
Some of the issues underlying the application of cost-benefit
analysis are conceptual, and the researcher may make a different
judgment in different circumstances. With respect to these issues,
being complete and explicit is important--the consensus in the
literature is that while there may be no single method for dealing
with these issues that is universally appropriate, the researcher
must be clear and direct in detailing how the issue was addressed in
the context of the analysis.
INTANGIBLE OR
UNQUANTIFIABLE FACTORS
CAN POSE DILEMMAS FOR
RESEARCHERS
---------------------------------------------------- Appendix IV:2.1.1
Ideally, a cost-benefit analysis involves translating each impact
into a common measurement (such as dollars) for comparison. However,
some effects may be difficult or impossible to measure or quantify.
For example, a researcher evaluating an alternative outpatient mental
health treatment program realized that the greater independence
afforded by the outpatient program could create increased
anxiety--or, alternatively, higher self-esteem--for some
participants. However, these intangible effects would be very
difficult to measure and even more difficult to quantify.\34 Some
individuals have criticized certain individual practitioners of
cost-benefit analysis for ignoring or de-emphasizing aspects of
proposed changes that cannot be easily quantified. Although it is
necessary for such effects to be described and emphasized
appropriately, the existence of such intangibles does not necessarily
limit the value of the analysis. For example, in the discussion of
mental health programs, the researchers found that patients in the
experimental program experienced higher satisfaction and reported
having a greater number of social relationships. These qualitative
benefits were important to the analysis even though they could not be
valued in dollars. In addition, if it can be shown explicitly that
the value of the intangibles would be unlikely to change the
conclusion, then cost-benefit analysis has played a valuable role by
considering intangible effects explicitly, even if it is not possible
to consider them quantitatively. (We addressed this issue in
questions 5 and 6 of table IV.1.)
--------------------
\34 See Burton A. Weisbrod, "Benefit-Cost Analysis of a Controlled
Experiment: Treating the Mentally Ill," Journal of Human Resources,
16 (4) (1981), pp. 523-48.
MEASURING INDIRECT AND
SECONDARY COSTS AND
BENEFITS CAN BE DIFFICULT
---------------------------------------------------- Appendix IV:2.1.2
Researchers must also decide whether or not to include indirect or
secondary effects that may result from the proposal. For example, a
change in tax rules may not only have an initial, direct effect on
individuals' income but may also create a secondary ripple (or
�multiplier�) effect on the economy as a whole. Similarly, a medical
treatment that prolongs life can be expected to have a secondary
effect on health care costs as individuals live longer.
A researcher who wants to include such secondary impacts will
frequently find a measurement challenge, because determining the
magnitude (or perhaps even the existence) of many secondary effects
involves answering a �what if?� type of question, and frequently
without relevant historical experience. For example, individuals who
receive improved treatment for hypertension may contract more
prolonged or costly illnesses in the future.\35 Any attempt to
measure the impact of these indirect future costs would require the
researcher to predict the future health status and health care costs
of the patient group in the event that the treatment had or had not
been available. In addition, the existence of some secondary effects
is dependent on one or more key assumptions. For example, an
economic multiplier effect is generally thought to take place only in
an economy operating below its productive capacity. A researcher who
wants to include such an effect must thus determine whether or not
this assumption holds, and then decide how large an effect should be
included.
Secondary impacts may be considered part of the cost-benefit analysis
or extraneous to it. The proper choice in each circumstance probably
depends on the likelihood that the secondary effects in fact exist,
their probable importance, and the ability to measure them. However,
like intangible effects, potential secondary or indirect effects
should be detailed even if they are not included quantitatively.
Furthermore, should such effects be included, the assumptions
underlying their existence and measurement must be revealed
explicitly to allow the reader to make an informed judgment. (We
addressed this issue in question 7 of table IV.1.)
--------------------
\35 For an example of such an analysis, see W.B. Stason and M.C.
Weinstein, "Allocation of Resources to Manage Hypertension," The New
England Journal of Medicine, 296 (13) (1977), pp. 732-39.
ISSUES OF "STANDING" MAY
REQUIRE CLARIFICATION OF
THE SCOPE OF THE ANALYSIS
---------------------------------------------------- Appendix IV:2.1.3
In some circumstances, it may be difficult to define the point of
view from which the cost-benefit analysis is calculated--that is, who
has "standing," or whose costs and benefits should be included in the
analysis. For most cost-benefit calculations involving government
policy, the analysis is appropriately done at the level of �society�
(rather than, for example, considering only the impact on individuals
in a particular state or locality), so that a wide range of
implications is considered. However, in some circumstances, it is
important to make the point of view explicit. For example, when
policy effects may cross national boundaries, �society� may be best
defined on a broader basis. If the cost-benefit analysis is
considering policies to reduce acid rain, for example, considering
the effects of acid rain and pollution reduction on the Canadians who
share U.S. weather patterns may be important. Similarly, analysts
may wish to consider the impact of product regulation on foreign
producers as well as on U.S. consumers.
Even when the unit of analysis is clear, occasionally some members of
that unit are not afforded standing under cost-benefit analysis. For
example, suppose that a particular policy proposal is likely to
result in a decrease in property crime. If a thief steals a
television, and both the victim and the thief are given standing,
then the gain to the thief could offset the loss to the victim in a
cost-benefit accounting. �Society,� however, is clearly worse off.
Unless we try to measure the psychological and sociological costs of
crime, it may make more sense not to afford the thief standing in the
cost-benefit analysis. Such issues arise infrequently but should be
made clear if they could affect the interpretation of the analysis.
(We addressed this issue in question 8 of table IV.1.)
TAKING THE STATUS QUO AS
BASELINE, COST-BENEFIT
ANALYSIS SHOULD ACCOUNT
FOR RELEVANT FACTORS AND
CONSTRAINTS
---------------------------------------------------- Appendix IV:2.1.4
Some cost-benefit analyses have been criticized for excluding
relevant factors on what appears to be an ad hoc basis. Without
including such factors, or providing an explicit justification for
excluding them, a researcher limits the value of the analysis.
However, while it is important to consider a variety of alternative
actions, it is equally crucial to adopt a realistic view of the
possible alternatives. For example, political and agency realities
often place meaningful constraints on options available for
consideration. If these constraints are not incorporated into the
analysis, the results could be �pie in the sky� recommendations that
would be of little use to the decisionmaker. (We addressed these
issues in questions 1, 2, 4, and 10 of table IV.1.)
Similarly, it is important that cost-benefit analysis be conducted in
a way that evaluates the change in aggregate costs and benefits, with
current conditions serving as a baseline. For the properties of
economic efficiency to hold, cost-benefit analysis must take what
economists call a marginal or incremental approach--that is, it must
consider only the changes that would result from the proposed
intervention. For example, a proposal to renovate a museum should be
evaluated, for cost-benefit purposes, on the basis of the incremental
cost of the renovation, not on the basis of the original cost of
constructing the entire museum. (We addressed this issue in question
9 of table IV.1.)
ISSUES OF DISTRIBUTION
MAY BE CONSIDERED WITHIN
OR OUTSIDE THE
COST-BENEFIT CONTEXT
---------------------------------------------------- Appendix IV:2.1.5
By definition, cost-benefit analysis is a method for considering the
aggregate effects of a given proposal. If the aggregate expected
benefits of the proposal exceed the aggregate expected costs, the
proposal is said to pass the cost-benefit test. This does not
necessarily mean that adopting the proposal would improve the
condition of each individual; instead, it implies only that when the
expected gains and losses to all individuals are added up, the total
expected gains exceed the total expected losses.\36
By adding up individual gains and losses to determine the effect on
society as a whole, cost-benefit analysis implicitly assumes that
each individual's gains or losses should be valued equally with any
other individual's gains or losses. As a result, cost-benefit
analysis is �neutral� with respect to the distribution of gains and
losses. To put it another way, the �fairness� of how gains and
losses are distributed is generally not included in the calculations
underlying a cost-benefit analysis.
Sometimes a decisionmaker might want to address such issues of
fairness. For example, if a proposal would involve redistributing
income, the loss of one dollar to a wealthy person may be viewed as
less consequential than the gain of one dollar to a poor person.\37
In order to address such an issue, the researcher or decisionmaker
can consider issues of distribution inside or outside the
cost-benefit context. One method of examining such issues might be
to analyze the distributional impact of a proposal separately, and
consider fairness issues--along with the cost-benefit calculation--as
another factor in the decision-making process. Under these
circumstances, if the proposal is viewed as having negative
distributional consequences, and it would be costly or difficult to
redistribute the gains and losses, then the proposal might be
rejected even though it would otherwise meet a cost-benefit test.
Occasionally a researcher who wants to consider distributional issues
explicitly chooses to incorporate distributional consequences into
the mathematics of the cost-benefit calculation. Instead of simply
adding up the costs and benefits accruing to each individual, the
researcher uses a mathematical formula that applies different weights
to different individuals. These weights could be based on a number
of factors, depending on the characteristics of the �fairness� issues
being addressed. For example, if the proposal would affect income
directly, weights could be based on income (with greater weight being
applied to gains in income experienced by low-income individuals).
Weights could also be based on other circumstances; for example,
greater weight could be placed on vulnerable populations or on future
generations. Similarly, if a change was proposed in a given program,
greater weight could be applied to the smaller number of program
participants (who would be more significantly affected) than the rest
of society (who would be affected only indirectly). A weighting
scheme would be most helpful to decisionmakers and outside observers,
however, if the formula was detailed explicitly and the analysis was
accompanied by a sensitivity analysis, so the effect of the
distributional weights was clear to the reader. (We addressed these
issues in questions 11 and 12 of table IV.1.)
--------------------
\36 Economists make a distinction between events that lead to a
"Pareto improvement" and those that lead only to a "potential Pareto
improvement." If a change leads to a Pareto improvement, then as a
result of the change no individual will be worse off, and at least
one individual will be better off. However, if a change leads to a
"potential Pareto improvement," then the aggregate gains exceed the
aggregate losses, but some individuals will gain and others will
lose. In this situation, it would be theoretically possible for the
"winners" to compensate the "losers" and leave no individual worse
off than before if redistribution itself is costless.
\37 There is some limited ground in utilitarian economic theory for
this assumption. Economic theory states that individuals will
generally, at some point, experience �diminishing marginal utility of
income�--that is, receiving one additional dollar of income brings
less additional satisfaction (or �utility�) as the individual becomes
wealthier. However, the amount of satisfaction deriving from any
good or service, including income, cannot be compared across
individuals. As a result, while economic theory suggests that it is
plausible that at some point giving one dollar to a rich person
produces less total �utility� than giving one dollar to a poor
person, economic theory also tells us that it is generally not
possible to measure the circumstances under which this assumption
might (or might not) be correct.
IMPLEMENTATION ISSUES IN
COST-BENEFIT ANALYSIS
------------------------------------------------------ Appendix IV:2.2
While a number of conceptual issues arise in cost-benefit analysis
for which the appropriate answer depends on circumstances, there are
also a number of methodological or implementation issues about which
there is widespread agreement. For example, years of debate on the
appropriate valuation method for risk reduction have largely been
resolved in the favor of �willingness to pay� measures, and the
importance of sensitivity analysis is generally recognized. While
the choice of discount rate has long been a subject of controversy,
with some practitioners arguing for the use of market rates and
others advocating a lower �social� discount rate, the literature
generally recognizes the value of multiple rates. Finally, there is
virtually universal agreement on the importance of reliable data and
careful risk measurement to cost-benefit analysis.
COSTS AND BENEFITS SHOULD
BE ADJUSTED FOR
DIFFERENCES IN TIMING
---------------------------------------------------- Appendix IV:2.2.1
Frequently, the various consequences of the proposal under discussion
will differ in when they occur. For example, changing the labeling
requirements on bags of charcoal could reduce carbon monoxide deaths
years later and also result in an immediate, one-time increase in
industry costs. As individuals and as a society, we generally prefer
to have dollars or resources now than at some time in the future
because we can benefit from them in the interim. In addition, if we
acquire $1 next year instead of today, we give up the opportunity to
invest that dollar and earn interest on our investment. As a result,
it is generally agreed that future dollar cost and benefit streams
should be reduced or "discounted" to reflect differences in timing.
The rate at which this adjustment is made is usually done on a
case-by-case basis. Many analysts choose to use the market rate of
interest--for example, the rate payable on government bonds with a
time horizon comparable to the analysis. Some researchers have
argued that the discount rate should be set somewhat higher because,
presumably, economic growth will leave future generations better off.
Others have advocated using a "social" discount rate, which is
generally lower than the market rate, on the grounds that society's
interest in the welfare of future generations implies that the
discount rate for all of society should be lower than the rates
chosen by individuals. Experts on cost-benefit analysis generally
encourage researchers to use multiple rates--both to assess the
sensitivity of the results to the chosen discount rate and to
facilitate comparison across studies. (We addressed issues of
discounting in question 18 of table IV.1.)
COMPARABILITY WITH OTHER
ANALYSES COULD INFLUENCE
SELECTION OF PARAMETERS
---------------------------------------------------- Appendix IV:2.2.2
Ideally, proponents of extensive cost-benefit analysis would like to
be able to order and prioritize regulatory interventions across
agencies. A potential mechanism for this type of coordination is
provided in Executive Order 12866, which requires agencies to submit
to the Office of Management and Budget (OMB) cost-benefit analyses
for regulations with an estimated impact on the economy of $100
million or more.\38 OMB has suggested or prescribed certain rules
(such as a presentation format and/or a specific discount rate) to
facilitate comparability across agencies and to help promote quality
analyses.\39 While a common set of assumptions or parameters may
facilitate comparability by a central government authority, so that
priorities may be set across as well as within agencies, some of
these assumptions may fit the circumstances of the given analyses
less well. For example, a given discount rate may be more
appropriate for one project (for which the benefits and costs are
spread out over a long period) than another (with costs and benefits
spread over a shorter period). Therefore, there may be a tension
between making an analysis comparable to others and customizing it to
fit a unique situation. (We addressed these issues in questions 10
and 18 of table IV.1.)
--------------------
\38 CPSC, as an independent agency, is not required to follow these
procedures.
\39 The evaluation factors we examined parallel OMB's suggestions; in
some areas, we have operationalized these concepts to create more
specific and objective evaluation questions.
POTENTIAL FUTURE PRICE
CHANGES CAN ADD
UNCERTAINTY TO ANALYSIS
---------------------------------------------------- Appendix IV:2.2.3
Typically, cost-benefit analysis assumes that the intervention
considered has only a small or negligible effect on relative prices
throughout the economy. Generally, this assumption makes sense;
however, for major changes (such as a big change in tax law) the
assumption is clearly inappropriate. In these cases, it is much
harder to put a value on the potential impact of the proposed change.
For example, a large cut in the capital gains tax could affect a wide
range of investment behavior, and may even affect how much some
individuals choose to work and how much they are paid. A researcher
trying to place a value on the potential consequences of a capital
gains tax cut might introduce an error by valuing changes in
productivity using the wage rates that prevailed before the cut,
because, in reality, once the new policy has been implemented, wage
rates might change. In such a situation, it is especially important
for the researcher to point out the potential mismeasurement, and (if
feasible) try to model its effect. (We addressed these issues in
question 13 of table IV.1.)
RESEARCHERS RECOGNIZE THE
IMPORTANCE OF SENSITIVITY
ANALYSIS TO INFORM THE
READER
---------------------------------------------------- Appendix IV:2.2.4
Every cost-benefit analysis will include some level of uncertainty or
imprecision, or reflect a methodological choice that not everyone
will necessarily agree with. Careful analysts will identify critical
sources of uncertainty or controversy, and revise or test the
analysis quantitatively or qualitatively to identify how or whether
these areas affect the conclusions reached. If large variations in
measurement or assumption do not alter the conclusions, then the
researcher and the decisionmaker can have greater confidence in the
original results. However, if the conclusions of the analysis can
change depending on methodological choices or variable measurement,
then the researcher may want to try to improve the measurement of the
original variables. If this is not possible, then the analysis may
be of more limited value--an implication the decisionmaker would need
to know. (We addressed these issues in question 21 of table IV.1.)
RELIABLE UNDERLYING DATA
CRUCIAL TO RELIABLE
COST-BENEFIT ANALYSIS
---------------------------------------------------- Appendix IV:2.2.5
Despite the importance of sensitivity analysis, even the most careful
and elaborate sensitivity analysis often cannot sufficiently
compensate for poor underlying data. Obviously, if key variables are
defined poorly or measured imprecisely, the quality of the
cost-benefit analysis will suffer. Because many variables used in
research are measured from surveys or samples, sampling error may be
unavoidable. The researcher can use a sensitivity analysis to test
the potential for sampling error to affect the conclusions; however,
if the survey instrument itself is flawed, the results may be
unreliable even beyond the degree indicated by sampling error.
Similarly, researchers may face a difficult problem when key data
(such as the cost of compliance) are provided for the analysis by an
interested party (such as the industry). If the interested party
provides the data, that party may have an incentive to provide
biased, inaccurate, or misleading information. In such cases, it is
important to try to verify the data to the extent possible; if not,
the researcher needs to at least note the source of the underlying
information and allow the reader to make an informed judgment about
the reliability of the final analysis. (We addressed potential data
issues in questions 19 and 20 of table IV.1.)
MEASURING AND VALUING RISK
REDUCTION UNDER COST-BENEFIT
ANALYSIS
------------------------------------------------------ Appendix IV:2.3
Frequently, the benefits of a particular proposal involve a reduction
in risk of injury or death. In these situations, the quality of the
cost-benefit analysis will also depend substantially on the estimates
of the proposal's impact on these risks. However, it is often
difficult to accurately measure and value risk reduction. A number
of issues arise, including the effect of the proposal on the level of
risk (especially when individual behavior is a factor in determining
the size of the risk) and measuring individuals' willingness to take
risks in exchange for rewards.
CONSIDERABLE UNCERTAINTY
OFTEN SURROUNDS RISK
MEASUREMENT AND RISK
ASSESSMENT
---------------------------------------------------- Appendix IV:2.3.1
Measuring the effect of a proposal on the level of risk can be
especially difficult. Generally, it is not realistic to assume that
any regulatory intervention will reduce the risk level to
zero--removing an environmental carcinogen, for example, will
probably reduce the number of cancer deaths, but is unlikely to
eliminate cancer entirely. Considerable uncertainty may be
inevitable when extrapolating a dose-response model. Sometimes large
doses of a potentially dangerous substance, given over a short period
of time, are used to predict the results of long-term exposure. When
the model moves across species--for example, predicting cases of
cancer in humans based on experiments with laboratory
animals--another source of uncertainty is introduced.
Measurement problems in the underlying data can make it hard to
predict future risk from epidemiological data as well. For example,
when longitudinal data are used, there may arise a "survival bias" in
that the analysis could be biased if it excludes individuals who die
or otherwise move out of the data set. Similarly, when some cases go
unreported, the data may understate the size and/or overstate the
severity of the hazard. For rare hazards, limited variation in the
epidemiological data can make measurement and prediction of risk more
difficult. In addition, some data may be known to produce biased
estimates (because of exclusions or potential double-counting, for
example). Data problems such as these (and the statistical or
analytical methods used to deal with them) should be pointed out to
the reader to increase the opportunity for informed judgment. (We
addressed these issues in questions 3 and 19 of table IV.1.)
RISK-RISK TRADE-OFFS AND
INDIVIDUALS' BEHAVIORAL
RESPONSES TO POLICY
CHANGES CAN AFFECT THE
LEVEL OF RISK
---------------------------------------------------- Appendix IV:2.3.2
Sometimes actions taken to reduce one risk can have the unintended
effect of increasing that or another risk. For example, unforeseen
consequences arose in the 1970s when CPSC issued regulations
requiring children's sleepwear to meet flammability standards.
Manufacturers used a chemical called Tris to meet these standards,
but later it was discovered that Tris posed a cancer risk. Changes
in individual behavior can also create uncertainty in predicting the
level of risk because valuations of costs and benefits of a proposed
action are often based on historical behavior. For example,
valuations of a proposed new highway may be based on the number of
individuals currently traveling the route, their commuting times, and
other factors. However, once the new highway is built, some people
may make extra trips in the area (shopping in different stores, for
example) than they did before. Some people who had driven at
off-hours to avoid congestion on the old road may now travel at peak
hours on the new one. Similarly, changes in public policy may cause
individuals or firms to change the technology they use, the amount of
time they spend at leisure or at work, or the amount they invest in
innovation.
A special type of behavioral change�often referred to as �offsetting
behavior�--occurs when individuals change the amount of precautions
they take in response to a change in policy. For example, having an
air bag in the car or being required to wear a motorcycle helmet
might make some drivers feel safer, so they exercise less caution on
the road. Sometimes this offsetting behavior can result in an
increase in the risk the intervention was designed to mitigate. For
example, because child-resistant medicine bottles can be difficult to
open, a grandparent may leave the cap off the bottle, creating an
even greater risk than would be posed with a non-child-resistant cap.
If such changes in consumer behavior are foreseeable, the analysis
will be improved if the researcher points out such possibilities, and
it will be even more useful if reasonable attempts can be made to
measure the impact. (We addressed these factors in question 14 of
table IV.1.)
WILLINGNESS-TO-PAY
MEASURES VIEWED AS
APPROPRIATE FOR VALUING
RISK BUT REQUIRE CAREFUL
ESTIMATION
---------------------------------------------------- Appendix IV:2.3.3
Measuring the benefit of risk reduction requires placing a value on
avoiding death and/or injury. Several approaches are available for
this task, but two have been used in cost-benefit analysis: (1) the
�human capital� approach, in which death or injury is valued at the
market value of the lost production it causes plus the medical costs
expended and (2) the �willingness-to-pay� approach, which attempts to
measure directly individuals' willingness to pay for reducing the
risk of death or injury.\40 In early cost-benefit analysis, the human
capital approach was used, largely because lost wages and medical
costs were relatively easy to measure. However, this approach is not
preferred today because of a number of shortcomings. First, if this
approach were taken literally to apply to individuals, then persons
who do not produce output in the marketplace (such as the elderly or
homemakers) are not valued�an assumption that is clearly ethically
and economically inappropriate. In addition, the human capital
approach is unable to take into account costs to the individual of
death or injury such as pain and suffering. Also, it is difficult to
apply an �average� value based on human capital calculations to a
statistically anonymous member of a large group. Finally, we are all
more valuable than the sum of what we produce, and these amounts
could not be included in a human-capital-based formulation. Thus,
human capital measurements are generally viewed as a lower boundary
on the value of avoiding death and injury, and are less preferred
than the more recent willingness-to-pay measures.
In part because of dissatisfaction with human capital measures as
used in cost-benefit analysis, and facilitated by newly available
large micro-level data sets, economists measure the benefits of
reducing death or injury by calculating the consumer's willingness to
pay for small reductions in the probability of injury or death.
These calculations have been done several ways. Some approaches
attempt to glean willingness-to-pay measures from observed behavior
in the marketplace. For example, one approach examines pay
differentials for jobs with different risks. Another approach looks
at the payoff to consumers who purchase safety devices, such as smoke
detectors and air bags in automobiles. These measures have the
advantage of being based on actual observed behavior in the
marketplace rather than on an artificial experimental situation.
However, the validity of such measures is based on the questionable
assumption that workers and consumers are sufficiently knowledgeable
about the risks they face and the potential of different occupations
or safety devices to alleviate those risks. For many purposes,
practitioners of cost-benefit analysis can select an appropriate
value from the range of research already done in this field without
performing the actual analysis themselves. However, for other
analyses, especially those involving unique risk-taking situations,
it may be wiser to gather new data to construct an estimate that is
based on circumstances as close as possible to those being studied.
Another common method for valuing risks is known as contingent
valuation, in which such values are elicited by observing responses
or behavior on a survey or in a controlled experiment. For example,
researchers surveyed individual shoppers on how much of a premium
they would be willing to pay for pesticide-free grapefruit.\41 These
methods have the advantage of being able to provide information on
areas that cannot be addressed with market data. However, this
characteristic could also be a weakness�the very artificiality of the
situation could make the consumer make a less deliberate choice or
could limit the usefulness of applying this measure to other
situations. This method does entail some technical requirements�for
example, it may be useful to perform statistical tests on the
distributional assumptions when constructing contingent valuation
measures. (We addressed these issues in questions 15 through 17 of
table IV.1.)
--------------------
\40 This terminology has been borrowed from W.K. Viscusi in "The
Valuation of Risks to Life and Health," in Benefits Assessment: The
State of the Art, eds., Judith D. Bentkover, Vincent T. Covello,
and Jeryl Mumpower (Dordrecht, Holland: Reidel Publishing Company,
1986), pp. 193-210.
\41 Jean C. Buzby, Richard C. Ready, and Jerry R. Skees,
"Contingent Valuation in Food Policy Analysis: A Case Study of a
Pesticide-Residue Risk Reduction," Journal of Agricultural and
Applied Economics, 27 (2) (Dec. 1995), pp. 613-25.
DEATH, INJURY, AND INCIDENT DATA
SOURCES USED BY CPSC
=========================================================== Appendix V
In order to target its resources and analyze the costs and benefits
of projects or potential projects, CPSC must obtain data on injuries
and deaths related to particular product hazards. CPSC relies on a
patchwork of independent data systems to address this need. However,
not only does each of these data sources have its own internal
limitations, but together CPSC's data sources present an
incomplete�and potentially distorted�picture of
consumer-product-related injuries and deaths. The implications of
this lack of data range from reducing CPSC's ability to apply
regulatory project selection criteria to limiting the agency's
ability to estimate the impact of its regulatory actions.
CPSC RELIES HEAVILY ON HOSPITAL
EMERGENCY ROOM REPORTS TO
OBTAIN ESTIMATES OF INJURIES
--------------------------------------------------------- Appendix V:1
NEISS SYSTEM
------------------------------------------------------- Appendix V:1.1
CPSC obtains most of its injury information from its National
Electronic Injury Surveillance System (NEISS), which gathers
information from hospital emergency room records. NEISS provides
national estimates about the number and severity of
emergency-room-treated injuries associated with, although not
necessarily caused by, consumer products in the United States. To
accomplish this, a stratified probability sample of hospitals is
drawn that is representative of all hospitals with emergency
departments in the United States and its territories.\42 CPSC
constructs national estimates of the number of injuries associated
with individual consumer products on the basis of reports from this
sample of 101 hospitals.
NEISS data result from information abstracted from hospital emergency
room records by coders trained by CPSC staff. The data are coded and
entered, at each site, into a personal computer programmed for this
purpose. The software checks the data entries for consistency. The
data collections are linked nightly to a CPSC permanent central
database. Data about product-related injuries are available to CPSC
staff within 72 hours after the accident for most of the injuries
reported. These daily inputs are reviewed by CPSC staff for quality
and for identifying possible emerging hazards. The timeliness of the
data also allows staff to observe seasonal or episodic variations.
For example, during a 30-day period surrounding the Fourth of July in
1990, CPSC gathered extra data through NEISS for a special study on
injuries involving fireworks.
The unit of analysis for NEISS is the injured person. Other key
characteristics coded include the date of treatment, age and gender
of patient, injury diagnosis, body part affected, disposition of
case, product involved, and accident location. In addition,
important details about the injury and the injured person are
provided in NEISS. For example, the address and phone number of the
injured person are included, permitting follow-up investigations
about the nature and cause of the injury. There are about 900
product codes, ranging from abrasive cleaners to youth chairs, which
the coders use to specify the product involved.\43 Consumer products
are coded to allow for a great deal of specificity in the estimates.
For example, a "hand saw" would be differentiated from a "portable
circular power saw," and a "bicycle-mounted baby carrier" would be
specified differently from a "backpack baby carrier." However, NEISS'
coding system that describes the diagnosis of the primary injury and
the body part injured is not overly specific, and because it is
unique it cannot be directly compared with similar data from other
databases.
Despite the extent of valuable information provided by NEISS, this
system also has significant limitations. One important consideration
relates to the nature and size of the NEISS sample design. NEISS,
throughout its history, has been designed so that only national--not
state, local, or regional--estimates can be made. Thus, NEISS cannot
detect interregional or interstate differences, and may also be
limited in its ability to identify emerging product hazard patterns
that are focused in specific states or regions.
A limitation in the NEISS data for which CPSC has received criticism
is the lack of information that would assist in assessing
causality�that is, information that would establish (or provide a
starting point to establishing) whether the product in question
caused the accident or merely was involved in the accident. NEISS
contains neither "N" codes--which describe the nature of the
injury--nor "E" codes--which help explain how the injury happened. E
codes briefly describe the circumstances of the accident that
produced the injury. For example, E codes could help distinguish
among falls that occurred on stairs, from a ladder, or off a roof.
NEISS' short narrative section sometimes contains this type of
information, but NEISS does not generally provide much information on
the circumstances surrounding the product's involvement.
--------------------
\42 The hospitals are sampled from a universe in which each hospital
has at least six beds and provides 24-hour emergency service.
\43 Customarily excluded from NEISS are injury data involving
automobiles and motorcycles, trains, boats, planes, firearms, food,
illegal drugs, pesticides, cosmetics, medical devices, assaults and
suicide attempts, and occupational injuries.
OTHER SOURCES OF DATA ON
INJURIES
------------------------------------------------------- Appendix V:1.2
CPSC augments the NEISS injury reports with other sources of
anecdotal information. For example, CPSC maintains an Injury or
Potential Injury Incident (IPII) database of reports the agency
receives about injuries or incidents involving consumer products.
These reports come from a variety of sources, including news clips;
consumer complaints, including calls to CPSC's hot line; and public
reports of product liability suits. Although the IPII can provide
only anecdotal information, this database sometimes contains more
detailed information than can be found in NEISS. For example, IPII
records can contain specific information about the product
involved�such as the manufacturer and date of purchase�that are
generally not found in NEISS. The IPII also serves as a source for
cases to investigate. This is particularly important for hazards for
which NEISS provides relatively few cases.
CPSC has also purchased data on poisonings in the form of the
American Association of Poison Control Centers (AAPCC) database.
This database is composed of reports from approximately 65 poison
control centers throughout the country.\44 Reports from the AAPCC
database contain such information as the number of phone calls
received by participating centers concerning possible ingestion of a
product and the number of individuals who reported experiencing
symptoms related to the ingestion. But these reports contain little
other information�for example, they do not show how much of the
poison was ingested.
Although NEISS provides some information on fire-related injuries,
CPSC obtains additional data on fires from the National Fire
Protection Association (NFPA) and the U.S. Fire Administration.
NFPA, a private organization, conducts an annual survey of fire
departments that is designed to make statistically valid national
estimates of the total number of fires experienced nationally each
year. However, the NFPA survey does not collect much detailed
information about the characteristics of individual fires. To
augment the NFPA information, CPSC relies on the more detailed
information provided in the National Fire Reporting System (NFIRS),
which is compiled by the U.S. Fire Administration.\45 NFIRS data can
provide information on the ages of the victim or victims, and the
type of dwelling (apartment versus single-family home, for example).
However, these data are available only with a long lag; a CPSC
official we interviewed in October 1996 told us that the most recent
data he had available at that time were from 1993.
--------------------
\44 The number of poison control centers reporting cases to the AAPCC
database varied over the years, ranging from 47 centers in 1984 to 67
in 1996.
\45 Although NFIRS is a large dataset, it does not capture data
uniformly across states. CPSC generally assumes that NFIRS data
accurately depict the distribution of characteristics across fires,
and then applies the NFIRS proportions to the NFPA data to
approximate these characteristics on a national level.
CPSC OBTAINS MOST OF ITS
FATALITY INFORMATION FROM
SELECTED DEATH CERTIFICATES
--------------------------------------------------------- Appendix V:2
CPSC obtains the majority of its information on fatalities from
purchasing death certificates from the states and culling information
from them to determine which deaths were related to consumer
products. (Like the NEISS emergency room reports, the death
certificates establish only involvement--not causality--of a consumer
product.) CPSC does not purchase the complete annual set of death
certificates from each state; instead, the agency buys death
certificates according to selected E codes.\46 CPSC purchases about
8,000 death certificates annually; of these, approximately 50 percent
will be related to a consumer product. Over the years, in response
to declining budgets, CPSC has reduced even further the number of E
codes for which it purchases certificates. Death certificates
include the date, place, cause of death, age, gender, race, and
residence of the deceased.
Although death certificates cannot indicate whether the consumer
product was "at fault" in a death, they do provide some information
on the underlying circumstances. For example, the E code would
specify that a person was killed in an accident caused by electric
current. Of course, the quality of the data depends on the care with
which causation is determined and reported. It is likely that this
differs from locality to locality as well as among individual
doctors. However, there are some objective indications that the
quality of reporting causes of death has been improving; for example,
the proportion of cases that are categorized under ill-defined
conditions has been falling.\47 Although death certificates do not
constitute a sample of known probability or a complete count of
fatalities, and thus statistically reliable estimates cannot be made,
the geographic information contained in death certificates may help
to identify state and regional patterns as well as those that are
national in scope.
However, death certificate data also have substantial limitations.
First, there is an extensive lag--usually 2 years--before data become
available. Therefore, death certificates are not very useful in
timely identifying emerging hazards. Furthermore, a number of
factors contribute to obtaining rather sketchy causal information.
For example, details are quite limited, which inhibit determining the
degree to which a consumer product was involved in the death, let
alone whether the product was defective or hazardous. Certificates
are frequently completed without the benefit of autopsy information
to establish the precise cause of death, either because an autopsy
was not performed or because the certificate was filled out before
the autopsy took place. There is substantial variation in coding
practices and the level of detail available from state to state.\48
The information available about the injured individual is very
limited, and the number of cases for most categories is very small;
such data therefore limit the kinds and amount of analyses CPSC can
perform and the conclusions it can draw.
CPSC augments its death certificate data with other sources where
possible. For example, CPSC has instituted a Medical Examiners' and
Coroners' Alert Project (MECAP), to provide more timely fatality
information that lends itself to follow-up investigations. CPSC has
engaged some 100 coroners and medical examiners from across the
country to report potential product-related hazards. This project
produces about 2,000 reports annually, and CPSC staff credit the
coroners' reports with alerting them to a suffocation hazard
concerning infant cushions, which eventually led CPSC to recall
existing products and ban future production. In addition to the
MECAP data, CPSC also records some reports of fatalities in the IPII,
NEISS, and NFIRS databases.
--------------------
\46 According to agency officials, the major categories of E codes
the agency does not purchase involve falls (which infrequently
involve consumer products) and fires (for which the agency relies on
other sources). This means that fall-related deaths associated with
consumer products--such as falls from playground equipment--will be
undercounted. In 1995, CPSC conducted a pilot study in which it
purchased a complete set of E codes from one state to determine
whether it should change its mix of E codes. CPSC decided that the
set it was buying was still the best choice given the budget for
death certificate data.
\47 See National Center for Health Statistics, Advance Report of
Final Mortality Statistics for 1994, 45 (3) (Sept. 30, 1996), p.
74.
\48 For more information on this variation, see New England Journal
of Medicine, Vol. 313 (1985), pp. 1285-6, and U.S. Department of
Health and Human Services, Morbidity and Mortality Weekly Report,
Vol. 37 (1988), pp. 191-4.
CPSC INVESTIGATES SELECTED
INCIDENTS TO OBTAIN DETAILED
INFORMATION ABOUT THE CAUSES OF
DEATHS AND INJURIES
--------------------------------------------------------- Appendix V:3
In order to effectively target resources, identify hazard patterns,
and determine the appropriate remedy for particular product hazards,
CPSC needs detailed analyses of the causes of reported incidents.
CPSC's data sets generally do not provide such information. The
NEISS data do not include E codes (standardized, if brief and
incomplete, descriptions of incident circumstances), nor do they
include detailed information about how the incident happened. Some
information may be provided in the short, free-text comment area of
the NEISS report, but generally few such details are recorded. Death
certificates do include E codes, but product involvement is often
difficult to ascertain.
As a result, CPSC staff perform follow-up investigations on selected
cases to develop additional information about each incident. Some of
these investigations are conducted entirely by telephone, while
others are conducted at the accident site. These investigations may
include detailed interviews with victims and/or witnesses, police or
fire reports, photographs of the product and/or the accident site,
laboratory testing of the product involved, or re-creations of the
incidents. For example, in 1996 CPSC staff investigated an incident
in which a baby's leg was scratched as it was caught between the
slats of her crib. As part of the on-site investigation, the CPSC
investigator interviewed the child's mother, examined the child,
examined and photographed the crib, and interviewed staff at the
store where the crib was purchased.
The CPSC staff we interviewed told us that investigations,
particularly on-site investigations, were an important source of
information on established projects. The additional detail these
investigations gathered helps determine causality and identify hazard
patterns, leading analysts to the appropriate remedies. For example,
investigations revealed that very few bicycle accidents were related
to mechanical problems, and as a result, CPSC staff decided not to
recommend any changes to existing bicycle standards. In addition,
investigations may provide key evidence to help identify and correct
compliance problems. For example, the investigator who reviewed the
crib incident found that the crib in question appeared to violate
several mandatory safety standards.
CPSC USES MODELING TECHNIQUES
AND SPECIAL SURVEYS TO ESTIMATE
EXPOSURE INFORMATION
--------------------------------------------------------- Appendix V:4
In order to produce a numerical risk assessment, CPSC must have some
information on the extent to which individuals are exposed to a
particular product hazard. Exposure information can take many
different forms, and the best measure of exposure will depend on the
characteristics of the particular product hazard. For example, one
measure of exposure might be the number of products in use, while
another measure might be the number of hours a person spends using
the product, and another measure might take into account the
intensity with which a product is used. If a product is for one-time
use only and is usually used soon after it is sold (such as
fireworks), number of products sold might be a reasonable proxy for
exposure. However, when a product is used for a longer period or
often "handed down" to another user, such as a baby high chair, it
would probably be more reasonable to base estimates of exposure on
the number of products in use. In cases where the potential hazard
is especially ubiquitous (like air pollution, for example),
population measures (the number living near the source or the number
of children under 5, for example) may be reasonable. For some
hazards, it may be important to account for the intensity of use.
For example, the probability of developing cancer from exposure to a
wood stove may depend on how often the stove is used, how large a
space it heats, and how many times the stove door is opened to add
wood. Similarly, bicycles could be used very frequently and very
intensely (every day in urban traffic) or infrequently and not
intensely (once a season on the bike path in the park); thus, a good
exposure measure would take such factors into account.
CPSC does not conduct a formal, numerical risk assessment for each
project it undertakes. Of the 115 CPSC projects we reviewed, only 24
included a numerical assessment of risk. CPSC most frequently relied
on estimates of products in use to provide the exposure information
for its risk assessments. In 65 percent of the cases where an
epidemiological risk assessment was performed, CPSC based its
exposure measure on an estimate of the number of products in use.\49
In an additional 5 percent, CPSC obtained information on the actual
number of products in use. In 30 percent of cases, CPSC used a
population-based measure, and in 10 percent of cases, CPSC used a
sales measure.\50 We did not evaluate whether the exposure measure
CPSC chose was appropriate for each case.
CPSC conducted special surveys on bicycles and on cigarette lighters
and matches, for instance, to develop exposure information. The
survey questions were designed to obtain information on the number of
products in the home, the intensity of use, the characteristics of
the households using the product, and the usual patterns of use. For
example, the bicycle survey included questions on the number of hours
spent biking, the ages, education, and income of household members,
whether riding was done most often on streets, sidewalks, or bike
paths, and whether riders used helmets.
Where special surveys are not practical--because of time, resource,
or other limitations--CPSC sometimes uses mathematical modeling
techniques to estimate the number of products in use. These models
can take sales information and information on the life of the product
to estimate the number of units in use. For example, CPSC used such
a model to estimate how many of the portable heaters made before a
1991 revision to a voluntary standard were still in use.
--------------------
\49 About half of these estimates of products in use were derived
from modeling and about half from special surveys.
\50 Percentages do not total 100 because in two cases CPSC used more
than one measure of exposure.
CPSC OBTAINS SOME INFORMATION
UNDER REPORTING REQUIREMENTS
FOR MANUFACTURERS AND OTHER
BUSINESSES
--------------------------------------------------------- Appendix V:5
CPSC may also receive information about potential product hazards
through industry reporting requirements. Although this information
is used mostly for identifying and addressing compliance problems, it
may also help identify new hazards. Companies are legally obligated
to report to CPSC information they receive that indicates a consumer
product they distribute is potentially hazardous. Under section 15
of the Consumer Product Safety Act, manufacturers (including
importers), distributors, and retailers of consumer products must
notify CPSC if they obtain information that a product (1) fails to
comply with a consumer product safety regulation or a voluntary
consumer product safety standard, (2) contains a defect that could
create a substantial product hazard, or (3) creates an unreasonable
risk of serious injury or death.
In addition to these reporting requirements, manufacturers and
importers of a consumer product must report to CPSC if (1) a
particular model of a consumer product is the subject of at least
three civil actions that have been filed in federal or state court,
(2) each suit alleges the involvement of that model in death or
grievous bodily injury, and (3) within a specified 2-year period at
least three of the actions resulted in a final settlement involving
the manufacturer or importer or in a judgment for the plaintiff.
Manufacturers must file a report within 30 days after the settlement
or judgment in the third such civil action.
CPSC may receive requests for information that firms have reported to
it under section 15 requirements. The law limits CPSC's disclosure
of any information identifying a manufacturer and further limits
CPSC's release of information that firms have provided under these
requirements. Reports on civil lawsuits may not be publicly
disclosed by CPSC or subpoenaed or otherwise obtained from CPSC
through discovery in any civil action or administrative procedure.
(See figure in printed edition.)Appendix VI
COMMENTS FROM THE CHAIRMAN AND
COMMISSIONER MOORE, CONSUMER
PRODUCT SAFETY COMMISSION, AND
GAO'S EVALUATION
=========================================================== Appendix V
(See figure in printed edition.)
(See figure in printed edition.)
(See figure in printed edition.)
(See figure in printed edition.)
(See figure in printed edition.)
(See figure in printed edition.)
(See figure in printed edition.)
(See figure in printed edition.)
(See figure in printed edition.)
(See figure in printed edition.)
(See figure in printed edition.)
(See figure in printed edition.)
(See figure in printed edition.)
(See figure in printed edition.)
(See figure in printed edition.)
(See figure in printed edition.)
The following are GAO's comments on Chairman Brown and Commissioner
Moore's letter dated July 24, 1997.
GAO COMMENTS
--------------------------------------------------------- Appendix V:6
1. As we discuss in the report, CPSC relies on a number of different
sources for injury incident data. (Descriptive information on all
CPSC's data sources is provided in app. V.) However, only one
source--the NEISS system--is capable of providing statistically
reliable, representative information, and the NEISS system covers
only those injuries treated in hospital emergency rooms. Although
CPSC obtains information from other sources, these data are anecdotal
and thus their usefulness is very limited in estimating injury
prevalence. As a result, we have recommended that CPSC consult with
experts both inside and outside the agency to prioritize its
additional data needs and to explore the feasibility of options for
obtaining these data. Changes over the past decade in the health
care market, including the growth of ambulatory care, changes in
reimbursement procedures, and improved health services research have
the potential to make such additional data collection more feasible
and less costly than it was a decade ago when some of these
assessments were last made.
2. Figure 3 correctly states that CPSC does not obtain systematic
surveillance data on injuries treated outside the emergency room. In
the text of the report, we state that the number of injuries treated
in each setting is unknown. We added a similar statement to the
figure to emphasize this point.
3. In several places in their comments, Chairman Brown and
Commissioner Moore refer to CPSC's investigations of selected
incidents to obtain more information. We discussed these
investigations in our report. We believe that these investigations
provide valuable information on causation, characteristics of
accident victims, and hazard patterns, and we agree that some
information gleaned from investigations is not obtainable from
surveillance data. However, according to CPSC staff, this
information is available to the agency only after a project is well
under way, not at the initial stage of project selection. As a
result, CPSC has little information on these important factors to
assist in project selection, and evaluation of these criteria at the
project selection stage is thus unavoidably speculative.
4. Descriptive information on this source of information was
provided in appendix V.
5. For reasons we discussed in the report, relying on aggregate data
(data not specific to a particular consumer product) to address the
limitations of CPSC's surveillance data is problematic. In addition,
in the briefing packages we reviewed, the estimates of injury
incidents usually referred only to NEISS cases and were not extended
with modeling techniques; because these techniques were used so
infrequently for this purpose, we did not assess their application in
this report.
6. In referring to the agency's monitoring of other aggregated
sources of injury data, Chairman Brown and Commissioner Moore state
that "Tracking of this type is sufficient to assure the adequacy of
our data, as we use it." We disagree. Examining trend information
from other sources (without a rigorous application to CPSC's own
specific needs) is not sufficient to compensate for (or even measure
the magnitude of) the limitations of CPSC's injury surveillance data.
7. We believe this comment may reflect a misunderstanding of our
point. We do not mean to suggest that CPSC incorrectly relies on
NEISS to provide information on chronic illnesses. Rather, we are
pointing out that CPSC has virtually no systematic data on chronic
illnesses. As we stated in the report, on page 20, we agree with
Chairman Brown and Commissioner Moore (and with the CPSC staff we
interviewed) that such data are often difficult to obtain. However,
chronic illness is listed as a criterion for CPSC project selection,
and CPSC has little information to assist in applying this criterion.
Accordingly, we are pleased with the statement by Brown and Moore
that CPSC will consider additional surveillance methods to obtain
more information on chronic illnesses.
8. We acknowledge in the report that CPSC uses other sources to
supplement its death certificate data. However, our interviews with
CPSC staff and our review of agency documents confirmed that death
certificates are the most important source of death data for CPSC.
In the briefing packages we reviewed, 80 percent of the calculations
for numbers of deaths were based in whole or in large part on death
certificate data, and death certificates were the sole source of
death information in 36 percent of all CPSC briefing packages (a far
greater percentage than for any other single source). Death data
were usually reported in CPSC briefing packages with a lag of 2 or
more years, especially when death certificates were the sole source
of data.
9. We agree with Chairman Brown and Commissioner Moore that there is
no single mold or perfect set of criteria for evaluating a
cost-benefit analysis, and we have revised the report to further
emphasize this point. We believe that Chairman Brown and
Commissioner Moore may have misunderstood the purpose of our
evaluation questions. We did not "derive a particular methodology"
for cost-benefit analysis, nor do we mean to suggest that CPSC should
follow some "formula" for conducting analyses that does not leave
room for competent professional judgment. However, to say that there
is no perfect "formula" for cost-benefit analysis does not imply that
all methodological choices are equally consistent with rigorous and
comprehensive professional work. Although no litmus test exists for
a "good" analysis, the professional literature offers some basic,
minimum elements that are commonly used in evaluating cost-benefit
analyses. These elements, which are based on the principles of
transparency and completeness, are generally considered necessary,
although not sufficient, for a good analysis. Although flexibility
may be sometimes necessary in the assumptions or models underlying a
cost-
benefit analysis, the elements we used--including full disclosure of
data limitations, sensitivity analysis, and incorporating all
important costs and benefits--are appropriate to a wide range of
situations.
10. Chairman Brown and Commissioner Moore are correct in stating
that we evaluated 29 CPSC cost-benefit analyses that were completed
between January 1, 1990, and September 30, 1996. We identified these
29 analyses as complete on the basis of CPSC's
statements--specifically, we considered a cost-benefit analysis to be
complete only if an explicit comparison was made between aggregate
costs and benefits. In addition, although the analyses may have been
prepared at different stages of the project, we based our review on
all available documentation on the project, and we reported results
only for the evaluation questions that applied to the majority of
cases and for which a clear determination could be made. We did not
report results separately for the eight regulatory analyses that were
required by law, because there were relatively few of these.
However, we found no substantial differences between these 8 and the
remaining 21 in terms of how they performed, compared with commonly
used elements of evaluation of cost-benefit analysis. Therefore, we
are confident that our results present an accurate assessment of
CPSC's cost-benefit analyses, and we recommend that the agency
implement changes to ensure that its analyses are comprehensive and
reported in sufficient detail.
11. We believe that, as now constructed, CPSC's method for tracking
projects operates at too high a level of generality and provides too
little information to give a comprehensive, accurate picture of the
agency's activities either at a given point or over a longer period.
CPSC staff told us, and our review of agency documentation confirmed,
that the Management Information System (MIS) usually tracks most
agency activities only at a very general level. For example, CPSC's
1996 year-end MIS report lists some specific projects such as
"upholstered furniture" and "range fires," but most projects are
accounted for under either broad umbrella codes such as "sports and
recreation" or "children's projects," or under activity codes such as
"investigations," "product safety assessment," or "emerging
problems." In addition, CPSC staff told us that reliable inferences
on resources spent cannot be drawn from MIS data because of
limitations in the computer system and because no consistent rule
exists about how staff time in different directorates is recorded to
project codes. As a result, CPSC staff were unable to generate a
comprehensive list of projects or to provide accurate information
about resources allocated to those projects. We recommend an
improved tracking system that would provide enough information to
monitor the projects selected and resources spent for each specific
consumer product hazard. We believe that as CPSC develops its
planned accounting system, it should attempt to make it as compatible
as practicable with the recommended tracking system. Nevertheless,
whether or not it implements its planned accounting system, we
believe that CPSC can and should improve its ability to track
projects.
12. Our interviews with present and former commissioners revealed a
pattern by which certain of CPSC's regulatory criteria have
historically been given greater emphasis in CPSC's project selection
process. Our objective was to describe the process as it was related
to us; we have not taken a position on whether this process is
appropriate. We have added a statement to our methodology section to
emphasize this point.
(See figure in printed edition.)Appendix VII
COMMENTS FROM COMMISSIONER GALL,
CONSUMER PRODUCT SAFETY
COMMISSION, AND GAO'S EVALUATION
=========================================================== Appendix V
(See figure in printed edition.)
(See figure in printed edition.)
(See figure in printed edition.)
The following are GAO's comments on Commissioner Gall's letter dated
July 23, 1997.
GAO COMMENTS
--------------------------------------------------------- Appendix V:7
1. Our interviews with present and former commissioners revealed a
pattern by which certain of CPSC's regulatory criteria have
historically been given greater emphasis in CPSC's project selection
process. Our objective was to describe the process as it was related
to us; we have not taken a position on whether this process is
appropriate. We have added a statement to our methodology section to
emphasize this point.
2. Commissioner Gall states that we did not analyze the relative
importance of the deficiencies we found in CPSC's data and
methodology. However, as we stated in the report, available
information does not permit us to determine the impact of
better-quality data on the decisions CPSC made. The limitations we
found in CPSC's data have a variety of potentially conflicting
impacts, precluding us from determining exactly how the results of
the analysis might change if improved data were available. For
example, because CPSC's injury estimates are often confined to
injuries treated in hospital emergency rooms, CPSC's estimates will
generally understate the actual number of injuries associated with a
consumer product. However, CPSC's systematic injury and death data
can generally tell only whether a product was involved in an
accident--not whether the product caused or contributed to the
accident. As a result, this can make the risks assessed by CPSC
appear larger than they might actually be. Similarly, we cannot
determine how improved exposure data would change the relative
importance of the risks assessed by CPSC. We agree with Commissioner
Gall that not all projects will merit the same level of data or
analysis. However, our review of CPSC raises questions about the
agency's ability to obtain and analyze data necessary to support
rigorous analysis of important agency projects.
3. We agree with Commissioner Gall that resource considerations
should enter into CPSC's decisions to undertake new data collection.
For this reason, we recommended an overall feasibility study for CPSC
to prioritize among its data needs and investigate new options for
obtaining additional information.
GAO CONTACTS AND STAFF
ACKNOWLEDGMENTS
======================================================== Appendix VIII
GAO CONTACTS
Lawrence J. Horinko, Assistant Director, (202) 512-7001
Sarah L. Glavin, Project Manager, (202) 512-7180
STAFF ACKNOWLEDGMENTS
In addition to those named above, the following individuals made
important contributions to this report: Sheila A. Nicholson,
Analyst, gathered and analyzed data on CPSC's information release
procedures; Nancy K. Kintner-Meyer, Senior Evaluator, compiled and
analyzed information on CPSC projects; George Bogart, Senior
Attorney, provided legal assistance; Harold Wallach, Senior Analyst,
assisted in the analysis of CPSC's data systems; and Charles Jeszeck
served as Assistant Director for the project in its early stages.
BIBLIOGRAPHY
============================================================ Chapter 0
Abkowitz, M., and P. Der-Ming Cheng. "Developing a Risk/Cost
Framework for Routing Truck Movements of Hazardous Materials."
Accident Analysis and Prevention, 20 (1988), pp. 39-51.
Abraham, John. "Negotiation and Accommodation in Expert Medical Risk
Assessment and Regulation: An Institutional Analysis of the
Benoxaprofen Case." Policy Sciences, 27 (1994), pp. 53-76.
Adams, John. "The Emperor's Old Clothes: The Curious Comeback of
Cost-Benefit Analysis." Environmental Values, 2 (1993), pp. 247-60.
Adler, Robert S. "From 'Model Agency' to Basket Case--Can the
Consumer Product Safety Commission Be Redeemed?" Administrative Law
Review, 42 (Winter 1989), pp. 61-129.
Alberini, Anna. "Testing Willingness-to-Pay Models of Discrete
Choice Contingent Valuation Survey Data." Land Economics, 71 (1)
(Feb. 1995), pp. 83-95.
Allen, Julius. Cost Benefit Analysis of Federal Regulation: A
Review and Analysis of Developments, 1978-1984. Report No. 84-74 E.
Washington, D.C.: Congressional Research Service, May 15, 1984.
Allison, Paul D. Event History Analysis: Regression for
Longitudinal Event Data, Sage University Paper Series: Quantitative
Applications in the Social Sciences, series no. 07-046. Newbury
Park, Calif.: Sage Publications, 1984.
American Academy of Pediatrics. "Hospital Discharge Data on Injury:
The Need for E Codes." Policy Statement in AAP News, Mar. 1992.
Arrow, Kenneth J., and others. Benefit-Cost Analysis in
Environmental, Health, and Safety Regulation: A Statement of
Principles. American Enterprise Institute, The Annapolis Center, and
Resources for the Future. Washington, D.C.: AEI Press, 1996.
Asch, Peter. Consumer Safety Regulation: Putting a Price on Life
and Limb. New York: Oxford University Press, 1988.
Ault, Eric B. "CPSC's Voluntary Standards: An Assessment and a
Paradox." The Frontier of Research in the Consumer Interest. Eds.
E. Scott Maynes and ACCI Research Committee. Columbia, Mo.:
American Council on Consumer Interests, 1986, pp. 77-81.
Bailey, Martin J. Reducing Risks to Life: Measurement of the
Benefits. Washington, D.C.: American Enterprise Institute, 1980.
Baker, S.P., and others. The Injury Fact Book, 2nd ed. New York:
Oxford University Press, 1992.
Battiato, Salvatore Enrico. "Cost-Benefit Analysis and the Theory of
Resource Allocation." Alan Williams and Emilio Giardina. Efficiency
in the Public Sector. Aldershot, Hants, England: Edward Elgar
Publishing, 1993, pp. 26-42.
Berlage, L., and R. Renard. "The Discount Rate in Cost-Benefit
Analysis and the Choice of a Numeraire." Oxford Economic Papers, 37
(1985), pp. 691-9.
Berlau, John. "Play (Regulated) Ball!" Reason, Dec. 1996, pp.
71-3.
Bhat, M., and S. Li. "Consumer-Product-Related Tooth Injuries
Treated in Hospital Emergency Rooms: United States, 1979-1987."
Community Dentistry and Oral Epidemiology, 18 (3) (June 1990), pp.
133-8.
Bird, Peter J.W.N. "A Note on Relative Price Changes in Cost-Benefit
Analysis." Public Finance, 35 (2) (1980), pp. 318-23.
Blackorby, Charles, and David Donaldson. "A Review Article: The
Case Against the Use of the Sum of Compensating Variations in
Cost-Benefit Analysis." Canadian Journal of Economics, 23 (3) (Aug.
1990), pp. 471-94.
_____. "Welfare Ratios and Distributionally Sensitive Cost-Benefit
Analysis." Journal of Public Economics, 34 (1987), pp. 265-90.
Boardman, Anthony, Aidan Vining, and W.G. Waters II. "Costs and
Benefits Through Bureaucratic Lenses: Example of a Highway Project."
Journal of Policy Analysis and Management, 12 (3) (1993), pp.
532-55.
Bordley, Robert F. "Making Social Trade-offs Among Lives,
Disabilities, and Costs." Journal of Risk and Uncertainty, 9 (1994),
pp. 135-49.
Boustead, Thomas. The Law and Economics of Administrative Law: A
Statistical Analysis of the Consumer Product Safety Commission's
Petition Process. Ph.D. Dissertation, Fordham University, May 1995.
Boyle, M.H., and others. "Economic Evaluation of Neonatal Intensive
Care of Very Low Birthweight Infants." New England Journal of
Medicine, 308 (1983), pp. 1330-37.
Bradfield, R.E. "Social Cost-Benefit Analysis in Project Evaluation:
A Practical Framework for Decision Making." Journal of Studies in
Economics and Econometrics, 13 (2) (1989), pp. 25-35.
Brent, Robert J. "The Cost-Benefit Analysis of Government Loans."
Public Finance Quarterly, 19 (1) (Jan. 1991), pp. 43-66.
Breyer, Stephen. Breaking the Vicious Circle. Cambridge, Mass.:
Harvard University Press, 1993.
Burt, Catharine E. "Injury-Related Visits to Hospital Emergency
Departments: United States, 1992." Advance Data, No. 261 (Feb. 1,
1995).
Buzby, Jean C., Richard C. Ready, and Jerry R. Skees. "Contingent
Valuation in Food Policy Analysis: A Case Study of a
Pesticide-Residue Risk Reduction." Journal of Agricultural and
Applied Economics, 27 (2) (Dec. 1995), pp. 613-25.
Campen, James T. Benefit, Cost, and Beyond: The Political Economy
of Benefit-Cost Analysis. Cambridge, Mass.: Ballinger Publishing
Company, 1986.
Cesario, Frank J. "Benefit-Cost Analysis Under Pricing Constraints."
Applied Economics, 13 (1981), pp. 215-24.
Chan, Yuk-Shee, and Anthony M. Marino. "Regulation of Product
Safety Characteristics Under Imperfect Observability." Journal of
Regulatory Economics, 6 (1994), pp. 177-95.
Chirinko, Robert S., and Edward P. Harper, Jr. "Buckle Up or Slow
Down? New Estimates of Offsetting Behavior and Their Implications
for Automobile Safety Regulation." Journal of Policy Analysis and
Management, 12 (2) (1993), pp. 270-96.
Cost-Benefit Analysis: Wonder Tool or Mirage? Report by the U.S.
House of Representatives Subcommittee on Oversight and Investigation,
Committee on Interstate and Foreign Commerce, Dec. 1980.
Cowen, Tyler. "The Scope and Limits of Preference Sovereignty."
Economics and Philosophy, 9 (1993), pp. 253-69.
Cox, Louis Anthony (Jr.). "Theory of Regulatory Benefits Assessment:
Econometric and Expressed Preference Approaches." Benefits
Assessment: The State of the Art. Eds. Judith D. Bentkover,
Vincent T. Covello, and Jeryl Mumpower. Dordrecht, Holland: D.
Reidel Publishing Company, 1986, pp. 85-160.
Crandall, Robert W. "The Use of Cost-Benefit Analysis in Product
Safety Regulation." The Frontier of Research in the Consumer
Interest. Eds. E. Scott Maynes and ACCI Research Committee.
Columbia, Mo.: American Council on Consumer Interests, 1986, pp.
61-75.
Crews, Clyde Wayne. "Ten Thousand Commandments: Regulatory Trends
1981-92 and the Prospect for Reform." Journal of Regulation and
Social Cost, Vol. 2 (Mar. 1993), pp. 105-50.
Cummings, Ronald G., Louis Anthony Cox, Jr., and A. Myrick Freeman
III. "General Methods for Risk Assessment." Benefits Assessment:
The State of the Art. Eds. Judith D. Bentkover, Vincent T.
Covello, and Jeryl Mumpower. Dordrecht, Holland: D. Reidel
Publishing Company, 1986, pp. 161-92.
Curtin, Leo, and Ronald Krystynak. "An Economic Framework for
Assessing Foodborne Disease Control Strategies With an Application to
Salmonella Control in Poultry." The Economics of Food Safety. Ed.
Julie A. Caswell. London: Elsevier, 1991, pp. 131-51.
Dasguptha, A.K., and D.W. Pearce. Cost-Benefit Analysis: Theory
and Practice. London: Macmillan, 1972.
Davis, Miriam. Health Risk Assessment Research at the Department of
Energy: Background Information. Washington, D.C.: U.S. Office of
Technology Assessment, Sept. 15, 1992.
Davis, Yvette, and others. "An Evaluation of the National Electronic
Injury Surveillance System for Use in Monitoring Nonfatal Firearm
Injuries and Obtaining National Estimates." Journal of Safety
Research, 27 (2) (Summer 1996), pp. 83-91.
Dehar, Mary Anne, Sally Casswell, and Paul Duignan. "Formative and
Process Evaluation of Health Promotion and Disease Prevention
Programs." Evaluation Review, 17 (2) (Apr. 1993), pp. 204-20.
Drummond, Michael F. "Cost Benefit Analysis in Health and Health
Care: Fine in Practice, but Does It Work in Theory?" Alan Williams
and Emilio Giardina. Efficiency in the Public Sector. Aldershot,
Hants, England: Edward Elgar Publishing, 1993, pp. 106-28.
Drummond, Michael F., Greg L. Stoddart, and George W. Torrance.
Methods for the Economic Evaluation of Health Care Programmes. New
York: Oxford University Press, 1996.
Drummond, Michael F., and others. "Users' Guide to the Medical
Literature: How to Use an Article on Economic Analysis of Clinical
Practice." Journal of the American Medical Association, 277 (19) (May
21, 1997), pp. 1552-57.
Evans, J.S., N.S. Hawkins, and J.D. Graham. "Uncertainty Analysis
and the Value of Information: Monitoring for Radon in the Home."
Journal of Air Pollution Control Association, 38 (1988), pp.
1380-85.
Evans, L., and M.C. Frick. "Helmet Effectiveness in Preventing
Motorcycle Driver and Passenger Fatalities." Accident Analysis and
Prevention, 20 (1988), pp. 447-58.
Fingar, Ann R., Richard S. Hopkins, and Marjorie Nelson.
"Work-Related Injuries in Athens County, 1982 to 1986: A Comparison
of Emergency Department and Workers' Compensation Data." Journal of
Occupational Medicine, 34 (8) (Aug. 1992), pp. 779-87.
Finkel, Adam M. Comparing Risks Thoughtfully. Washington, D.C.:
U.S. Office of Technology Assessment, Sept. 1994.
_____. "Who's Exaggerating?" Discover, May 1996, pp. 45-54.
Fischhoff, Baruch. Ranking Risks. Washington, D.C.: U.S. Office
of Technology Assessment, Feb. 1994.
_____, and Louis Anthony Cox, Jr. "Conceptual Framework for
Regulatory Benefits Assessment." Benefits Assessment: The State of
the Art. Eds. Judith D. Bentkover, Vincent T. Covello, and Jeryl
Mumpower. Dordrecht, Holland: D. Reidel Publishing Company, 1986,
pp. 51-84.
Fise, Mary Ellen. "Consumer Product Safety Regulation," Chapter 11
in Regulation and Consumer Protection. Dame Publications, Inc.,
1995, pp. 277-94.
_____. The CPSC: Guiding or Hiding From Product Safety?
Washington, D.C.: Consumer Federation of America, May 1987.
_____, and M. Kristen Rand. Pennies for Consumer Protection? A
Report on the CPSC's 1991 Budget. Washington, D.C.: Consumer
Federation of America, May 1990.
Fisher, Ann, Maria Pavlova, and Vincent Covello, eds. Evaluation and
Effective Risk Communication Workshop Proceedings. Washington, D.C.:
Interagency Task Force on Environmental Cancer and Heart and Lung
Disease, Committee on Public Education and Communication, Jan. 1991.
Fisher, Ann, Lauraine G. Chestnut, and Daniel M. Violette. "The
Value of Reducing Risks of Death: A Note on New Evidence." Journal
of Policy Analysis and Management, 8 (1) (1989), pp. 88-100.
Fuchs, Edward Paul, and James Anderson. "Institutionalizing
Cost-Benefit Analysis in Regulatory Agencies." Research in Public
Policy Analysis and Management, 4 (1987), pp. 187-211.
Fung, V.A., J.C. Barrett, and J. Huff. "Reviews and Commentaries:
The Carcinogenesis Bioassay in Perspective: Application in
Identifying Human Cancer Hazards." Environmental Health Perspectives,
103 (1995), pp. 7-8.
Gerner, Jennifer L. "Product Safety: A Review." The Frontier of
Research in the Consumer Interest, eds. E. Scott Maynes and ACCI
Research Committee. Columbia, Mo.: American Council on Consumer
Interests, 1986, pp. 37-59.
Gilroy, John Martin, and Maurice Wade, eds. The Moral Dimensions of
Public Policy Choice: Beyond the Market Paradigm. Pittsburgh, Pa.:
University of Pittsburgh Press, 1992.
Gold, Marthe R., and others. Cost-Effectiveness in Health and
Medicine. New York: Oxford University Press, 1996.
Goldenhar, Linda, and Paul A. Schulte. "Intervention Research in
Occupational Health and Safety." Journal of Occupational Medicine, 36
(7), pp. 763-75.
Goldstein, B. "Risk Management Will Not Be Improved by Mandating
Numerical Uncertainty Analysis for Risk Assessment." University of
Cincinnati Law Review, Vol. 63, pp. 1599-1610.
Goodstein, Eban. "Benefit-Cost Analysis at the EPA." Journal of
Socio-Economics, 24 (2) (1995), pp. 375-89.
Graham, John D. "Making Sense of Risk: An Agenda for Congress."
Risks, Costs and Lives Saved: Getting Better Results from
Regulation. Ed. Robert W. Hahn. Washington, D.C.: AEI Press,
1996, pp. 183-207.
_____. "Technology, Behavior, and Safety: An Empirical Study of
Automobile Occupant-Protection Regulation." Policy Sciences, 17
(1984), pp. 141-51.
_____, and Younghee Lee. "Behavioral Response to Safety Regulation:
The Case of Motorcycle Helmet-Wearing Legislation." Policy Sciences,
19 (1986), pp. 253-73.
Graves, Edmund J., and Brenda S. Gillum. "1994 Summary: National
Hospital Discharge Survey." Advance Data, No. 278 (Oct. 3, 1996).
Gray, George M. The Challenge of Risk Characterization. Washington,
D.C.: U.S. Office of Technology Assessment, Nov. 1992.
Greenberg, David H. "Conceptual Issues in Cost/Benefit Analysis of
Welfare-to-Work Programs." Contemporary Policy Issues, 10 (Oct.
1992), pp. 51-64.
Gregory, Robin, Thomas C. Brown, and Jack L. Knetsch. "Valuing
Risks to the Environment." The Annals of the American Academy of
Political and Social Science. Eds. Howard Kunreuther and Paul
Slovic. Vol. 545 (May 1996), pp. 54-63.
Griffin, Ronald. "On the Meaning of Economic Efficiency in Policy
Analysis." Land Economics, 71 (1) (Feb. 1995), pp. 1-15.
Guyer, Bernard, and Susan S. Gallagher. "An Approach to the
Epidemiology of Childhood Injuries." Pediatric Clinics of North
America, 32 (1) (Feb. 1985), pp. 5-15.
Haas, C.N., and others. "Risk Assessment of Viruses in Drinking
Water." Risk Analysis, Vol. 13, pp. 545-52.
Hadden, Susan G. Read the Label: Reducing Risk by Providing
Information. Boulder, Co.: Westview Press, 1986.
Haddix, A.C., and others, eds. Prevention Effectiveness: A Guide to
Decision Analysis and Economic Evaluation. New York: Oxford
University Press, 1996.
Hahn, Robert W. "Improving Regulation: Steps Toward Reform."
Statement before the U.S. Senate Subcommittee on Financial
Management and Accountability, Committee on Govermental Affairs,
Sept. 25, 1996.
_____. "A Preliminary Estimate of Some Indirect Costs of
Environmental Regulation." Unpublished paper, Feb. 1995.
_____. "Regulatory Reform: A Legislative Agenda." Statement before
the U.S. Senate Committee on Governmental Affairs, Feb. 8, 1995.
_____. "Regulatory Reform: What Do the Government's Numbers Tell
Us?" Risks, Costs and Lives Saved: Getting Better Results From
Regulation. Ed. Robert W. Hahn. Washington, D.C.: AEI Press,
1996, pp. 208-54.
_____. "Why We Need to Balance the Costs and Benefits of
Regulation." Statement before the U.S. House of Representatives,
Subcommittee on Oversight and Investigations, Committee on Economic
and Educational Opportunities, and the Subcommittee on Paperwork and
Regulation, Committee on Small Business, Feb. 2, 1995.
Hall, Margaret Jean, and Maria F. Owings. "Hospitalizations for
Injury and Poisoning in the United States, 1991." Advance Data, No.
252 (Oct. 7, 1994).
Hammitt, J.K., and J.A.K. Cave. Research Planning for Food Safety:
A Value of Information Approach. RAND R-3946-ASPE/NCTR. Santa
Monica, Calif.: RAND, 1991.
Hanke, Steve. "On the Feasibility of Benefit-Cost Analysis." Public
Policy, 29 (2) (Spring 1981), pp. 147-57.
Haskins, Jack B. "Evaluative Research on the Effects of Mass
Communication Safety Campaigns: A Methodological Critique." Journal
of Safety Research, 2 (2) (June 1970), pp. 86-8.
Haupt, Barbara. "1982 Summary: National Hospital Discharge Survey."
Advance Data, No. 95 (Dec. 27, 1983).
Havrilesky, Thomas. "The Persistent Misapplication of the Hedonic
Damages Concept to Wrongful Death and Personal Injury Litigation."
Journal of Forensic Economics, 8 (1) (1995), pp. 49-54.
Heimann, Christopher M., and others. "Project: The Impact of
Cost-Benefit Analysis on Federal Administrative Law." Administrative
Law Review, 42 (Fall 1990), pp. 545-654.
Henderson, J. Vernon. "Peak Shifting and Cost-Benefit
Miscalculations." Regional Science and Urban Economics, 22 (1992),
pp. 103-121.
Hensler, Deborah R., and others. Compensation for Accidental
Injuries in the United States. Santa Monica, Calif.: RAND, The
Institute for Civil Justice, 1991.
Hildred, William, and Fred Beauvais. "An Instrumentalist Critique of
`Cost-Utility Analysis.'" Journal of Economic Issues, 29 (4) (Dec.
1995), pp. 1083-96.
Hoehn, John P., and Alan Randall. "A Satisfactory Benefit Cost
Indicator From Contingent Valuation." Journal of Environmental
Economics and Management, 14 (1987), pp. 226-47.
_____. "Too Many Proposals Pass the Benefit-Cost Test." American
Economic Review, 79 (3) (June 1989), pp. 544-51.
_____. "Too Many Proposals Pass the Benefit-Cost Test: Reply."
American Economic Review, 81 (5) (Dec. 1991), pp. 1450-2.
Hoffer, George E., Stephen W. Pruitt, and Robert J. Reilly. "The
Impact of Product Recalls on the Wealth of Sellers: A
Reexamination." Journal of Political Economy, 96 (3) (1988).
Hopkins, Thomas D. "The Costs of Federal Regulation." Journal of
Regulation and Social Costs, Mar. 1992, pp. 5-31.
Horowitz, John K., and Richard T. Carson. "Discounting Statistical
Lives." Journal of Risk and Uncertainty, 3 (1990), pp. 403-13.
Hubin, Donald C. "The Moral Justification of Benefit-Cost Analysis."
Economics and Philosophy, 10 (1994), pp. 169-94.
The Institute of Medicine and the National Research Council,
Commission on Life Sciences, Committee on Trauma Research. Injury in
America: A Continuing Public Health Problem. Washington, D.C.:
National Academy Press, 1985.
Jarrell, Gregg A., and Sam Peltzman. "The Impact of Product Recalls
on the Wealth of Sellers." George J. Stigler Chicago Studies in
Political Economy. Chicago, Ill.: Chicago Press, 1988, pp. 612-34.
Johansson, Per-Olov. "Altruism in Cost-Benefit Analysis."
Environmental and Resource Economics, 2 (1992), pp. 605-13.
Joksch, H. "Critique of Peltzman's Study: The Effects of Automobile
Safety Regulation." Accident Analysis and Prevention, 8 (1976), pp.
213-14.
Keeney, Ralph L. "The Role of Values in Risk Management." The Annals
of the American Academy of Political and Social Science. Eds.
Howard Kunreuther and Paul Slovic. Vol. 545 (May 1996).
Philadelphia: Sage Periodicals Press, pp. 126-34.
Kemper, Peter, David A. Long, and Craig Thornton. "A Benefit-Cost
Analysis of the Supported Work Experiment." The National Supported
Work Demonstration. Eds. Robinson G. Hollister, Peter Kemper, and
Rebecca A. Maynard, Madison, Wis.: University of Wisconsin Press,
1984, pp. 239-85.
Kerton, Robert, and Richard Bodell. "Quality, Choice and the
Economics of Concealment: The Marketing of Lemons." Journal of
Consumer Affairs, 29 (1) (Summer 1995), pp. 1-28.
Kopp, Raymond J. "Why Existence Value SHOULD Be Used in Cost-Benefit
Analysis." Journal of Policy Analysis and Management, 11 (1) (1992),
pp. 123-30.
Langlois, Jean A., and others. "Improving the E Coding of
Hospitalizations for Injury: Do Hospital Records Contain Adequate
Documentation?" American Journal of Public Health, 85 (9) (Sept.
1995), pp. 1261-65.
Laughery, Kenneth R., David R. Lovvoll, and Meredith L. McQuilkin.
"Allocation of Responsibility for Child Safety." Proceedings of the
Human Factors and Ergonomics Society 40th Annual Meeting. Santa
Monica, Calif.: Human Factors and Ergonomics Society, 1996, pp.
810-13.
Lave, Lester B. "Benefit Cost Analysis: Do the Benefits Exceed the
Costs?" Risks, Costs and Lives Saved: Getting Better Results From
Regulation. Ed. Robert W. Hahn. Washington, D.C.: AEI Press,
1996, pp. 104-34.
Layard, Richard. Cost-Benefit Analysis. Middlesex, England:
Penguin Books, 1972.
Lescohier, Ilana, Susan S. Gallagher, and Bernard Guyer. "Not by
Accident." Issues in Science and Technology, Summer 1990, pp. 35-42.
Lichtenberg, Erik. "Conservatism in Risk Assessment and Food Safety
Policy." The Economics of Food Safety. Ed. Julie A. Caswell.
London: Elsevier, 1991, pp. 89-102.
Linneman, Peter. "The Effects of Consumer Safety Standards: The
1973 Mattress Flammability Standard." George J. Stigler Chicago
Studies in Political Economy. Chicago, Ill.: University of Chicago
Press, 1988, pp. 441-60.
Litan, Robert E., and William D. Nordhaus. Reforming Federal
Regulation. New Haven: Yale University Press, 1983.
Loomis, John B., and Pierre H. duVair. "Evaluating the Effect of
Alternative Risk Communication Devices on Willingness to Pay:
Results From a Dichotomous Choice Contingent Valuation Experiment."
Land Economics, 69 (3) (Aug. 1993), pp. 287-98.
Lovvoll, David R., and others. "Responsibility for Product Safety in
the Work Environment." Proceedings of the Human Factors and
Ergonomics Society 40th Annual Meeting. Santa Monica, Calif.: Human
Factors and Ergonomics Society, 1996, pp. 814-17.
MacLean, D. Values at Risk. Totowa, N.J.: Rowman and Allanheld,
1986.
Magat, Wesley A., and Michael J. Moore. "Consumer Product Safety
Regulation in the United States and the United Kingdom: The Case of
Bicycles." Rand Journal of Economics, 27 (1) (Spring 1996), pp.
148-64.
Magat, Wesley A., and W. Kip Viscusi. Informational Approaches to
Regulation. Cambridge, Mass.: MIT Press, 1992.
Mayer, Michelle, and Felicia B. LeClere. "Injury Prevention
Measures in Households with Children in the United States, 1990."
Advance Data, No. 250 (May 31, 1994).
McCaig, Linda F. "National Hospital Ambulatory Medical Care Survey:
1992 Emergency Department Summary." Advance Data, No. 245 (Mar. 2,
1994).
McGarity, Thomas O. "Media Quality, Technology, and Cost Benefit
Balancing Strategies for Health and Environmental Regulation." Law
and Contemporary Problems, 46 (3) (1983), pp. 159-233.
_____, and Sidney A. Shapiro. Workers at Risk: The Failed Promise
of the Occupational Safety and Health Administration. Westport,
Conn.: Praeger Press, 1993.
McGinnis, J.M., and W.H. Foege. "Actual Causes of Death in the
United States." Journal of the American Medical Association, Vol.
270, pp. 2207-12.
McKean, Kevin. "They Fly in the Face of Danger." Discover, Apr.
1986, pp. 48-58.
Miller, James C. III, and Bruce Yandle, eds. Benefit-Cost Analyses
of Social Regulation: Case Studies From the Council on Wage and
Price Stability. Washington, D.C.: American Enterprise Institute,
1979.
Miller, Ted R., and Diane C. Lestina. "Patterns in U.S. Medical
Expenditures and Utilization for Injury, 1987." American Journal of
Public Health, 86 (1) (Jan. 1996), pp. 89-93.
Mishan, E.J. Cost-Benefit Analysis. London: Unwin Hyman, 1988.
Moore, John L. Cost Benefit Analysis: Issues in Its Use in
Regulation. Report No. 95-760 ENR. Washington, D.C.:
Congressional Research Service, June 28, 1995.
Morgan, M. Granger, and Max Henrion. Uncertainty: A Guide to
Dealing With Uncertainty in Quantitative Risk and Policy Analysis.
Cambridge, Mass.: Cambridge University Press, 1990.
Nash, Chris. "Cost-benefit Analysis of Transport Projects." Alan
Williams and Emilio Giardina. Efficiency in the Public Sector.
Aldershot, Hants, England: Edward Elgar Publishing, 1993, pp.
83-105.
National Committee for Injury Prevention and Control. Injury
Prevention: Meeting the Challenge. New York: Oxford University
Press, 1989.
Needleman, Carolyn, and Martin L. Needleman. "Qualitative Methods
for Intervention Research." American Journal of Industrial Medicine,
29 (1996), pp. 329-37.
Nelson, Cheryl R., and Barbara J. Stussman. "Alcohol- and
Drug-Related Visits to Hospital Emergency Departments: 1992 National
Hospital Ambulatory Medical Care Survey." Advance Data, No. 251
(Aug. 10, 1994).
Newbery, David. "The Isolation Paradox and the Discount Rate for
Benefit-Cost Analysis: A Comment." Quarterly Journal of Economics,
Feb. 1990, pp. 235-8.
Oi, Walter Y. "Safety at What Price?" American Economic Review, AEA
Papers and Proceedings, 85 (2) (May 1995), pp. 68-71.
Ozonoff, Victoria Vespe, Susan Tan-Torres, and Catherine W. Barber.
"Assessment of E-Coding Practices and Costs in Massachusetts
Hospitals." Public Health Reports, 108 (5) (Sept.-Oct. 1993), pp.
633-6.
Page, Talbot. "A Framework for Unreasonable Risk in the Toxic
Substances Control Act (TSCA)." Management of Assessed Risk for
Carcinogens. Ed. William J. Nicholson. Annals of the New York
Academy of Sciences, New York: 1981.
_____, and Paolo F. Ricci. "A Cost-Benefit Perspective for Risk
Assessment." Principles of Health Risk Assessment. Ed. Paolo F.
Ricci. Englewood Cliffs, N.J.: Prentice-Hall, Inc., 1985, pp.
37-65.
Pease, William S. The Impact of Scientific Research on Federal
Agency Guidelines for Conducting Health Risk Assessments.
Washington, D.C.: U.S. Office of Technology Assessment, Dec. 18,
1992.
Peltzman, Sam. "The Effect of Automobile Safety Regulation." George
J. Stigler Chicago Studies in Political Economy. Chicago, Ill. :
University of Chicago Press, 1988, pp. 349-403 (originally published
in Journal of Political Economy, 83, 1975, pp. 677-725).
Petty, Ross D. "Bicycle Safety: A Case Study in Regulatory Review."
Regulation, 17 (2) (1994), pp. 22-24.
_____. "Regulating Product Safety: The Informational Role of the
U.S. Federal Trade Commission." Journal of Consumer Policy, 18
(1995), pp. 387-415.
Pollack, E.S., and D.G. Keimig, eds. Counting Injuries and
Illnesses in the Workplace: Proposals for a Better System. New
York: National Academy Press, 1987.
Price, Colin. "Does Social Cost-Benefit Analysis Measure Overall
Utility Change?" Economics Letters, 26 (1988), pp. 357-61.
Proulx, Guylene, and Joelle Pineau. "The Impact of Age on Occupants'
Behavior During a Residential Fire." Unpublished paper, 1996.
Quiggin, John. "Existence Value and Cost-Benefit Analysis: A Third
View." Journal of Policy Analysis and Management, 12 (1) (1993), pp.
195-9.
_____. "Too Many Proposals Pass the Benefit-Cost Test: Comment."
American Economic Review, 81 (5) (Dec. 1991), pp. 1446-9.
Ramirez, Jorge, and others. "Ex-Post Analysis of Flood Control:
Benefit-Cost Analysis and the Value of Information." Water Resources
Research, 24 (8) (Aug. 1988), pp. 1397-1405.
Rauchschwalbe, Renae, and N. Clay Mann. "Pediatric Window-Cord
Strangulations in the United States, 1981-1995." Journal of the
American Medical Association, 277 (21) (June 4, 1997), pp. 1696-98.
Rice, Dorothy P., and Wendy Max. "Annotation: The High Cost of
Injuries in the United States." American Journal of Public Health, 86
(1)
(Jan. 1996), pp. 14-15.
Risa, Alf Erling. "Public Regulation of Private Accident Risk: The
Moral Hazard of Technological Improvements." Journal of Regulatory
Economics, 4 (1992), pp. 335-46.
Rivara, Frederick P., and others. "Cost Estimates for Statewide
Reporting of Injuries by E Coding Hospital Discharge Abstract Data
Base Systems." Public Health Reports, 105 (6) (Nov.-Dec. 1990), pp.
635-38.
Robins, Marcia P. "An Examination of the Effect of CPSC Prescription
Drug Child-Resistant Packaging Requirements on Child Fatalities."
Unpublished paper, Nov. 1989.
Rodgers, Gregory B. "The Effectiveness of Helmets in Reducing
All-Terrain Vehicle Injuries and Deaths." Accident Analysis and
Prevention, 22 (1) (1990), pp. 47-58.
_____. "Evaluating Product-Related Hazards at the Consumer Product
Safety Commission: The Case of All-Terrain Vehicles." Evaluation
Review, Feb. 1990, pp. 3-21.
_____, and P. Rubin. "Cost-Benefit Analysis of All-Terrain Vehicles
at the CPSC." Risk Analysis, 9 (1989), pp. 63-9.
Rosenthal, Donald H., and Robert H. Nelson. "Why Existence Value
Should NOT Be Used in Cost-Benefit Analysis." Journal of Policy
Analysis and Management, 11 (1) (1992), pp. 116-22.
Rubin, Paul H., R. Dennis Murphy, and Gregg Jarrell. "Risky
Products, Risky Stocks." Regulation, No. 1 (1988), pp. 35-9.
Rubinfeld, Daniel L., and Gregory B. Rodgers. "Evaluating the
Injury Risk Associated With All-Terrain Vehicles: An Application of
Bayes' Rule." Journal of Risk and Uncertainty, 5 (1992), pp. 145-58.
Schappert, Susan M. "National Ambulatory Medical Care Survey: 1992
Summary." Advance Data, No. 253 (Aug. 18, 1994).
Schierow, Linda-Jo. "Risk Analysis and Cost-Benefit Analysis of
Environmental Regulations. 94-961 ENR. Washington, D.C. : U.S.
Congressional Research Service, Dec. 2, 1994.
Schroeder, Elinor P., and Sidney A. Shapiro. "Responses to
Occupational Disease: The Role of Markets, Regulation, and
Information." Georgetown Law Journal, 72 (1983), pp. 1231-1306.
Selbst, S., M.D. Baker, and M. Shames. "Bunk Bed Injuries."
American Journal of Diseases in Children, 144 (6) (June 1990), pp.
721-3.
Sen, Amartya. "Approaches to the Choice of Discount Rates for Social
Benefit-Cost Analysis." Chapter 8 in Resources, Values, and
Development. Cambridge, Mass.: Harvard University Press, 1984, pp.
172-203.
Senturia, Yvonne D., and others. "Exposure-Corrected Risk Estimates
for Childhood Product-Related Injuries." American Journal of Public
Health, 1993, pp. 473-7.
Shew, Russel, and Rachel Dardis. "An Economic Analysis of Child
Restraints." Journal of Consumer Policy, 18 (1995), pp. 417-31.
Smith, V. Kerry. "A Conceptual Overview of the Foundations of
Benefit-Cost Analysis." Benefits Assessment: The State of the Art.
Eds. Judith D. Bentkover, Vincent T. Covello, and Jeryl Mumpower.
Dordrecht, Holland: D. Reidel Publishing Company, 1986, pp. 13-34.
_____. "Uncertainty, Benefit-Cost Analysis, and the Treatment of
Option Value." Journal of Environmental Economics and Management, 14
(1987), pp. 283-92.
_____, and others. "Can Public Information Programs Affect Risk
Perceptions?" Journal of Policy Analysis and Management, 9 (1)
(1990), pp. 41-59.
Spitzer, Hugh. Case Studies to Demonstrate the Impact of Research on
Assessment of Carcinogenic Risk. Washington, D.C.: U.S. Office of
Technology Assessment, Dec. 1, 1992.
Stason, W.B., and M.C. Weinstein. "Allocation of Resources to
Manage Hypertension." New England Journal of Medicine, 296 (13)
(1977), pp. 732-9.
Stayner, Leslie, and others. "Approaches for Assessing the Efficacy
of Occupational Health and Safety Standards." American Journal of
Industrial Medicine, 29 (1996), pp. 353-7.
_____. "National Hospital Ambulatory Medical Care Survey: 1994
Emergency Department Summary." Advance Data, No. 275 (May 17, 1996).
Stussman, Barbara J. "National Hospital Ambulatory Medical Care
Survey: 1993 Emergency Department Summary." Advance Data, No. 271
(Jan. 25, 1996).
Tengs, Tammy O., and John D. Graham. "The Opportunity Costs of
Haphazard Social Investments in Life Saving." Risks, Costs and Lives
Saved: Getting Better Results from Regulation. Ed. Robert W.
Hahn. Washington, D.C.: AEI Press, 1996, pp. 167-82.
Thacker, Stephen B., and others. "Assessing Prevention Effectiveness
Using Data to Drive Program Decisions." Public Health Reports, 109
(2) (Mar.-Apr. 1994), pp. 187-94.
Thomas, L.G. "Revealed Bureaucratic Preference: Priorities of the
Consumer Product Safety Commission." Rand Journal of Economics, 19
(1) (Spring 1988), pp. 102-13.
Thompson, M.S., J.L. Read, and M. Laing. "Feasibility of
Willingness-to-Pay Measurement in Chronic Arthritis." Medical
Decision Making, 4 (2)
(1984), pp. 195-215.
Tolchin, Susan J., and Martin Tolchin. "Particles of Truth: Cotton
Dust and Cost-Benefit Analysis." Dismantling America: The Rush to
Deregulate, Boston, Mass.: Houghton Mifflin Company, 1983, pp.
111-41.
Tolley, G., K. Kenkel, and R. Rabian, eds. Valuing Health for
Policy: An Economic Approach. Chicago, Ill.: University of Chicago
Press, 1994.
Trinkoff, Alison, and Peggy L. Parks. "Prevention Strategies for
Infant Walker-Related Injuries." Public Health Reports, 108 (6)
(Nov.-Dec. 1993), pp. 784-8.
Trumbull, William. "Reply to Whittington and MacRae." Journal of
Policy Analysis and Management, 9 (4) (1990), pp. 548-50.
_____. "Who Has Standing in Cost-Benefit Analysis?" Journal of
Policy Analysis and Management, 9 (2) (1990), pp. 201-18.
U.S. Commission on Risk Assessment and Risk Management. Risk
Assessment and Risk Management in Regulatory Decision-Making.
Washington, D.C.: 1997.
U.S. Department of Health and Human Services, Centers for Disease
Control and Prevention. "A Framework for Assessing the Effectiveness
of Disease and Injury Prevention." Morbidity and Mortality Weekly
Report, Recommendations and Reports, Vol. 41, No. RR-3 (Mar. 27,
1992), pp. 1-13.
_____. "Guidelines for Death Scene Investigation of Sudden
Unexplained Infant Deaths." Morbidity and Mortality Weekly Report,
Recommendations and Reports, Vol. 45, No. RR-10 (June 21, 1996),
pp. 1-31.
_____. "Injuries and Deaths Associated With the Use of
Snowmobiles--Maine, 1991-1996." Morbidity and Mortality Weekly
Report, Vol. 46, No. 1 (Jan. 10, 1997), pp. 1-4.
_____. Inventory of Federal Data Systems in the United States for
Injury Surveillance, Research and Prevention Activities. Washington,
D.C.: May 1996.
U.S. Department of Health and Human Services, Centers for Disease
Control and Prevention, National Center for Health Statistics.
"Advance Report of Final Mortality Statistics, 1994." Monthly Vital
Statistics Report, 45 (3) (Supplement, Sept. 30, 1996).
_____. "Hospitalizations for Injury and Poisoning in the United
States, 1991." Advance Data, No. 252 (Oct. 7, 1994).
U.S. Department of Health and Human Services, Public Health Service.
Healthy People 2000: Midcourse Review and 1995 Revisions.
Washington, D.C.: 1995.
_____. Healthy People 2000: National Health Promotion and Disease
Objectives. Washington, D.C.: 1991.
U.S. General Accounting Office. Peer Review: EPA's Implementation
Remains Uneven. Sept. 24, 1996, GAO/RCED-96-236.
U.S. Office of Management and Budget. Guidelines and Discount Rates
for Benefit-Cost Analysis of Federal Programs. Circular A-94.
Washington, D.C.: Oct. 29, 1992.
_____. Regulatory Program of the United States Government. Bulletin
No. 91-04. Washington, D.C.: Nov. 26, 1990.
U.S. Office of Technology Assessment. Researching Health Risks.
OTA-BBS-570. Washington, D.C.: Jan. 1993.
_____. Risks to Students in School. OTA-ENV-633. Washington, D.C.:
Sept. 1995.
U.S. Physician Payment Review Commission. A Comparison of
Alternative Approaches to Risk Measurement. Washington, D.C.: Dec.
1994.
van Ravenswaay, Eileen O., and John P. Hoehn. "The Impact of Health
Risk Information on Food Demand: A Case Study of Alar and Apples."
The Economics of Food Safety. Ed. Julie A. Caswell. London:
Elsevier, 1991, pp. 155-174.
Veasie, M.A., and others. "Epidemiologic Research on the Etiology of
Injuries at Work." Annual Review of Public Health, 15 (1994), pp.
203-21.
Viscusi, W. Kip. "The Dangers of Unbounded Commitments to Regulate
Risk." Risks, Costs and Lives Saved: Getting Better Results From
Regulation. Ed. Robert W. Hahn. Washington, D.C.: AEI Press,
1996, pp. 135-66.
_____. Regulating Consumer Product Safety. Washington, D.C.:
American Enterprise Institute, 1984.
_____. Tax Policy and the Economy. Ed. James M. Porterba. Vol.
9, 1995, p. 180.
_____. "The Valuation of Risks to Life and Health: Guidelines for
Policy Analysis." Benefits Assessment: The State of the Art. Eds.
Judith D. Bentkover, Vincent T. Covello, and Jeryl Mumpower.
Dordrecht, Holland: D. Reidel Publishing Company, 1986, pp.
193-210.
_____. "The Value of Risks to Life and Health." Journal of Economic
Literature, 31 (Dec. 1993), pp. 1912-46.
_____, and Richard J. Zeckhauser. "Hazard Communication: Warnings
and Risk." The Annals of the American Academy of Political and Social
Science. Eds. Howard Kunreuther and Paul Slovic. Vol. 545 (May
1996). Philadelphia: Sage Periodicals Press, pp. 106-15.
_____, and Wesley A. Magat, eds. Learning About Risk: Consumer and
Worker Responses to Hazard Information. Cambridge, Mass.: Harvard
University Press, 1987.
Waller, Julian A. "Reflections on a Half Century of Injury Control."
American Journal of Public Health, 84 (4) (Apr. 1994), pp. 664-70.
_____, Joan M. Skelly, and John H. Davis. "Treated Injuries in
Northern Vermont." Accident Analysis and Prevention, 27 (6) (1995),
pp. 819-28.
Warr, Peter G., and Brian D. Wright. "The Isolation Paradox and the
Discount Rate for Benefit-Cost Analysis." Quarterly Journal of
Economics, 96 (1) (Feb. 1981), pp. 129-45.
Weinstein, M.C., and others. "Recommendations of the Panel on
Cost-Effectiveness in Health and Medicine." Journal of the American
Medical Association, 276 (15) (1996), pp. 1253-8.
Weisbrod, Burton A. "Benefit-Cost Analysis of a Controlled
Experiment: Treating the Mentally Ill." Journal of Human Resources,
16 (4) (1981), pp. 523-48.
Weiss, K.B., P.J. Gergen, and T.A. Hodgson. "An Economic
Evaluation of Asthma in the United States." New England Journal of
Medicine, 326, pp. 862-6.
Wheelwright, Jeff. "The Air of Ostrava: Pollution and Risk
Assessment in the Czech Republic." Discover, 17 (5) (May 1996), pp.
56-64.
Whittington, Dale, and Duncan MacRae, Jr. "Comment: Judgments About
Who Has Standing in Cost-Benefit Analysis." Journal of Policy
Analysis and Management, 9 (4) (1990), pp. 536-47.
Wildasin, David E. "Indirect Distributional Effects in Benefit-Cost
Analysis of Small Projects." The Economic Journal, 98 (Sept. 1988),
pp. 801-7.
Williams, Alan. "Cost-Benefit Analysis: Applied Welfare Economics
or General Decision Aid?" Alan Williams and Emilio Giardina.
Efficiency in the Public Sector. Aldershot, Hants, England: Edward
Elgar Publishing, 1993, pp. 65-82.
Wood, William W. "Cost-Benefit Analysis of Small Business
Assistance: Do Entrepreneurs Really Need `Assisting'?" Journal of
Private Enterprise, 10 (1) (Summer 1994), pp. 13-21.
Zeckhauser, Richard J., and W. Kip Viscusi. "The Risk Management
Dilemma." The Annals of the American Academy of Political and Social
Science. Eds. Howard Kunreuther and Paul Slovic. Vol. 545 (May
1996). Philadelphia: Sage Periodicals Press, pp. 144-55.
Zerbe, Richard O. "Comment: Does Benefit Cost Analysis Stand Alone?
Rights and Standing." Journal of Policy Analysis and Management, 10
(1) (1991), pp. 96-105.
*** End of document. ***