    Consumer Product Safety Commission: Better Data Needed to Help Identify
    and Analyze Potential Hazards (Letter Report, 09/29/97, GAO/HEHS-97-147).

    Pursuant to a congressional request, GAO reviewed the Consumer Product
    Safety Commission's (CPSC) project selection, use of cost-benefit
    analysis and risk assessment, and information release procedures,
    focusing on: (1) the criteria CPSC uses to select projects and the
    information it relies upon in making these choices; (2) the information
    CPSC draws on to perform risk assessment and cost-benefit analyses and
    CPSC's methodology for conducting cost-benefit analyses; and (3) CPSC's
    procedures for releasing manufacturer-specific information to the public
    and whether evidence exists that CPSC violated its statutory
    requirements concerning the release of such information.

    GAO noted that: (1) although CPSC has established criteria to help
    select new projects, with the agency's current data these criteria can
    be measured only imprecisely, if at all; (2) although CPSC has described
    itself as "data driven," its information on product-related injuries and
    deaths is often sketchy; (3) this makes it more difficult not only for
    agency management to monitor current projects but also for staff and
    commissioners to assess and prioritize the need for new projects in
    different hazard areas; (4) CPSC has insufficient data on both internal
    agency efforts and external product hazards to assess the impact and
    cost of each project; (5) to help evaluate alternative methods of
    addressing potential hazards, CPSC may perform a risk assessment to
    estimate the likelihood of injury associated with a hazard or conduct a
    cost-benefit analysis to assess the potential effects of a proposed
    regulation; (6) although CPSC does not complete either a risk assessment
    or cost-benefit analysis for every project, the agency conducts these
    analyses more often than it is required to by law; (7) nevertheless,
    CPSC's data are often insufficient to support a thorough application of
    these analytical techniques; (8) to evaluate relative risks, it is
    usually necessary to have information on how many consumers use the
    product--information that CPSC frequently does not have; (9) risk
    assessment of consumer products requires measurement of the number of
    harmful incidents; (10) CPSC's imprecise and incomplete death and injury
    data make risk assessment and cost-benefit analysis at best less
    reliable and at worst impossible to do; (11) the cost-benefit analyses
    conducted by CPSC between 1990 and 1996 were often not comprehensive;
    (12) CPSC has established procedures to implement statutory requirements
    concerning the release of manufacturer-specific information; (13) when
    releasing information to the public that identifies a specific
    manufacturer, CPSC is required to verify the information and allow the
    manufacturer an opportunity to comment; (14) evidence from the industry
    and from legal cases suggests that CPSC has met its statutory
    requirements in this area; (15) individuals within CPSC, as well as some
    industry representatives and consumer groups, expressed dissatisfaction
    with the requirements of this law; and (16) some of these individuals
    have proposed statutory changes that range from reducing to expanding
    the current requirements.

    --------------------------- Indexing Terms -----------------------------

     REPORTNUM:  HEHS-97-147
         TITLE:  Consumer Product Safety Commission: Better Data Needed to
                 Help Identify and Analyze Potential Hazards
          DATE:  09/29/97
       SUBJECT:  Consumer protection
                 Statistical data
                 Data collection
                 Data integrity
                 Statistical methods
                 Cost effectiveness analysis
                 Safety regulation
                 Reporting requirements
                 Product safety
                 Management information systems
    IDENTIFIER:  CPSC National Electronic Injury Surveillance System

    ******************************************************************
    ** This file contains an ASCII representation of the text of a  **
    ** GAO report.  Delineations within the text indicating chapter **
    ** titles, headings, and bullets are preserved.  Major          **
    ** divisions and subdivisions of the text, such as Chapters,    **
    ** Sections, and Appendixes, are identified by double and       **
    ** single lines.  The numbers on the right end of these lines   **
    ** indicate the position of each of the subsections in the      **
    ** document outline.  These numbers do NOT correspond with the  **
    ** page numbers of the printed product.                         **
    **                                                              **
    ** No attempt has been made to display graphic images, although **
    ** figure captions are reproduced.  Tables are included, but    **
    ** may not resemble those in the printed version.               **
    **                                                              **
    ** Please see the PDF (Portable Document Format) file, when     **
    ** available, for a complete electronic file of the printed     **
    ** document's contents.                                         **
    **                                                              **
    ** A printed copy of this report may be obtained from the GAO   **
    ** Document Distribution Center.  For further details, please   **
    ** send an e-mail message to:                                   **
    **                                                              **
    **                                            **
    **                                                              **
    ** with the message 'info' in the body.                         **
    ******************************************************************

    Cover
    ================================================================ COVER

    Report to the Chairmen, Committee on Commerce, Science, and
    Transportation, U.S.  Senate, and Committee on Commerce, House of
    Representatives

    September 1997

    CONSUMER PRODUCT SAFETY COMMISSION
    - BETTER DATA NEEDED TO HELP
    IDENTIFY AND ANALYZE POTENTIAL
    HAZARDS

    GAO/HEHS-97-147

    Consumer Product Safety Commission

    (205333)

    Abbreviations
    =============================================================== ABBREV

      AAPCC - American Association of Poison Control Centers
      ANPR - Advance Notice of Proposed Rulemaking
      ATV - all-terrain vehicle
      CO - carbon monoxide
      CPSC - Consumer Product Safety Commission
      FOIA - Freedom of Information Act
      HMO - health maintenance organization
      IPII - Injury or Potential Injury Incident
      LP - liquified petroleum
      MECAP - Medical Examiners' and Coroners' Alert Program
      MIS - management information system
      NEISS - National Electronic Injury Surveillance System
      NFIRS - National Fire Reporting System
      NFPA - National Fire Protection Association
      OMB - Office of Management and Budget

    Letter
    =============================================================== LETTER

    B-274042

    September 29, 1997

    The Honorable John McCain
    Chairman, Committee on Commerce,
     Science, and Transportation
    United States Senate

    The Honorable Thomas J.  Bliley, Jr.
    Chairman, Committee on Commerce
    House of Representatives

    Every year, children are hurt while using playground equipment; users
    or spectators are injured when fireworks explode unexpectedly; and
    homeowners are cut while operating chain saws and lawn mowers.
    Created to protect consumers from "unreasonable risk of injury," the
    U.S.  Consumer Product Safety Commission (CPSC) oversees these and
    about 15,000 other consumer products ranging from kitchen appliances
    and children's toys to hot tubs and garage door openers.  With a
    budget of about $42.5 million, CPSC carries out its mission by (1)
    enforcing federal consumer product regulations (by recalling products
    from store shelves, for example) and (2) conducting projects to
    address products with potential hazards not covered by existing
    regulations.\1

    These projects may result in CPSC issuing new regulations concerning
    specific products, assisting in the development of voluntary industry
    standards, or providing information to consumers about how to use the
    products safely.

    Contending that the agency is ineffectively allocating its resources,
    CPSC's critics have voiced dissatisfaction with the selection of
    certain agency projects and have questioned the validity of CPSC's
    risk assessment and cost-benefit analyses supporting those projects.
    In addition, congressional and interest group critics have questioned
    the agency's procedures for ensuring the accuracy of
    manufacturer-specific information before releasing it to the public,
    contending that such releases can mar the reputation of responsible
    corporate citizens.

    In light of these concerns, you asked us to review CPSC's project
    selection, use of cost-benefit analysis and risk assessment, and
    information release procedures.  Specifically, this report (1)
    identifies the criteria CPSC uses to select projects and reviews the
    information it relies upon in making these choices, (2) assesses the
    information CPSC draws on to perform risk assessment and cost-benefit
    analyses and evaluates the agency's methodology for conducting
    cost-benefit analyses, and (3) describes CPSC's procedures for
    releasing manufacturer-specific information to the public and reports
    whether evidence exists that CPSC violated its statutory requirements
    concerning the release of such information.

    To address these objectives, we reviewed internal CPSC documents,
    relevant legislation and regulations, and the literature on
    cost-benefit analysis and on consumer product safety issues.  We
    interviewed CPSC commissioners and staff, four former commissioners,
    consumer advocates, industry representatives, and outside experts to
    obtain their perspectives on CPSC's work.  We identified CPSC
    projects by compiling from various agency documents a list of 115
    potential product hazards examined by the agency from January 1990 to
    September 30, 1996, and we reviewed available agency documentation on
    each of these projects.  We examined the agency's internal databases
    to obtain project information and to assess the agency's information
    on product hazards.  In reviewing CPSC's cost-benefit analyses, we
    consulted with experts to develop objective criteria that reflected
    elements commonly used in the evaluation of cost-benefit analyses.
    These evaluation questions were designed to elicit whether CPSC
    conducted comprehensive analyses and reported them in sufficient
    detail.  However, they do not make up a complete measure of the
    quality of a cost-benefit analysis; for example, our evaluation
    assessed whether these elements were included but not how well they
    were measured or incorporated.  We reviewed CPSC's internal
    procedures concerning information clearance and release, and we
    examined relevant legal cases in this area.  For more detailed
    information on our scope and methodology, see appendix I.

    --------------------
    \1 Projects vary widely in scope, and CPSC has no standard definition
    of what constitutes a project.  For our review, we defined a
    "project" as work CPSC conducted on any specific consumer product
    that was associated with a potential hazard or hazards not covered by
    existing regulation.

       RESULTS IN BRIEF
    ------------------------------------------------------------ Letter :1

    Although CPSC has established criteria to help select new projects,
    with the agency's current data these criteria can be measured only
    imprecisely, if at all.  The criteria for selecting projects include
    the number of product-related injuries, chronic illnesses, and
    deaths.  However, although CPSC has described itself as "data
    driven," its information on product-related injuries and deaths is
    often sketchy.  For example, because the agency's measure of injuries
    generally includes only hospital emergency room reports, CPSC has an
    incomplete picture of injuries.  Similarly, CPSC's data on
    product-related deaths understate the total number of deaths and are
    generally available to the agency only with a 2-year time lag.  In
    addition, CPSC does not maintain a comprehensive list or description
    of either its past or ongoing projects.  This makes it more difficult
    not only for agency management to monitor current projects but also
    for staff and commissioners to assess and prioritize the need for new
    projects in different hazard areas.  As a result, CPSC has
    insufficient data on both internal agency efforts and external
    product hazards to assess the impact and cost of each project, either
    when it is selected or after it has been implemented.

    To help evaluate alternative methods of addressing potential hazards,
    CPSC may perform a risk assessment to estimate the likelihood of
    injury associated with a hazard or conduct a cost-benefit analysis to
    assess the potential effects of a proposed regulation.  Although CPSC
    does not complete either a risk assessment or cost-benefit analysis
    for every project, the agency conducts these analyses more often than
    it is required to by law.  Nevertheless, CPSC's data are often
    insufficient to support a thorough application of these analytical
    techniques--a problem that frequently arises in doing both risk
    assessment and cost-benefit analysis.  To evaluate relative risks, it
    is usually necessary to have information on how many consumers use
    the product--information that CPSC frequently does not have.  In
    addition, risk assessment in a consumer product context requires
    measurement of the number of harmful incidents.  CPSC's imprecise and
    incomplete death and injury data make risk assessment and
    cost-benefit analysis at best less reliable and at worst impossible
    to do.  Furthermore, the cost-benefit analyses conducted by CPSC
    between 1990 and 1996 were often not comprehensive, and the reports
    on these analyses were not sufficiently detailed.  For example,
    experts generally agree that sensitivity analysis--a technique that
    enables the reader to determine which assumptions, data limitations,
    or parameters are most important to the conclusions--should be
    incorporated in cost-benefit analyses.  Most of CPSC's cost-benefit
    analyses did not include such information.  We are making
    recommendations to the Chairman of CPSC to improve the agency's
    project selection and cost-benefit analyses.

    CPSC has established procedures to implement statutory requirements
    concerning the release of manufacturer-specific information.  When
    releasing information to the public that identifies a specific
    manufacturer--for example, in a safety alert or recall notice--CPSC
    is required to verify the information and allow the manufacturer an
    opportunity to comment.  Evidence from the industry and from legal
    cases suggests that CPSC has met its statutory requirements in this
    area.  Individuals within CPSC, as well as some industry
    representatives and consumer groups, expressed dissatisfaction with
    the requirements of this law.  Some of these individuals have
    proposed statutory changes that range from reducing to expanding the
    current requirements.

       BACKGROUND
    ------------------------------------------------------------ Letter :2

    CPSC was created in 1972 under the Consumer Product Safety Act (P.L.
    92-573) to regulate consumer products that pose an unreasonable risk
    of injury, to assist consumers in using products safely, and to
    promote research and investigation into product-related deaths,
    injuries, and illnesses.  CPSC currently has three commissioners, who
    are responsible for establishing agency policy.\2

    One of these commissioners is designated the chairman; the chairman
    directs all the executive and administrative functions of the agency.

    The Consumer Product Safety Act consolidated federal safety
    regulatory activity relating to consumer products within CPSC.  As a
    result, in addition to its responsibilities for protecting against
    product hazards in general, CPSC also administers four laws that
    authorize various performance standards for specific consumer
    products.  These laws are the Flammable Fabrics Act (June 3, 1953,
    c.164), which authorizes flammability standards for clothing,
    upholstery, and other fabrics; the Federal Hazardous Substances Act
    (P.L.  86-613), which authorizes the regulation of substances that
    are toxic, corrosive, combustible, or otherwise hazardous; the Poison
    Prevention Packaging Act of 1970 (P.L.  91-601), which authorizes
    requirements for child-resistant packaging for certain drugs and
    other household substances; and the Refrigerator Safety Act of 1956
    (Aug.  2, 1956, c.890), which establishes safety standards for
    household refrigerators.

    In fiscal year 1997, CPSC carries out this broad mission with a
    budget of about $42.5 million and a full-time-equivalent staff of
    480.  As figure 1 shows, after adjusting for inflation, the agency's
    budget has decreased by about 60 percent since 1974.  Similarly,
    CPSC's current staffing level represents 43 percent fewer positions
    as compared with the agency's 1974 staff.

       Figure 1:  CPSC Funding, in
       Inflation-Adjusted Dollars,
       1974-96

       (See figure in printed
       edition.)

    Notes:  Budget figures were obtained from CPSC.  The 1993 budget
    figure excludes a special appropriation for office space relocation.
    Budget figures were adjusted for inflation using the Gross Domestic
    Product deflator for federal nondefense spending, with 1992 as the
    base year.

    CPSC uses a number of regulatory and nonregulatory tools to reduce
    injuries and deaths associated with consumer products.  Under several
    of the laws that it administers, CPSC has the authority to issue
    regulations that establish performance or labeling standards for
    consumer products.  For example, in 1993, CPSC issued regulations
    under the Consumer Product Safety Act requiring disposable cigarette
    lighters to be child-resistant.  If CPSC determines that there is no
    feasible standard that would sufficiently address the danger, CPSC
    may issue regulations to ban the manufacture and distribution of the
    product.  In addition, under the Consumer Product Safety Act, if a
    product violates a safety regulation or presents a "substantial
    hazard," CPSC may order a product recall, in which the item is
    removed from store shelves and consumers are alerted to return the
    item for repair, replacement, or refund.\3 CPSC can also impose civil
    penalties for violations of federal safety standards.

    Although CPSC has these broad regulatory powers, much of the agency's
    efforts are carried out using nonregulatory methods.  In addition to
    federally mandated product safety standards, many consumer products
    are covered by voluntary standards.  These voluntary standards, which
    are often established by private standard-setting groups, do not have
    the force of law.  However, many voluntary standards are widely
    accepted by industry.\4 The 1981 amendments to the Consumer Product
    Safety Act require CPSC to defer to a voluntary standard--rather than
    issue a mandatory regulation--if CPSC determines that the voluntary
    standard adequately addresses the hazard and that there is likely to
    be substantial compliance with the voluntary standard.  As a result,
    voluntary standards development is an important tool in CPSC's
    hazard-reduction efforts.  For example, in 1996 CPSC helped a private
    group develop a voluntary standard to address the risk of children
    getting their heads stuck between the slats of toddler beds, and in
    1995 CPSC assisted a standard-setting group in upgrading safety
    standards to prevent fires associated with Christmas tree lights.

    CPSC also addresses product hazards by providing information to
    consumers on safety practices that can help prevent product-related
    accidents.  For example, to encourage consumers to use electricity
    safely--and particularly to promote the use of ground fault circuit
    interrupters--CPSC conducted a far-reaching publicity campaign that
    included radio public service announcements, messages printed on
    carryout bags for hardware stores, a joint press conference with
    industry representatives, presentations on television's Home Shopping
    Network, and promotional letters to real estate and home inspection
    associations.  In addition to its own active efforts to disseminate
    information, CPSC provides considerable amounts of information in
    response to requests from the public.  Like other federal agencies,
    CPSC must comply with the Freedom of Information Act (FOIA) when
    responding to requests from the public for information.  A notable
    feature of FOIA is its presumption in favor of disclosure:  any
    person has the right to inspect and copy any government records
    unless the documents requested fall within one of the exemptions to
    the act (for example, disclosure of trade secrets).  FOIA requests
    may come to CPSC from regulated industries, the press, consumer
    groups, or individuals.  During calendar year 1995, CPSC responded to
    16,424 formal requests made under FOIA.

    CPSC's resource base and extensive jurisdiction require the agency to
    select among potential product hazards.  New initiatives may come to
    CPSC in several ways.  First, any person may file a petition asking
    CPSC to issue, amend, or revoke a regulation.  Petitions, which can
    be as simple as a letter or as formal and detailed as a legal brief,
    have come to CPSC from doctors and nurses, consumers and advocacy
    groups, and industry representatives.  CPSC may grant or deny a
    petition either in full or in part.  Even when CPSC denies a petition
    and declines to issue a regulation, it may still begin a project to
    address the hazard by promoting a voluntary standard or conducting a
    consumer education campaign.  For example, a project on heat tapes
    (heated wraps for exposed pipes) originated with a petition from a
    concerned consumer.  CPSC denied the petition for a mandatory
    standard, but conducted a research study and review of the existing
    voluntary standard for heat tapes.

    Second, CPSC receives some product hazard projects from the Congress.
    The Congress may require CPSC to study a wide-ranging product area.
    For example, the Consumer Product Safety Improvement Act of 1990
    resulted in a large body of work on products affecting indoor air
    quality, including wood stoves, kerosene heaters, and carpets.  The
    Congress may also direct CPSC to impose a specific regulation, such
    as when it directed CPSC to require additional labeling on toys
    intended for children aged 3 to 6 warning parents of possible choking
    hazards when the toy is used by children under age 3.\5

    Finally, CPSC commissioners and agency staff may initiate projects or
    suggest areas to address.  CPSC gathers death and injury information
    to help identify potential product hazards and also obtains input
    from the public.  The agency maintains a toll-free hot line and an
    Internet site that the public can use to notify agency staff of a
    possible product hazard.  In addition, CPSC holds public meetings to
    get input on possible hazards to address and on which hazards should
    receive priority.  For example, CPSC increased its efforts to remove
    drawstrings from children's clothing after receiving a letter from a
    woman whose daughter was strangled when her jacket string caught on a
    playground slide.

    The selection of projects to address new product hazards takes place
    at different levels of the agency throughout the year.  For a
    petition, the commissioners decide whether the product hazard
    warrants further agency involvement, and the commissioners vote on
    whether to grant or deny the petition.  If a project is believed to
    have a high potential for regulatory action or involve a substantial
    amount of agency resources, the commissioners decide whether to
    pursue it.  Projects of this caliber are often noted in the agency's
    annual budget and operating plan, which must also be approved by
    commissioner vote.\6 Staff request a decision on such a project by
    preparing a briefing package about the product hazard for the
    commissioners, who vote to begin the regulatory process, take some
    other action, or terminate the project.  Agency staff generally may
    decide to initiate projects that are unlikely to result in
    regulation, and no briefing package is sent to the commissioners for
    a vote.\7

    (Of the 115 CPSC projects we identified, 80 (70 percent) were
    detailed in briefing packages.)

    The scope of the agency's projects varies greatly, and CPSC has no
    standard definition of what constitutes a project.  A project might
    cover general product areas, such as fire hazards, or address only a
    specific product, like cigarette lighters.  A project might require
    undertaking an extensive research study or providing technical
    assistance to a group that is developing a voluntary standard.

    --------------------
    \2 The Consumer Product Safety Act provides for the appointment of
    five commissioners by the President of the United States for
    staggered 7-year terms.  However, since 1986, no more than three
    commissioners have served at one time.

    \3 In practice, CPSC rarely uses its regulatory power to order a
    recall, but works cooperatively with manufacturers to carry out
    recalls.

    \4 Voluntary standards may benefit manufacturers by giving consumers
    added confidence in a product, providing some degree of protection
    from product liability, and allowing manufacturers to benefit from
    the safety expertise developed by voluntary standards groups.  In
    addition, although federal law does not compel manufacturers to
    comply with voluntary standards, state or local regulations may
    incorporate some voluntary standards regarding consumer products, and
    some retailers prefer to carry only those goods that comply with the
    applicable voluntary standards.

    \5 This mandate was imposed in the Child Safety Protection Act (P.L.
    103-267, June 16, 1994).

    \6 Commission votes are also taken in several other instances, such
    as initiating a regulation, the use of the agency logo by outside
    groups, and the application of civil penalties.

    \7 For more information about the agency's organization and
    structure, see app.  II.

       CPSC HAS LIMITED INFORMATION
       AVAILABLE TO ASSIST IN PROJECT
       SELECTION
    ------------------------------------------------------------ Letter :3

    The bulk of CPSC's workload is made up of projects selected by the
    agency rather than by the Congress.  CPSC has established criteria to
    help in project selection, such as the numbers of deaths and injuries
    associated with a product.  However, CPSC is unable to accurately
    measure these criteria because its data on potential hazards are
    incomplete.  In addition, CPSC does not maintain systematic
    information on past and ongoing projects, which makes it difficult to
    assess and prioritize the need for new projects in different hazard
    areas.  The lack of comprehensive data on individual product hazards
    and on agency initiatives raises questions about CPSC's ability to
    evaluate its own effectiveness--which it is now required to do under
    the Government Performance and Results Act of 1993 (the Results Act).

          AGENCY HAS CONSIDERABLE
          DISCRETION IN PROJECT
          SELECTION
    ---------------------------------------------------------- Letter :3.1

    CPSC has wide latitude over which potential product hazards it
    targets for regulatory and nonregulatory action.  Although it has
    little or no discretion over projects mandated by the Congress, CPSC
    can choose to accept or reject suggestions that are submitted by
    petition or proposed by the agency staff.  As shown in figure 2, 59
    percent of CPSC projects were initiated by CPSC, 30 percent
    originated from a petition, and about 11 percent resulted from
    congressional mandates.  Of the 115 projects the agency worked on
    from January 1, 1990, to September 30, 1996, 97 (about 90 percent)
    were chosen by the agency.\8 Data were unavailable to assess the
    extent to which staff suggestions for projects were accepted or
    rejected.  Of the petitions filed with CPSC between January 1, 1990,
    and September 30, 1996, 60 percent resulted in projects (32 percent
    by granting the petition in whole or in part, and 27 percent by
    denying the petition to establish a mandatory regulation but creating
    a nonregulatory project).  In 27 percent of cases, CPSC decided that
    no action was needed or that existing actions or standards were
    sufficient to address the issue raised by the petition.  In the
    remaining cases, a decision is still pending or the petition was
    withdrawn before a decision was rendered.

       Figure 2:  Origin of CPSC
       Projects, January 1,
       1990-September 30, 1996

       (See figure in printed
       edition.)

    --------------------
    \8 For a listing of these 115 projects, see app.  III.

          CPSC PROJECT SELECTION
          CRITERIA OPEN TO DIFFERENCES
          IN EMPHASIS AND
          INTERPRETATION
    ---------------------------------------------------------- Letter :3.2

    CPSC has established criteria for setting agency priorities and
    selecting potential hazards to address.  These criteria, which are
    incorporated in agency regulations, include the following:

      -- the frequency of injuries and deaths resulting from the hazard;

      -- the severity of the injuries resulting from the hazard;

      -- addressability--that is, the extent to which the hazard is
         likely to be reduced through CPSC action--agency regulations
         note that the cause of the hazard should be analyzed to help
         determine the extent to which injuries can reasonably be
         expected to be reduced or eliminated through CPSC action;

      -- the number of chronic illnesses and future injuries predicted to
         result from the hazard;

      -- preliminary estimates of the costs and benefits to society
         resulting from CPSC action;

      -- unforeseen nature of the risk--that is, the degree to which
         consumers are aware of the hazard and its consequences;

      -- vulnerability of the population at risk--whether some
         individuals (such as children) may be less able to recognize or
         escape from potential hazards and therefore may require a
         relatively higher degree of protection;

      -- probability of exposure to the product hazard--that is, how many
         consumers are exposed to the potential hazard, or how likely a
         typical consumer is to be exposed to the hazard; and

      -- other--additional criteria to be considered at the discretion of
         CPSC.

             VULNERABLE POPULATIONS,
             NUMBERS OF DEATHS AND
             INJURIES, AND CAUSALITY
             EMPHASIZED OVER OTHER
             SELECTION CRITERIA
    -------------------------------------------------------- Letter :3.2.1

    CPSC's regulations allow for considerable freedom in applying these
    criteria; commissioners and staff can base their project selections
    on what they perceive as the most important factors.  For example,
    the regulations do not specify whether any criterion should be given
    more weight than the others, nor must all criteria be applied to
    every potential project.  Indeed, our interviews with present and
    former commissioners and our review of CPSC briefing packages
    revealed a pattern in which three criteria--the numbers of deaths and
    injuries, the causality of injuries, and the vulnerability of the
    population at risk--were more strongly emphasized than the others.
    In addition, each of the commissioners we interviewed identified some
    criteria as being more important than others for project selection.
    For example, one commissioner indicated that the number of deaths and
    injuries was most important, while another commissioner included
    awareness of the hazard in a list of several criteria she believed
    were most important.  However, there was considerable agreement among
    the commissioners about the importance of several criteria.  The
    commissioners cited two criteria--vulnerability of population and
    number of deaths and injuries--as especially important for project
    selection.  In addition, several--but not all--commissioners
    emphasized causality of injuries.  None of the other criteria was
    emphasized by more than one or two commissioners.

    Because the commissioners use their judgment in applying these
    criteria, there is no systematic checklist or scoring system that
    would enable us to determine which factors were considered most
    important for a particular product.  However, information related to
    some or all of these criteria is sometimes contained in briefing
    packages and other documents.  Our review of CPSC project
    documentation showed that information on vulnerable populations and
    the numbers of deaths and injuries associated with the product was
    likely to be compiled at some time during the project, but
    information associated with other criteria was less likely to be
    documented.  For example, of the 115 projects we reviewed, death and
    injury information was available in 97 cases.  However, only 26 cases
    included information on exposure to the hazard, a less-emphasized
    criterion.

    Although data were insufficient to compare the universe of possible
    projects with the ones selected by CPSC, the characteristics of CPSC
    projects appear generally consistent with the stronger emphasis on
    death and injury data and on vulnerable populations expressed by the
    current and former commissioners.  For example, while 76 of the 115
    projects we examined were directed at least partially at a vulnerable
    population group, only 13 projects were associated with a chronic
    illness.  However, although the number of deaths and injuries
    associated with product hazards was almost always available in
    project documentation, there was no pattern of only those projects
    with high numbers of injuries or deaths being selected.  Of the 97
    projects that had death and injury statistics, 19 showed fewer than
    50 injuries and/or deaths associated with the product.  The estimated
    number of annual injuries associated with product hazards ranged from
    1 to 162,100 (for baseball injuries), and the estimated number of
    deaths associated with product hazards ranged from zero to a high of
    3,600 annually (for smoke detectors).  This wide range is consistent
    with CPSC staff's statement that there is no threshold for the number
    of deaths and injuries that would require acceptance or rejection of
    a project.

             COMMISSIONERS INTERPRET
             SOME SELECTION CRITERIA
             DIFFERENTLY
    -------------------------------------------------------- Letter :3.2.2

    Although the commissioners and former commissioners we interviewed
    generally agreed on the criteria they emphasized for project
    selection, they expressed very different views on how some of these
    criteria should be interpreted.  For example, several commissioners
    viewed vulnerable populations as focusing on children, while others
    highlighted additional segments of the population that they
    considered vulnerable.  One commissioner also listed low-income and
    poorly educated consumers as vulnerable populations, and another
    expressed concern that the elderly were especially vulnerable to
    injury from product hazards.  Project documentation focused on
    children more frequently than on other population segments thought to
    be at special risk.  Many projects we examined contained no
    information in the documentation that indicated a particular
    population was being considered vulnerable.  However, of the 76
    projects for which information was available on special populations,
    69 (91 percent) mentioned children.\9

    Industry observers, consumer advocates, current and former
    commissioners, and others expressed widely diverging views on how to
    apply causality of injuries in selecting projects.  All seven
    commissioners we interviewed mentioned this criterion, and several
    stressed causal factors.  A major issue surrounding the application
    of causality is determining the appropriate level of protection the
    agency should provide when a product hazard results, at least in
    part, from consumer behavior.  For example, a consumer advocate
    stated that regulatory action may be necessary whatever the cause of
    the incident if children who were incapable of protecting themselves
    get hurt.  Similarly, another individual told us that CPSC should
    deal with potential hazards on the basis of the behavior that
    actually took place, not the behavior that might be expected or
    considered reasonable.  However, other individuals asserted that CPSC
    should address only those hazards that result from products that are
    defective--that is, products that create a hazard even when used as
    intended by the manufacturer.  Some industry representatives stated
    that it was inappropriate for CPSC to take action concerning a
    product if the product was "misused" by the consumer.

    Complicating this debate is the difficulty of defining misuse of the
    product or negligence of the consumer.  For example, the appropriate
    degree of parental supervision is frequently an issue with children's
    products.  One of the agency's more controversial projects
    illustrates this point.  CPSC staff conducted a project to
    investigate the deaths of children using baby bath seats or rings.
    In these incidents, infants slipped out of the seat and drowned in
    the bathtub when the parent or caregiver stepped out of the room and
    left the child unsupervised, despite warning signs on the seats not
    to leave children unattended.  The Commission disagreed on the proper
    course of action, largely because of differing views on causality.
    In 1994, the staff recommended that the Commission issue an Advance
    Notice of Proposed Rulemaking (ANPR), the first step in the
    regulatory process.  The staff argued (and one commissioner agreed)
    that some parents will leave a young child alone in the bathtub
    regardless of a warning not to.  However, in voting against issuing
    an ANPR, the other two commissioners stated that they believed
    regulation was not appropriate because the lack of supervision, not
    the product, caused the tragedies.

    CPSC staff have also encountered other instances in which the
    behavior of consumers might be viewed as inappropriate.  For example,
    the role of alcohol and drug use in accidents can also raise
    questions about the appropriate level of regulatory protection.\10 In
    addition, a 1991 CPSC study found that at least 33 percent of bicycle
    accidents involved behaviors such as performing stunts and going too
    fast.  Similarly, a 1991 CPSC study of fires associated with heat
    tapes found that at least 38 percent of the heat tapes had been
    installed improperly.  In each of these cases, no regulatory action
    was taken; in the case of bicycles, the staff did recommend
    increasing efforts to encourage consumers to take safety precautions
    such as using lights at night and wearing helmets.

    --------------------
    \9 Many of these projects involved children's products, for which
    only children would be affected, such as cribs or baby walkers.
    Other projects dealt with products the general public would be
    exposed to but for which children would be more likely to receive
    injury, including mouthwashes with high concentrations of alcohol and
    automatic garage doors that could crush a child when they close.

    \10 Alcohol and drug use contributed to an estimated 1.35 million
    injury-related visits to hospital emergency departments in 1992.  See
    Cheryl R.  Nelson and Barbara J.  Stussman, "Alcohol- and
    Drug-Related Visits to Hospital Emergency Departments:  1992 National
    Hospital Ambulatory Medical Care Survey," Advance Data, No.  251
    (Aug.  10, 1994).

          DATA SYSTEMS PROVIDE
          INSUFFICIENT INFORMATION TO
          MEASURE SELECTION CRITERIA,
          MONITOR ALL PROJECTS, OR
          EVALUATE RESULTS
    ---------------------------------------------------------- Letter :3.3

    CPSC uses data from internal management systems and from external
    sources to assist in project selection.  CPSC collects information on
    product-related deaths and injuries to provide information for
    project selection as well as to perform risk assessments and
    cost-benefit analyses.  Furthermore, the agency maintains a
    computerized management information system (MIS) that contains
    information on some of its major activities and is used by the agency
    to develop its annual budget.  Both these internal and external data
    are of limited value.  The inadequacy of the information raises
    questions about CPSC's ability to make informed project selection
    decisions so that agency resources are being spent efficiently.

             SIGNIFICANT GAPS EXIST IN
             CPSC'S DATA ON
             PRODUCT-RELATED INJURIES
             AND DEATHS
    -------------------------------------------------------- Letter :3.3.1

    CPSC has developed a patchwork of independent data systems to provide
    information on deaths and injuries associated with consumer products.
    To obtain estimates of the number of injuries associated with
    specific consumer products, CPSC relies on its National Electronic
    Injury Surveillance System (NEISS).  NEISS gathers information from
    the emergency room records of a nationally representative sample of
    101 hospitals.  CPSC also obtains information on fatalities by
    purchasing a selected group of death certificates from the states.
    It supplements this information with anecdotal reports from
    individual consumers and with data from private organizations such as
    fire-prevention groups and poison control centers.  Because neither
    NEISS nor death certificate data provide detailed information on
    hazard patterns or injury causes, CPSC investigates selected
    incidents to obtain more detailed information.  In addition, CPSC
    sometimes uses mathematical modeling techniques or conducts special
    surveys to obtain information on product exposure.  (For more
    information on CPSC's data sources, see app.  V.)

    CPSC's data give the agency only limited assistance in applying its
    project selection criteria.  (These criteria, the measures used for
    each, and major data limitations are given in table 1.) CPSC's injury
    and death data allow the agency to piece together at best an
    incomplete view of the incidents that result from consumer product
    hazards.  Product-related injuries may be treated in a variety of
    ways--in an emergency room, in a physician's office, or through an
    outpatient clinic, for example.  As figure 3 illustrates, CPSC
    obtains systematic surveillance information only on deaths and on
    injuries treated in the emergency room; injuries treated in other
    settings (such as physicians' offices) are not represented in CPSC's
    surveillance data.

                              Table 1

                  CPSC Regulatory Priority-Setting
                Criteria and Systematic Information
                             Available

    Criterion       Measure used by CPSC  Major limitations
    --------------  --------------------  --------------------
    Number of       Number of reported    Incomplete because
    deaths          deaths where          not all certificates
                    consumer product      are gathered and not
                    involvement can be    all product-related
                    inferred.             incidents are coded.

    Number of       Estimated number of   Generally omits
    injuries        injuries involving    injuries treated in
                    consumer products     other settings.
                    treated in emergency
                    rooms.

    Severity of     Estimated percentage  Not representative
    injuries        of emergency-room-    of the severity of
                    treated injuries      all injuries treated
                    requiring             in all settings.
                    hospitalization.

    Chronic         Some limited          Little systematic
    illnesses       information from      information.
                    emergency room
                    diagnosis
                    information.

    Predicted       Prediction based on   Of questionable
    future          NEISS data.           validity because of
    injuries                              changes in medical
                                          care over time.

    Vulnerable      Percentage of         Incomplete.
    populations     emergency-room-       Information
                    treated injuries or   available only on
                    deaths involving      age, not on other
                    children or the       vulnerable
                    elderly.              populations, such as
                                          persons with
                                          disabilities.

    Exposure        Various measures,     Exposure surveys are
                    including sales,      time consuming and
                    estimated products    expensive; not done
                    in use, and           for all projects;
                    population.           done only after
                                          project is well
                                          under way.

    Addressability  Judgment based on     Often impossible to
    /causation      cause of incident,    make an informed
                    hazard pattern, and   judgment until
                    information on        project is well
                    product design;       under way;
                    investigations of     investigations are
                    selected incidents    time consuming and
                    may be conducted to   expensive.
                    obtain this
                    information.

    Preliminary     Results of analysis.  Quality data are
    cost-benefit                          frequently not
    analysis                              available; limited
                                          accuracy in early
                                          stages of project.
    ----------------------------------------------------------
       Figure 3:  Types of Death and
       Injury Data Covered by CPSC's
       Systematic Surveillance
       Information

       (See figure in printed
       edition.)

    Notes:  Death data are considered partially included because CPSC
    obtains death certificates for selected causes of death.  For the
    other sources of treatment, CPSC may obtain some anecdotal
    information from consumer reports or other sources (see app.  V).
    The actual numbers of injuries treated in each setting is unknown.

    A "near miss" refers to an incident in which a product-related injury
    nearly occurred but was narrowly averted.  In its regulations that
    address priority-setting, CPSC states that such incidents can be as
    important as actual injuries in identifying potential hazards.

    CPSC staff identified the lack of data on injuries treated in
    physicians' offices and other settings as a key concern.  Because
    CPSC's data sets reveal only a portion of the injury picture, the
    agency underestimates the total numbers of deaths and injuries
    associated with any given consumer product.  The extent of this
    undercount is unknown.  For example, researchers report widely
    varying estimates of the percentage of injuries that are treated in
    emergency rooms as opposed to other medical settings.  A 1991 study
    by researchers at RAND found that approximately 65 percent of
    injuries were treated in the emergency room.\11 However, recent data
    indicate that the number of injury-related visits to physicians'
    offices alone was more than double the number of injury-related
    visits to the emergency room.\12 CPSC's estimates of product-related
    deaths are also undercounted, for two reasons.  First, for budgetary
    reasons, the agency purchases only a subset of the total number of
    death certificates from states.  Second, CPSC death counts include
    only those cases in which product involvement can be inferred from
    the information on the death certificate, and in some cases,
    product-related information is not recorded.

    Even if a reliable figure was available to determine the exact
    percentage of product-related injuries that were treated in emergency
    rooms, this percentage would not necessarily apply to any specific
    type of product-related injury.  For example, even if it was
    established that 40 percent of all product-related injuries were
    treated in emergency rooms, the percentage of bunk bed injuries
    treated in emergency rooms might be much larger or smaller.  The
    setting in which injuries are treated depends on a wide array of
    factors that vary among individuals, across geographic regions, and
    among different types of injuries.  Research indicates that African
    Americans are more likely to use the emergency room than Caucasians
    are.\13 Access to the emergency room or to a physician also depends
    on the type of medical insurance a person has.  For example, health
    maintenance organizations (HMO) often place restrictions on
    reimbursement for emergency room care, and HMO membership as a
    percentage of the total population varies widely from state to
    state.\14 In addition, injuries that occur at night, when most
    physicians' offices are closed, may be more likely to be treated in
    the emergency room.  As a result, it is unlikely that CPSC could
    approximate the number of injuries associated with a specific product
    by using data that apply to all consumer products as a group.

    The incompleteness of CPSC's injury information also hampers its
    ability to reliably discern long-term trends in injuries, which is
    not only a criterion for project selection but also an important
    factor for evaluating the success of CPSC's injury-reduction efforts
    and determining the need for follow-up actions.  The relative sizes
    of the pieces of the injury puzzle in figure 3 are unknown but appear
    to change over time.  For example, hospitalizations decreased by 5
    percent on a per capita basis between 1982 and 1994, while between
    1983 and 1993, hospital outpatient clinics saw a 53-percent increase
    in visits on a per capita basis.\15 As a result, it is impossible to
    determine whether any change in the number of emergency room visits
    represents a true change in injuries or a shift to other medical
    settings.\16

    According to CPSC staff, identifying chronic illnesses associated
    with consumer products is nearly impossible.  CPSC staff stated that
    little is known about many chronic illness hazards that may be
    associated with potentially dangerous substances, and even less
    information is available about which consumer products may contain
    these substances.  Chronic illnesses are especially likely to be
    underestimated in CPSC's NEISS data because they are underrepresented
    among emergency room visits and because product involvement is more
    difficult to ascertain.  Similarly, consumer product involvement is
    very seldom recorded on death certificates in cases of chronic
    illnesses.

    CPSC's surveillance data also give an incomplete picture of the
    severity of incidents.  Although the data capture many relatively
    severe injuries�that is, those that result in death or require
    treatment in an emergency medical facility--data are missing for
    individuals who are admitted to the hospital through their physician
    rather than through the emergency room.  Potentially less severe
    cases�for example, those treated in physicians' offices, walk-in
    medical centers, or hospital outpatient clinics�are not represented
    at all in CPSC's systematic surveillance data systems.  Consequently,
    CPSC has no data on some consumer product problems that may result in
    numerous but potentially less severe injuries.

    Sketchy information about accident victims also limits CPSC's ability
    to assess which consumer product hazards have a disproportionate
    impact on vulnerable populations.  NEISS and death certificates
    provide only the age of the victim; no systematic or comprehensive
    information is available to determine whether a given hazard has a
    special impact on other vulnerable populations, such as persons with
    disabilities.  A former commissioner told us that the lack of other
    demographic information such as race, income, and disability status
    made it difficult for her to know which subpopulations were
    predominantly affected by a particular hazard.  Another commissioner
    echoed this concern, and said that such information would be useful
    in targeting public information campaigns on certain hazards to those
    groups that needed the information most.

    CPSC staff identified the need for additional exposure data as a
    major concern.  However, they also told us that obtaining information
    on exposure to products and establishing causation require special
    efforts that can be time consuming and costly.  Although CPSC's
    priority-setting criteria include exposure to the hazard, exposure
    data are generally not included in CPSC's ongoing data collection
    efforts.  As a result, exposure is assessed either not at all or
    further along in the project, precluding the use of exposure as an
    effective criterion for project selection.  Similarly, CPSC's
    emergency room and death certificate data provide little information
    on the circumstances surrounding the incident.  As a result, CPSC
    staff perform follow-up investigations of selected incidents to
    obtain additional detail.  These investigations may include detailed
    interviews with victims and/or witnesses, police or fire reports,
    photographs of the product and/or accident site, laboratory testing
    of the product, or recreations of the incident.  As with exposure
    data, these investigations are not conducted for every project and
    are completed only after a project is well under way.  Thus,
    assessment of causation at the project selection stage is unavoidably
    speculative.

    --------------------
    \11 Deborah R.  Hensler, and others, Compensation for Accidental
    Injuries in the United States (Santa Monica, Calif.:  RAND, 1991).

    \12 See Susan M.  Schappert, "National Ambulatory Medical Care
    Survey:  1992 Summary,"Advance Data No.  253 (Aug.  18, 1994), and
    Linda F.  McCaig, "National Hospital Ambulatory Medical Care Survey:
    1992 Emergency Department Summary," Advance Data, No.  245 (Mar.  2,
    1994).

    \13 See Linda F.  McCaig, "National Hospital Ambulatory Medical Care
    Survey:  1992 Emergency Department Summary," p.  1.

    \14 1996 figures from InterStudy show that while only 1.2 percent of
    individuals in Mississippi belonged to HMOs, 44.8 percent of
    individuals in Oregon were HMO members.  See The InterStudy
    Competitive Edge:  HMO Industry Report 6.2, InterStudy Publications
    (1996), p.  28.

    \15 These figures were derived from data provided by the American
    Hospital Association.

    \16 Precise estimates on the impact of changing treatment patterns
    are rare, and the impact may vary among local areas.  However, one
    research team found that because of the growth in alternative
    facilities in their local area, emergency rooms handled about half of
    outpatient injury cases--down from a previous estimate of 75 percent.
    See Julian A.  Waller, Joan M.  Skelly, and John H.  Davis, "Treated
    Injuries in Northern Vermont," Accident Analysis and Prevention, 27
    (6) (1995), pp.  819-28.

             AGENCY PROJECT MANAGEMENT
             INFORMATION INCOMPLETE OR
             UNAVAILABLE
    -------------------------------------------------------- Letter :3.3.2

    CPSC conducts a number of projects annually, but staff were unable to
    provide a comprehensive list of projects the agency had worked on in
    the 6-year period we examined.  CPSC was also unable to verify the
    completeness of the project list we compiled from agency documents
    and interviews with staff.  According to CPSC staff, internal
    management systems do not contain this information and such a list
    could be compiled only by relying on institutional memories of staff
    members who had been with the agency long enough to know which
    products the agency had addressed.  Without systematic and
    comprehensive information on its past efforts, CPSC cannot assess
    whether some hazard areas have been overrepresented and whether
    agency resources might be more efficiently employed.

    CPSC also lacks information on the characteristics of, resources used
    on, or outcomes of individual projects.  CPSC's MIS tracks contract
    dollars and staff time by accounting codes that cover some specific
    projects and general categories, such as compliance work, which are
    composed of numerous activities.  According to agency officials,
    CPSC's MIS generally cannot provide descriptive information on
    individual projects, such as when a project was started or concluded;
    the number of staff days used; what aspect of the product was
    addressed; whether the project originated from a petition,
    congressional mandate or other source; or what action was taken to
    address the hazard (mandatory standard, voluntary standard, or public
    information campaign, for example).  In addition, CPSC staff told us
    that two separate projects involving the same or similar products at
    different times may be assigned the same MIS code.  As a result, even
    if a project appears to be tracked in the MIS, reliable inferences
    cannot be drawn from MIS data.

             CPSC HAS LIMITED ABILITY
             TO EVALUATE AGENCY IMPACT
    -------------------------------------------------------- Letter :3.3.3

    CPSC's limited data on deaths and injuries, combined with its lack of
    information on projects, reduce the agency's ability to evaluate the
    impact of its work, a process it is now required to undertake under
    recently passed legislation.  The Results Act requires every federal
    agency to evaluate the effectiveness of its efforts starting in
    fiscal year 1999.  The Results Act is aimed at increasing the
    investment return of tax dollars by improving agencies' performance.
    Under the Results Act, an agency is to set mission-related goals and
    measure progress toward these goals to evaluate agency impact.  CPSC
    has preliminarily identified results-oriented goals in four areas:
    (1) reducing head injuries to children, (2) reducing deaths from
    fires, (3) reducing deaths from carbon monoxide poisoning, and (4)
    reducing deaths from electrocutions.  However, the limitations in
    CPSC's injury and death data raise a question about how well CPSC
    will be able to evaluate the effectiveness of agency actions in these
    and other areas.

       CPSC USES COST-BENEFIT ANALYSIS
       AND RISK ASSESSMENT, BUT
       IMPROVEMENTS ARE NEEDED IN DATA
       AND METHODOLOGY
    ------------------------------------------------------------ Letter :4

    CPSC uses two analytical tools--risk assessment and cost-benefit
    analysis--to assist in making decisions on regulatory and
    nonregulatory methods to address potential hazards.  Risk assessment
    involves estimating the likelihood of an adverse event (such as
    injury or death).  For example, CPSC estimated that the risk of death
    from an accident involving an all-terrain vehicle (ATV) was about 1
    death for every 10,000 ATVs in use in 1994.  Cost-benefit analysis
    details and compares the expected effects of a proposed regulation or
    policy, including both the positive results (benefits) and the
    negative consequences (costs).  Although cost-benefit analysis may
    not be applicable to every decision and may not be the only factor
    appropriately considered in a decision, it can be a useful
    decision-making tool.  The Congress requires CPSC to perform
    cost-benefit analysis before issuing certain regulations, and CPSC
    has conducted cost-benefit analysis for these regulations and in
    other situations in which it was not required.  Although perfectly
    complete and accurate data are rarely available for any analysis,
    CPSC's data are frequently inadequate to support detailed, thorough,
    and careful risk assessment and cost-benefit analysis.  In addition,
    CPSC's cost-benefit analyses are frequently not comprehensive, and
    the reports on these analyses are not sufficiently detailed.
    Improvements in the agency's methodology and in the quality of the
    underlying data are necessary to ensure the clarity and accuracy of
    CPSC's risk assessments and cost-benefit analyses.

          CPSC PERFORMS COST-BENEFIT
          ANALYSIS MORE OFTEN THAN
          REQUIRED BY LAW
    ---------------------------------------------------------- Letter :4.1

    Cost-benefit analysis can help decisionmakers by organizing and
    aggregating all the relevant information to clarify the nature of the
    trade-offs involved in a decision.  Although cost-benefit analysis
    may not be appropriately used as the sole criterion for making a
    decision, a well-constructed cost-benefit analysis can highlight
    crucial factors, expose possible biases, and facilitate informed
    decisions even when it is impossible to measure all the potential
    effects of a specific regulatory proposal.  The Congress has required
    CPSC to perform and publish a cost-benefit analysis when issuing a
    regulation (such as a mandatory standard or product ban) under the
    Consumer Product Safety Act.  In addition, CPSC is also required to
    conduct cost-benefit analyses before issuing regulations under the
    authority of portions of the Federal Hazardous Substances Act
    (specified labeling provisions are exempt from this requirement) and
    the Flammable Fabrics Act.\17

    Because most of the agency's projects do not involve mandatory
    regulation, relatively few CPSC projects conducted between January 1,
    1990, and September 30, 1996, were subject to these requirements.  We
    identified 8 cost-benefit analyses that CPSC performed in accordance
    with these requirements, and an additional 21 analyses it conducted
    in situations in which it was not required.  For example, CPSC
    performed cost-benefit analyses in eight instances in which it was
    considering issuing requirements for child-resistant packaging under
    the Poison Prevention Packaging Act, which does not require
    cost-benefit analysis.\18 CPSC frequently conducts cost-benefit
    analysis with respect to regulatory procedures, whether or not it is
    required to do so.  However, a complete cost-benefit analysis is done
    less frequently for voluntary standards projects or information and
    education efforts, although some economic information may be
    generated to assist such projects.  In addition to the complete
    cost-benefit analyses, we identified an additional 23 cases in which
    some information was provided on some economic benefits or costs.

    Before issuing a mandatory regulation, CPSC is required to consider
    the degree and nature of the risk of injury the regulation is
    designed to eliminate or reduce.  However, CPSC usually does not
    conduct a formal numerical risk assessment before issuing a
    regulation, and the law does not require it.  We found 24 risk
    assessments conducted by CPSC between January 1, 1990, and September
    30, 1996; only 4 of these were associated with regulatory action.

    --------------------
    \17 As an independent agency, CPSC is not required to comply with
    Executive Order 12866, which requires federal agencies to perform a
    cost-benefit analysis and submit it to the Office of Management and
    Budget (OMB) for review whenever the agency issues a regulation that
    is economically significant.

    \18 For child-resistant packaging regulations, CPSC is legally
    required to consider only whether requiring child-resistant packaging
    is "technically feasible, practicable, and appropriate."

          AGENCY DATA ARE GENERALLY
          INSUFFICIENT TO SUPPORT
          THOROUGH AND DETAILED
          ANALYSIS
    ---------------------------------------------------------- Letter :4.2

    Both risk assessment and cost-benefit analysis require extensive
    data.  Risk assessment requires information both on the adverse event
    and on exposure to the precipitating circumstances.  For example,
    when CPSC performed a risk assessment on floor furnaces, the agency
    estimated the number of previous injuries associated with floor
    furnaces and the number of floor furnaces in use.\19 Similarly, when
    CPSC performed a risk assessment to examine the risk of contracting
    cancer from dioxin traces in common paper products, the agency used
    information from laboratory studies on dioxin's link to cancer and
    also incorporated data on exposure to paper products.\20 Because
    cost-benefit analysis includes a comprehensive delineation of the
    expected effects of a given proposal, a careful and thorough
    cost-benefit analysis will also be very data intensive and rich with
    detail.

    CPSC's data systems are frequently unable to adequately meet the
    extensive demands for information posed by risk assessment and
    cost-benefit analysis.  As a result, the agency's estimates of risks,
    costs, and benefits are less accurate because they reflect the
    substantial limitations of the underlying data.  Available
    information does not permit us to determine the potential impact of
    better data on the results of CPSC's cost-benefit analyses and risk
    assessments.  Some of these data weaknesses tend to make product
    hazard risks seem larger, and other problems tend to make the same
    risks appear smaller.  For example, because CPSC's data undercount
    the deaths and injuries associated with particular consumer products,
    estimates of risk--and the potential benefits of reducing that
    risk--appear smaller.  However, CPSC's surveillance data provide
    information only on whether a product was involved in an accident,
    not on whether the product caused the accident.  At least at the
    initial stages of a project, this can make the risks assessed by
    CPSC--and the benefits of reducing those risks--appear larger.

    For risk assessment, CPSC must also obtain information on exposure to
    the hazard.  For example, to assess the risk associated with aluminum
    ladders, CPSC obtained estimates of the number of ladders available
    for use and on the number of times each year the ladder was used.
    Obtaining exposure data presents special challenges for CPSC.
    Because the product definition that relates to a particular hazard is
    often relatively narrow, existing data sources frequently offer
    insufficient detail.  For example, CPSC was unable to use Census
    sources to determine the number of saunas in the United States
    because saunas were included in a broader classification of products
    when government data were collected.  In addition, CPSC staff told us
    that it is often difficult to find accurate information on the number
    of products that are in households and available for use.

    CPSC sometimes responds to these challenges by using mathematical
    modeling techniques or easier-to-obtain proxy measures (such as
    population) to estimate product exposure.  In addition, for a few
    large-scale projects, CPSC has incurred the substantial expenses
    necessary to conduct its own detailed exposure survey.  For example,
    CPSC conducted a survey of households that asked detailed questions
    on matches and disposable cigarette lighters--the number purchased,
    where they are generally kept, how they are used, and other details.
    However, for the majority of the projects we reviewed, CPSC did not
    gather any data on exposure.  Of the 80 projects we reviewed for
    which briefing packages were prepared, only 26 included information
    on exposure to the hazard, and CPSC's risk assessments were confined
    to 24 cases between 1990 and 1996--approximately 21 percent of all
    projects conducted over that time period.

    --------------------
    \19 CPSC found that the risk of injury associated with floor furnaces
    was small in terms of the number of incidents but relatively higher
    when considered in relation to the number of products in use.

    \20 CPSC's study found that the risk from this type of dioxin
    exposure was negligible--at most, 5 cases per billion people
    exposed--and as a result no action was taken.

          CPSC COST-BENEFIT ANALYSES
          ARE OFTEN NOT COMPREHENSIVE
          AND NOT REPORTED IN
          SUFFICIENT DETAIL
    ---------------------------------------------------------- Letter :4.3

    The methodology used to conduct a cost-benefit analysis will
    frequently depend on the circumstances and the context of the
    analysis.  For this reason, there is no complete set of standards for
    evaluating the quality of an individual cost-benefit analysis.
    However, the professional literature offers some guidance for
    analysts, and certain specific elements are frequently mentioned as
    essential for cost-benefit analysis.  For example, because
    cost-benefit analysis is meant to be a complete delineation of the
    expected effects of a proposed action, all potential impacts (even
    those that cannot be quantified) should be discussed.  To ensure that
    the reader is able to make an informed judgment, it is important to
    be explicit about the underlying data, methodology, and assumptions.
    Accordingly, the literature suggests that all methodological choices
    and assumptions should be detailed, all limitations pertaining to the
    data should be revealed, and measures of uncertainty should be
    provided to allow the reader to take into account the precision of
    the underlying data.  Similarly, the literature calls for sensitivity
    analysis, which enables the reader to determine which assumptions,
    values, and parameters of the cost-benefit analysis are most
    important to the conclusions.

    On the basis of our review of the cost-benefit literature, we
    developed a list of the elements that are frequently used in
    evaluating cost-benefit analyses.  This list, and a description of
    all the factors we examined, are in appendix IV.\21

    Although we compared each of these elements with each of CPSC's
    analyses, not all elements were applicable to each case.  For
    example, in some cases, the circumstances indicated by a given
    element--such as reliance on statistical data--were not found, and
    those cases were treated as not applicable to that element.  In
    addition, for some elements it was not always possible to determine
    whether CPSC's analysis was consistent with the element.  For these
    reasons, and to emphasize those areas that we viewed as most
    critical, we reported only the evaluation results that relate to key
    elements, applied to the majority of CPSC's analyses, and for which a
    determination was possible in all or nearly all cases.

    Our review of all the cost-benefit analyses that CPSC conducted
    between January 1, 1990, and October 31, 1996, showed that for
    many--but not all--elements, CPSC's analyses were not comprehensive
    and not reported in sufficient detail (see table 2).  For example,
    CPSC provided descriptive information on proposals and also provided
    information on a variety of reasonable alternatives in almost 100
    percent of cases.  However, CPSC analyses generally did not provide
    measures of uncertainty for the underlying data.  Estimates derived
    from samples are subject to sampling error, which can be especially
    large when the estimates are projected from relatively fewer cases.
    In only 17 percent of its analyses did CPSC provide any statistical
    information on the precision of the underlying estimates.  Similarly,
    when estimates are based on a relatively small sample size,
    projections are generally not considered reliable.  CPSC analysts
    cautioned the reader against making conclusions based on small sample
    data only 45 percent of the time.  In addition, some of CPSC's data
    sets have a known upward or downward bias because of the way the data
    were constructed.  For example, CPSC's estimates of deaths based on
    CPSC's death certificate database will be understated, and when
    estimates of incidents are based only on investigated or reported
    cases (such as cases reported to CPSC's hot line), two potential
    biases are likely to be introduced into the analysis:  (1) the
    estimates are likely to be biased downward by nonreporting and (2)
    the incidents reported tend to be the more severe ones.  In only 53
    percent of applicable cases did CPSC's analysis inform the reader of
    known limitations inherent in the data being used for cost-benefit
    analysis.

                              Table 2

                 Evaluation of CPSC Analyses Shows
              Inconsistencies With Several Evaluation
                              Elements

                                                 Percentage of
                                                        CPSC's
                                                 analyses that
                                                          were
                                                    consistent
                                                     with this
    Evaluation element                                 element
    ------------------------------------------  --------------
    Provided descriptive information about a                98
     well-defined proposal
    Addressed multiple alternatives                         95
    Reported measures of precision for                      17
     underlying data
    Cautioned reader about making inferences                45
     from data with a small sample size
    Reported known biases in underlying data                53
    Provided any sensitivity analysis                       26
     information
    Included all important categories of                    54
     benefits and costs
    Considered "risk-risk" trade-offs\a                     49
    ----------------------------------------------------------
    \a A "risk-risk" trade-off refers to an action to decrease a hazard's
    risk that unintentionally increases that or another risk.

    We identified several other areas in which CPSC analyses could
    benefit from improvement.  For example, researchers agree that
    sensitivity analysis--a technique that enables the reader to
    determine which assumptions, data limitations, or parameters are most
    important to the conclusions--should be incorporated in cost-benefit
    analyses.  CPSC usually did not provide sensitivity analysis
    information.  For example, agency briefing packages did not include
    any information on how CPSC's injury cost estimates were derived or
    what factors were the largest components of injury costs.  CPSC
    applies a statistical model to injury estimates to derive a figure
    for injury cost.  The model that computed injury cost estimates
    accounts for a number of components, including medical costs, forgone
    wages, and pain and suffering.  With only one exception, CPSC
    briefing packages provided only the total cost, without any
    information on the derivation of those costs or the individual
    components.  In addition, CPSC provided only an average injury cost,
    not a range of injury cost estimates.  For situations in which
    injuries differ in severity, or for projects in which severity is
    probably overstated or understated in the data, the reader would find
    such information useful.

    Forty-six percent of CPSC analyses did not consider the full range of
    costs and benefits likely to result from regulation.  For example,
    CPSC analysts frequently omitted mentioning intangible costs and/or
    benefits (costs or benefits that are difficult to quantify, such as
    loss of consumer enjoyment) or potential indirect effects (such as
    changes in the prices of related goods).  In addition, CPSC
    frequently excluded risk-risk considerations from its evaluation of
    the costs and benefits of potential actions.  Sometimes actions taken
    to reduce one risk can have the unintended effect of increasing that
    or another risk.  Individuals may take more or fewer precautions in
    response to a change in a product's safety features, and this
    behavior can result in an increase in the risk the intervention was
    designed to mitigate.  For example, in establishing a standard for
    child-resistant packaging that was also "senior-friendly," CPSC
    considered that because child-resistant medicine bottles can be
    difficult to open, a grandparent may leave the cap off the bottle,
    creating an even greater risk than would be the case with the
    original cap.  Although CPSC considered such factors in some cases,
    only 49 percent of its analyses reflected potential risk-risk
    trade-offs.

    CPSC has not established internal procedures that require analysts to
    conduct comprehensive analyses and report them in sufficient detail.
    For example, according to CPSC staff, the agency has little written
    guidance about what factors should be included in cost-benefit
    analyses, what methodology should be used to incorporate these
    factors, and how the results should be presented.  Staff also told us
    that CPSC analyses are not generally subject to external peer review.
    Such reviews can serve as an important mechanism for enhancing the
    quality and credibility of the analyses that are used to help make
    key agency decisions.

    --------------------
    \21 These elements are also similar to OMB's guidance for preparing
    economic analyses of significant regulatory actions.

       CPSC HAS ESTABLISHED PROCEDURES
       FOR COMPLYING WITH STATUTORY
       REQUIREMENTS FOR RELEASING
       MANUFACTURER-SPECIFIC
       INFORMATION
    ------------------------------------------------------------ Letter :5

    To help minimize the possibility that a product might be unfairly
    disparaged, in section 6(b) of the Consumer Product Safety Act the
    Congress imposed restrictions on the disclosure of
    manufacturer-specific information by CPSC.\22

    Before CPSC can disclose any information that identifies a
    manufacturer,\23 the agency must

      -- take "reasonable steps" to verify the accuracy of the
         information and to ensure that disclosure is fair,

      -- notify the manufacturer that the information is subject to
         release, and

      -- provide the manufacturer with an opportunity to comment on the
         information.

    If the manufacturer requests that its comments be included in CPSC's
    disclosure of the information, CPSC can release the information only
    if it is accompanied by the manufacturer's comments.  If the
    manufacturer objects to the release even if its comments are
    included, it can challenge CPSC in U.S.  district court to block
    disclosure.  These restrictions on the release of information apply
    not only to information the agency issues on its own--such as a press
    release--but also to information disclosed in response to a request
    under FOIA.  In addition, section 6(b) also requires CPSC to
    establish procedures to ensure that releases of information that
    reflect on the safety of a consumer product or class of products are
    accurate and not misleading, regardless of whether the information
    disclosed identifies a specific manufacturer.

    CPSC has established procedures to implement these requirements,
    including requiring technical staff to "sign off" on information
    releases and notifying manufacturers.  Evidence from several
    sources--industry sources, published legal decisions, and agency
    retractions--suggests that CPSC has complied with its statutory
    requirements.  CPSC staff and commissioners, industry
    representatives, and consumer advocates expressed a wide variety of
    opinions on the effectiveness of these requirements, and some
    individuals favored specific changes.

    --------------------
    \22 An exception to these restrictions is given if CPSC has declared
    that the product is an "imminent hazard" under section 12 of the
    Consumer Product Safety Act.

    \23 These restrictions also apply even if the manufacturer is not
    named but the information would allow the reader to readily identify
    the manufacturer from the context.  For example, if there is only one
    manufacturer of a product that is identified in the information, the
    information may be subject to restriction even if the manufacturer's
    name is not given.

          CPSC HAS ESTABLISHED
          PROCEDURES FOR VERIFYING AND
          CLEARING INFORMATION FOR
          RELEASE
    ---------------------------------------------------------- Letter :5.1

    Part of CPSC's mission is to provide the public with information to
    help individuals use consumer products safely.  CPSC disseminates
    information through its own initiatives and also in response to
    requests from the public.  For example, CPSC informs both consumers
    and businesses about product hazards through product recall notices,
    provision of information at trade shows and special events, and a
    telephone hot line and an Internet site.  In addition, CPSC responds
    to thousands of telephone and written requests for information each
    year.

    CPSC's mission and its responsibility under FOIA require the agency
    to disseminate a great deal of information.  However, because much of
    this information is about specific products or manufacturers, CPSC's
    information disclosure is often restricted under section 6(b).  In
    its regulation implementing section 6(b), CPSC established several
    measures designed to ensure compliance with the statutory
    requirements.  These measures include obtaining written verification
    from consumers of the information they report to the agency,
    notifying manufacturers by certified mail when manufacturer-specific
    information is requested, and giving manufacturers the option of
    having their comments published along with any information being
    disclosed.

    CPSC's procedures outline several steps to verify all information
    before it is released.  For example, CPSC checks each report the
    agency receives from consumers about incidents involving potentially
    hazardous products to ensure that CPSC's records accurately reflect
    the consumer's version of the incident.  Agency procedures require
    staff to send a written description of each incident back to the
    person who reported it with a request that he or she review it and
    state if any information needs to be corrected or supplemented.\24
    The commission staff review each of these incident reports for
    discrepancies or any obvious inaccuracies.  Once they have been
    checked and confirmed with the consumer, incident reports are made
    available to the public upon request.  If the reports contain
    information that would identify a specific manufacturer, they are
    subject to 6(b) requirements regarding disclosure.

    CPSC also investigates events surrounding selected product-related
    injuries or incidents.  Investigation reports provide details about
    incident sequence, human behavior factors, and product involvement.
    The reports generally contain the consumer's version of what happened
    and the observations of witnesses, fire and police investigators, and
    others.  Investigations may also include follow-up inspections at
    retail stores or service centers.  However, neither investigations
    nor incident reports include the manufacturer's view of the incident.
    Its point of view may be expressed in comments it submits before the
    report is released.  Like incident reports, investigation files are
    available to the public upon request and are subject to 6(b)
    requirements.

    CPSC has issued clearance procedures to cover situations in which
    commissioners or staff initiate public disclosures--for example, when
    the Commission publishes the results of agency research.  These
    procedures are intended to verify any information--oral or
    written--released by the Commission, regardless of whether the
    information identifies a manufacturer.  Under CPSC's guidelines, each
    assistant or associate executive director whose area of
    responsibility is involved must review the information and indicate
    approval for the release in writing.\25 After all other review has
    been completed, the Office of the General Counsel must also review
    and approve the release.  Press releases with respect to product
    recalls are written and issued jointly by CPSC and the affected
    manufacturer.  In addition, CPSC's clearance procedures for press
    releases state that final clearance must be obtained from the Office
    of the Chairman of the Commission.\26 In addition, CPSC staff told us
    that the current chairman's policy of coordinating media inquiries
    through the Office of Public Affairs is intended to ensure that
    information provided is in compliance with section 6(b).

    CPSC has also established procedures to implement the notification
    and comment provisions of section 6(b).  Before CPSC releases
    information in response to an FOIA request, an information specialist
    determines whether a manufacturer could be readily identified.  CPSC
    staff said that agency policy is to clearly and narrowly identify
    hazardous products (including by manufacturer) whenever possible, in
    order to prevent the person receiving the information from confusing
    safe products with unsafe products of the same type.  However, if an
    information request is broad, like "all bicycle accidents," names of
    manufacturers are removed before the information is released,
    according to CPSC staff.  If the requested information could identify
    a manufacturer, then staff review the information for appropriate
    exemptions (such as trade secrets), and delete portions as
    appropriate.  The manufacturer is given 20 calendar days in which to
    review and comment on a summary of the information CPSC plans to
    release.\27 Because CPSC often receives multiple requests for the
    same information, the agency informs manufacturers that it will not
    send them copies of subsequent requests for the same information
    unless specifically requested to do so.  However, according to CPSC
    staff, more than 80 percent of the manufacturers that submit 6(b)
    comments routinely request such notification.  In calendar year 1993
    (the most recent year for which data were available), CPSC sent out
    487 notices to manufacturers and received 154 responses (32 percent).
    Twenty-five manufacturers (5 percent) contested the accuracy of the
    information or claimed that the proposed disclosure would be unfair.

    If a manufacturer fails to comment, the information can be released
    30 days from the date CPSC notified the manufacturer.  After taking
    the manufacturer's comments into account, CPSC may decide to disclose
    incident information despite the firm's objection if, for example,
    the comments lack specific information to support a claim of
    inaccuracy or a request for confidentiality.  If CPSC chooses to
    disclose information over a manufacturer's objection, it must release
    the manufacturer's comments along with the other information, unless
    the manufacturer requests otherwise.  In addition, if CPSC decides to
    release information and the manufacturer objects, the manufacturer
    has 10 working days to go to court and seek to enjoin CPSC from
    disclosing the information.  Manufacturers have sued CPSC to prohibit
    disclosure of records only 11 times since the agency was founded, and
    CPSC was prohibited from releasing the information in 2 of these
    cases.\28

    --------------------
    \24 The cover letter that accompanies the incident report also asks
    recipients if they would like to have their names withheld if
    information about the incident is made public.

    \25 For a description of CPSC's directorates, see app.  II.

    \26 An exception is made for those hazards considered by the
    Compliance staff as most serious, for which clearance must be
    obtained from a majority of the commissioners.

    \27 Since small companies may lack the technical ability to comment,
    CPSC staff told us that they will sometimes help small businesses to
    formulate their comments.

    \28 Currently, Daisy Manufacturing, Inc.  v.  CPSC is on appeal,
    after a judge ruled in CPSC's favor in U.S.  district court.

          EVIDENCE SUGGESTS THAT CPSC
          COMPLIES WITH LEGAL
          REQUIREMENTS REGARDING THE
          RELEASE OF INFORMATION
    ---------------------------------------------------------- Letter :5.2

    Information from three sources of evidence--industry, published legal
    cases, and data on retractions--suggests that CPSC complies with its
    statutory requirements concerning information release.  Industry
    sources, even those otherwise critical of the agency, told us that
    CPSC generally does a good job of keeping proprietary information
    confidential as required by law.  Our review of published legal
    decisions found no rulings that CPSC violated its statutory
    requirements concerning the release of information.  Retractions by
    CPSC are also rare.  If CPSC finds that it has disclosed inaccurate
    or misleading information that reflects adversely on any consumer
    product or class of consumer products or on any manufacturer, it must
    publish a retraction.  Any retraction must be made in the same manner
    in which CPSC released the original information.  According to CPSC,
    it has published only three such retractions.  Two of these
    retractions, in 1984 and 1994, were made in response to requests from
    firms.  A third retraction, in 1990, was issued after CPSC discovered
    that a report in its public reading room had mistakenly included
    inaccurate information.

          INDUSTRY OBSERVERS, CONSUMER
          GROUPS, AND OTHERS SUGGESTED
          CHANGES TO 6(B) REQUIREMENTS
    ---------------------------------------------------------- Letter :5.3

    Industry observers, CPSC staff, and consumer groups expressed a wide
    range of opinions on the effectiveness of section 6(b).  In response
    to our inquiries, some CPSC commissioners and former commissioners
    said that these restrictions serve a useful purpose and should not be
    changed.  However, CPSC's current chairman, industry and advocacy
    group representatives, and others expressed dissatisfaction with
    6(b), and some of these suggested possible changes.  Although these
    individuals raised issues about the extent of the protection afforded
    to manufacturers and the resources necessary to ensure compliance, we
    did not assess whether the specific suggestions were necessary or
    feasible.

    CPSC's chairman, other CPSC officials, former commissioners, and the
    representative of a consumer advocacy group stated that compliance
    with 6(b) is costly for CPSC and delays the agency in getting
    information out to the public.  Although CPSC has not estimated the
    cost of complying with 6(b), agency staff told us that it takes much
    more staff time to respond to FOIA requests that come under 6(b) than
    it does to respond to FOIA requests that do not involve company
    names.  To reduce the burden of complying with these requirements,
    CPSC staff have suggested that the notification requirement that
    gives manufacturers 20 days in which to comment should apply only the
    first time an item is released.  Some have suggested that instead of
    requiring CPSC to verify information from consumer complaints, the
    agency should be allowed to issue such information with an explicit
    disclaimer that CPSC has not taken a position on the validity of the
    consumer's report.

    Instead of reducing CPSC's verification requirements, some industry
    representatives suggested expanding them.  These manufacturers stated
    that before CPSC releases incident information, it should
    substantiate the information rather than relying on a consumer's
    testimony.  Industry representatives stated--and CPSC staff
    confirmed--that many of the requests for CPSC information come from
    attorneys for plaintiffs in product liability suits.  As a result,
    some industry representatives expressed concern that unsubstantiated
    consumer complaints could be used against them in product liability
    litigation.  They suggested that 6(b) should require CPSC to
    substantiate all incident reports by investigating them before they
    can be disclosed, instead of merely checking with the consumer as it
    does now.  However, CPSC officials told us that investigations--which
    are time consuming and costly--can be conducted only on a small
    proportion of specially selected cases because of limited resources.

    Industry representatives also said that the current restrictions do
    not provide sufficient protection when information is released on
    product groups instead of on the products of an individual
    manufacturer.  Several industry representatives expressed concern
    that producers of safer products may be unfairly maligned when CPSC
    releases information about a group of products, only some of which
    may be associated with a safety problem.  According to some of these
    industry representatives, CPSC should extend protection to product
    groups similar to the safeguards manufacturers receive under 6(b).

    Retailers' representatives also suggested specific changes to CPSC's
    information release requirements.  They said that retailers do not
    receive timely notice of recalls because CPSC has interpreted the law
    to prohibit advance notification of retailers.  Consequently, the
    retailers said that they sometimes receive notice of recalls at the
    same time as their customers and have no time to prepare.  For
    example, when consumers come in with recalled products, the retailer
    may not yet know whether the manufacturer has agreed to replace the
    product, refund the purchase price, or provide some other remedy.
    Retailers' representatives suggested amending 6(b) to give 5 business
    days' advance notice to retailers before the public announcement of a
    recall.  CPSC officials said that typically manufacturers are and
    should be the ones to contact the retailers and make all arrangements
    for a recall.  Although they disagreed on the need for a statutory
    change, both CPSC staff and a major retailers' association said that
    they were trying to work out a more satisfactory arrangement.

       CONCLUSIONS
    ------------------------------------------------------------ Letter :6

    CPSC's current data provide insufficient information to monitor
    ongoing projects and to determine whether potential projects adhere
    to the agency's selection criteria.  Moreover, inadequate agency data
    often prevent CPSC from conducting risk assessments on projects,
    potentially limiting the agency's ability to target resources to the
    hazards presenting the greatest risks.  The lack of sufficient data,
    combined with methodological problems, also makes CPSC's cost-benefit
    analyses less useful than they could be.  With more detailed
    information on both internal resources and external product hazards,
    CPSC would be better able to assure the Congress and taxpayers that
    its resources are expended wisely.

    We identified several key areas where CPSC management could improve
    its collection and analysis of external data.  For example, CPSC
    would be better able to make informed decisions on potential agency
    projects if it had additional statistically reliable and timely data
    in several areas, including (1) injuries treated outside of hospital
    emergency rooms; (2) exposure to consumer products and
    product-related hazards; (3) chronic illnesses related to consumer
    products; and (4) hazards that disproportionately affect certain
    vulnerable populations, such as low-income individuals and consumers
    with disabilities.

    In addition, project selection and implementation could be improved
    if commissioners and staff had tracking information on CPSC projects,
    such as starting and ending dates, project origin, project costs
    (including staff days and contract costs), and agency actions taken
    to address the potential hazard.  Such information could assist the
    commissioners in monitoring ongoing projects, targeting new efforts
    on the basis of previous work, and assessing the allocation of
    resources across current projects and among major hazard areas.

    CPSC could also benefit from an improved methodology for cost-benefit
    analysis.  A stronger methodological base for CPSC's cost-benefit
    analyses, including more complete documentation, would promote sound
    regulatory decision-making and could improve the quality of the input
    and comment CPSC receives during the regulatory process.

    Without these improved data, CPSC will remain unable to accurately
    apply measurable criteria in choosing projects or to rigorously
    assess relative risks among potential hazards.  Some of this
    necessary information�the need for more representative and complete
    injury and exposure data, for example�may require a significant
    investment of resources, so CPSC may need to prioritize these data
    needs.  In doing so, is important for CPSC to draw on the insight of
    individuals outside the agency to ensure that all available
    alternatives for obtaining these data are explored.  Some of the
    other information CPSC needs, however, could be compiled internally
    at relatively little additional cost and effort.  For example, more
    detailed information on individual projects could be collected within
    the agency.  Similarly, the methodological problems we identified in
    CPSC's cost-benefit analysis could be remedied without additional
    data.

       RECOMMENDATIONS
    ------------------------------------------------------------ Letter :7

    We recommend that the Chairman of CPSC take the following actions:

      -- Improve the quality of CPSC's injury, death, and exposure data
         by consulting with experts both within and outside the agency to
         (1) prioritize CPSC's needs for additional statistically valid
         surveillance data on injuries and deaths related to consumer
         products and on exposure to consumer products and
         product-related hazards, (2) investigate the feasibility and
         cost of alternative means of obtaining these data, and (3)
         design data systems to collect and analyze this information.

      -- Direct agency staff to develop and implement a project
         management tracking system to compile information on current
         agency projects.  For each project, such a system should
         include, at a minimum, a description of the hazard addressed,
         start and end dates, project origin, and major agency action
         resulting from it.

      -- Direct agency staff to develop and implement procedures to
         ensure that all cost-benefit analyses performed on behalf of
         CPSC are comprehensive and reported in sufficient detail,
         including providing measures of precision for underlying data,
         incorporating information on all important costs and benefits,
         and performing sensitivity analysis.

       AGENCY COMMENTS
    ------------------------------------------------------------ Letter :8

    We received two separate sets of comments from CPSC's
    commissioners--one from Chairman Brown and Commissioner Moore and one
    from Commissioner Gall.  CPSC staff also submitted some technical
    comments, which we incorporated in the report as appropriate.

    Chairman Brown and Commissioner Moore stated that they are
    considering our recommendations and that in some respects our
    recommendations parallel efforts already under way at the Commission.
    However, they disagreed with some of the specific findings of our
    report.  They stated that CPSC's actions are based on solid injury
    and death estimates and that CPSC (1) employs sound economic analyses
    that are appropriate for the circumstances, (2) tracks projects to
    monitor the progress of its work, and (3) has been successful in
    dramatically reducing the threat to consumers from unsafe products.
    We concluded that CPSC's death and injury data are generally
    insufficient to support the agency's project selection process for
    two reasons:  (1) CPSC has little or no data on several project
    selection criteria and (2) CPSC's data on its other project selection
    criteria exhibit significant gaps.  Similarly, our finding that
    CPSC's cost-benefit analyses were not comprehensive and not reported
    in sufficient detail supports our conclusion that these analyses were
    less useful than they could have been in the agency's decision-making
    process.  Because CPSC's current management information system
    operates at a very high level of generality and (according to CPSC
    staff) does not produce consistent, reliable information, we
    recommend that the agency implement an improved tracking system that
    would provide enough information to monitor the projects selected and
    the resources spent for each hazard.  We did not review whether
    CPSC's actions had been successful in reducing the number of injuries
    and deaths associated with consumer products.  The full text of
    Chairman Brown and Commissioner Moore's detailed comments and our
    response are in appendix VI.

    Commissioner Gall stated in her comments that she is looking forward
    to implementing many of the reforms we recommended.  She also stated
    that she agrees with our conclusion that CPSC could improve the way
    it gathers information, including reassessing the need for injury
    data gathered outside of hospital emergency rooms.  Commissioner Gall
    also agreed that CPSC needs an improved system to track ongoing
    activities or projects and that sensitivity analysis, measures of
    uncertainty, and risk-risk analysis should be incorporated into CPSC
    analyses.  However, she also commented that additional GAO analysis
    of the implications of inadequate data could have been helpful.
    Unfortunately, available information does not permit us to determine
    the impact of better-quality data on the decisions made by CPSC.  In
    addition, Commissioner Gall said that she believes the Compliance
    function of CPSC also warrants further review.  Such a review is
    outside the scope of this report.  Commissioner Gall's detailed
    comments and our response are in appendix VII.

    ---------------------------------------------------------- Letter :8.1

    As agreed with your office, unless you publicly announce its contents
    earlier, we plan no further distribution of this report until 30 days
    after its issue date.  At that time, we will send copies to the
    commissioners of the Consumer Product Safety Commission and make
    copies available to others upon request.

    If you or your staff have any questions about this report, please
    contact me at (202) 512-7014.  Major contributors are listed in
    appendix VIII.

    Carlotta C.  Joyner
    Director, Education
     and Employment Issues

    SCOPE AND METHODOLOGY
    =========================================================== Appendix I

    This report provides information on project selection, analytical
    research, and information release procedures at the Consumer Product
    Safety Commission (CPSC), which we gathered from a variety of sources
    inside and outside the agency.  For example, we interviewed current
    CPSC staff, including analysts and line managers, to obtain
    information about CPSC's data and procedures.  We also interviewed
    the three current CPSC commissioners and four former commissioners.
    In addition, we interviewed representatives of manufacturers,
    retailers, trade associations, consumer groups, and academic and
    other experts to obtain their perspectives on CPSC's activities.  We
    reviewed the legislation governing CPSC's activities and the
    associated regulations.  We also reviewed statements by public and
    private organizations and the academic research literature dealing
    with consumer protection issues and the technical literature on
    cost-benefit analysis and risk assessment.

    We gathered extensive information from CPSC documents.  We reviewed
    every agency budget request, operating plan, mid-year review, and
    regulatory agenda from 1990 to 1996.  We reviewed all CPSC's annual
    reports from fiscal years 1986 to 1996.  We also obtained all CPSC
    Federal Register notices from January 1993 to May 1996 and selected
    Federal Register notices from January 1990 to December 1992.  We
    reviewed all CPSC press releases from September 1992 to February
    1996.  We also reviewed 644 CPSC briefing packages prepared from
    January 1, 1990, to September 30, 1996, including all briefing
    packages that concerned a specific, identifiable potential product
    hazard under consideration for regulatory, voluntary, or
    information-gathering activities.  We excluded matters exclusively
    related to compliance with and enforcement of existing standards,
    civil penalties, and internal management issues as well as other
    items not related to a specific potential product hazard.  In
    addition, we reviewed other agency documents, including
    information-clearing procedures, documentation of data systems,
    consumer education materials, internal memorandums and
    correspondence, and selected documents downloaded from CPSC's
    Internet site.

    In describing CPSC's project selection, we drew on our interviews
    with commissioners, former commissioners, and agency staff, and we
    reviewed the project selection criteria in CPSC's regulations.
    Although we described the criteria used by the agency, we did not
    make judgments about the merit of individual criteria or about
    whether any criterion was appropriately applied in any specific case.
    We compiled a list of agency projects on the basis of the documents
    we reviewed.  We included not only major regulatory efforts but also
    smaller-scale projects.  We gathered information on the
    characteristics of these projects; for example, we obtained
    information on how the projects originated, what action CPSC took,
    and (for most) how many deaths and injuries were associated with the
    hazard.  CPSC officials examined our list and provided some
    information that was not readily available in agency documents.
    However, because CPSC's internal data management system does not
    track agency projects, neither we nor CPSC were able to verify the
    accuracy and comprehensiveness of this information.  We also reviewed
    agency documentation and the technical literature and interviewed
    outside experts to obtain information on CPSC's data-gathering
    systems.

    To examine CPSC's cost-benefit analysis, we reviewed the technical
    literature and developed a set of objective evaluation questions to
    elicit descriptions of and to evaluate the analytical work.  These
    evaluation questions were designed to indicate only whether CPSC's
    cost-benefit analyses were consistent with elements that are commonly
    used to evaluate whether cost-benefit analyses are comprehensive and
    reported in sufficient detail.  Our review did not assess whether
    CPSC's analyses were the best that could be done on any particular
    topic.  We reviewed these evaluation questions with two leading
    experts in the field of risk assessment and cost-benefit analysis:
    Professor John Mendeloff of the University of Pittsburgh and
    Professor John Graham of Harvard University.  Then we examined all
    CPSC's cost-benefit analyses from January 1990 through September 1996
    to see how they measured up against these evaluation questions.  In
    making these assessments, we reviewed all the information available
    to the commissioners in the written record; that is, we did not
    confine our analysis to the portion of the briefing packages that
    dealt explicitly with cost-benefit analysis or risk assessment, and
    we examined all the briefing packages that pertained to that
    particular project.  Although we evaluated CPSC's cost-benefit
    analyses using a wide-ranging set of evaluation questions, we
    reported results of our analysis only for those questions where:  (1)
    the question was applicable to the majority of the CPSC analyses we
    reviewed; and (2) we were able to make a determination of whether the
    analyses were consistent with the evaluation question in applicable
    cases.

    To review CPSC's information release process, we reviewed internal
    agency procedures, discussed the information release requirements
    with agency officials and industry representatives, and reviewed the
    relevant legal cases.  We provided information on CPSC's information
    release procedures but did not audit whether CPSC complied with these
    procedures in releasing information.  Instead, we relied on other
    sources--industry sources, published legal decisions, and
    retractions--to assess whether this readily available evidence
    suggested that CPSC had violated its statutory requirements in
    releasing manufacturer-specific information.

    Our review was conducted between August 1996 and May 1997 in
    accordance with generally accepted government auditing standards.

    AGENCY ORGANIZATION AND STRUCTURE
    ========================================================== Appendix II

    The Consumer Product Safety Act (P.L.  92-573) provides for the
    appointment of five commissioners by the President of the United
    States for staggered 7-year terms.  Not more than three of the
    commissioners may be affiliated with the same political party.  The
    President appoints the commissioners, subject to Senate confirmation.
    However, the President cannot directly overrule an agency decision or
    fire a commissioner for making an unpopular decision.  Since 1986,
    there have been no more than three commissioners at one time.  In
    fulfilling the provisions of the government in the Sunshine Act (P.L.
    94-409),\29 in general, commissioners must open to the public any
    meeting held for the purpose of disposing of CPSC business--that is,
    if any two of the three commissioners want to discuss agency
    business, generally public notice must first be given.  On a daily
    basis, communication among CPSC offices takes place at the staff
    level.

    The chairman, the principal executive officer of the Commission,
    directs all executive and administrative functions of the agency.
    The chairman oversees the appointment and supervision of all agency
    personnel except those employed in the immediate offices of other
    commissioners.  CPSC annually elects a vice chairman to act in the
    absence or disability of the chairman or in case of a vacancy in the
    office of chairman.

    As figure II.1 shows, six offices report directly to the chairman:

      -- the Office of the Secretary manages the agency's records,
         publishes CPSC's public meetings calendar, and administers the
         requirements of the Freedom of Information Act;

      -- the Office of Congressional Relations serves as CPSC's liaison
         with the Congress;

      -- the Office of General Counsel provides advice and counsel to the
         agency on legal matters;

      -- the Office of the Inspector General is an independent office
         that undertakes activities to prevent and detect waste, fraud,
         and abuse; and

      -- the Office of Equal Opportunity and Minority Enterprise monitors
         compliance with equal opportunity employment laws.

    The agency's executive director, who is appointed by and reports
    directly to the chairman, is responsible for overseeing the
    day-to-day management of the agency.  The executive director's office
    also houses the agency's small business ombudsman, who acts as a
    liaison to small businesses, providing technical assistance
    concerning CPSC programs and regulations, among other things.

    The executive director has line authority over nine operating
    directorates:

      -- The Office of Compliance is responsible for compliance with and
         administrative enforcement of CPSC regulations, including
         product standards.  The office initiates investigations on
         safety hazards of products already in the consumer marketplace
         or being offered for import.  It negotiates and subsequently
         monitors corrective action plans designed to give public notice
         of hazards and recall defective or noncomplying products.

      -- The Office of Hazard Identification and Reduction manages many
         of CPSC's activities that involve identifying, examining, and
         remedying new product hazards.  The office's responsibilities
         include collecting and analyzing data to identify hazards and
         hazard patterns, carrying out CPSC's regulatory and voluntary
         standards development projects, and coordinating international
         activities related to consumer product safety.  Serving in
         various groups under the director of hazard identification and
         reduction are the epidemiologists and statisticians, who provide
         information on particular products; engineers and human factors
         specialists, who help develop design remedies for product
         hazards; and economists, who provide market information and
         perform cost-benefit analysis on commission projects.

      -- The Office of Information and Public Affairs is CPSC's
         touchstone with consumers and the media.  It prepares and
         publishes brochures, booklets, fact sheets, and safety alerts
         providing safety information to consumers on products used in
         the home environment.  The Office of Information and Public
         Affairs also handles requests from the media for information and
         access to CPSC staff and prepares press releases announcing CPSC
         actions or decisions.

      -- The Office of Information Services manages the agency's
         toll-free hot line, Internet, and fax-on-demand services.
         Within the Office of Information Services, CPSC's Information
         Clearinghouse provides summary information on product-related
         injuries in response to requests from the public.\30

      -- The Office of the Budget is responsible for overseeing the
         development of CPSC's budget.

      -- The Office of Human Resources Management provides support to the
         agency in recruitment and placement, position classification,
         training, and other personnel areas.

      -- The Office of Planning and Evaluation assists with long-term
         planning efforts and manages and ensures agency compliance with
         paperwork reduction regulations.  In addition, the Office of
         Planning and Evaluation is currently preparing for the
         implementation of the Government Performance and Results Act
         (the Results Act) and reviewing the effectiveness of the
         agency's outreach efforts.

      -- The Directorate for Field Operations coordinates the activities
         of CPSC's 128 field staff located in 38 cities across the
         country.  CPSC field staff carry out a wide range of agency
         activities, including conducting investigations of injury
         incidents; acting as liaisons with state and local
         organizations; working with the local press to support consumer
         education campaigns; and inspecting manufacturers, importers,
         distributors, and retailers to ensure compliance with safety
         regulations and standards.  To complement the efforts of the
         field staff at the local level, CPSC contracts some product
         safety work to state and local entities; commissions state and
         local officials to function as officials of CPSC for the purpose
         of conducting investigations, inspections, recalls, and sample
         collections; and transmits information on CPSC programs and
         activities to states.

      -- The Directorate for Administration is responsible for executing
         general administrative policies.

       Figure II.1:  CPSC Organization
       Chart

       (See figure in printed
       edition.)

    --------------------
    \29 The purpose of the Sunshine Act is to provide the public with
    information regarding the decision-making processes of the federal
    government while protecting both the rights of individuals and the
    ability of the government to carry out its responsibilities.

    \30 If such a request cannot be filled without identifying the
    manufacturer, however, the request is sent to the freedom of
    information officer for processing.

    CPSC PROJECTS, 1990-96
    ========================================================= Appendix III

    CPSC has addressed many different product hazards.  We identified 115
    projects the agency worked on between January 1, 1990, and September
    30, 1996.  Because CPSC does not maintain a list of projects it has
    worked on or any project characteristics, such as project origin or
    result, we used various agency documents, such as budget requests,
    annual operating plans, regulatory agendas, and project briefing
    documents, to compile our list.

    We attempted to determine how each project came to the attention of
    the agency, for example, through a petition, congressional direction,
    or internal initiation.  This was not always clear.  For example, in
    three cases, a petition was submitted and subsequently denied, but
    the issue was later mandated by the Congress as a project the agency
    must address.  For example, a petition concerning the safety of
    bicycle helmets submitted in 1990 was denied.  In 1994, the Congress
    passed the Children's Bicycle Helmet Safety Act of 1994, which
    requires CPSC to establish a mandatory standard.  We counted such
    projects as originating through the Congress.  In other cases, we
    categorized projects on the basis of information in CPSC documents.

    We also attempted to identify the most significant action that
    resulted from the agency's efforts.  Some projects were ongoing for
    several years and involved more than one agency action, such as
    information campaigns conducted in conjunction with voluntary
    standards efforts.  For example, agency work on carbon monoxide (CO)
    detectors resulted in a voluntary standard in 1992, a "CO Awareness
    Week" information campaign in 1994, and public hearings on the issue
    in 1996.  In cases such as these, we attempted to identify the
    activity with the greatest long-lasting impact; usually this was a
    voluntary or mandatory standard.  Also, projects classified as
    "voluntary standard activity" include those for which a new voluntary
    standard was created, an existing voluntary standard was revised, or
    staff are currently participating in voluntary standard activities.

    In addition, we identified whether a risk assessment or cost-benefit
    analysis was completed for a given project, using CPSC briefing
    packages.  We defined a formal risk assessment to include only those
    cases for which a numerical estimate of unit risk was calculated; we
    defined a complete cost-benefit analysis to include those projects
    for which both economic costs and benefits were explicitly compared,
    even if quantitative estimates were not made for all economic factors
    considered.  Some risk-related or economic information was provided
    for many of the projects for which no formal, complete risk
    assessment or cost-benefit analysis was performed.

                                                                         Table III.1

                                                                    CPSC Projects, 1990-96

                                       Origin                                           Agency action                              Analytical efforts
                      ----------------------------------------  -------------------------------------------------------------  --------------------------
                                                                                                                                      Cost-
                                                                                           Voluntary   Information                  benefit          Risk
                      Initiated by                Initiated by               Mandatory      standard           and                 analysis    assessment
    Product           the Congress      Petition        agency       Ban      standard    activity\a     education    Other\b          done          done
    ----------------  ------------  ------------  ------------  --------  ------------  ------------  ------------  ---------  ------------  ------------
    Fire/gas codes and standards
    -----------------------------------------------------------------------------------------------------------------------------------------------------
    Camping heaters                                          x                                     x
    Carbon monoxide                                          x                                     x
     detectors
    Children's                                               x                       x                                                    x
     sleep-wear
     flammability
    Cigarette                                  x                                     x                                                    x             x
     lighters
    Cigarette safety             x                                                                                          x
    Elderly                                                  x                                                              x
     sleepwear
     flammability
    Fire safety                                              x                                                              x
     devices
    Fireworks, fuse-                         x\c                                     x                                                    x
     burn time
    Fireworks, large                                         x         x                                                                  x             x
     and small
     reloadable
     shell
    Fireworks,                                               x                       x                                                    x
     multiple tube
     mine and shell
    Flexible gas                                             x                                     x
     connectors
    Floor furnaces                             x                                                                            x                           x
    Fuel gas                                                 x                                                              x
     detectors
    Gas control                                              x                                     x
     valves,
     automatic
    Gas furnaces                                             x                                     x
    Gas grills, 20-                                          x                                     x
     lb. systems
    Gas systems,                                             x                                     x
     over-
     pressurization
    Heaters,                                                 x                                     x
     unvented gas
     space heaters
    Liquified                                                x                                     x
     petroleum (LP)
     gas odorant
     fade
    Mattress/                                                x                                                              x
     bedding fires
    Multipurpose                               x                                                                          x\d             x
     lighters
    Range/oven fires                                         x                                                              x
    Smoke detectors                                          x                                     x
    Upholstered                                x                                                 x\e
     furniture
    Water heaters                                            x                                     x                                      x

    Sports and Recreation
    -----------------------------------------------------------------------------------------------------------------------------------------------------
    All-terrain                                              x                                                            x\f                           x
     vehicles (ATV)
    Baseball                                                 x                                                              x
     injuries\g
    Baseball bases                                           x                                     x
    Baseball, chest                                          x                                     x
     protectors for
     young batters
    Baseball, face-                            x                                                   x                                      x             x
     guards on
     helmets
    Baseball, safety                                         x                                     x
     baseballs
     (soft)
    Bicycles                                                 x                                                   x                                      x
    Bicycle                                                  x                                                              x
     reflectors
    Bicycle/                   x\h                                                   x                                                                  x
     recreation
     helmets
    In-line skates                                           x                                                   x
    Model rocket                               x                                                                            x
     motors
    Pools, barriers                                          x                                     x
     for swimming
     pools and spas
    Pools, diving                                            x                                     x
     injuries
    Pools, swimming                            x                                                   x
     pool covers
    Saunas                                     x                                                   x
    Soccer injuries                                          x                                                              x
    Soccer goals                                             x                                     x
    Spas and hot                               x                                                   x                                                    x
     tubs

    Electrical/power codes and standards
    -----------------------------------------------------------------------------------------------------------------------------------------------------
    Garage door                  x                                                   x                                                  x\i
     openers
    Go-carts                                                 x                                     x
    Hair dryers                                x                                                   x
    Heaters,                                   x                                                                            x
     portable
     electric
    Heat tapes                                 x                                                                            x             x
    Holiday lights                                           x                                     x
    Home electrical                                          x                                     x
     system fires
    Ladders,                                   x                                                                            x             x             x
     portable
     aluminum
    Lamps, portable                                          x                                     x
    Microwave ovens                                          x                                     x
    Mowers, riding                                           x                                     x                                      x             x
    Mowers, walk-                                            x                                                              x
     behind
    Receptacle                                               x                                     x
     outlets
    Shock protection                                         x                                     x                                                    x
     devices (GFCI)
    Thermoplastics                                           x                                                              x
     in electrical
     products

    Children's products
    -----------------------------------------------------------------------------------------------------------------------------------------------------
    Baby bath seats                                          x                                                   x
    Baby walkers                               x                                                 x\d                                      x             x
    Bean bag chairs                                          x                                     x
    Bunk beds                                  x                                                   x
    Child restraints                           x                                                                 x                        x             x
     on grocery
     carts
    Choking hazards-             x                                                   x                                                    x
     -balloons,
     balls, marbles,
     small figures,
     pom-poms
    Clacker balls                              x                                     x                                                    x
    Crib corner post                           x                                                   x                                                    x
     extensions
    Cribs, non-                                              x                                     x
     full-size
    Crib toys                                  x                                                   x
    Drawstrings on                                           x                                     x
     children's
     clothing
    Electronic video                           x                                     x                                                    x
     games
    5-gallon buckets                                         x                                     x                                                    x
    Furniture                                                x                                     x
     tipover
    High chairs                                              x                                     x
    Infant cushions                                          x         x                                                                  x
    Infant                                                   x                                                   x
     suffocation
    Lead in paint                                            x                                                              x
    Playground                                 x                                                   x                                      x             x
     equipment, home
    Playground                                               x                                     x                                                  x\j
     equipment,
     public
    Playground                                               x                                     x
     equipment, soft
    Playground                                               x                                     x
     surfacing
    Play yards                                               x                                     x
    Strollers                                                x                                     x
    Toddler beds                                             x                                     x
    Window blind                                             x                                     x
     cords
    Window guards                                            x                                     x

    Chemical/poison prevention
    -----------------------------------------------------------------------------------------------------------------------------------------------------
    Acetone in 1-                            x\k                                                                            x
     gallon
     containers
    Architectural                              x                                                                            x             x
     glazing
    Art materials                x                                                   x
    Charcoal                                   x                                     x                                                    x             x
     labeling
    Child-                                                   x                       x                                                    x
     resistant,
     adult-friendly
     packaging of
     drugs
    Child-resistant                            x                                     x
     packaging of
     certain dietary
     products with
     iron powders
    Child-resistant                                          x                       x                                                    x
     packaging of
     dibucaine
    Child-resistant                                          x                       x                                                    x
     packaging of
     ibuprofen
    Child-resistant                                          x                       x                                                    x
     packaging of
     drugs with
     loperamide
    Child-resistant                          x\l                                     x                                                    x
     packaging of
     glue removers
     with
     acetonitrile
    Child-resistant                          x\l                                     x                                                    x
     packaging of
     hair wave
     neutralizers
    Child-resistant                                          x                       x                                                    x
     packaging of
     lidocaine
    Child-resistant                            x                                     x                                                    x
     packaging of
     mouthwashes
     with ethanol
    Child-resistant                                          x                       x
     packaging of
     naproxen
    Child-resistant                          x\m                                                                            x
     packaging of
     spot remover
     with naphtha
    Coal-and                     x                                                                                          x                           x
     woodburning
     stove emissions
    Dichlorobenzene                                          x                                                              x                           x
    Dioxin in paper                                          x                                                              x                           x
     products
    Exemption to                               x                                                                            x
     child-
     resistant
     packaging of
     mouthwashes
     with ethanol
    Indoor air                   x                                                                                          x
     quality
    Indoor air                   x                                                                                          x                           x
     quality--
     carpet
     emissions
    Indoor air                   x                                                                 x
     quality--
     formaldehyde in
     pressed wood
     products
    Indoor air                   x                                                                                          x
     quality--
     glycol ethers
    Indoor air                   x                                                                                          x
     quality--
     heating,
     ventilation,
     and air-
     conditioning
     systems
    Indoor air                   x                                                                 x
     quality--
     kerosene
     heaters
    Indoor air                   x                                                                 x
     quality--
     portable room
     humidifiers
    Lead poisoning                                           x                                     x
     abatement
    Methylene                                  x                                                                            x                           x
     chloride in
     paint strippers
    Sulfuric acid                              x                                                                            x
     drain cleaners

    Other
    -----------------------------------------------------------------------------------------------------------------------------------------------------
    Waterbeds                                  x                                                   x                                                    x
    =====================================================================================================================================================
    Total 115                   13            34            68         2            21            55             5         32            29            24
    -----------------------------------------------------------------------------------------------------------------------------------------------------
    \a "Voluntary standard activity" includes efforts in which agency
    staff participated in the development of a new voluntary standard or
    revision of an existing voluntary standard.  This participation may
    have included regularly attending meetings of a standards development
    group; taking an active part in discussions; and/or providing
    research or other support, such as information and education programs
    or administrative assistance.

    \b "Other" includes projects that were completed with no agency
    action, terminated, currently in process or pending, or completed
    with a study or report.

    \c Two petitions were submitted for fuse-burn time.  The first,
    submitted in 1991, was denied.  The second, submitted in 1996,
    resulted in a change to an existing mandatory standard.

    \d CPSC published an Advance Notice of Proposed Rulemaking (ANPR).
    This is the first step toward enacting a mandatory standard.

    \e CPSC published an ANPR in 1994, which is the first step toward
    enacting a mandatory standard.  The ANPR also solicited offers to
    develop or modify a voluntary standard.  During fiscal year 1996,
    CPSC worked with an industry group on voluntary standards
    development.

    \f CPSC has worked with states to develop state ATV legislation.

    \g The baseball injuries project started in the 1980s and resulted in
    a special report on baseball injuries and protective equipment.  The
    projects--baseball bases, chest protectors, faceguards on helmets,
    and safety baseballs--all grew out of this umbrella project and, as
    such, are counted separately.

    \h A petition was also submitted in 1990.  The Congress mandated this
    project in the Children's Bicycle Helmet Safety Act of 1994.  Project
    origin is counted as congressionally initiated.

    \i The cost-benefit analysis for garage door openers applied to the
    labeling and record-keeping requirements only, not to the provisions
    of the safety standard.

    \j The risk assessment for playground equipment examined the hazard
    associated with exposure to arsenic from wooden equipment.  The risk
    assessment concluded that the risk was small.  Most of CPSC's efforts
    concerning playground equipment are related to injuries from falls or
    impacts of equipment and are not related to chemical exposure.

    \k Petition, submitted in 1995, was withdrawn.

    \l One petition was submitted in 1988 by the American Association of
    Poison Control Centers (AAPCC) requesting CPSC to require
    child-resistant packaging both for household glue removers containing
    acetonitrile and home cold wave permanent neutralizers containing
    sodium bromate or potassium bromate.

    \m Petition, submitted in 1991, was withdrawn.

    A REVIEW OF THE LITERATURE ON
    COST-BENEFIT ANALYSIS
    ========================================================== Appendix IV

    Cost-benefit analysis can be described as an analytical technique
    that details the expected positive and negative effects (expected
    benefits and costs) of a given policy proposal.  To construct this
    framework, researchers approach a policy question needing
    considerable information about the potential remedy.  Because
    cost-benefit analysis requires an inclusive approach to evaluating a
    proposal's impact, the proposal itself must be well defined, and some
    information must be known about its impact.  Once the expected
    benefits and costs of the proposal are thoroughly identified and
    delineated, the next step is to place a value on each benefit or
    cost.  Frequently, these expected costs and benefits are expressed in
    numerical or monetary terms to facilitate a comparison of the
    aggregate costs and benefits.  If all relevant factors can be
    translated into monetary terms, the decision rule suggested by this
    proceeding is to accept the proposal if its aggregate benefits exceed
    its aggregate costs.\31

    Although at first the concept behind cost-benefit analysis seems
    relatively straightforward, the application of cost-benefit analysis
    is not.  The practical difficulties associated with measuring
    effects, quantifying results, and accounting for uncertainty (to name
    only a few issues) can create a gap between the way cost-benefit
    analysis is described in theory and the way it is implemented in
    practice.  For this reason, experts generally agree that analysts
    should be comprehensive in including all important factors and be
    explicit in their description of the underlying data, methodology,
    and assumptions.

    This appendix describes many of the major methodological issues that
    often arise in cost-benefit analysis and outlines specific elements
    that are frequently used to evaluate cost-benefit analyses.  The
    methodology used to conduct a cost-benefit analysis frequently varies
    depending on the circumstances and the context of the analysis.  For
    this reason, there is no complete set of standards for evaluating the
    quality of an individual cost-benefit analysis.  However, the
    professional literature offers some guidance for analysts, and
    certain specific elements are frequently used to determine whether a
    given analysis meets a minimum threshold of comprehensiveness and
    openness.  These elements are necessary, but not sufficient, for a
    quality analysis.

    From the extensive cost-benefit literature, we developed objective
    evaluation questions to evaluate cost-benefit analyses performed by
    CPSC.  These evaluation questions are summarized in table IV.1.
    Although these evaluation questions are not a comprehensive measure
    of the quality of an analysis, they were designed to reflect whether
    an analysis is comprehensive and reported in sufficient detail.  We
    applied these questions to each of CPSC's analyses, but not all
    evaluation questions were applicable to each case.  In addition, for
    some questions it was not always possible to determine whether CPSC's
    analysis complied with the particular element reflected in the
    question.  For these reasons, and to emphasize those areas that we
    considered as most critical, we reported only the evaluation results
    that related to key elements and that applied to the majority of
    CPSC's analyses; these eight are shown in bold in table IV.1.

                             Table IV.1

               Frequently Used Elements of Evaluation
                      of Cost-Benefit Analysis

    Number    Evaluation question
    --------  ------------------------------------------------
    Evaluating potential courses of action
    ----------------------------------------------------------
    1         Was descriptive information given about well-
              defined alternative courses of action?

    2         Was more than one alternative course of action
              considered in the analysis?

    3         Was evidence given to support the degree to
              which the proposal was assumed to mitigate the
              problem?

    Considering all important factors for analysis
    ----------------------------------------------------------
    4         Were all important categories of potential costs
              and benefits included in the analysis? (For
              example, did the analysis include [where
              applicable] intangible benefits and costs,
              health and safety benefits, costs of compliance,
              upfront costs, price changes, and changes in
              consumer surplus?)

    5         Were the effects of intangible costs and
              benefits detailed explicitly, even if they could
              not be quantified? (For example, did the
              analysis include [where applicable] consumer
              utility?)

    6         Did the analysis show how the existence of
              intangible costs or benefits could affect the
              conclusions?

    7a        Were possible indirect effects discussed? (For
              example, did the analysis include [where
              applicable] price changes of related goods or
              likely changes in market concentration?)

    7b        If indirect effects were measured
              quantitatively, were the calculations and
              assumptions behind this measurement discussed in
              detail?

    7c        If indirect effects were measured qualitatively,
              did the analysis show how the existence of these
              factors could affect the conclusions?

    Considering interdependent factors
    ----------------------------------------------------------
    8         Were possible issues of standing identified?

    9         Was an incremental (marginal) analysis of costs
              and benefits performed?

    10        Were major policy interdependencies identified?
              (For example, did the analysis discuss the
              implications if another agency also had
              jurisdiction over the product, or if the
              proposal could create a conflict with existing
              government policy?)

    Considering distribution issues
    ----------------------------------------------------------
    11        Did the analysis make clear the distribution of
              gains and losses? (For example, who would pay
              the costs and who would get the benefits of the
              proposal?)

    12a       Was the impact of distribution of gains and
              losses assessed? (For example, would the
              proposal be likely to have a greater adverse
              impact on small businesses, potentially leading
              to increased market concentration?)

    12b       Were distributional weights employed in the
              analysis?

    12c       If weights were used, was information provided
              about the basis for those weights?

    13a       Is the standard assumption that relative market
              prices are insensitive to the policy change
              likely to hold in this case--that is, are there
              likely to be few or no macroeconomic effects?

    13b       If no, was this impact discussed or considered
              in the analysis?

    Considering risk-risk factors and potential offsetting
    behavi
    ----------------------------------------------------------
    14        Were risk-risk or offsetting behavior concerns
              identified and considered in the analysis?

    Using the willingness-to-pay approach to value risk
    reduction
    ----------------------------------------------------------
    15        Were willingness-to-pay measures used to value
              reductions in the risk of death or injury?

    16        Is the numerical value of a statistical life
              used in this analysis consistent with the
              literature?

    17        Is the numerical value of a reduction in the
              chance of injury consistent with the literature?

    Discounting future costs and benefits
    ----------------------------------------------------------
    18a       Are costs and benefits that occur at different
              points discounted for differences in timing?

    18b       Does the discount rate or rates used include the
              suggested rate by the Office of Management and
              Budget?

    18c       Does the discount rate or rates used include the
              Treasury bill rate for the time horizon of the
              analysis?

    18d       Does the discount rate or rates used include a
              lower "social" discount rate?

    Considering data issues
    ----------------------------------------------------------
    19a       Where applicable, were the implications of
              unverified or unverifiable data provided by an
              interested party identified and discussed?

    19b       Where applicable, were the implications of
              uncertainty surrounding a dose-response model
              identified and discussed?

    19c       Where applicable, were the implications of
              uncertainty surrounding cross-species
              extrapolation identified and discussed?

    19d       Where applicable, were the implications of the
              use of data relying on investigated or reported
              cases identified and discussed?

    19e       Where applicable, were the implications of
              survival bias in the underlying data identified
              and discussed?

    19f       Where applicable, were the implications of small
              sample sizes identified and discussed?

    19g       Where applicable, were the implications of other
              known biases in the underlying data identified
              and discussed?

    20        If the underlying data were derived from a
              statistical sample, were appropriate measures of
              precision provided?

    Performing sensitivity analysis
    ----------------------------------------------------------
    21a       Was sensitivity analysis performed on any
              parameter in the analysis?

    21b       Was sensitivity analysis performed on the value
              of a statistical life?

    21c       Was sensitivity analysis performed on the value
              of injury reduction?

    21d       Was sensitivity analysis performed on the
              discount rate?

    21e       Was sensitivity analysis performed on the
              precision of the underlying data?

    21f       Was sensitivity analysis performed on other
              important parameters?
    ----------------------------------------------------------
    Note:  We reported results for questions that are in bold.  We viewed
    these measures as especially important, they applied to the majority
    of CPSC analyses, and a determination could be made for these
    questions in each case.

    --------------------
    \31 Variations of cost-benefit analysis include cost-effectiveness
    analysis, which relates the costs and outcomes of alternative methods
    to achieve the same desired goal.  For example, one
    cost-effectiveness analysis compared the cost per case detected for
    different methods of treating the same medical condition.  Because
    only cost-benefit analysis is relevant to CPSC's activities, related
    techniques are not considered here.  For more information on these
    and other related analytical tools, see Michael F.  Drummond, Greg L.
    Stoddart, and George W.  Torrance, Methods for the Economic
    Evaluation of Health Care Programmes (New York:  Oxford University
    Press, 1996).

       BACKGROUND
    -------------------------------------------------------- Appendix IV:1

    As an organizing framework, cost-benefit analysis can help a
    decisionmaker to organize and aggregate all the relevant information
    in a way that can clarify the nature of the trade-offs involved in a
    decision.  At the same time, by providing a framework to convert
    dissimilar effects to a common measurement, cost-benefit analysis can
    allow this information to be weighed and aggregated to help make a
    decision.  A well-constructed cost-benefit analysis can highlight
    crucial factors, expose possible biases, and perhaps expand the
    openness of the decision-making process by clarifying the factors on
    which the decision was based--whether these factors are purely
    economic criteria or include other social factors.  A key advantage
    of carefully built cost-benefit analysis is that it promotes explicit
    rather than implicit decision-making, even when it is impossible to
    monetize or even quantify all the potential effects of a given
    regulatory proposal.

    Despite the value of this analytical tool, in some situations using
    cost-benefit analysis as the sole basis for decision-making may be
    inappropriate.  For example, alternatives that would eliminate basic
    human rights or dramatically increase income inequality may be viewed
    as morally unacceptable.  Similarly, when uncertainty about possible
    effects is so pervasive that attempting to identify potential costs
    and benefits would amount to nothing more than uninformed
    speculation, cost-benefit analysis probably has little to offer.\32
    In addition, for minor decisions it may not be necessary to employ a
    rigorous, time- and resource-consuming decision-making process, and a
    less detailed cost-benefit analysis--or none at all--may be more
    appropriate.  Virtually all observers agree that the appropriate role
    for cost-benefit analysis--sole decision-making rule, input into
    decision-making, or not done at all--will depend on the context in
    which the particular decision is being considered.

    To some individuals, application of an abstract analytical technique
    such as cost-benefit analysis is especially unpalatable in certain
    situations.  For example, the idea of applying a value to �saving
    life� may be distasteful, because we regard our own lives as
    �priceless� or of infinite value.  However, although our lives may be
    priceless, avoiding risks often means forgoing time, convenience,
    enjoyment, or other opportunities--all of which do have a price.
    Thus, policy interventions that can affect life expectancy pose an
    unavoidable problem--to refuse to consider the value of potentially
    lifesaving interventions is to implicitly value them at zero, and to
    consider any potentially lifesaving activity as infinitely valuable
    implies that individuals would never take any action that involves
    risk (driving a car, for example).  The literature on cost-benefit
    analysis makes a key distinction in this area.  For ethical reasons,
    most practitioners consider it inappropriate to use cost-benefit
    analysis to evaluate alternatives that would (with certainty) affect
    the life expectancy of a given, known individual.  But government
    policy does not usually involve making such decisions.  Instead,
    policy questions typically center on actions that may bring about
    small changes in the statistical life expectancy of anonymous members
    of a large group.  For example, when CPSC considered imposing a
    mandatory regulation on large, multiple-tube fireworks, the agency
    estimated that such a standard could reduce the number of individuals
    that die in related accidents by one anonymous consumer over a 3-year
    period.  On a daily basis, each of us makes such trade-offs between
    perfect safety and other things we value, such as the convenience of
    driving a car or the excitement of skiing down a mountain.  Such
    assessments, which place a value on reductions in risk, are viewed as
    appropriate, whereas valuing a given person's life may be viewed as
    less reasonable.

    Similarly, a decision to surrender basic rights--such as liberty--may
    be unacceptable on moral grounds, and so cost-benefit analysis might
    not be applicable in a situation involving such rights.  In addition,
    some individuals object to cost-benefit analysis in circumstances in
    which other species may be affected adversely.  For example, one
    individual objected to a proposal that would allow increased
    pollution in a particular river because the proposal would adversely
    affect the fish and other species in the river.  Cost-benefit
    analysis would probably be unable to address this objection because
    there is no accepted method to place a value on the losses suffered
    by the fish.

    --------------------
    \32 However, one should use caution in assuming that this situation
    applies.  Sometimes the conclusions of a cost-benefit analysis can be
    quite robust even in the face of substantial uncertainty.  For
    example, a recent analysis of the benefits and costs of reducing
    emissions of acrylonitrile concluded that imposing strict standards
    on acrylonitrile emissions fails reasonable cost-benefit tests,
    despite as much as a threefold factor of uncertainty in acrylonitrile
    emissions, additional uncertainty in the amount of exposure to
    acrylonitrile, and a probable but unmeasurable upward bias in the
    estimate of cancer risk.  (See John A.  Haigh, David Harrison, Jr.,
    and Albert L.  Nichols, "Benefit-Cost Analysis of Environmental
    Regulation:  Case Studies of Hazardous Air Pollution," ch.  1 in The
    Moral Dimensions of Public Policy Choice:  Beyond the Market
    Paradigm, eds., John Martin Gilroy and Maurice Wade (Pittsburgh, Pa.:
    University of Pittsburgh Press, 1992), pp.  15-57.

       CONCEPTUAL, ANALYTICAL, AND
       TECHNICAL ISSUES ARISE IN
       APPLICATION OF COST-BENEFIT
       ANALYSIS
    -------------------------------------------------------- Appendix IV:2

    Over the past 3 decades, a substantial economic literature has
    explored the conceptual, analytical, and technical issues posed by
    the application of cost-benefit analysis.  In this literature, some
    elements that cover a wide range of research methods are generally
    accepted.  As one experienced researcher has said, �Good studies
    follow procedures that are in accord with economic theory for
    estimating benefits and costs, provide a clear statement of all
    assumptions, point out uncertainties where they exist, and suggest
    realistic margins of error.�\33 Even when undertaken by careful and
    competent researchers, cost-benefit analysis can sometimes be
    difficult to interpret, especially when uncertainty is substantial
    and information is incomplete.

    --------------------
    \33 See Eban Goodstein, "Benefit-Cost Analysis at the EPA," The
    Journal of Socio-Economics, 24 (2) (1995), pp.  375-89.

          CONCEPTUAL ISSUES IN
          COST-BENEFIT ANALYSIS
    ------------------------------------------------------ Appendix IV:2.1

    Some of the issues underlying the application of cost-benefit
    analysis are conceptual, and the researcher may make a different
    judgment in different circumstances.  With respect to these issues,
    being complete and explicit is important--the consensus in the
    literature is that while there may be no single method for dealing
    with these issues that is universally appropriate, the researcher
    must be clear and direct in detailing how the issue was addressed in
    the context of the analysis.

             INTANGIBLE OR
             UNQUANTIFIABLE FACTORS
             CAN POSE DILEMMAS FOR
             RESEARCHERS
    ---------------------------------------------------- Appendix IV:2.1.1

    Ideally, a cost-benefit analysis involves translating each impact
    into a common measurement (such as dollars) for comparison.  However,
    some effects may be difficult or impossible to measure or quantify.
    For example, a researcher evaluating an alternative outpatient mental
    health treatment program realized that the greater independence
    afforded by the outpatient program could create increased
    anxiety--or, alternatively, higher self-esteem--for some
    participants.  However, these intangible effects would be very
    difficult to measure and even more difficult to quantify.\34 Some
    individuals have criticized certain individual practitioners of
    cost-benefit analysis for ignoring or de-emphasizing aspects of
    proposed changes that cannot be easily quantified.  Although it is
    necessary for such effects to be described and emphasized
    appropriately, the existence of such intangibles does not necessarily
    limit the value of the analysis.  For example, in the discussion of
    mental health programs, the researchers found that patients in the
    experimental program experienced higher satisfaction and reported
    having a greater number of social relationships.  These qualitative
    benefits were important to the analysis even though they could not be
    valued in dollars.  In addition, if it can be shown explicitly that
    the value of the intangibles would be unlikely to change the
    conclusion, then cost-benefit analysis has played a valuable role by
    considering intangible effects explicitly, even if it is not possible
    to consider them quantitatively.  (We addressed this issue in
    questions 5 and 6 of table IV.1.)

    --------------------
    \34 See Burton A.  Weisbrod, "Benefit-Cost Analysis of a Controlled
    Experiment:  Treating the Mentally Ill," Journal of Human Resources,
    16 (4) (1981), pp.  523-48.

             MEASURING INDIRECT AND
             SECONDARY COSTS AND
             BENEFITS CAN BE DIFFICULT
    ---------------------------------------------------- Appendix IV:2.1.2

    Researchers must also decide whether or not to include indirect or
    secondary effects that may result from the proposal.  For example, a
    change in tax rules may not only have an initial, direct effect on
    individuals' income but may also create a secondary ripple (or
    �multiplier�) effect on the economy as a whole.  Similarly, a medical
    treatment that prolongs life can be expected to have a secondary
    effect on health care costs as individuals live longer.

    A researcher who wants to include such secondary impacts will
    frequently find a measurement challenge, because determining the
    magnitude (or perhaps even the existence) of many secondary effects
    involves answering a �what if?� type of question, and frequently
    without relevant historical experience.  For example, individuals who
    receive improved treatment for hypertension may contract more
    prolonged or costly illnesses in the future.\35 Any attempt to
    measure the impact of these indirect future costs would require the
    researcher to predict the future health status and health care costs
    of the patient group in the event that the treatment had or had not
    been available.  In addition, the existence of some secondary effects
    is dependent on one or more key assumptions.  For example, an
    economic multiplier effect is generally thought to take place only in
    an economy operating below its productive capacity.  A researcher who
    wants to include such an effect must thus determine whether or not
    this assumption holds, and then decide how large an effect should be
    included.

    Secondary impacts may be considered part of the cost-benefit analysis
    or extraneous to it.  The proper choice in each circumstance probably
    depends on the likelihood that the secondary effects in fact exist,
    their probable importance, and the ability to measure them.  However,
    like intangible effects, potential secondary or indirect effects
    should be detailed even if they are not included quantitatively.
    Furthermore, should such effects be included, the assumptions
    underlying their existence and measurement must be revealed
    explicitly to allow the reader to make an informed judgment.  (We
    addressed this issue in question 7 of table IV.1.)

    --------------------
    \35 For an example of such an analysis, see W.B.  Stason and M.C.
    Weinstein, "Allocation of Resources to Manage Hypertension," The New
    England Journal of Medicine, 296 (13) (1977), pp.  732-39.

             ISSUES OF "STANDING" MAY
             REQUIRE CLARIFICATION OF
             THE SCOPE OF THE ANALYSIS
    ---------------------------------------------------- Appendix IV:2.1.3

    In some circumstances, it may be difficult to define the point of
    view from which the cost-benefit analysis is calculated--that is, who
    has "standing," or whose costs and benefits should be included in the
    analysis.  For most cost-benefit calculations involving government
    policy, the analysis is appropriately done at the level of �society�
    (rather than, for example, considering only the impact on individuals
    in a particular state or locality), so that a wide range of
    implications is considered.  However, in some circumstances, it is
    important to make the point of view explicit.  For example, when
    policy effects may cross national boundaries, �society� may be best
    defined on a broader basis.  If the cost-benefit analysis is
    considering policies to reduce acid rain, for example, considering
    the effects of acid rain and pollution reduction on the Canadians who
    share U.S.  weather patterns may be important.  Similarly, analysts
    may wish to consider the impact of product regulation on foreign
    producers as well as on U.S.  consumers.

    Even when the unit of analysis is clear, occasionally some members of
    that unit are not afforded standing under cost-benefit analysis.  For
    example, suppose that a particular policy proposal is likely to
    result in a decrease in property crime.  If a thief steals a
    television, and both the victim and the thief are given standing,
    then the gain to the thief could offset the loss to the victim in a
    cost-benefit accounting.  �Society,� however, is clearly worse off.
    Unless we try to measure the psychological and sociological costs of
    crime, it may make more sense not to afford the thief standing in the
    cost-benefit analysis.  Such issues arise infrequently but should be
    made clear if they could affect the interpretation of the analysis.
    (We addressed this issue in question 8 of table IV.1.)

             TAKING THE STATUS QUO AS
             BASELINE, COST-BENEFIT
             ANALYSIS SHOULD ACCOUNT
             FOR RELEVANT FACTORS AND
             CONSTRAINTS
    ---------------------------------------------------- Appendix IV:2.1.4

    Some cost-benefit analyses have been criticized for excluding
    relevant factors on what appears to be an ad hoc basis.  Without
    including such factors, or providing an explicit justification for
    excluding them, a researcher limits the value of the analysis.
    However, while it is important to consider a variety of alternative
    actions, it is equally crucial to adopt a realistic view of the
    possible alternatives.  For example, political and agency realities
    often place meaningful constraints on options available for
    consideration.  If these constraints are not incorporated into the
    analysis, the results could be �pie in the sky� recommendations that
    would be of little use to the decisionmaker.  (We addressed these
    issues in questions 1, 2, 4, and 10 of table IV.1.)

    Similarly, it is important that cost-benefit analysis be conducted in
    a way that evaluates the change in aggregate costs and benefits, with
    current conditions serving as a baseline.  For the properties of
    economic efficiency to hold, cost-benefit analysis must take what
    economists call a marginal or incremental approach--that is, it must
    consider only the changes that would result from the proposed
    intervention.  For example, a proposal to renovate a museum should be
    evaluated, for cost-benefit purposes, on the basis of the incremental
    cost of the renovation, not on the basis of the original cost of
    constructing the entire museum.  (We addressed this issue in question
    9 of table IV.1.)

             ISSUES OF DISTRIBUTION
             MAY BE CONSIDERED WITHIN
             OR OUTSIDE THE
             COST-BENEFIT CONTEXT
    ---------------------------------------------------- Appendix IV:2.1.5

    By definition, cost-benefit analysis is a method for considering the
    aggregate effects of a given proposal.  If the aggregate expected
    benefits of the proposal exceed the aggregate expected costs, the
    proposal is said to pass the cost-benefit test.  This does not
    necessarily mean that adopting the proposal would improve the
    condition of each individual; instead, it implies only that when the
    expected gains and losses to all individuals are added up, the total
    expected gains exceed the total expected losses.\36

    By adding up individual gains and losses to determine the effect on
    society as a whole, cost-benefit analysis implicitly assumes that
    each individual's gains or losses should be valued equally with any
    other individual's gains or losses.  As a result, cost-benefit
    analysis is �neutral� with respect to the distribution of gains and
    losses.  To put it another way, the �fairness� of how gains and
    losses are distributed is generally not included in the calculations
    underlying a cost-benefit analysis.

    Sometimes a decisionmaker might want to address such issues of
    fairness.  For example, if a proposal would involve redistributing
    income, the loss of one dollar to a wealthy person may be viewed as
    less consequential than the gain of one dollar to a poor person.\37
    In order to address such an issue, the researcher or decisionmaker
    can consider issues of distribution inside or outside the
    cost-benefit context.  One method of examining such issues might be
    to analyze the distributional impact of a proposal separately, and
    consider fairness issues--along with the cost-benefit calculation--as
    another factor in the decision-making process.  Under these
    circumstances, if the proposal is viewed as having negative
    distributional consequences, and it would be costly or difficult to
    redistribute the gains and losses, then the proposal might be
    rejected even though it would otherwise meet a cost-benefit test.

    Occasionally a researcher who wants to consider distributional issues
    explicitly chooses to incorporate distributional consequences into
    the mathematics of the cost-benefit calculation.  Instead of simply
    adding up the costs and benefits accruing to each individual, the
    researcher uses a mathematical formula that applies different weights
    to different individuals.  These weights could be based on a number
    of factors, depending on the characteristics of the �fairness� issues
    being addressed.  For example, if the proposal would affect income
    directly, weights could be based on income (with greater weight being
    applied to gains in income experienced by low-income individuals).
    Weights could also be based on other circumstances; for example,
    greater weight could be placed on vulnerable populations or on future
    generations.  Similarly, if a change was proposed in a given program,
    greater weight could be applied to the smaller number of program
    participants (who would be more significantly affected) than the rest
    of society (who would be affected only indirectly).  A weighting
    scheme would be most helpful to decisionmakers and outside observers,
    however, if the formula was detailed explicitly and the analysis was
    accompanied by a sensitivity analysis, so the effect of the
    distributional weights was clear to the reader.  (We addressed these
    issues in questions 11 and 12 of table IV.1.)

    --------------------
    \36 Economists make a distinction between events that lead to a
    "Pareto improvement" and those that lead only to a "potential Pareto
    improvement." If a change leads to a Pareto improvement, then as a
    result of the change no individual will be worse off, and at least
    one individual will be better off.  However, if a change leads to a
    "potential Pareto improvement," then the aggregate gains exceed the
    aggregate losses, but some individuals will gain and others will
    lose.  In this situation, it would be theoretically possible for the
    "winners" to compensate the "losers" and leave no individual worse
    off than before if redistribution itself is costless.

    \37 There is some limited ground in utilitarian economic theory for
    this assumption.  Economic theory states that individuals will
    generally, at some point, experience �diminishing marginal utility of
    income�--that is, receiving one additional dollar of income brings
    less additional satisfaction (or �utility�) as the individual becomes
    wealthier.  However, the amount of satisfaction deriving from any
    good or service, including income, cannot be compared across
    individuals.  As a result, while economic theory suggests that it is
    plausible that at some point giving one dollar to a rich person
    produces less total �utility� than giving one dollar to a poor
    person, economic theory also tells us that it is generally not
    possible to measure the circumstances under which this assumption
    might (or might not) be correct.

          IMPLEMENTATION ISSUES IN
          COST-BENEFIT ANALYSIS
    ------------------------------------------------------ Appendix IV:2.2

    While a number of conceptual issues arise in cost-benefit analysis
    for which the appropriate answer depends on circumstances, there are
    also a number of methodological or implementation issues about which
    there is widespread agreement.  For example, years of debate on the
    appropriate valuation method for risk reduction have largely been
    resolved in the favor of �willingness to pay� measures, and the
    importance of sensitivity analysis is generally recognized.  While
    the choice of discount rate has long been a subject of controversy,
    with some practitioners arguing for the use of market rates and
    others advocating a lower �social� discount rate, the literature
    generally recognizes the value of multiple rates.  Finally, there is
    virtually universal agreement on the importance of reliable data and
    careful risk measurement to cost-benefit analysis.

             COSTS AND BENEFITS SHOULD
             BE ADJUSTED FOR
             DIFFERENCES IN TIMING
    ---------------------------------------------------- Appendix IV:2.2.1

    Frequently, the various consequences of the proposal under discussion
    will differ in when they occur.  For example, changing the labeling
    requirements on bags of charcoal could reduce carbon monoxide deaths
    years later and also result in an immediate, one-time increase in
    industry costs.  As individuals and as a society, we generally prefer
    to have dollars or resources now than at some time in the future
    because we can benefit from them in the interim.  In addition, if we
    acquire $1 next year instead of today, we give up the opportunity to
    invest that dollar and earn interest on our investment.  As a result,
    it is generally agreed that future dollar cost and benefit streams
    should be reduced or "discounted" to reflect differences in timing.

    The rate at which this adjustment is made is usually done on a
    case-by-case basis.  Many analysts choose to use the market rate of
    interest--for example, the rate payable on government bonds with a
    time horizon comparable to the analysis.  Some researchers have
    argued that the discount rate should be set somewhat higher because,
    presumably, economic growth will leave future generations better off.
    Others have advocated using a "social" discount rate, which is
    generally lower than the market rate, on the grounds that society's
    interest in the welfare of future generations implies that the
    discount rate for all of society should be lower than the rates
    chosen by individuals.  Experts on cost-benefit analysis generally
    encourage researchers to use multiple rates--both to assess the
    sensitivity of the results to the chosen discount rate and to
    facilitate comparison across studies.  (We addressed issues of
    discounting in question 18 of table IV.1.)

             COMPARABILITY WITH OTHER
             ANALYSES COULD INFLUENCE
             SELECTION OF PARAMETERS
    ---------------------------------------------------- Appendix IV:2.2.2

    Ideally, proponents of extensive cost-benefit analysis would like to
    be able to order and prioritize regulatory interventions across
    agencies.  A potential mechanism for this type of coordination is
    provided in Executive Order 12866, which requires agencies to submit
    to the Office of Management and Budget (OMB) cost-benefit analyses
    for regulations with an estimated impact on the economy of $100
    million or more.\38 OMB has suggested or prescribed certain rules
    (such as a presentation format and/or a specific discount rate) to
    facilitate comparability across agencies and to help promote quality
    analyses.\39 While a common set of assumptions or parameters may
    facilitate comparability by a central government authority, so that
    priorities may be set across as well as within agencies, some of
    these assumptions may fit the circumstances of the given analyses
    less well.  For example, a given discount rate may be more
    appropriate for one project (for which the benefits and costs are
    spread out over a long period) than another (with costs and benefits
    spread over a shorter period).  Therefore, there may be a tension
    between making an analysis comparable to others and customizing it to
    fit a unique situation.  (We addressed these issues in questions 10
    and 18 of table IV.1.)

    --------------------
    \38 CPSC, as an independent agency, is not required to follow these
    procedures.

    \39 The evaluation factors we examined parallel OMB's suggestions; in
    some areas, we have operationalized these concepts to create more
    specific and objective evaluation questions.

             POTENTIAL FUTURE PRICE
             CHANGES CAN ADD
             UNCERTAINTY TO ANALYSIS
    ---------------------------------------------------- Appendix IV:2.2.3

    Typically, cost-benefit analysis assumes that the intervention
    considered has only a small or negligible effect on relative prices
    throughout the economy.  Generally, this assumption makes sense;
    however, for major changes (such as a big change in tax law) the
    assumption is clearly inappropriate.  In these cases, it is much
    harder to put a value on the potential impact of the proposed change.
    For example, a large cut in the capital gains tax could affect a wide
    range of investment behavior, and may even affect how much some
    individuals choose to work and how much they are paid.  A researcher
    trying to place a value on the potential consequences of a capital
    gains tax cut might introduce an error by valuing changes in
    productivity using the wage rates that prevailed before the cut,
    because, in reality, once the new policy has been implemented, wage
    rates might change.  In such a situation, it is especially important
    for the researcher to point out the potential mismeasurement, and (if
    feasible) try to model its effect.  (We addressed these issues in
    question 13 of table IV.1.)

             RESEARCHERS RECOGNIZE THE
             IMPORTANCE OF SENSITIVITY
             ANALYSIS TO INFORM THE
             READER
    ---------------------------------------------------- Appendix IV:2.2.4

    Every cost-benefit analysis will include some level of uncertainty or
    imprecision, or reflect a methodological choice that not everyone
    will necessarily agree with.  Careful analysts will identify critical
    sources of uncertainty or controversy, and revise or test the
    analysis quantitatively or qualitatively to identify how or whether
    these areas affect the conclusions reached.  If large variations in
    measurement or assumption do not alter the conclusions, then the
    researcher and the decisionmaker can have greater confidence in the
    original results.  However, if the conclusions of the analysis can
    change depending on methodological choices or variable measurement,
    then the researcher may want to try to improve the measurement of the
    original variables.  If this is not possible, then the analysis may
    be of more limited value--an implication the decisionmaker would need
    to know.  (We addressed these issues in question 21 of table IV.1.)

             RELIABLE UNDERLYING DATA
             CRUCIAL TO RELIABLE
             COST-BENEFIT ANALYSIS
    ---------------------------------------------------- Appendix IV:2.2.5

    Despite the importance of sensitivity analysis, even the most careful
    and elaborate sensitivity analysis often cannot sufficiently
    compensate for poor underlying data.  Obviously, if key variables are
    defined poorly or measured imprecisely, the quality of the
    cost-benefit analysis will suffer.  Because many variables used in
    research are measured from surveys or samples, sampling error may be
    unavoidable.  The researcher can use a sensitivity analysis to test
    the potential for sampling error to affect the conclusions; however,
    if the survey instrument itself is flawed, the results may be
    unreliable even beyond the degree indicated by sampling error.
    Similarly, researchers may face a difficult problem when key data
    (such as the cost of compliance) are provided for the analysis by an
    interested party (such as the industry).  If the interested party
    provides the data, that party may have an incentive to provide
    biased, inaccurate, or misleading information.  In such cases, it is
    important to try to verify the data to the extent possible; if not,
    the researcher needs to at least note the source of the underlying
    information and allow the reader to make an informed judgment about
    the reliability of the final analysis.  (We addressed potential data
    issues in questions 19 and 20 of table IV.1.)

          MEASURING AND VALUING RISK
          REDUCTION UNDER COST-BENEFIT
          ANALYSIS
    ------------------------------------------------------ Appendix IV:2.3

    Frequently, the benefits of a particular proposal involve a reduction
    in risk of injury or death.  In these situations, the quality of the
    cost-benefit analysis will also depend substantially on the estimates
    of the proposal's impact on these risks.  However, it is often
    difficult to accurately measure and value risk reduction.  A number
    of issues arise, including the effect of the proposal on the level of
    risk (especially when individual behavior is a factor in determining
    the size of the risk) and measuring individuals' willingness to take
    risks in exchange for rewards.

             CONSIDERABLE UNCERTAINTY
             OFTEN SURROUNDS RISK
             MEASUREMENT AND RISK
             ASSESSMENT
    ---------------------------------------------------- Appendix IV:2.3.1

    Measuring the effect of a proposal on the level of risk can be
    especially difficult.  Generally, it is not realistic to assume that
    any regulatory intervention will reduce the risk level to
    zero--removing an environmental carcinogen, for example, will
    probably reduce the number of cancer deaths, but is unlikely to
    eliminate cancer entirely.  Considerable uncertainty may be
    inevitable when extrapolating a dose-response model.  Sometimes large
    doses of a potentially dangerous substance, given over a short period
    of time, are used to predict the results of long-term exposure.  When
    the model moves across species--for example, predicting cases of
    cancer in humans based on experiments with laboratory
    animals--another source of uncertainty is introduced.

    Measurement problems in the underlying data can make it hard to
    predict future risk from epidemiological data as well.  For example,
    when longitudinal data are used, there may arise a "survival bias" in
    that the analysis could be biased if it excludes individuals who die
    or otherwise move out of the data set.  Similarly, when some cases go
    unreported, the data may understate the size and/or overstate the
    severity of the hazard.  For rare hazards, limited variation in the
    epidemiological data can make measurement and prediction of risk more
    difficult.  In addition, some data may be known to produce biased
    estimates (because of exclusions or potential double-counting, for
    example).  Data problems such as these (and the statistical or
    analytical methods used to deal with them) should be pointed out to
    the reader to increase the opportunity for informed judgment.  (We
    addressed these issues in questions 3 and 19 of table IV.1.)

             RISK-RISK TRADE-OFFS AND
             INDIVIDUALS' BEHAVIORAL
             RESPONSES TO POLICY
             CHANGES CAN AFFECT THE
             LEVEL OF RISK
    ---------------------------------------------------- Appendix IV:2.3.2

    Sometimes actions taken to reduce one risk can have the unintended
    effect of increasing that or another risk.  For example, unforeseen
    consequences arose in the 1970s when CPSC issued regulations
    requiring children's sleepwear to meet flammability standards.
    Manufacturers used a chemical called Tris to meet these standards,
    but later it was discovered that Tris posed a cancer risk.  Changes
    in individual behavior can also create uncertainty in predicting the
    level of risk because valuations of costs and benefits of a proposed
    action are often based on historical behavior.  For example,
    valuations of a proposed new highway may be based on the number of
    individuals currently traveling the route, their commuting times, and
    other factors.  However, once the new highway is built, some people
    may make extra trips in the area (shopping in different stores, for
    example) than they did before.  Some people who had driven at
    off-hours to avoid congestion on the old road may now travel at peak
    hours on the new one.  Similarly, changes in public policy may cause
    individuals or firms to change the technology they use, the amount of
    time they spend at leisure or at work, or the amount they invest in
    innovation.

    A special type of behavioral change�often referred to as �offsetting
    behavior�--occurs when individuals change the amount of precautions
    they take in response to a change in policy.  For example, having an
    air bag in the car or being required to wear a motorcycle helmet
    might make some drivers feel safer, so they exercise less caution on
    the road.  Sometimes this offsetting behavior can result in an
    increase in the risk the intervention was designed to mitigate.  For
    example, because child-resistant medicine bottles can be difficult to
    open, a grandparent may leave the cap off the bottle, creating an
    even greater risk than would be posed with a non-child-resistant cap.
    If such changes in consumer behavior are foreseeable, the analysis
    will be improved if the researcher points out such possibilities, and
    it will be even more useful if reasonable attempts can be made to
    measure the impact.  (We addressed these factors in question 14 of
    table IV.1.)

             WILLINGNESS-TO-PAY
             MEASURES VIEWED AS
             APPROPRIATE FOR VALUING
             RISK BUT REQUIRE CAREFUL
             ESTIMATION
    ---------------------------------------------------- Appendix IV:2.3.3

    Measuring the benefit of risk reduction requires placing a value on
    avoiding death and/or injury.  Several approaches are available for
    this task, but two have been used in cost-benefit analysis:  (1) the
    �human capital� approach, in which death or injury is valued at the
    market value of the lost production it causes plus the medical costs
    expended and (2) the �willingness-to-pay� approach, which attempts to
    measure directly individuals' willingness to pay for reducing the
    risk of death or injury.\40 In early cost-benefit analysis, the human
    capital approach was used, largely because lost wages and medical
    costs were relatively easy to measure.  However, this approach is not
    preferred today because of a number of shortcomings.  First, if this
    approach were taken literally to apply to individuals, then persons
    who do not produce output in the marketplace (such as the elderly or
    homemakers) are not valued�an assumption that is clearly ethically
    and economically inappropriate.  In addition, the human capital
    approach is unable to take into account costs to the individual of
    death or injury such as pain and suffering.  Also, it is difficult to
    apply an �average� value based on human capital calculations to a
    statistically anonymous member of a large group.  Finally, we are all
    more valuable than the sum of what we produce, and these amounts
    could not be included in a human-capital-based formulation.  Thus,
    human capital measurements are generally viewed as a lower boundary
    on the value of avoiding death and injury, and are less preferred
    than the more recent willingness-to-pay measures.

    In part because of dissatisfaction with human capital measures as
    used in cost-benefit analysis, and facilitated by newly available
    large micro-level data sets, economists measure the benefits of
    reducing death or injury by calculating the consumer's willingness to
    pay for small reductions in the probability of injury or death.
    These calculations have been done several ways.  Some approaches
    attempt to glean willingness-to-pay measures from observed behavior
    in the marketplace.  For example, one approach examines pay
    differentials for jobs with different risks.  Another approach looks
    at the payoff to consumers who purchase safety devices, such as smoke
    detectors and air bags in automobiles.  These measures have the
    advantage of being based on actual observed behavior in the
    marketplace rather than on an artificial experimental situation.
    However, the validity of such measures is based on the questionable
    assumption that workers and consumers are sufficiently knowledgeable
    about the risks they face and the potential of different occupations
    or safety devices to alleviate those risks.  For many purposes,
    practitioners of cost-benefit analysis can select an appropriate
    value from the range of research already done in this field without
    performing the actual analysis themselves.  However, for other
    analyses, especially those involving unique risk-taking situations,
    it may be wiser to gather new data to construct an estimate that is
    based on circumstances as close as possible to those being studied.

    Another common method for valuing risks is known as contingent
    valuation, in which such values are elicited by observing responses
    or behavior on a survey or in a controlled experiment.  For example,
    researchers surveyed individual shoppers on how much of a premium
    they would be willing to pay for pesticide-free grapefruit.\41 These
    methods have the advantage of being able to provide information on
    areas that cannot be addressed with market data.  However, this
    characteristic could also be a weakness�the very artificiality of the
    situation could make the consumer make a less deliberate choice or
    could limit the usefulness of applying this measure to other
    situations.  This method does entail some technical requirements�for
    example, it may be useful to perform statistical tests on the
    distributional assumptions when constructing contingent valuation
    measures.  (We addressed these issues in questions 15 through 17 of
    table IV.1.)

    --------------------
    \40 This terminology has been borrowed from W.K.  Viscusi in "The
    Valuation of Risks to Life and Health," in Benefits Assessment:  The
    State of the Art, eds., Judith D.  Bentkover, Vincent T.  Covello,
    and Jeryl Mumpower (Dordrecht, Holland:  Reidel Publishing Company,
    1986), pp.  193-210.

    \41 Jean C.  Buzby, Richard C.  Ready, and Jerry R.  Skees,
    "Contingent Valuation in Food Policy Analysis:  A Case Study of a
    Pesticide-Residue Risk Reduction," Journal of Agricultural and
    Applied Economics, 27 (2) (Dec.  1995), pp.  613-25.

    DEATH, INJURY, AND INCIDENT DATA
    SOURCES USED BY CPSC
    =========================================================== Appendix V

    In order to target its resources and analyze the costs and benefits
    of projects or potential projects, CPSC must obtain data on injuries
    and deaths related to particular product hazards.  CPSC relies on a
    patchwork of independent data systems to address this need.  However,
    not only does each of these data sources have its own internal
    limitations, but together CPSC's data sources present an
    incomplete�and potentially distorted�picture of
    consumer-product-related injuries and deaths.  The implications of
    this lack of data range from reducing CPSC's ability to apply
    regulatory project selection criteria to limiting the agency's
    ability to estimate the impact of its regulatory actions.

       CPSC RELIES HEAVILY ON HOSPITAL
       EMERGENCY ROOM REPORTS TO
       OBTAIN ESTIMATES OF INJURIES
    --------------------------------------------------------- Appendix V:1

          NEISS SYSTEM
    ------------------------------------------------------- Appendix V:1.1

    CPSC obtains most of its injury information from its National
    Electronic Injury Surveillance System (NEISS), which gathers
    information from hospital emergency room records.  NEISS provides
    national estimates about the number and severity of
    emergency-room-treated injuries associated with, although not
    necessarily caused by, consumer products in the United States.  To
    accomplish this, a stratified probability sample of hospitals is
    drawn that is representative of all hospitals with emergency
    departments in the United States and its territories.\42 CPSC
    constructs national estimates of the number of injuries associated
    with individual consumer products on the basis of reports from this
    sample of 101 hospitals.

    NEISS data result from information abstracted from hospital emergency
    room records by coders trained by CPSC staff.  The data are coded and
    entered, at each site, into a personal computer programmed for this
    purpose.  The software checks the data entries for consistency.  The
    data collections are linked nightly to a CPSC permanent central
    database.  Data about product-related injuries are available to CPSC
    staff within 72 hours after the accident for most of the injuries
    reported.  These daily inputs are reviewed by CPSC staff for quality
    and for identifying possible emerging hazards.  The timeliness of the
    data also allows staff to observe seasonal or episodic variations.
    For example, during a 30-day period surrounding the Fourth of July in
    1990, CPSC gathered extra data through NEISS for a special study on
    injuries involving fireworks.

    The unit of analysis for NEISS is the injured person.  Other key
    characteristics coded include the date of treatment, age and gender
    of patient, injury diagnosis, body part affected, disposition of
    case, product involved, and accident location.  In addition,
    important details about the injury and the injured person are
    provided in NEISS.  For example, the address and phone number of the
    injured person are included, permitting follow-up investigations
    about the nature and cause of the injury.  There are about 900
    product codes, ranging from abrasive cleaners to youth chairs, which
    the coders use to specify the product involved.\43 Consumer products
    are coded to allow for a great deal of specificity in the estimates.
    For example, a "hand saw" would be differentiated from a "portable
    circular power saw," and a "bicycle-mounted baby carrier" would be
    specified differently from a "backpack baby carrier." However, NEISS'
    coding system that describes the diagnosis of the primary injury and
    the body part injured is not overly specific, and because it is
    unique it cannot be directly compared with similar data from other
    databases.

    Despite the extent of valuable information provided by NEISS, this
    system also has significant limitations.  One important consideration
    relates to the nature and size of the NEISS sample design.  NEISS,
    throughout its history, has been designed so that only national--not
    state, local, or regional--estimates can be made.  Thus, NEISS cannot
    detect interregional or interstate differences, and may also be
    limited in its ability to identify emerging product hazard patterns
    that are focused in specific states or regions.

    A limitation in the NEISS data for which CPSC has received criticism
    is the lack of information that would assist in assessing
    causality�that is, information that would establish (or provide a
    starting point to establishing) whether the product in question
    caused the accident or merely was involved in the accident.  NEISS
    contains neither "N" codes--which describe the nature of the
    injury--nor "E" codes--which help explain how the injury happened.  E
    codes briefly describe the circumstances of the accident that
    produced the injury.  For example, E codes could help distinguish
    among falls that occurred on stairs, from a ladder, or off a roof.
    NEISS' short narrative section sometimes contains this type of
    information, but NEISS does not generally provide much information on
    the circumstances surrounding the product's involvement.

    --------------------
    \42 The hospitals are sampled from a universe in which each hospital
    has at least six beds and provides 24-hour emergency service.

    \43 Customarily excluded from NEISS are injury data involving
    automobiles and motorcycles, trains, boats, planes, firearms, food,
    illegal drugs, pesticides, cosmetics, medical devices, assaults and
    suicide attempts, and occupational injuries.

          OTHER SOURCES OF DATA ON
          INJURIES
    ------------------------------------------------------- Appendix V:1.2

    CPSC augments the NEISS injury reports with other sources of
    anecdotal information.  For example, CPSC maintains an Injury or
    Potential Injury Incident (IPII) database of reports the agency
    receives about injuries or incidents involving consumer products.
    These reports come from a variety of sources, including news clips;
    consumer complaints, including calls to CPSC's hot line; and public
    reports of product liability suits.  Although the IPII can provide
    only anecdotal information, this database sometimes contains more
    detailed information than can be found in NEISS.  For example, IPII
    records can contain specific information about the product
    involved�such as the manufacturer and date of purchase�that are
    generally not found in NEISS.  The IPII also serves as a source for
    cases to investigate.  This is particularly important for hazards for
    which NEISS provides relatively few cases.

    CPSC has also purchased data on poisonings in the form of the
    American Association of Poison Control Centers (AAPCC) database.
    This database is composed of reports from approximately 65 poison
    control centers throughout the country.\44 Reports from the AAPCC
    database contain such information as the number of phone calls
    received by participating centers concerning possible ingestion of a
    product and the number of individuals who reported experiencing
    symptoms related to the ingestion.  But these reports contain little
    other information�for example, they do not show how much of the
    poison was ingested.

    Although NEISS provides some information on fire-related injuries,
    CPSC obtains additional data on fires from the National Fire
    Protection Association (NFPA) and the U.S.  Fire Administration.
    NFPA, a private organization, conducts an annual survey of fire
    departments that is designed to make statistically valid national
    estimates of the total number of fires experienced nationally each
    year.  However, the NFPA survey does not collect much detailed
    information about the characteristics of individual fires.  To
    augment the NFPA information, CPSC relies on the more detailed
    information provided in the National Fire Reporting System (NFIRS),
    which is compiled by the U.S.  Fire Administration.\45 NFIRS data can
    provide information on the ages of the victim or victims, and the
    type of dwelling (apartment versus single-family home, for example).
    However, these data are available only with a long lag; a CPSC
    official we interviewed in October 1996 told us that the most recent
    data he had available at that time were from 1993.

    --------------------
    \44 The number of poison control centers reporting cases to the AAPCC
    database varied over the years, ranging from 47 centers in 1984 to 67
    in 1996.

    \45 Although NFIRS is a large dataset, it does not capture data
    uniformly across states.  CPSC generally assumes that NFIRS data
    accurately depict the distribution of characteristics across fires,
    and then applies the NFIRS proportions to the NFPA data to
    approximate these characteristics on a national level.

       CPSC OBTAINS MOST OF ITS
       FATALITY INFORMATION FROM
       SELECTED DEATH CERTIFICATES
    --------------------------------------------------------- Appendix V:2

    CPSC obtains the majority of its information on fatalities from
    purchasing death certificates from the states and culling information
    from them to determine which deaths were related to consumer
    products.  (Like the NEISS emergency room reports, the death
    certificates establish only involvement--not causality--of a consumer
    product.) CPSC does not purchase the complete annual set of death
    certificates from each state; instead, the agency buys death
    certificates according to selected E codes.\46 CPSC purchases about
    8,000 death certificates annually; of these, approximately 50 percent
    will be related to a consumer product.  Over the years, in response
    to declining budgets, CPSC has reduced even further the number of E
    codes for which it purchases certificates.  Death certificates
    include the date, place, cause of death, age, gender, race, and
    residence of the deceased.

    Although death certificates cannot indicate whether the consumer
    product was "at fault" in a death, they do provide some information
    on the underlying circumstances.  For example, the E code would
    specify that a person was killed in an accident caused by electric
    current.  Of course, the quality of the data depends on the care with
    which causation is determined and reported.  It is likely that this
    differs from locality to locality as well as among individual
    doctors.  However, there are some objective indications that the
    quality of reporting causes of death has been improving; for example,
    the proportion of cases that are categorized under ill-defined
    conditions has been falling.\47 Although death certificates do not
    constitute a sample of known probability or a complete count of
    fatalities, and thus statistically reliable estimates cannot be made,
    the geographic information contained in death certificates may help
    to identify state and regional patterns as well as those that are
    national in scope.

    However, death certificate data also have substantial limitations.
    First, there is an extensive lag--usually 2 years--before data become
    available.  Therefore, death certificates are not very useful in
    timely identifying emerging hazards.  Furthermore, a number of
    factors contribute to obtaining rather sketchy causal information.
    For example, details are quite limited, which inhibit determining the
    degree to which a consumer product was involved in the death, let
    alone whether the product was defective or hazardous.  Certificates
    are frequently completed without the benefit of autopsy information
    to establish the precise cause of death, either because an autopsy
    was not performed or because the certificate was filled out before
    the autopsy took place.  There is substantial variation in coding
    practices and the level of detail available from state to state.\48
    The information available about the injured individual is very
    limited, and the number of cases for most categories is very small;
    such data therefore limit the kinds and amount of analyses CPSC can
    perform and the conclusions it can draw.

    CPSC augments its death certificate data with other sources where
    possible.  For example, CPSC has instituted a Medical Examiners' and
    Coroners' Alert Project (MECAP), to provide more timely fatality
    information that lends itself to follow-up investigations.  CPSC has
    engaged some 100 coroners and medical examiners from across the
    country to report potential product-related hazards.  This project
    produces about 2,000 reports annually, and CPSC staff credit the
    coroners' reports with alerting them to a suffocation hazard
    concerning infant cushions, which eventually led CPSC to recall
    existing products and ban future production.  In addition to the
    MECAP data, CPSC also records some reports of fatalities in the IPII,
    NEISS, and NFIRS databases.

    --------------------
    \46 According to agency officials, the major categories of E codes
    the agency does not purchase involve falls (which infrequently
    involve consumer products) and fires (for which the agency relies on
    other sources).  This means that fall-related deaths associated with
    consumer products--such as falls from playground equipment--will be
    undercounted.  In 1995, CPSC conducted a pilot study in which it
    purchased a complete set of E codes from one state to determine
    whether it should change its mix of E codes.  CPSC decided that the
    set it was buying was still the best choice given the budget for
    death certificate data.

    \47 See National Center for Health Statistics, Advance Report of
    Final Mortality Statistics for 1994, 45 (3) (Sept.  30, 1996), p.
    74.

    \48 For more information on this variation, see New England Journal
    of Medicine, Vol.  313 (1985), pp.  1285-6, and U.S.  Department of
    Health and Human Services, Morbidity and Mortality Weekly Report,
    Vol.  37 (1988), pp.  191-4.

       CPSC INVESTIGATES SELECTED
       INCIDENTS TO OBTAIN DETAILED
       INFORMATION ABOUT THE CAUSES OF
       DEATHS AND INJURIES
    --------------------------------------------------------- Appendix V:3

    In order to effectively target resources, identify hazard patterns,
    and determine the appropriate remedy for particular product hazards,
    CPSC needs detailed analyses of the causes of reported incidents.
    CPSC's data sets generally do not provide such information.  The
    NEISS data do not include E codes (standardized, if brief and
    incomplete, descriptions of incident circumstances), nor do they
    include detailed information about how the incident happened.  Some
    information may be provided in the short, free-text comment area of
    the NEISS report, but generally few such details are recorded.  Death
    certificates do include E codes, but product involvement is often
    difficult to ascertain.

    As a result, CPSC staff perform follow-up investigations on selected
    cases to develop additional information about each incident.  Some of
    these investigations are conducted entirely by telephone, while
    others are conducted at the accident site.  These investigations may
    include detailed interviews with victims and/or witnesses, police or
    fire reports, photographs of the product and/or the accident site,
    laboratory testing of the product involved, or re-creations of the
    incidents.  For example, in 1996 CPSC staff investigated an incident
    in which a baby's leg was scratched as it was caught between the
    slats of her crib.  As part of the on-site investigation, the CPSC
    investigator interviewed the child's mother, examined the child,
    examined and photographed the crib, and interviewed staff at the
    store where the crib was purchased.

    The CPSC staff we interviewed told us that investigations,
    particularly on-site investigations, were an important source of
    information on established projects.  The additional detail these
    investigations gathered helps determine causality and identify hazard
    patterns, leading analysts to the appropriate remedies.  For example,
    investigations revealed that very few bicycle accidents were related
    to mechanical problems, and as a result, CPSC staff decided not to
    recommend any changes to existing bicycle standards.  In addition,
    investigations may provide key evidence to help identify and correct
    compliance problems.  For example, the investigator who reviewed the
    crib incident found that the crib in question appeared to violate
    several mandatory safety standards.

       CPSC USES MODELING TECHNIQUES
       AND SPECIAL SURVEYS TO ESTIMATE
       EXPOSURE INFORMATION
    --------------------------------------------------------- Appendix V:4

    In order to produce a numerical risk assessment, CPSC must have some
    information on the extent to which individuals are exposed to a
    particular product hazard.  Exposure information can take many
    different forms, and the best measure of exposure will depend on the
    characteristics of the particular product hazard.  For example, one
    measure of exposure might be the number of products in use, while
    another measure might be the number of hours a person spends using
    the product, and another measure might take into account the
    intensity with which a product is used.  If a product is for one-time
    use only and is usually used soon after it is sold (such as
    fireworks), number of products sold might be a reasonable proxy for
    exposure.  However, when a product is used for a longer period or
    often "handed down" to another user, such as a baby high chair, it
    would probably be more reasonable to base estimates of exposure on
    the number of products in use.  In cases where the potential hazard
    is especially ubiquitous (like air pollution, for example),
    population measures (the number living near the source or the number
    of children under 5, for example) may be reasonable.  For some
    hazards, it may be important to account for the intensity of use.
    For example, the probability of developing cancer from exposure to a
    wood stove may depend on how often the stove is used, how large a
    space it heats, and how many times the stove door is opened to add
    wood.  Similarly, bicycles could be used very frequently and very
    intensely (every day in urban traffic) or infrequently and not
    intensely (once a season on the bike path in the park); thus, a good
    exposure measure would take such factors into account.

    CPSC does not conduct a formal, numerical risk assessment for each
    project it undertakes.  Of the 115 CPSC projects we reviewed, only 24
    included a numerical assessment of risk.  CPSC most frequently relied
    on estimates of products in use to provide the exposure information
    for its risk assessments.  In 65 percent of the cases where an
    epidemiological risk assessment was performed, CPSC based its
    exposure measure on an estimate of the number of products in use.\49
    In an additional 5 percent, CPSC obtained information on the actual
    number of products in use.  In 30 percent of cases, CPSC used a
    population-based measure, and in 10 percent of cases, CPSC used a
    sales measure.\50 We did not evaluate whether the exposure measure
    CPSC chose was appropriate for each case.

    CPSC conducted special surveys on bicycles and on cigarette lighters
    and matches, for instance, to develop exposure information.  The
    survey questions were designed to obtain information on the number of
    products in the home, the intensity of use, the characteristics of
    the households using the product, and the usual patterns of use.  For
    example, the bicycle survey included questions on the number of hours
    spent biking, the ages, education, and income of household members,
    whether riding was done most often on streets, sidewalks, or bike
    paths, and whether riders used helmets.

    Where special surveys are not practical--because of time, resource,
    or other limitations--CPSC sometimes uses mathematical modeling
    techniques to estimate the number of products in use.  These models
    can take sales information and information on the life of the product
    to estimate the number of units in use.  For example, CPSC used such
    a model to estimate how many of the portable heaters made before a
    1991 revision to a voluntary standard were still in use.

    --------------------
    \49 About half of these estimates of products in use were derived
    from modeling and about half from special surveys.

    \50 Percentages do not total 100 because in two cases CPSC used more
    than one measure of exposure.

       CPSC OBTAINS SOME INFORMATION
       UNDER REPORTING REQUIREMENTS
       FOR MANUFACTURERS AND OTHER
       BUSINESSES
    --------------------------------------------------------- Appendix V:5

    CPSC may also receive information about potential product hazards
    through industry reporting requirements.  Although this information
    is used mostly for identifying and addressing compliance problems, it
    may also help identify new hazards.  Companies are legally obligated
    to report to CPSC information they receive that indicates a consumer
    product they distribute is potentially hazardous.  Under section 15
    of the Consumer Product Safety Act, manufacturers (including
    importers), distributors, and retailers of consumer products must
    notify CPSC if they obtain information that a product (1) fails to
    comply with a consumer product safety regulation or a voluntary
    consumer product safety standard, (2) contains a defect that could
    create a substantial product hazard, or (3) creates an unreasonable
    risk of serious injury or death.

    In addition to these reporting requirements, manufacturers and
    importers of a consumer product must report to CPSC if (1) a
    particular model of a consumer product is the subject of at least
    three civil actions that have been filed in federal or state court,
    (2) each suit alleges the involvement of that model in death or
    grievous bodily injury, and (3) within a specified 2-year period at
    least three of the actions resulted in a final settlement involving
    the manufacturer or importer or in a judgment for the plaintiff.
    Manufacturers must file a report within 30 days after the settlement
    or judgment in the third such civil action.

    CPSC may receive requests for information that firms have reported to
    it under section 15 requirements.  The law limits CPSC's disclosure
    of any information identifying a manufacturer and further limits
    CPSC's release of information that firms have provided under these
    requirements.  Reports on civil lawsuits may not be publicly
    disclosed by CPSC or subpoenaed or otherwise obtained from CPSC
    through discovery in any civil action or administrative procedure.

    (See figure in printed edition.)Appendix VI
    COMMENTS FROM THE CHAIRMAN AND
    COMMISSIONER MOORE, CONSUMER
    PRODUCT SAFETY COMMISSION, AND
    GAO'S EVALUATION
    =========================================================== Appendix V

    (See figure in printed edition.)

    (See figure in printed edition.)

    (See figure in printed edition.)

    (See figure in printed edition.)

    (See figure in printed edition.)

    (See figure in printed edition.)

    (See figure in printed edition.)

    (See figure in printed edition.)

    (See figure in printed edition.)

    (See figure in printed edition.)

    (See figure in printed edition.)

    (See figure in printed edition.)

    (See figure in printed edition.)

    (See figure in printed edition.)

    (See figure in printed edition.)

    (See figure in printed edition.)

    The following are GAO's comments on Chairman Brown and Commissioner
    Moore's letter dated July 24, 1997.

       GAO COMMENTS
    --------------------------------------------------------- Appendix V:6

    1.  As we discuss in the report, CPSC relies on a number of different
    sources for injury incident data.  (Descriptive information on all
    CPSC's data sources is provided in app.  V.) However, only one
    source--the NEISS system--is capable of providing statistically
    reliable, representative information, and the NEISS system covers
    only those injuries treated in hospital emergency rooms.  Although
    CPSC obtains information from other sources, these data are anecdotal
    and thus their usefulness is very limited in estimating injury
    prevalence.  As a result, we have recommended that CPSC consult with
    experts both inside and outside the agency to prioritize its
    additional data needs and to explore the feasibility of options for
    obtaining these data.  Changes over the past decade in the health
    care market, including the growth of ambulatory care, changes in
    reimbursement procedures, and improved health services research have
    the potential to make such additional data collection more feasible
    and less costly than it was a decade ago when some of these
    assessments were last made.

    2.  Figure 3 correctly states that CPSC does not obtain systematic
    surveillance data on injuries treated outside the emergency room.  In
    the text of the report, we state that the number of injuries treated
    in each setting is unknown.  We added a similar statement to the
    figure to emphasize this point.

    3.  In several places in their comments, Chairman Brown and
    Commissioner Moore refer to CPSC's investigations of selected
    incidents to obtain more information.  We discussed these
    investigations in our report.  We believe that these investigations
    provide valuable information on causation, characteristics of
    accident victims, and hazard patterns, and we agree that some
    information gleaned from investigations is not obtainable from
    surveillance data.  However, according to CPSC staff, this
    information is available to the agency only after a project is well
    under way, not at the initial stage of project selection.  As a
    result, CPSC has little information on these important factors to
    assist in project selection, and evaluation of these criteria at the
    project selection stage is thus unavoidably speculative.

    4.  Descriptive information on this source of information was
    provided in appendix V.

    5.  For reasons we discussed in the report, relying on aggregate data
    (data not specific to a particular consumer product) to address the
    limitations of CPSC's surveillance data is problematic.  In addition,
    in the briefing packages we reviewed, the estimates of injury
    incidents usually referred only to NEISS cases and were not extended
    with modeling techniques; because these techniques were used so
    infrequently for this purpose, we did not assess their application in
    this report.

    6.  In referring to the agency's monitoring of other aggregated
    sources of injury data, Chairman Brown and Commissioner Moore state
    that "Tracking of this type is sufficient to assure the adequacy of
    our data, as we use it." We disagree.  Examining trend information
    from other sources (without a rigorous application to CPSC's own
    specific needs) is not sufficient to compensate for (or even measure
    the magnitude of) the limitations of CPSC's injury surveillance data.

    7.  We believe this comment may reflect a misunderstanding of our
    point.  We do not mean to suggest that CPSC incorrectly relies on
    NEISS to provide information on chronic illnesses.  Rather, we are
    pointing out that CPSC has virtually no systematic data on chronic
    illnesses.  As we stated in the report, on page 20, we agree with
    Chairman Brown and Commissioner Moore (and with the CPSC staff we
    interviewed) that such data are often difficult to obtain.  However,
    chronic illness is listed as a criterion for CPSC project selection,
    and CPSC has little information to assist in applying this criterion.
    Accordingly, we are pleased with the statement by Brown and Moore
    that CPSC will consider additional surveillance methods to obtain
    more information on chronic illnesses.

    8.  We acknowledge in the report that CPSC uses other sources to
    supplement its death certificate data.  However, our interviews with
    CPSC staff and our review of agency documents confirmed that death
    certificates are the most important source of death data for CPSC.
    In the briefing packages we reviewed, 80 percent of the calculations
    for numbers of deaths were based in whole or in large part on death
    certificate data, and death certificates were the sole source of
    death information in 36 percent of all CPSC briefing packages (a far
    greater percentage than for any other single source).  Death data
    were usually reported in CPSC briefing packages with a lag of 2 or
    more years, especially when death certificates were the sole source
    of data.

    9.  We agree with Chairman Brown and Commissioner Moore that there is
    no single mold or perfect set of criteria for evaluating a
    cost-benefit analysis, and we have revised the report to further
    emphasize this point.  We believe that Chairman Brown and
    Commissioner Moore may have misunderstood the purpose of our
    evaluation questions.  We did not "derive a particular methodology"
    for cost-benefit analysis, nor do we mean to suggest that CPSC should
    follow some "formula" for conducting analyses that does not leave
    room for competent professional judgment.  However, to say that there
    is no perfect "formula" for cost-benefit analysis does not imply that
    all methodological choices are equally consistent with rigorous and
    comprehensive professional work.  Although no litmus test exists for
    a "good" analysis, the professional literature offers some basic,
    minimum elements that are commonly used in evaluating cost-benefit
    analyses.  These elements, which are based on the principles of
    transparency and completeness, are generally considered necessary,
    although not sufficient, for a good analysis.  Although flexibility
    may be sometimes necessary in the assumptions or models underlying a
    cost-

    benefit analysis, the elements we used--including full disclosure of
    data limitations, sensitivity analysis, and incorporating all
    important costs and benefits--are appropriate to a wide range of
    situations.

    10.  Chairman Brown and Commissioner Moore are correct in stating
    that we evaluated 29 CPSC cost-benefit analyses that were completed
    between January 1, 1990, and September 30, 1996.  We identified these
    29 analyses as complete on the basis of CPSC's
    statements--specifically, we considered a cost-benefit analysis to be
    complete only if an explicit comparison was made between aggregate
    costs and benefits.  In addition, although the analyses may have been
    prepared at different stages of the project, we based our review on
    all available documentation on the project, and we reported results
    only for the evaluation questions that applied to the majority of
    cases and for which a clear determination could be made.  We did not
    report results separately for the eight regulatory analyses that were
    required by law, because there were relatively few of these.
    However, we found no substantial differences between these 8 and the
    remaining 21 in terms of how they performed, compared with commonly
    used elements of evaluation of cost-benefit analysis.  Therefore, we
    are confident that our results present an accurate assessment of
    CPSC's cost-benefit analyses, and we recommend that the agency
    implement changes to ensure that its analyses are comprehensive and
    reported in sufficient detail.

    11.  We believe that, as now constructed, CPSC's method for tracking
    projects operates at too high a level of generality and provides too
    little information to give a comprehensive, accurate picture of the
    agency's activities either at a given point or over a longer period.
    CPSC staff told us, and our review of agency documentation confirmed,
    that the Management Information System (MIS) usually tracks most
    agency activities only at a very general level.  For example, CPSC's
    1996 year-end MIS report lists some specific projects such as
    "upholstered furniture" and "range fires," but most projects are
    accounted for under either broad umbrella codes such as "sports and
    recreation" or "children's projects," or under activity codes such as
    "investigations," "product safety assessment," or "emerging
    problems." In addition, CPSC staff told us that reliable inferences
    on resources spent cannot be drawn from MIS data because of
    limitations in the computer system and because no consistent rule
    exists about how staff time in different directorates is recorded to
    project codes.  As a result, CPSC staff were unable to generate a
    comprehensive list of projects or to provide accurate information
    about resources allocated to those projects.  We recommend an
    improved tracking system that would provide enough information to
    monitor the projects selected and resources spent for each specific
    consumer product hazard.  We believe that as CPSC develops its
    planned accounting system, it should attempt to make it as compatible
    as practicable with the recommended tracking system.  Nevertheless,
    whether or not it implements its planned accounting system, we
    believe that CPSC can and should improve its ability to track
    projects.

    12.  Our interviews with present and former commissioners revealed a
    pattern by which certain of CPSC's regulatory criteria have
    historically been given greater emphasis in CPSC's project selection
    process.  Our objective was to describe the process as it was related
    to us; we have not taken a position on whether this process is
    appropriate.  We have added a statement to our methodology section to
    emphasize this point.

    (See figure in printed edition.)Appendix VII
    COMMENTS FROM COMMISSIONER GALL,
    CONSUMER PRODUCT SAFETY
    COMMISSION, AND GAO'S EVALUATION
    =========================================================== Appendix V

    (See figure in printed edition.)

    (See figure in printed edition.)

    (See figure in printed edition.)

    The following are GAO's comments on Commissioner Gall's letter dated
    July 23, 1997.

       GAO COMMENTS
    --------------------------------------------------------- Appendix V:7

    1.  Our interviews with present and former commissioners revealed a
    pattern by which certain of CPSC's regulatory criteria have
    historically been given greater emphasis in CPSC's project selection
    process.  Our objective was to describe the process as it was related
    to us; we have not taken a position on whether this process is
    appropriate.  We have added a statement to our methodology section to
    emphasize this point.

    2.  Commissioner Gall states that we did not analyze the relative
    importance of the deficiencies we found in CPSC's data and
    methodology.  However, as we stated in the report, available
    information does not permit us to determine the impact of
    better-quality data on the decisions CPSC made.  The limitations we
    found in CPSC's data have a variety of potentially conflicting
    impacts, precluding us from determining exactly how the results of
    the analysis might change if improved data were available.  For
    example, because CPSC's injury estimates are often confined to
    injuries treated in hospital emergency rooms, CPSC's estimates will
    generally understate the actual number of injuries associated with a
    consumer product.  However, CPSC's systematic injury and death data
    can generally tell only whether a product was involved in an
    accident--not whether the product caused or contributed to the
    accident.  As a result, this can make the risks assessed by CPSC
    appear larger than they might actually be.  Similarly, we cannot
    determine how improved exposure data would change the relative
    importance of the risks assessed by CPSC.  We agree with Commissioner
    Gall that not all projects will merit the same level of data or
    analysis.  However, our review of CPSC raises questions about the
    agency's ability to obtain and analyze data necessary to support
    rigorous analysis of important agency projects.

    3.  We agree with Commissioner Gall that resource considerations
    should enter into CPSC's decisions to undertake new data collection.
    For this reason, we recommended an overall feasibility study for CPSC
    to prioritize among its data needs and investigate new options for
    obtaining additional information.

    GAO CONTACTS AND STAFF
    ACKNOWLEDGMENTS
    ======================================================== Appendix VIII

    GAO CONTACTS

    Lawrence J.  Horinko, Assistant Director, (202) 512-7001
    Sarah L.  Glavin, Project Manager, (202) 512-7180

    STAFF ACKNOWLEDGMENTS

    In addition to those named above, the following individuals made
    important contributions to this report:  Sheila A.  Nicholson,
    Analyst, gathered and analyzed data on CPSC's information release
    procedures; Nancy K.  Kintner-Meyer, Senior Evaluator, compiled and
    analyzed information on CPSC projects; George Bogart, Senior
    Attorney, provided legal assistance; Harold Wallach, Senior Analyst,
    assisted in the analysis of CPSC's data systems; and Charles Jeszeck
    served as Assistant Director for the project in its early stages.

    BIBLIOGRAPHY
    ============================================================ Chapter 0

    Abkowitz, M., and P.  Der-Ming Cheng.  "Developing a Risk/Cost
    Framework for Routing Truck Movements of Hazardous Materials."
    Accident Analysis and Prevention, 20 (1988), pp.  39-51.

    Abraham, John.  "Negotiation and Accommodation in Expert Medical Risk
    Assessment and Regulation:  An Institutional Analysis of the
    Benoxaprofen Case." Policy Sciences, 27 (1994), pp.  53-76.

    Adams, John.  "The Emperor's Old Clothes:  The Curious Comeback of
    Cost-Benefit Analysis." Environmental Values, 2 (1993), pp.  247-60.

    Adler, Robert S.  "From 'Model Agency' to Basket Case--Can the
    Consumer Product Safety Commission Be Redeemed?" Administrative Law
    Review, 42 (Winter 1989), pp.  61-129.

    Alberini, Anna.  "Testing Willingness-to-Pay Models of Discrete
    Choice Contingent Valuation Survey Data." Land Economics, 71 (1)
    (Feb.  1995), pp.  83-95.

    Allen, Julius.  Cost Benefit Analysis of Federal Regulation:  A
    Review and Analysis of Developments, 1978-1984.  Report No.  84-74 E.
    Washington, D.C.:  Congressional Research Service, May 15, 1984.

    Allison, Paul D.  Event History Analysis:  Regression for
    Longitudinal Event Data, Sage University Paper Series:  Quantitative
    Applications in the Social Sciences, series no.  07-046.  Newbury
    Park, Calif.:  Sage Publications, 1984.

    American Academy of Pediatrics.  "Hospital Discharge Data on Injury:
    The Need for E Codes." Policy Statement in AAP News, Mar.  1992.

    Arrow, Kenneth J., and others.  Benefit-Cost Analysis in
    Environmental, Health, and Safety Regulation:  A Statement of
    Principles.  American Enterprise Institute, The Annapolis Center, and
    Resources for the Future.  Washington, D.C.:  AEI Press, 1996.

    Asch, Peter.  Consumer Safety Regulation:  Putting a Price on Life
    and Limb.  New York:  Oxford University Press, 1988.

    Ault, Eric B.  "CPSC's Voluntary Standards:  An Assessment and a
    Paradox." The Frontier of Research in the Consumer Interest.  Eds.
    E.  Scott Maynes and ACCI Research Committee.  Columbia, Mo.:
    American Council on Consumer Interests, 1986, pp.  77-81.

    Bailey, Martin J.  Reducing Risks to Life:  Measurement of the
    Benefits.  Washington, D.C.:  American Enterprise Institute, 1980.

    Baker, S.P., and others.  The Injury Fact Book, 2nd ed.  New York:
    Oxford University Press, 1992.

    Battiato, Salvatore Enrico.  "Cost-Benefit Analysis and the Theory of
    Resource Allocation." Alan Williams and Emilio Giardina.  Efficiency
    in the Public Sector.  Aldershot, Hants, England:  Edward Elgar
    Publishing, 1993, pp.  26-42.

    Berlage, L., and R.  Renard.  "The Discount Rate in Cost-Benefit
    Analysis and the Choice of a Numeraire." Oxford Economic Papers, 37
    (1985), pp.  691-9.

    Berlau, John.  "Play (Regulated) Ball!" Reason, Dec.  1996, pp.
    71-3.

    Bhat, M., and S.  Li.  "Consumer-Product-Related Tooth Injuries
    Treated in Hospital Emergency Rooms:  United States, 1979-1987."
    Community Dentistry and Oral Epidemiology, 18 (3) (June 1990), pp.
    133-8.

    Bird, Peter J.W.N.  "A Note on Relative Price Changes in Cost-Benefit
    Analysis." Public Finance, 35 (2) (1980), pp.  318-23.

    Blackorby, Charles, and David Donaldson.  "A Review Article:  The
    Case Against the Use of the Sum of Compensating Variations in
    Cost-Benefit Analysis." Canadian Journal of Economics, 23 (3) (Aug.
    1990), pp.  471-94.

    _____.  "Welfare Ratios and Distributionally Sensitive Cost-Benefit
    Analysis." Journal of Public Economics, 34 (1987), pp.  265-90.

    Boardman, Anthony, Aidan Vining, and W.G.  Waters II.  "Costs and
    Benefits Through Bureaucratic Lenses:  Example of a Highway Project."
    Journal of Policy Analysis and Management, 12 (3) (1993), pp.
    532-55.

    Bordley, Robert F.  "Making Social Trade-offs Among Lives,
    Disabilities, and Costs." Journal of Risk and Uncertainty, 9 (1994),
    pp.  135-49.

    Boustead, Thomas.  The Law and Economics of Administrative Law:  A
    Statistical Analysis of the Consumer Product Safety Commission's
    Petition Process.  Ph.D.  Dissertation, Fordham University, May 1995.

    Boyle, M.H., and others.  "Economic Evaluation of Neonatal Intensive
    Care of Very Low Birthweight Infants." New England Journal of
    Medicine, 308 (1983), pp.  1330-37.

    Bradfield, R.E.  "Social Cost-Benefit Analysis in Project Evaluation:
    A Practical Framework for Decision Making." Journal of Studies in
    Economics and Econometrics, 13 (2) (1989), pp.  25-35.

    Brent, Robert J.  "The Cost-Benefit Analysis of Government Loans."
    Public Finance Quarterly, 19 (1) (Jan.  1991), pp.  43-66.

    Breyer, Stephen.  Breaking the Vicious Circle.  Cambridge, Mass.:
    Harvard University Press, 1993.

    Burt, Catharine E.  "Injury-Related Visits to Hospital Emergency
    Departments:  United States, 1992." Advance Data, No.  261 (Feb.  1,
    1995).

    Buzby, Jean C., Richard C.  Ready, and Jerry R.  Skees.  "Contingent
    Valuation in Food Policy Analysis:  A Case Study of a
    Pesticide-Residue Risk Reduction." Journal of Agricultural and
    Applied Economics, 27 (2) (Dec.  1995), pp.  613-25.

    Campen, James T.  Benefit, Cost, and Beyond:  The Political Economy
    of Benefit-Cost Analysis.  Cambridge, Mass.:  Ballinger Publishing
    Company, 1986.

    Cesario, Frank J.  "Benefit-Cost Analysis Under Pricing Constraints."
    Applied Economics, 13 (1981), pp.  215-24.

    Chan, Yuk-Shee, and Anthony M.  Marino.  "Regulation of Product
    Safety Characteristics Under Imperfect Observability." Journal of
    Regulatory Economics, 6 (1994), pp.  177-95.

    Chirinko, Robert S., and Edward P.  Harper, Jr.  "Buckle Up or Slow
    Down?  New Estimates of Offsetting Behavior and Their Implications
    for Automobile Safety Regulation." Journal of Policy Analysis and
    Management, 12 (2) (1993), pp.  270-96.

    Cost-Benefit Analysis:  Wonder Tool or Mirage?  Report by the U.S.
    House of Representatives Subcommittee on Oversight and Investigation,
    Committee on Interstate and Foreign Commerce, Dec.  1980.

    Cowen, Tyler.  "The Scope and Limits of Preference Sovereignty."
    Economics and Philosophy, 9 (1993), pp.  253-69.

    Cox, Louis Anthony (Jr.).  "Theory of Regulatory Benefits Assessment:
    Econometric and Expressed Preference Approaches." Benefits
    Assessment:  The State of the Art.  Eds.  Judith D.  Bentkover,
    Vincent T.  Covello, and Jeryl Mumpower.  Dordrecht, Holland:  D.
    Reidel Publishing Company, 1986, pp.  85-160.

    Crandall, Robert W.  "The Use of Cost-Benefit Analysis in Product
    Safety Regulation." The Frontier of Research in the Consumer
    Interest.  Eds.  E.  Scott Maynes and ACCI Research Committee.
    Columbia, Mo.:  American Council on Consumer Interests, 1986, pp.
    61-75.

    Crews, Clyde Wayne.  "Ten Thousand Commandments:  Regulatory Trends
    1981-92 and the Prospect for Reform." Journal of Regulation and
    Social Cost, Vol.  2 (Mar.  1993), pp.  105-50.

    Cummings, Ronald G., Louis Anthony Cox, Jr., and A.  Myrick Freeman
    III.  "General Methods for Risk Assessment." Benefits Assessment:
    The State of the Art.  Eds.  Judith D.  Bentkover, Vincent T.
    Covello, and Jeryl Mumpower.  Dordrecht, Holland:  D.  Reidel
    Publishing Company, 1986, pp.  161-92.

    Curtin, Leo, and Ronald Krystynak.  "An Economic Framework for
    Assessing Foodborne Disease Control Strategies With an Application to
    Salmonella Control in Poultry." The Economics of Food Safety.  Ed.
    Julie A.  Caswell.  London:  Elsevier, 1991, pp.  131-51.

    Dasguptha, A.K., and D.W.  Pearce.  Cost-Benefit Analysis:  Theory
    and Practice.  London:  Macmillan, 1972.

    Davis, Miriam.  Health Risk Assessment Research at the Department of
    Energy:  Background Information.  Washington, D.C.:  U.S.  Office of
    Technology Assessment, Sept.  15, 1992.

    Davis, Yvette, and others.  "An Evaluation of the National Electronic
    Injury Surveillance System for Use in Monitoring Nonfatal Firearm
    Injuries and Obtaining National Estimates." Journal of Safety
    Research, 27 (2) (Summer 1996), pp.  83-91.

    Dehar, Mary Anne, Sally Casswell, and Paul Duignan.  "Formative and
    Process Evaluation of Health Promotion and Disease Prevention
    Programs." Evaluation Review, 17 (2) (Apr.  1993), pp.  204-20.

    Drummond, Michael F.  "Cost Benefit Analysis in Health and Health
    Care:  Fine in Practice, but Does It Work in Theory?" Alan Williams
    and Emilio Giardina.  Efficiency in the Public Sector.  Aldershot,
    Hants, England:  Edward Elgar Publishing, 1993, pp.  106-28.

    Drummond, Michael F., Greg L.  Stoddart, and George W.  Torrance.
    Methods for the Economic Evaluation of Health Care Programmes.  New
    York:  Oxford University Press, 1996.

    Drummond, Michael F., and others.  "Users' Guide to the Medical
    Literature:  How to Use an Article on Economic Analysis of Clinical
    Practice." Journal of the American Medical Association, 277 (19) (May
    21, 1997), pp.  1552-57.

    Evans, J.S., N.S.  Hawkins, and J.D.  Graham.  "Uncertainty Analysis
    and the Value of Information:  Monitoring for Radon in the Home."
    Journal of Air Pollution Control Association, 38 (1988), pp.
    1380-85.

    Evans, L., and M.C.  Frick.  "Helmet Effectiveness in Preventing
    Motorcycle Driver and Passenger Fatalities." Accident Analysis and
    Prevention, 20 (1988), pp.  447-58.

    Fingar, Ann R., Richard S.  Hopkins, and Marjorie Nelson.
    "Work-Related Injuries in Athens County, 1982 to 1986:  A Comparison
    of Emergency Department and Workers' Compensation Data." Journal of
    Occupational Medicine, 34 (8) (Aug.  1992), pp.  779-87.

    Finkel, Adam M.  Comparing Risks Thoughtfully.  Washington, D.C.:
    U.S.  Office of Technology Assessment, Sept.  1994.

    _____.  "Who's Exaggerating?" Discover, May 1996, pp.  45-54.

    Fischhoff, Baruch.  Ranking Risks.  Washington, D.C.:  U.S.  Office
    of Technology Assessment, Feb.  1994.

    _____, and Louis Anthony Cox, Jr.  "Conceptual Framework for
    Regulatory Benefits Assessment." Benefits Assessment:  The State of
    the Art.  Eds.  Judith D.  Bentkover, Vincent T.  Covello, and Jeryl
    Mumpower.  Dordrecht, Holland:  D.  Reidel Publishing Company, 1986,
    pp.  51-84.

    Fise, Mary Ellen.  "Consumer Product Safety Regulation," Chapter 11
    in Regulation and Consumer Protection.  Dame Publications, Inc.,
    1995, pp.  277-94.

    _____.  The CPSC:  Guiding or Hiding From Product Safety?
    Washington, D.C.:  Consumer Federation of America, May 1987.

    _____, and M.  Kristen Rand.  Pennies for Consumer Protection?  A
    Report on the CPSC's 1991 Budget.  Washington, D.C.:  Consumer
    Federation of America, May 1990.

    Fisher, Ann, Maria Pavlova, and Vincent Covello, eds.  Evaluation and
    Effective Risk Communication Workshop Proceedings.  Washington, D.C.:
    Interagency Task Force on Environmental Cancer and Heart and Lung
    Disease, Committee on Public Education and Communication, Jan.  1991.

    Fisher, Ann, Lauraine G.  Chestnut, and Daniel M.  Violette.  "The
    Value of Reducing Risks of Death:  A Note on New Evidence." Journal
    of Policy Analysis and Management, 8 (1) (1989), pp.  88-100.

    Fuchs, Edward Paul, and James Anderson.  "Institutionalizing
    Cost-Benefit Analysis in Regulatory Agencies." Research in Public
    Policy Analysis and Management, 4 (1987), pp.  187-211.

    Fung, V.A., J.C.  Barrett, and J.  Huff.  "Reviews and Commentaries:
    The Carcinogenesis Bioassay in Perspective:  Application in
    Identifying Human Cancer Hazards." Environmental Health Perspectives,
    103 (1995), pp.  7-8.

    Gerner, Jennifer L.  "Product Safety:  A Review." The Frontier of
    Research in the Consumer Interest, eds.  E.  Scott Maynes and ACCI
    Research Committee.  Columbia, Mo.:  American Council on Consumer
    Interests, 1986, pp.  37-59.

    Gilroy, John Martin, and Maurice Wade, eds.  The Moral Dimensions of
    Public Policy Choice:  Beyond the Market Paradigm.  Pittsburgh, Pa.:
    University of Pittsburgh Press, 1992.

    Gold, Marthe R., and others.  Cost-Effectiveness in Health and
    Medicine.  New York:  Oxford University Press, 1996.

    Goldenhar, Linda, and Paul A.  Schulte.  "Intervention Research in
    Occupational Health and Safety." Journal of Occupational Medicine, 36
    (7), pp.  763-75.

    Goldstein, B.  "Risk Management Will Not Be Improved by Mandating
    Numerical Uncertainty Analysis for Risk Assessment." University of
    Cincinnati Law Review, Vol.  63, pp.  1599-1610.

    Goodstein, Eban.  "Benefit-Cost Analysis at the EPA." Journal of
    Socio-Economics, 24 (2) (1995), pp.  375-89.

    Graham, John D.  "Making Sense of Risk:  An Agenda for Congress."
    Risks, Costs and Lives Saved:  Getting Better Results from
    Regulation.  Ed.  Robert W.  Hahn.  Washington, D.C.:  AEI Press,
    1996, pp.  183-207.

    _____.  "Technology, Behavior, and Safety:  An Empirical Study of
    Automobile Occupant-Protection Regulation." Policy Sciences, 17
    (1984), pp.  141-51.

    _____, and Younghee Lee.  "Behavioral Response to Safety Regulation:
    The Case of Motorcycle Helmet-Wearing Legislation." Policy Sciences,
    19 (1986), pp.  253-73.

    Graves, Edmund J., and Brenda S.  Gillum.  "1994 Summary:  National
    Hospital Discharge Survey." Advance Data, No.  278 (Oct.  3, 1996).

    Gray, George M.  The Challenge of Risk Characterization.  Washington,
    D.C.:  U.S.  Office of Technology Assessment, Nov.  1992.

    Greenberg, David H.  "Conceptual Issues in Cost/Benefit Analysis of
    Welfare-to-Work Programs." Contemporary Policy Issues, 10 (Oct.
    1992), pp.  51-64.

    Gregory, Robin, Thomas C.  Brown, and Jack L.  Knetsch.  "Valuing
    Risks to the Environment." The Annals of the American Academy of
    Political and Social Science.  Eds.  Howard Kunreuther and Paul
    Slovic.  Vol.  545 (May 1996), pp.  54-63.

    Griffin, Ronald.  "On the Meaning of Economic Efficiency in Policy
    Analysis." Land Economics, 71 (1) (Feb.  1995), pp.  1-15.

    Guyer, Bernard, and Susan S.  Gallagher.  "An Approach to the
    Epidemiology of Childhood Injuries." Pediatric Clinics of North
    America, 32 (1) (Feb.  1985), pp.  5-15.

    Haas, C.N., and others.  "Risk Assessment of Viruses in Drinking
    Water." Risk Analysis, Vol.  13, pp.  545-52.

    Hadden, Susan G.  Read the Label:  Reducing Risk by Providing
    Information.  Boulder, Co.:  Westview Press, 1986.

    Haddix, A.C., and others, eds.  Prevention Effectiveness:  A Guide to
    Decision Analysis and Economic Evaluation.  New York:  Oxford
    University Press, 1996.

    Hahn, Robert W.  "Improving Regulation:  Steps Toward Reform."
    Statement before the U.S.  Senate Subcommittee on Financial
    Management and Accountability, Committee on Govermental Affairs,
    Sept.  25, 1996.

    _____.  "A Preliminary Estimate of Some Indirect Costs of
    Environmental Regulation." Unpublished paper, Feb.  1995.

    _____.  "Regulatory Reform:  A Legislative Agenda." Statement before
    the U.S.  Senate Committee on Governmental Affairs, Feb.  8, 1995.

    _____.  "Regulatory Reform:  What Do the Government's Numbers Tell
    Us?" Risks, Costs and Lives Saved:  Getting Better Results From
    Regulation.  Ed.  Robert W.  Hahn.  Washington, D.C.:  AEI Press,
    1996, pp.  208-54.

    _____.  "Why We Need to Balance the Costs and Benefits of
    Regulation." Statement before the U.S.  House of Representatives,
    Subcommittee on Oversight and Investigations, Committee on Economic
    and Educational Opportunities, and the Subcommittee on Paperwork and
    Regulation, Committee on Small Business, Feb.  2, 1995.

    Hall, Margaret Jean, and Maria F.  Owings.  "Hospitalizations for
    Injury and Poisoning in the United States, 1991." Advance Data, No.
    252 (Oct.  7, 1994).

    Hammitt, J.K., and J.A.K.  Cave.  Research Planning for Food Safety:
    A Value of Information Approach.  RAND R-3946-ASPE/NCTR.  Santa
    Monica, Calif.:  RAND, 1991.

    Hanke, Steve.  "On the Feasibility of Benefit-Cost Analysis." Public
    Policy, 29 (2) (Spring 1981), pp.  147-57.

    Haskins, Jack B.  "Evaluative Research on the Effects of Mass
    Communication Safety Campaigns:  A Methodological Critique." Journal
    of Safety Research, 2 (2) (June 1970), pp.  86-8.

    Haupt, Barbara.  "1982 Summary:  National Hospital Discharge Survey."
    Advance Data, No.  95 (Dec.  27, 1983).

    Havrilesky, Thomas.  "The Persistent Misapplication of the Hedonic
    Damages Concept to Wrongful Death and Personal Injury Litigation."
    Journal of Forensic Economics, 8 (1) (1995), pp.  49-54.

    Heimann, Christopher M., and others.  "Project:  The Impact of
    Cost-Benefit Analysis on Federal Administrative Law." Administrative
    Law Review, 42 (Fall 1990), pp.  545-654.

    Henderson, J.  Vernon.  "Peak Shifting and Cost-Benefit
    Miscalculations." Regional Science and Urban Economics, 22 (1992),
    pp.  103-121.

    Hensler, Deborah R., and others.  Compensation for Accidental
    Injuries in the United States.  Santa Monica, Calif.:  RAND, The
    Institute for Civil Justice, 1991.

    Hildred, William, and Fred Beauvais.  "An Instrumentalist Critique of
    `Cost-Utility Analysis.'" Journal of Economic Issues, 29 (4) (Dec.
    1995), pp.  1083-96.

    Hoehn, John P., and Alan Randall.  "A Satisfactory Benefit Cost
    Indicator From Contingent Valuation." Journal of Environmental
    Economics and Management, 14 (1987), pp.  226-47.

    _____.  "Too Many Proposals Pass the Benefit-Cost Test." American
    Economic Review, 79 (3) (June 1989), pp.  544-51.

    _____.  "Too Many Proposals Pass the Benefit-Cost Test:  Reply."
    American Economic Review, 81 (5) (Dec.  1991), pp.  1450-2.

    Hoffer, George E., Stephen W.  Pruitt, and Robert J.  Reilly.  "The
    Impact of Product Recalls on the Wealth of Sellers:  A
    Reexamination." Journal of Political Economy, 96 (3) (1988).

    Hopkins, Thomas D.  "The Costs of Federal Regulation." Journal of
    Regulation and Social Costs, Mar.  1992, pp.  5-31.

    Horowitz, John K., and Richard T.  Carson.  "Discounting Statistical
    Lives." Journal of Risk and Uncertainty, 3 (1990), pp.  403-13.

    Hubin, Donald C.  "The Moral Justification of Benefit-Cost Analysis."
    Economics and Philosophy, 10 (1994), pp.  169-94.

    The Institute of Medicine and the National Research Council,
    Commission on Life Sciences, Committee on Trauma Research.  Injury in
    America:  A Continuing Public Health Problem.  Washington, D.C.:
    National Academy Press, 1985.

    Jarrell, Gregg A., and Sam Peltzman.  "The Impact of Product Recalls
    on the Wealth of Sellers." George J.  Stigler Chicago Studies in
    Political Economy.  Chicago, Ill.:  Chicago Press, 1988, pp.  612-34.

    Johansson, Per-Olov.  "Altruism in Cost-Benefit Analysis."
    Environmental and Resource Economics, 2 (1992), pp.  605-13.

    Joksch, H.  "Critique of Peltzman's Study:  The Effects of Automobile
    Safety Regulation." Accident Analysis and Prevention, 8 (1976), pp.
    213-14.

    Keeney, Ralph L.  "The Role of Values in Risk Management." The Annals
    of the American Academy of Political and Social Science.  Eds.
    Howard Kunreuther and Paul Slovic.  Vol.  545 (May 1996).
    Philadelphia:  Sage Periodicals Press, pp.  126-34.

    Kemper, Peter, David A.  Long, and Craig Thornton.  "A Benefit-Cost
    Analysis of the Supported Work Experiment." The National Supported
    Work Demonstration.  Eds.  Robinson G.  Hollister, Peter Kemper, and
    Rebecca A.  Maynard, Madison, Wis.:  University of Wisconsin Press,
    1984, pp.  239-85.

    Kerton, Robert, and Richard Bodell.  "Quality, Choice and the
    Economics of Concealment:  The Marketing of Lemons." Journal of
    Consumer Affairs, 29 (1) (Summer 1995), pp.  1-28.

    Kopp, Raymond J.  "Why Existence Value SHOULD Be Used in Cost-Benefit
    Analysis." Journal of Policy Analysis and Management, 11 (1) (1992),
    pp.  123-30.

    Langlois, Jean A., and others.  "Improving the E Coding of
    Hospitalizations for Injury:  Do Hospital Records Contain Adequate
    Documentation?" American Journal of Public Health, 85 (9) (Sept.
    1995), pp.  1261-65.

    Laughery, Kenneth R., David R.  Lovvoll, and Meredith L.  McQuilkin.
    "Allocation of Responsibility for Child Safety." Proceedings of the
    Human Factors and Ergonomics Society 40th Annual Meeting.  Santa
    Monica, Calif.:  Human Factors and Ergonomics Society, 1996, pp.
    810-13.

    Lave, Lester B.  "Benefit Cost Analysis:  Do the Benefits Exceed the
    Costs?" Risks, Costs and Lives Saved:  Getting Better Results From
    Regulation.  Ed.  Robert W.  Hahn.  Washington, D.C.:  AEI Press,
    1996, pp.  104-34.

    Layard, Richard.  Cost-Benefit Analysis.  Middlesex, England:
    Penguin Books, 1972.

    Lescohier, Ilana, Susan S.  Gallagher, and Bernard Guyer.  "Not by
    Accident." Issues in Science and Technology, Summer 1990, pp.  35-42.

    Lichtenberg, Erik.  "Conservatism in Risk Assessment and Food Safety
    Policy." The Economics of Food Safety.  Ed.  Julie A.  Caswell.
    London:  Elsevier, 1991, pp.  89-102.

    Linneman, Peter.  "The Effects of Consumer Safety Standards:  The
    1973 Mattress Flammability Standard." George J.  Stigler Chicago
    Studies in Political Economy.  Chicago, Ill.:  University of Chicago
    Press, 1988, pp.  441-60.

    Litan, Robert E., and William D.  Nordhaus.  Reforming Federal
    Regulation.  New Haven:  Yale University Press, 1983.

    Loomis, John B., and Pierre H.  duVair.  "Evaluating the Effect of
    Alternative Risk Communication Devices on Willingness to Pay:
    Results From a Dichotomous Choice Contingent Valuation Experiment."
    Land Economics, 69 (3) (Aug.  1993), pp.  287-98.

    Lovvoll, David R., and others.  "Responsibility for Product Safety in
    the Work Environment." Proceedings of the Human Factors and
    Ergonomics Society 40th Annual Meeting.  Santa Monica, Calif.:  Human
    Factors and Ergonomics Society, 1996, pp.  814-17.

    MacLean, D.  Values at Risk.  Totowa, N.J.:  Rowman and Allanheld,
    1986.

    Magat, Wesley A., and Michael J.  Moore.  "Consumer Product Safety
    Regulation in the United States and the United Kingdom:  The Case of
    Bicycles." Rand Journal of Economics, 27 (1) (Spring 1996), pp.
    148-64.

    Magat, Wesley A., and W.  Kip Viscusi.  Informational Approaches to
    Regulation.  Cambridge, Mass.:  MIT Press, 1992.

    Mayer, Michelle, and Felicia B.  LeClere.  "Injury Prevention
    Measures in Households with Children in the United States, 1990."
    Advance Data, No.  250 (May 31, 1994).

    McCaig, Linda F.  "National Hospital Ambulatory Medical Care Survey:
    1992 Emergency Department Summary." Advance Data, No.  245 (Mar.  2,
    1994).

    McGarity, Thomas O.  "Media Quality, Technology, and Cost Benefit
    Balancing Strategies for Health and Environmental Regulation." Law
    and Contemporary Problems, 46 (3) (1983), pp.  159-233.

    _____, and Sidney A.  Shapiro.  Workers at Risk:  The Failed Promise
    of the Occupational Safety and Health Administration.  Westport,
    Conn.:  Praeger Press, 1993.

    McGinnis, J.M., and W.H.  Foege.  "Actual Causes of Death in the
    United States." Journal of the American Medical Association, Vol.
    270, pp.  2207-12.

    McKean, Kevin.  "They Fly in the Face of Danger." Discover, Apr.
    1986, pp.  48-58.

    Miller, James C.  III, and Bruce Yandle, eds.  Benefit-Cost Analyses
    of Social Regulation:  Case Studies From the Council on Wage and
    Price Stability.  Washington, D.C.:  American Enterprise Institute,
    1979.

    Miller, Ted R., and Diane C.  Lestina.  "Patterns in U.S.  Medical
    Expenditures and Utilization for Injury, 1987." American Journal of
    Public Health, 86 (1) (Jan.  1996), pp.  89-93.

    Mishan, E.J.  Cost-Benefit Analysis.  London:  Unwin Hyman, 1988.

    Moore, John L.  Cost Benefit Analysis:  Issues in Its Use in
    Regulation.  Report No.  95-760 ENR.  Washington, D.C.:
    Congressional Research Service, June 28, 1995.

    Morgan, M.  Granger, and Max Henrion.  Uncertainty:  A Guide to
    Dealing With Uncertainty in Quantitative Risk and Policy Analysis.
    Cambridge, Mass.:  Cambridge University Press, 1990.

    Nash, Chris.  "Cost-benefit Analysis of Transport Projects." Alan
    Williams and Emilio Giardina.  Efficiency in the Public Sector.
    Aldershot, Hants, England:  Edward Elgar Publishing, 1993, pp.
    83-105.

    National Committee for Injury Prevention and Control.  Injury
    Prevention:  Meeting the Challenge.  New York:  Oxford University
    Press, 1989.

    Needleman, Carolyn, and Martin L.  Needleman.  "Qualitative Methods
    for Intervention Research." American Journal of Industrial Medicine,
    29 (1996), pp.  329-37.

    Nelson, Cheryl R., and Barbara J.  Stussman.  "Alcohol- and
    Drug-Related Visits to Hospital Emergency Departments:  1992 National
    Hospital Ambulatory Medical Care Survey." Advance Data, No.  251
    (Aug.  10, 1994).

    Newbery, David.  "The Isolation Paradox and the Discount Rate for
    Benefit-Cost Analysis:  A Comment." Quarterly Journal of Economics,
    Feb.  1990, pp.  235-8.

    Oi, Walter Y.  "Safety at What Price?" American Economic Review, AEA
    Papers and Proceedings, 85 (2) (May 1995), pp.  68-71.

    Ozonoff, Victoria Vespe, Susan Tan-Torres, and Catherine W.  Barber.
    "Assessment of E-Coding Practices and Costs in Massachusetts
    Hospitals." Public Health Reports, 108 (5) (Sept.-Oct.  1993), pp.
    633-6.

    Page, Talbot.  "A Framework for Unreasonable Risk in the Toxic
    Substances Control Act (TSCA)." Management of Assessed Risk for
    Carcinogens.  Ed.  William J.  Nicholson.  Annals of the New York
    Academy of Sciences, New York:  1981.

    _____, and Paolo F.  Ricci.  "A Cost-Benefit Perspective for Risk
    Assessment." Principles of Health Risk Assessment.  Ed.  Paolo F.
    Ricci.  Englewood Cliffs, N.J.:  Prentice-Hall, Inc., 1985, pp.
    37-65.

    Pease, William S.  The Impact of Scientific Research on Federal
    Agency Guidelines for Conducting Health Risk Assessments.
    Washington, D.C.:  U.S.  Office of Technology Assessment, Dec.  18,
    1992.

    Peltzman, Sam.  "The Effect of Automobile Safety Regulation." George
    J.  Stigler Chicago Studies in Political Economy.  Chicago, Ill.  :
    University of Chicago Press, 1988, pp.  349-403 (originally published
    in Journal of Political Economy, 83, 1975, pp.  677-725).

    Petty, Ross D.  "Bicycle Safety:  A Case Study in Regulatory Review."
    Regulation, 17 (2) (1994), pp.  22-24.

    _____.  "Regulating Product Safety:  The Informational Role of the
    U.S.  Federal Trade Commission." Journal of Consumer Policy, 18
    (1995), pp.  387-415.

    Pollack, E.S., and D.G.  Keimig, eds.  Counting Injuries and
    Illnesses in the Workplace:  Proposals for a Better System.  New
    York:  National Academy Press, 1987.

    Price, Colin.  "Does Social Cost-Benefit Analysis Measure Overall
    Utility Change?" Economics Letters, 26 (1988), pp.  357-61.

    Proulx, Guylene, and Joelle Pineau.  "The Impact of Age on Occupants'
    Behavior During a Residential Fire." Unpublished paper, 1996.

    Quiggin, John.  "Existence Value and Cost-Benefit Analysis:  A Third
    View." Journal of Policy Analysis and Management, 12 (1) (1993), pp.
    195-9.

    _____.  "Too Many Proposals Pass the Benefit-Cost Test:  Comment."
    American Economic Review, 81 (5) (Dec.  1991), pp.  1446-9.

    Ramirez, Jorge, and others.  "Ex-Post Analysis of Flood Control:
    Benefit-Cost Analysis and the Value of Information." Water Resources
    Research, 24 (8) (Aug.  1988), pp.  1397-1405.

    Rauchschwalbe, Renae, and N.  Clay Mann.  "Pediatric Window-Cord
    Strangulations in the United States, 1981-1995." Journal of the
    American Medical Association, 277 (21) (June 4, 1997), pp.  1696-98.

    Rice, Dorothy P., and Wendy Max.  "Annotation:  The High Cost of
    Injuries in the United States." American Journal of Public Health, 86
    (1)
    (Jan.  1996), pp.  14-15.

    Risa, Alf Erling.  "Public Regulation of Private Accident Risk:  The
    Moral Hazard of Technological Improvements." Journal of Regulatory
    Economics, 4 (1992), pp.  335-46.

    Rivara, Frederick P., and others.  "Cost Estimates for Statewide
    Reporting of Injuries by E Coding Hospital Discharge Abstract Data
    Base Systems." Public Health Reports, 105 (6) (Nov.-Dec.  1990), pp.
    635-38.

    Robins, Marcia P.  "An Examination of the Effect of CPSC Prescription
    Drug Child-Resistant Packaging Requirements on Child Fatalities."
    Unpublished paper, Nov.  1989.

    Rodgers, Gregory B.  "The Effectiveness of Helmets in Reducing
    All-Terrain Vehicle Injuries and Deaths." Accident Analysis and
    Prevention, 22 (1) (1990), pp.  47-58.

    _____.  "Evaluating Product-Related Hazards at the Consumer Product
    Safety Commission:  The Case of All-Terrain Vehicles." Evaluation
    Review, Feb.  1990, pp.  3-21.

    _____, and P.  Rubin.  "Cost-Benefit Analysis of All-Terrain Vehicles
    at the CPSC." Risk Analysis, 9 (1989), pp.  63-9.

    Rosenthal, Donald H., and Robert H.  Nelson.  "Why Existence Value
    Should NOT Be Used in Cost-Benefit Analysis." Journal of Policy
    Analysis and Management, 11 (1) (1992), pp.  116-22.

    Rubin, Paul H., R.  Dennis Murphy, and Gregg Jarrell.  "Risky
    Products, Risky Stocks." Regulation, No.  1 (1988), pp.  35-9.

    Rubinfeld, Daniel L., and Gregory B.  Rodgers.  "Evaluating the
    Injury Risk Associated With All-Terrain Vehicles:  An Application of
    Bayes' Rule." Journal of Risk and Uncertainty, 5 (1992), pp.  145-58.

    Schappert, Susan M.  "National Ambulatory Medical Care Survey:  1992
    Summary." Advance Data, No.  253 (Aug.  18, 1994).

    Schierow, Linda-Jo.  "Risk Analysis and Cost-Benefit Analysis of
    Environmental Regulations.  94-961 ENR.  Washington, D.C.  :  U.S.
    Congressional Research Service, Dec.  2, 1994.

    Schroeder, Elinor P., and Sidney A.  Shapiro.  "Responses to
    Occupational Disease:  The Role of Markets, Regulation, and
    Information." Georgetown Law Journal, 72 (1983), pp.  1231-1306.

    Selbst, S., M.D.  Baker, and M.  Shames.  "Bunk Bed Injuries."
    American Journal of Diseases in Children, 144 (6) (June 1990), pp.
    721-3.

    Sen, Amartya.  "Approaches to the Choice of Discount Rates for Social
    Benefit-Cost Analysis." Chapter 8 in Resources, Values, and
    Development.  Cambridge, Mass.:  Harvard University Press, 1984, pp.
    172-203.

    Senturia, Yvonne D., and others.  "Exposure-Corrected Risk Estimates
    for Childhood Product-Related Injuries." American Journal of Public
    Health, 1993, pp.  473-7.

    Shew, Russel, and Rachel Dardis.  "An Economic Analysis of Child
    Restraints." Journal of Consumer Policy, 18 (1995), pp.  417-31.

    Smith, V.  Kerry.  "A Conceptual Overview of the Foundations of
    Benefit-Cost Analysis." Benefits Assessment:  The State of the Art.
    Eds.  Judith D.  Bentkover, Vincent T.  Covello, and Jeryl Mumpower.
    Dordrecht, Holland:  D.  Reidel Publishing Company, 1986, pp.  13-34.

    _____.  "Uncertainty, Benefit-Cost Analysis, and the Treatment of
    Option Value." Journal of Environmental Economics and Management, 14
    (1987), pp.  283-92.

    _____, and others.  "Can Public Information Programs Affect Risk
    Perceptions?" Journal of Policy Analysis and Management, 9 (1)
    (1990), pp.  41-59.

    Spitzer, Hugh.  Case Studies to Demonstrate the Impact of Research on
    Assessment of Carcinogenic Risk.  Washington, D.C.:  U.S.  Office of
    Technology Assessment, Dec.  1, 1992.

    Stason, W.B., and M.C.  Weinstein.  "Allocation of Resources to
    Manage Hypertension." New England Journal of Medicine, 296 (13)
    (1977), pp.  732-9.

    Stayner, Leslie, and others.  "Approaches for Assessing the Efficacy
    of Occupational Health and Safety Standards." American Journal of
    Industrial Medicine, 29 (1996), pp.  353-7.

    _____.  "National Hospital Ambulatory Medical Care Survey:  1994
    Emergency Department Summary." Advance Data, No.  275 (May 17, 1996).

    Stussman, Barbara J.  "National Hospital Ambulatory Medical Care
    Survey:  1993 Emergency Department Summary." Advance Data, No.  271
    (Jan.  25, 1996).

    Tengs, Tammy O., and John D.  Graham.  "The Opportunity Costs of
    Haphazard Social Investments in Life Saving." Risks, Costs and Lives
    Saved:  Getting Better Results from Regulation.  Ed.  Robert W.
    Hahn.  Washington, D.C.:  AEI Press, 1996, pp.  167-82.

    Thacker, Stephen B., and others.  "Assessing Prevention Effectiveness
    Using Data to Drive Program Decisions." Public Health Reports, 109
    (2) (Mar.-Apr.  1994), pp.  187-94.

    Thomas, L.G.  "Revealed Bureaucratic Preference:  Priorities of the
    Consumer Product Safety Commission." Rand Journal of Economics, 19
    (1) (Spring 1988), pp.  102-13.

    Thompson, M.S., J.L.  Read, and M.  Laing.  "Feasibility of
    Willingness-to-Pay Measurement in Chronic Arthritis." Medical
    Decision Making, 4 (2)
    (1984), pp.  195-215.

    Tolchin, Susan J., and Martin Tolchin.  "Particles of Truth:  Cotton
    Dust and Cost-Benefit Analysis." Dismantling America:  The Rush to
    Deregulate, Boston, Mass.:  Houghton Mifflin Company, 1983, pp.
    111-41.

    Tolley, G., K.  Kenkel, and R.  Rabian, eds.  Valuing Health for
    Policy:  An Economic Approach.  Chicago, Ill.:  University of Chicago
    Press, 1994.

    Trinkoff, Alison, and Peggy L.  Parks.  "Prevention Strategies for
    Infant Walker-Related Injuries." Public Health Reports, 108 (6)
    (Nov.-Dec.  1993), pp.  784-8.

    Trumbull, William.  "Reply to Whittington and MacRae." Journal of
    Policy Analysis and Management, 9 (4) (1990), pp.  548-50.

    _____.  "Who Has Standing in Cost-Benefit Analysis?" Journal of
    Policy Analysis and Management, 9 (2) (1990), pp.  201-18.

    U.S.  Commission on Risk Assessment and Risk Management.  Risk
    Assessment and Risk Management in Regulatory Decision-Making.
    Washington, D.C.:  1997.

    U.S.  Department of Health and Human Services, Centers for Disease
    Control and Prevention.  "A Framework for Assessing the Effectiveness
    of Disease and Injury Prevention." Morbidity and Mortality Weekly
    Report, Recommendations and Reports, Vol.  41, No.  RR-3 (Mar.  27,
    1992), pp.  1-13.

    _____.  "Guidelines for Death Scene Investigation of Sudden
    Unexplained Infant Deaths." Morbidity and Mortality Weekly Report,
    Recommendations and Reports, Vol.  45, No.  RR-10 (June 21, 1996),
    pp.  1-31.

    _____.  "Injuries and Deaths Associated With the Use of
    Snowmobiles--Maine, 1991-1996." Morbidity and Mortality Weekly
    Report, Vol.  46, No.  1 (Jan.  10, 1997), pp.  1-4.

    _____.  Inventory of Federal Data Systems in the United States for
    Injury Surveillance, Research and Prevention Activities.  Washington,
    D.C.:  May 1996.

    U.S.  Department of Health and Human Services, Centers for Disease
    Control and Prevention, National Center for Health Statistics.
    "Advance Report of Final Mortality Statistics, 1994." Monthly Vital
    Statistics Report, 45 (3) (Supplement, Sept.  30, 1996).

    _____.  "Hospitalizations for Injury and Poisoning in the United
    States, 1991." Advance Data, No.  252 (Oct.  7, 1994).

    U.S.  Department of Health and Human Services, Public Health Service.
    Healthy People 2000:  Midcourse Review and 1995 Revisions.
    Washington, D.C.:  1995.

    _____.  Healthy People 2000:  National Health Promotion and Disease
    Objectives.  Washington, D.C.:  1991.

    U.S.  General Accounting Office.  Peer Review:  EPA's Implementation
    Remains Uneven.  Sept.  24, 1996, GAO/RCED-96-236.

    U.S.  Office of Management and Budget.  Guidelines and Discount Rates
    for Benefit-Cost Analysis of Federal Programs.  Circular A-94.
    Washington, D.C.:  Oct.  29, 1992.

    _____.  Regulatory Program of the United States Government.  Bulletin
    No.  91-04.  Washington, D.C.:  Nov.  26, 1990.

    U.S.  Office of Technology Assessment.  Researching Health Risks.
    OTA-BBS-570.  Washington, D.C.:  Jan.  1993.

    _____.  Risks to Students in School.  OTA-ENV-633.  Washington, D.C.:
    Sept.  1995.

    U.S.  Physician Payment Review Commission.  A Comparison of
    Alternative Approaches to Risk Measurement.  Washington, D.C.:  Dec.
    1994.

    van Ravenswaay, Eileen O., and John P.  Hoehn.  "The Impact of Health
    Risk Information on Food Demand:  A Case Study of Alar and Apples."
    The Economics of Food Safety.  Ed.  Julie A.  Caswell.  London:
    Elsevier, 1991, pp.  155-174.

    Veasie, M.A., and others.  "Epidemiologic Research on the Etiology of
    Injuries at Work." Annual Review of Public Health, 15 (1994), pp.
    203-21.

    Viscusi, W.  Kip.  "The Dangers of Unbounded Commitments to Regulate
    Risk." Risks, Costs and Lives Saved:  Getting Better Results From
    Regulation.  Ed.  Robert W.  Hahn.  Washington, D.C.:  AEI Press,
    1996, pp.  135-66.

    _____.  Regulating Consumer Product Safety.  Washington, D.C.:
    American Enterprise Institute, 1984.

    _____.  Tax Policy and the Economy.  Ed.  James M.  Porterba.  Vol.
    9, 1995, p.  180.

    _____.  "The Valuation of Risks to Life and Health:  Guidelines for
    Policy Analysis." Benefits Assessment:  The State of the Art.  Eds.
    Judith D.  Bentkover, Vincent T.  Covello, and Jeryl Mumpower.
    Dordrecht, Holland:  D.  Reidel Publishing Company, 1986, pp.
    193-210.

    _____.  "The Value of Risks to Life and Health." Journal of Economic
    Literature, 31 (Dec.  1993), pp.  1912-46.

    _____, and Richard J.  Zeckhauser.  "Hazard Communication:  Warnings
    and Risk." The Annals of the American Academy of Political and Social
    Science.  Eds.  Howard Kunreuther and Paul Slovic.  Vol.  545 (May
    1996).  Philadelphia:  Sage Periodicals Press, pp.  106-15.

    _____, and Wesley A.  Magat, eds.  Learning About Risk:  Consumer and
    Worker Responses to Hazard Information.  Cambridge, Mass.:  Harvard
    University Press, 1987.

    Waller, Julian A.  "Reflections on a Half Century of Injury Control."
    American Journal of Public Health, 84 (4) (Apr.  1994), pp.  664-70.

    _____, Joan M.  Skelly, and John H.  Davis.  "Treated Injuries in
    Northern Vermont." Accident Analysis and Prevention, 27 (6) (1995),
    pp.  819-28.

    Warr, Peter G., and Brian D.  Wright.  "The Isolation Paradox and the
    Discount Rate for Benefit-Cost Analysis." Quarterly Journal of
    Economics, 96 (1) (Feb.  1981), pp.  129-45.

    Weinstein, M.C., and others.  "Recommendations of the Panel on
    Cost-Effectiveness in Health and Medicine." Journal of the American
    Medical Association, 276 (15) (1996), pp.  1253-8.

    Weisbrod, Burton A.  "Benefit-Cost Analysis of a Controlled
    Experiment:  Treating the Mentally Ill." Journal of Human Resources,
    16 (4) (1981), pp.  523-48.

    Weiss, K.B., P.J.  Gergen, and T.A.  Hodgson.  "An Economic
    Evaluation of Asthma in the United States." New England Journal of
    Medicine, 326, pp.  862-6.

    Wheelwright, Jeff.  "The Air of Ostrava:  Pollution and Risk
    Assessment in the Czech Republic." Discover, 17 (5) (May 1996), pp.
    56-64.

    Whittington, Dale, and Duncan MacRae, Jr.  "Comment:  Judgments About
    Who Has Standing in Cost-Benefit Analysis." Journal of Policy
    Analysis and Management, 9 (4) (1990), pp.  536-47.

    Wildasin, David E.  "Indirect Distributional Effects in Benefit-Cost
    Analysis of Small Projects." The Economic Journal, 98 (Sept.  1988),
    pp.  801-7.

    Williams, Alan.  "Cost-Benefit Analysis:  Applied Welfare Economics
    or General Decision Aid?" Alan Williams and Emilio Giardina.
    Efficiency in the Public Sector.  Aldershot, Hants, England:  Edward
    Elgar Publishing, 1993, pp.  65-82.

    Wood, William W.  "Cost-Benefit Analysis of Small Business
    Assistance:  Do Entrepreneurs Really Need `Assisting'?" Journal of
    Private Enterprise, 10 (1) (Summer 1994), pp.  13-21.

    Zeckhauser, Richard J., and W.  Kip Viscusi.  "The Risk Management
    Dilemma." The Annals of the American Academy of Political and Social
    Science.  Eds.  Howard Kunreuther and Paul Slovic.  Vol.  545 (May
    1996).  Philadelphia:  Sage Periodicals Press, pp.  144-55.

    Zerbe, Richard O.  "Comment:  Does Benefit Cost Analysis Stand Alone?
    Rights and Standing." Journal of Policy Analysis and Management, 10
    (1) (1991), pp.  96-105.

    *** End of document. ***