Dietary Supplements: Uncertainties in Analyses Underlying FDA's Proposed
Rule on Ephedrine Alkaloids (Letter Report, 07/02/1999,
GAO/HEHS/GGD-99-90).

In 1997, the Food and Drug Administration (FDA) published a proposed
rule that would establish a dosing regimen, require warning statements,
and affect other aspects of product labeling for dietary supplements
containing ephedrine alkaloids, which are promoted as helping
individuals lose weight and increase energy levels. GAO is concerned
that the proposed dosing level was based on information associated with
only 13 adverse event reports and that the proposed duration-of-use
limits were based on scientific studies showing problems with extended
use well beyond the proposal's seven-day limit. FDA did not establish a
causal link between the ingestion of ephedrine alkaloids and adverse
events for either of these two aspects of its rule. FDA did not document
the basis for its estimate of benefits from the rule sufficiently to
allow GAO to determine the estimate's accuracy. FDA has no internal
guidance on using adverse events reports for rulemaking related to
dietary supplements, and its use of reports for this rule was different
from its use in earlier rulemaking. FDA generally complied with
statutory and executive order requirements for rulemaking but did not
disclose why it made key assumptions in its cost-benefit analysis, the
degree of their uncertainty, or alternative assumptions that would have
dramatically affected its estimate of benefits.

--------------------------- Indexing Terms -----------------------------

 REPORTNUM:  HEHS/GGD-99-90
     TITLE:  Dietary Supplements: Uncertainties in Analyses Underlying
	     FDA's Proposed Rule on Ephedrine Alkaloids
      DATE:  07/02/1999
   SUBJECT:  Nutrition research
	     Health statistics
	     Agency proceedings
	     Food and drug legislation
	     Proposed legislation
	     Cost effectiveness analysis
	     Health hazards
	     Safety standards
	     Product safety
	     Safety regulation
IDENTIFIER:  CPSC National Electronic Injury Surveillance System

******************************************************************
** This file contains an ASCII representation of the text of a  **
** GAO report.  This text was extracted from a PDF file.        **
** Delineations within the text indicating chapter titles,      **
** headings, and bullets have not been preserved, and in some   **
** cases heading text has been incorrectly merged into          **
** body text in the adjacent column.  Graphic images have       **
** not been reproduced, but figure captions are included.       **
** Tables are included, but column deliniations have not been   **
** preserved.                                                   **
**                                                              **
** Please see the PDF (Portable Document Format) file, when     **
** available, for a complete electronic file of the printed     **
** document's contents.                                         **
**                                                              **
** A printed copy of this report may be obtained from the GAO   **
** Document Distribution Center.  For further details, please   **
** send an e-mail message to:                                   **
**                                                              **
**                                            **
**                                                              **
** with the message 'info' in the body.                         **
******************************************************************

    United States General Accounting Office GAO
    Report to the Chairman and Ranking Minority Member, Committee on
    Science, House of Representatives July 1999             DIETARY
    SUPPLEMENTS Uncertainties in Analyses Underlying FDA's Proposed
    Rule on Ephedrine Alkaloids GAO/HEHS/GGD-99-90 GAO    United
    States General Accounting Office Washington, D.C. 20548 Health,
    Education, and Human Services Division B-281581 July 2, 1999 The
    Honorable James Sensenbrenner, Jr. Chairman The Honorable George
    E. Brown, Jr. Ranking Minority Member Committee on Science House
    of Representatives The dietary supplement industry estimates that
    as many as 2 billion doses of dietary supplements containing
    botanical ephedrine alkaloids are consumed each year in the United
    States. The principal source of botanical ephedrine alkaloids is
    the Chinese herb ma huang. Traditionally, this substance was used
    by Chinese physicians to treat conditions such as hay fever and
    asthma. In Western medicine, synthetic ephedrine alkaloids have
    been used in products such as over-the-counter asthma and cold
    medicines, while botanical ephedrine alkaloids have been used in
    dietary supplements that are promoted to provide health benefits,
    including helping individuals lose weight and increase energy
    levels. In June 1997, the Food and Drug Administration (FDA)
    published a proposed rule that would establish a dosing regimen,
    require warning statements, and affect other aspects of product
    labeling for dietary supplements containing ephedrine alkaloids.
    FDA developed its proposed rule in response to what it termed
    "serious illnesses and injuries, including multiple deaths,
    associated with the use of dietary supplement products that
    contain ephedrine alkaloids." FDA based its rule, in part, on a
    number of adverse events reports (AER) it received that indicated
    that some health problems could have been associated with use of
    dietary supplements containing ephedrine alkaloids. FDA determined
    that its proposed rule on dietary supplements containing ephedrine
    alkaloids was an economically significant rule and, therefore, was
    required to conduct a cost-benefit analysis pursuant to the
    requirements in Executive Order 12866.1 Other regulatory analysis
    requirements for federal rulemaking are those under the Regulatory
    Flexibility Act (RFA), which directs agencies to consider the
    potential impact of regulation on small businesses and other small
    entities,2 and the Unfunded Mandates Reform Act (UMRA), which
    generally requires agencies 1E.O. 12866 was issued on September
    30, 1993, and covers all agencies except independent regulatory
    agencies. 25 U.S.C. 601-612. Page 1
    GAO/HEHS/GGD-99-90 FDA's Proposed Rule on Ephedrine Alkaloids B-
    281581 to prepare cost-benefit analyses for any proposed
    regulation that would impose mandates likely to result in
    expenditures of $100 million or more in any 1 year.3 Industry
    groups and the Small Business Administration's (SBA) Office of
    Advocacy have challenged FDA's proposed rule, claiming that the
    scientific information and AERs supporting the rule were poor and
    unreliable. Further, industry groups and the Office of Advocacy
    claimed that the cost-benefit analysis performed by FDA
    understated the costs of the regulation and overestimated the
    benefits. They also had concerns about compliance with RFA's and
    UMRA's procedural requirements. In light of the concerns expressed
    by SBA and industry groups, you asked us to examine (1) the
    scientific basis for FDA's proposed rule and (2) the agency's
    adherence to the regulatory analysis requirements for federal
    rulemaking. In examining the basis for the rule, you asked that we
    examine scientific evidence, FDA's past use of and internal
    guidance on AERs in rulemaking, and information contained in the
    AERs for determining dosing regimens and benefits that would arise
    from the proposed rule. Regarding the rulemaking requirements, you
    asked that we determine the extent to which FDA's analysis
    contained elements expected in a federal agency's cost-benefit
    analysis and to analyze FDA's compliance with the requirements in
    RFA and UMRA. To meet these objectives, we interviewed
    representatives of and obtained documents from FDA, SBA's Office
    of Advocacy, the Office of Information and Regulatory Affairs
    (OIRA) in the Office of Management and Budget (OMB), and the
    dietary supplement industry. We also examined the public docket of
    the proposed rule, performed a content analysis of a random sample
    of AERs, and reviewed scientific literature and case reports of
    adverse events from products containing ephedrine alkaloids. We
    conducted our review from September 1998 to May 1999 in accordance
    with generally accepted government auditing standards. (See app. I
    for further information on our scope and methodology.) Results in
    Brief    FDA based its proposed rule on numerous reports of
    adverse events associated with products thought to contain
    ephedrine alkaloids; it also used evidence from scientific
    literature indicating that ingestion of ephedrine alkaloids
    adversely affects some individuals. The number and type of AERs
    warranted FDA's consideration of steps to address safety 32 U.S.C.
    638 and 1532. Page 2                     GAO/HEHS/GGD-99-90 FDA's
    Proposed Rule on Ephedrine Alkaloids B-281581 issues. However, we
    have concerns about the strength of some of the information FDA
    used to support two aspects of the proposed rule: the dosing level
    and duration of use limits. While there was scientific evidence
    showing adverse events at levels above 20 mg per serving, FDA's
    dosing level proposal of 8 mg per serving was based on information
    associated with only 13 AERs-the quality of which is questionable.
    For the duration of use limits, FDA relied on scientific studies
    that showed problems associated with extended use, well beyond the
    7-day limit proposed. Moreover, FDA did not establish a causal
    link between the ingestion of ephedrine alkaloids and the
    occurrence of adverse events for either its proposed dosing level
    or duration of use. FDA also based its estimate of the benefits of
    the proposed rule on the annual number of adverse events reported
    to FDA. However, because FDA did not document which AERs it
    identified as containing "serious" events, we could not determine
    the accuracy of FDA's estimated benefits. In addition, FDA has no
    internal guidance on the use of AERs for rulemaking related to
    foods and dietary supplements, and the AERs were used differently
    in this proposed rule than in prior rulemaking. The agency
    generally complied with the statutory and executive order
    requirements for rulemaking, but the cost-benefit analysis that
    accompanied the rule does not reflect the full range of
    uncertainty associated with the proposed rule. FDA's cost-benefit
    analysis and other analyses included the primary elements required
    under E.O. 12866 and related "best practices" guidance and RFA.
    UMRA's requirements did not appear to apply to the rule. Although
    FDA disclosed the basic methodology, data, and assumptions used in
    its cost-benefit analysis, the agency did not always disclose why
    certain key assumptions were made or the degree of uncertainty
    involved in those assumptions. It also did not disclose that
    alternative assumptions would have had a dramatic effect on the
    agency's estimate of the benefits of the proposed actions. While
    FDA's conclusions regarding the desirability of the proposed
    action may be valid, we believe these conclusions are open to
    question because of limitations and uncertainties associated with
    the agency's scientific and economic analyses. Given these
    uncertainties, we recommend that FDA obtain additional information
    to support the proposed dosing levels and duration of use limits
    and improve the transparency of its cost-benefit analysis before
    proceeding to final rulemaking. Page 3              GAO/HEHS/GGD-
    99-90 FDA's Proposed Rule on Ephedrine Alkaloids B-281581
    Background    FDA is responsible for overseeing the federal
    government's regulation of drugs, medical devices, food safety,
    veterinary medicine, and biological products. Unlike many of these
    products, dietary supplements do not have to undergo preapproval
    by FDA to determine their safety or efficacy. The Dietary
    Supplement Health and Education Act of 1994 created a new
    framework for FDA's regulation of dietary supplements as part of
    its oversight of food safety.4 The act allows dietary supplement
    products to bear a statement describing how consumption of the
    supplement can affect humans,5 but manufacturers of dietary
    supplements cannot make a drug claim for the product-that is, a
    statement claiming to diagnose, mitigate, treat, cure, or prevent
    a specific disease or class of diseases.6 Dietary supplements
    containing botanical sources of ephedrine alkaloids are currently
    marketed for weight loss and increased energy. These products are
    marketed in a variety of forms, including pills, powders, liquid
    drops, and teas. Currently, FDA has no requirements that the
    packages for these products include dosing regimens and warning
    labels. Over-the-counter products containing synthetic ephedrine
    alkaloids are considered safe and effective for people ages 12 and
    older to temporarily relieve shortness of breath, tightness of
    chest, and wheezing due to bronchial asthma, if recommended
    dosages are followed. The recommended dosages-which range from
    12.5 mg to 25 mg every 4 hours, not to exceed 150 mg per day-are
    provided on the label. Labels for over-the-counter drug products
    containing ephedrine alkaloids list side effects, such as
    nervousness, sleeplessness, nausea, and loss of appetite. The
    labels also warn against using the product if, for example, the
    potential user has high blood pressure, heart disease, thyroid
    disease, or diabetes, or has been hospitalized for asthma or is
    taking a prescription drug for asthma, unless directed by a
    doctor. Labels also note that if symptoms persist or become worse,
    users should discontinue taking the drug and consult a doctor.
    After reports of stroke in asthmatics who had taken ephedrine
    along with prescription monoamine oxidase inhibitors,7 precautions
    against using both drugs concurrently were added to the label.
    4P.L. 103-417. 5Such statements may (1) claim a benefit related to
    a classical nutritional deficiency disease, (2) describe the role
    of a nutrient or dietary ingredient intended to affect the
    structure or function in humans, (3) characterize the documented
    mechanism by which the ingredient acts to maintain such structure
    or function, or (4) describe general well-being from the
    consumption of the ingredient. 621 U.S.C. 321 (g)(1). 7Monoamine
    oxidase inhibitors are drugs used to treat depression, psychiatric
    or emotional conditions, or Parkinson's disease. Page 4
    GAO/HEHS/GGD-99-90 FDA's Proposed Rule on Ephedrine Alkaloids B-
    281581 FDA has no authority to require submission of reports of
    adverse events. FDA must generally rely on consumers or their
    friends and family members, physicians or other health care
    professionals, product manufacturers, and state health agencies to
    voluntarily report adverse events. FDA uses these AERs as a
    passive monitoring tool to identify potentially serious public
    health issues that may be associated with the use of a particular
    product or type of products. Like all passive surveillance
    systems, AERs have certain limitations such as underreporting and
    poor report quality (see app. II for a further discussion of the
    adverse event monitoring system). FDA's Proposed Rule on    FDA's
    analysis of AERs and information from the scientific literature
    relating Ephedrine Alkaloids       to ephedrine alkaloids
    indicated to the agency that there was cause for concern regarding
    a potential public health problem associated with dietary
    supplements containing ephedrine alkaloids. At the time of the
    proposed rule, FDA had received over 800 AERs for products thought
    to contain ephedrine alkaloids-more AERs than the agency had
    received for any other dietary supplements. According to agency
    officials, FDA found that, unlike most AERs related to foods, the
    AERs relating to ephedrine alkaloids commonly involved visits to a
    physician or emergency room. Some of these reports were very
    serious, including effects such as strokes, and the events were
    occurring in a population of young adults in which such serious
    events are not expected. FDA officials also stated that the
    adverse physiological and pharmacological effects from dietary
    supplements believed to contain botanical sources of ephedrine
    alkaloids were similar to those reported for drugs containing
    synthetic sources of ephedrine alkaloids, but these effects were
    being seen at lower doses and potencies for botanicals than in the
    drug products. From 1994 through 1997, FDA took a series of steps
    to respond to what it perceived was a public health concern
    related to these products. In 1994, FDA's Center for Food Safety
    and Applied Nutrition (CFSAN), which oversees dietary supplements,
    issued a medical bulletin outlining potential adverse reactions
    from dietary supplements containing ephedrine alkaloids. The
    agency issued a press release warning consumers not to purchase a
    particular brand of dietary supplement containing ephedrine
    alkaloids that the agency determined could cause severe injury or
    death in some people. Page 5                GAO/HEHS/GGD-99-90
    FDA's Proposed Rule on Ephedrine Alkaloids B-281581 In October
    1995, FDA convened a special work group composed of
    pharmacologists, physicians, and industry representatives to
    address concerns related to the use of dietary supplements
    containing ephedrine alkaloids, but no consensus developed on how
    to address the public health concerns outlined by FDA. In August
    1996, FDA's Food Advisory Committee was asked to provide opinions
    on and rationale for specific ways to address problems associated
    with dietary supplements containing ephedrine alkaloids. Over half
    the committee members concluded that no safe level for ephedrine
    alkaloids could be identified and recommended that these products
    be removed from the market. Most of the other members felt that a
    fairly low level of ephedrine alkaloids would be "reasonably
    safe." The Food Advisory Committee was unable to identify a
    benefit for ephedrine alkaloids in terms of supplementing the
    diet. In June 1997, FDA published a proposed rule regarding
    dietary supplements containing ephedrine alkaloids.8 In two
    subsequent notices the agency reopened and extended the comment
    period on the proposed rule until December 1997.9 The proposed
    rule * defines levels of the amount of ephedrine alkaloids in a
    serving of dietary supplements at and above which the product
    would be deemed adulterated (8 mg), * places restrictions on the
    frequency of use and daily dosages (24 mg or more), * requires
    that labels on these supplements contain a statement warning that
    the product should not be used for more than 7 days, * prohibits
    the use of ephedrine alkaloids with ingredients that have a known
    stimulant effect (for example, caffeine), * prohibits certain
    labeling claims that require long-term intake of the supplements
    to achieve the purported purpose, * requires a warning statement
    in conjunction with claims that encourage short-term excessive
    intake, and * requires a specific warning statement to appear on
    product labels. Federal Regulatory         E.O. 12866 establishes
    certain rulemaking responsibilities for covered Analysis
    Requirements      agencies. Among other things, the order states
    that, in deciding whether and how to regulate, agencies should
    assess all costs and benefits of available regulatory
    alternatives, including both quantifiable and qualitative effects.
    The order also states that, in choosing among 862 Fed. Reg. 30678.
    962 Fed. Reg. 44247 and 62 Fed. Reg. 48968. Page 6
    GAO/HEHS/GGD-99-90 FDA's Proposed Rule on Ephedrine Alkaloids B-
    281581 alternatives, an agency should select those approaches that
    maximize net benefits and "base its decisions on the best
    reasonably obtainable scientific, technical, economic, and other
    information concerning the need for, and consequences of, the
    intended regulation." The order requires agencies to conduct cost-
    benefit analyses for all regulatory actions that are likely to
    result in a $100 million annual effect on the economy or are
    otherwise economically significant. In January 1996, the
    Administrator of OIRA issued "best practices" guidance on
    preparing cost-benefit analyses under the order. The guidance
    indicates that an analysis should contain certain basic elements
    and should be "transparent"-disclosing how the analysis was
    conducted, what assumptions were used, and the implications of
    plausible alternative assumptions. During the past 20 years, the
    Congress has attempted to improve the federal rulemaking process
    by enacting a number of statutes that impose certain analytical
    requirements on agencies issuing proposed regulations. For
    example, RFA of 1980 was enacted in response to concerns about the
    effect that federal regulations could have on small entities. RFA
    directs all agencies to give particular attention to the potential
    impact of regulation on small businesses and other small entities
    and requires consideration of regulatory alternatives that are
    less burdensome to small entities. Under RFA, an agency must
    prepare an initial regulatory flexibility analysis at the time
    proposed rules are issued unless the head of the agency determines
    that the proposed rule would not have a "significant economic
    impact upon a substantial number of small entities." The act also
    requires agencies to ensure that small entities have an
    opportunity to participate in the rulemaking process. Other
    statutory rulemaking requirements are set forth in UMRA. UMRA
    generally requires agencies (other than independent regulatory
    agencies) to prepare cost-benefit and other analyses for any
    regulations imposing mandates that are likely to result in
    expenditures of $100 million or more in any 1 year either by
    state, local, and tribal governments, in the aggregate, or by the
    private sector. Although UMRA's scope and requirements differ from
    E.O. 12866, the provisions on economic analysis are very similar.
    Accordingly, the guidance for implementing the executive order
    states that the economic analysis that the agency prepares should
    also satisfy the requirements of UMRA. Page 7
    GAO/HEHS/GGD-99-90 FDA's Proposed Rule on Ephedrine Alkaloids B-
    281581 FDA Analyses Relied         To develop its proposed rule,
    FDA used a combination of scientific on Poorly
    evidence on the effects of ephedrine alkaloids and a set of
    reports it received on adverse events associated with dietary
    supplements Documented Reports          containing ephedrine
    alkaloids. While the signs and symptoms described of Adverse
    Events           in the AERs were consistent with available
    scientific evidence and known physiologic and pharmacologic
    effects of ephedrine alkaloids, the AERs were poorly documented.
    FDA also used the AERs differently than in its past rulemaking.
    Specifically, the agency used AERs as the sole source of support
    for specific dosing levels, relied on weak information to set
    limits on duration of use, and did not perform a causal analysis
    to determine whether ingestion of a dietary supplement containing
    ephedrine alkaloids caused or contributed to the adverse events.
    FDA also used these AERs to determine the number of serious events
    that could be attributed to the dietary supplements and the
    expected benefits that would arise if the proposed rule were
    implemented. However, FDA did not document which AERs it
    determined to be serious. Moreover, it did not establish criteria
    to determine which events were serious and did not perform any
    reliability assessments of its analyses. Scientific Information
    Our review of the scientific literature found case reports that
    suggested Indicates Ephedrine         that ephedrine alkaloids
    could increase blood pressure in persons with Alkaloids Can Affect
    the    normal and high blood pressure;10 predispose certain
    individuals to Cardiovascular and          tachycardia (rapid
    heart rate);11 and cause cardiomyopathy (disease of the 12
    13 Nervous Systems             heart muscle),  stroke,  or
    myocardial necrosis (death of cells in the heart).14 We also found
    descriptions of adverse events associated with 10S. Chua and S.
    Benrimoj, "Non-Prescription Sympathomimetic Agents and
    Hypertension," Medical Toxicology, Vol. 3 (1988), pp. 387-417.
    11D. McCleave and others, "Compartmental Shift of Potassium-A
    Result of Sympathomimetic Overdose," Australian and New Zealand
    Journal of Medicine, Vol. 8 (1978), pp. 180-83. 12J. Gualtieri and
    C. Harris, "Dilated Cardiomyopathy in a Heavy Ephedrine Abuser,"
    (abstract) Journal of Toxicology, Clinical Toxicology, Vol. 34
    (1996), pp. 581-82. 13A. Bruno and others, "Stroke Associated With
    Ephedrine Use," Neurology, Vol. 43 (1993), pp. 1313-16. 14P.
    Pentel and others, "Myocardial Necrosis Due to Intraperitoneal
    Administration of Phenylpropanolamine in Rats," Fundamental and
    Applied Toxicology, Vol. 9 (1987), pp. 167-72. Page 8
    GAO/HEHS/GGD-99-90 FDA's Proposed Rule on Ephedrine Alkaloids B-
    281581 ephedrine alkaloids that affected the central nervous
    system, such as mania,15 paranoid psychoses,16 and seizures.17 Our
    review found only one clinical study examining the effects of a
    botanical source of ephedrine alkaloids, ma huang, on heart rate
    and blood pressure. The study of adults with normal blood pressure
    found a statistically-but not clinically-significant increase in
    heart rate but variable effects on blood pressure.18 However, "in
    combination with other stimulants and at higher doses, effects of
    magnification of the heart rate and blood pressure response could
    be expected." One report also highlighted a case of mania as a
    result of ingestion of a dietary supplement containing ma huang.19
    We also reviewed a number of studies using clinical trials that
    evaluated the efficacy of ephedrine (a synthetic ephedrine
    alkaloid)-alone or in combination with caffeine-for weight loss.
    Although these trials were not designed to determine whether
    ephedrine is safe to use during weight loss programs, FDA
    concluded that this body of literature showed "clinically
    significant adverse effects in populations with no known risk
    factors with the use of ephedrine and that synergistic adverse
    effects can result when ephedrine and caffeine are combined." FDA
    further concluded that "The patterns and types of the adverse
    effects reported in these trials are consistent with the known
    effects of sympathomimetic agents20-that is, they mainly involved
    nervous and cardiovascular system effects." AERs Were Used
    To develop FDA's proposed rule on dietary supplements containing
    Differently for Developing    ephedrine alkaloids, CFSAN used AERs
    differently than in prior FDA the Proposed Rule Than
    rulemaking. FDA officials acknowledge that the agency does not
    have any for Prior Rulemaking          formal internal guidelines
    on the use of AERs for rulemaking and provided five examples of
    past rulemaking in which CFSAN used several sources of information
    in addition to AERs to support the agency's proposals. For the
    15W. Clovis, "Mania and Cough Syrup," Journal of Clinical
    Psychiatry, Vol. 54 (1993), p. 200. 16M. T. Lambert, "Paranoid
    Psychoses After Abuse of Proprietary Cold Remedies," British
    Journal of Psychiatry, Vol. 151 (1987), pp. 548-50. 17S. M.
    Mueller and E. B. Solow, "Seizures Associated With a New
    Combination Pick-Me-Up Pill," Annals of Neurology, Vol. 11 (1982),
    p. 322. 18L. White and others, "Pharmacokinetics and
    Cardiovascular Effects of Ma-huang (Ephedra sinica) in
    Normotensive Adults," Journal of Clinical Pharmacology, Vol. 37
    (1997), pp. 116-22. 19R. R. Capwell, "Ephedrine-Induced Mania From
    an Herbal Diet Supplement" (letter), American Journal of
    Psychiatry, Vol. 152 (1995), p. 647. 20Such agents produce
    physiological responses resembling those that are caused by the
    action of the sympathetic nervous system, as in tending to reduce
    digestive secretions and speeding up the heart. Page 9
    GAO/HEHS/GGD-99-90 FDA's Proposed Rule on Ephedrine Alkaloids B-
    281581 proposed rule on dietary supplements containing ephedrine
    alkaloids, CFSAN used AERs alone to determine specific dosing
    levels.21 In a prior rulemaking, FDA used prevalence estimates
    among people with asthma to determine sensitive populations who
    might be affected by sulfiting agents22 and convened a scientific
    panel to examine exposure estimates and evidence from clinical
    experiments.23 For its rulemaking on protein supplements, FDA used
    data collected through a telephone survey, conducted jointly with
    the Centers for Disease Control and Prevention (CDC), and
    information on death rates from the National Center for Health
    Statistics. FDA also initiated its own experimental protocols to
    examine the basic metabolic mechanisms of protein diets.24 FDA's
    proposed rule on supplements and drugs containing iron relied on
    data from the American Association of Poison Control Centers and
    the U.S. Consumer Product Safety Commission to determine
    fatalities from accidental iron poisonings. Data from the National
    Electronic Injury Surveillance System were also incorporated to
    estimate the number of cases of hospital emergency room treatment
    for iron ingestion.25 FDA did not establish dosing levels in any
    of these rulemakings. Although AERs were used as early warning
    signals in many of these cases, the proposed rule on ephedrine
    alkaloids relied more heavily on AERs than did prior rulemaking
    efforts. (See app. III for additional information on the use of
    AERs in prior rulemaking.) Shortcomings of AERs and    The
    inherent weaknesses of AERs-along with FDA's heavy reliance on
    FDA's Reliance on Them      them-lead to uncertainty regarding the
    dosing regimen outlined in the Add Uncertainty to FDA's
    proposed rule. The AERs used in the rule lacked or had
    inconsistent Proposed Rule               information relevant to
    FDA's analysis, such as the amount of product used, how often it
    was used, or for how long it was used. The limitations of a
    passive surveillance system such as AERs call into question FDA's
    ability to 21FDA also used AERs to show evidence of dechallenge
    (signs and symptoms resolve or improve when a consumer stops using
    a product) and rechallenge (symptoms recur when the consumer
    resumes using the product). 22Sulfiting agents are one type of
    chemical preservative that serves to prevent or to delay the
    process of browning and deterioration of raw fruits and
    vegetables. 2351 Fed. Reg. 25021. 2449 Fed. Reg. 13679. 2562 Fed.
    Reg. 2218. Page 10                    GAO/HEHS/GGD-99-90 FDA's
    Proposed Rule on Ephedrine Alkaloids B-281581 determine a specific
    dosing level based solely on these reports.26 FDA did not perform
    a causal analysis to determine if, in fact, the 13 AERs it used to
    set dosing levels were caused by supplements containing ephedrine
    alkaloids. FDA indicated in the proposed rule that 10 to 73
    percent of reported adverse events might not be related to
    consumption of dietary supplements containing ephedrine alkaloids.
    FDA's support for its recommended limit on duration of use was
    also weak. FDA's lack of documentation on which adverse events it
    classified as serious makes it difficult to determine the validity
    of FDA's estimates of the benefits that would arise from the
    proposed rule. In addition, FDA did not have specific criteria to
    determine which events should be considered serious and did not
    perform a reliability assessment to ascertain the validity of its
    estimates of serious events. Adverse Event Reports Were     Prior
    to the proposed rulemaking, FDA received 864 AERs on dietary
    Incomplete and Inconsistent    supplements containing ephedrine
    alkaloids-more AERs than for any other single dietary supplement.
    However, the AERs that we examined often lacked important
    information, and the information that they did contain was
    sometimes inconsistent. These problems suggest that AERs should be
    used with caution, and their use can contribute to uncertainty in
    FDA's conclusions. In examining a random sample of the AERs (92
    out of 864 reports), we found that 39 percent lacked information
    on the amount of product consumed, 41 percent lacked information
    on the frequency with which the product was consumed, and 28
    percent lacked information on the duration for which the product
    was consumed. A total of 45 percent of the AERs lacked information
    on either dose, frequency, or duration, and 24 percent lacked
    information on all three dimensions. Finally, 62 percent of the
    AERs in our sample did not contain medical records, which are
    important in determining potential underlying conditions that
    might have caused the adverse event (rather than assuming
    ingestion of dietary supplements containing ephedrine alkaloids
    caused the event). We also found cases where the amount of product
    consumed or the duration for which it was consumed was listed
    differently in multiple locations within an AER. In addition, the
    name of the product consumed sometimes varied within an AER. As a
    result, it is difficult to identify the 26FDA evaluated the
    scientific literature to determine that ephedrine alkaloids could
    pose hazards at 20 mg per dose. However, FDA relied solely on AERs
    in setting a specific dose level of 8 mg per dose and 24 mg per
    day. Page 11                    GAO/HEHS/GGD-99-90 FDA's Proposed
    Rule on Ephedrine Alkaloids B-281581 correct information that
    corresponds to the event being reported and to make inferences
    from such reports.27 FDA Relied Solely on AERs for    In its
    proposed rule, FDA concluded that one possible strategy for
    Setting Specific Dosing          addressing these adverse events
    was to restrict the level of ephedrine Regimens and Did Not
    alkaloids in dietary supplements. To determine a possible dosing
    level, FDA Determine If These Events        reviewed clinical
    trials on therapeutic uses for ephedrine alkaloids used Were
    Caused by Ephedrine         alone and in combination with other
    pharmaceutical substances to treat Alkaloids
    obesity. These trials indicated that 20 mg ephedrine alkaloids per
    dose could cause adverse events to occur in the subjects taking
    part in the clinical trial.28 Thus, FDA concluded that 20 mg per
    serving of ephedrine alkaloids presented potential risks for some
    individuals. However, the studies did not provide information on
    risk at levels below 20 mg per serving for use in the general
    population, and lean or moderately overweight persons have been
    shown to be more sensitive to substances like ephedrine alkaloids
    than are obese individuals. To determine if serious adverse events
    were occurring at levels below 20 mg per serving, FDA obtained and
    tested the dietary supplement products associated with AERs that
    were serious, where available.29 FDA then identified the levels of
    ephedrine alkaloids in those AERs by performing laboratory
    analyses of ephedrine alkaloid levels for dietary supplements
    turned over to the agency by the consumers who suffered adverse
    events (consumer samples). FDA also performed tests on samples
    obtained from the marketplace for situations in which the consumer
    experiencing the adverse event no longer possessed the product
    (consumer-related samples). In all, from more than 800 AERs
    submitted to the agency between 1993 and the time of the proposed
    rule, FDA collected and tested 34 samples of consumer or consumer-
    related products. From the 34 samples tested, 13 product samples
    met two criteria: the results of the analytical tests were valid
    and supportable and the products contained less than 20 mg of
    ephedrine alkaloids. FDA then set a specific dosing level of 8 mg
    per serving. In other words, FDA relied on these 13 AERs where the
    products tested yielded ephedrine alkaloid levels below 20 mg per
    serving as the sole source of support for the specific dosing
    level 27While these types of problems do not occur uniquely among
    reports involving ephedrine alkaloids, they reinforce the fact
    that there are inherent weaknesses of passive surveillance systems
    such as AERs. 28See, for example, A. Astrup and others, "The
    Effect and Safety of an Ephedrine/Caffeine Compound Compared to
    Ephedrine, Caffeine and Placebo in Obese Subjects on an Energy
    Restricted Diet. A Double Blind Trial," International Journal of
    Obesity, Vol. 16 (1992), pp. 269-77. 29Testing was necessary
    because there are no requirements that product contents be
    labeled. Page 12                     GAO/HEHS/GGD-99-90 FDA's
    Proposed Rule on Ephedrine Alkaloids B-281581 found in the
    proposed rule.30 Table 1 provides information on these 13 cases.
    Table 1: Summary of 13 AERs Used for Specific Dosing Level Total
    ephedrine alkaloids (mg)a AER number Initial           Recheck
    Type of sample         Symptoms as described in AER 10088
    1.6            2.0        Consumer               Elevated blood
    pressure, severe headache, nausea, heavy perspiration, passed out
    11233         1.8            Trace      Consumer
    Hair loss, sleeplessness, chest pain, increased energy 11106
    Trace          1.9        Consumer               Sweating,
    trembling, high blood pressure, menstrual bleeding, suffered
    stroke 9754          8.8            8.8        Consumer
    Heat stroke, chest and back pain, rapid heart rate, hyperthermia
    8475          9.1            9.9        Consumer-related Tremors,
    tics, insomnia, hyperactive reflexes 10974         11.0
    13.1       Consumer-related Seizure 11144b        10.0, 11.7 9.0,
    15.3      Consumer-related Transient amnesia, lost consciousness
    10946         10.5           8.3        Consumer
    Rash, increased blood pressure 9751          9.2            11.1
    Consumer               Attempted suicide 10919         11.2
    14.4       Consumer               Weakness, dizziness, elevated
    blood pressure 11134c        14.8                      Consumer
    Death 11619         15.1           18.5       Consumer
    Rapid heart rate, headache, numbness, face droop, dizziness 11298
    16.6           20.6       Consumer               Blood in urine,
    elevated blood pressure aLevels are based on the amount the
    consumer reported having used per serving. Where consumer intake
    was unknown, the levels were based on the label directions for
    use. FDA performed an initial analysis and a recheck analysis for
    the samples it obtained. bTwo consumer-related samples were
    tested. cRecheck analysis was not conducted because original
    analysis was performed by a national or international expert.
    While FDA used these 13 AERs to set a dosing level, the agency did
    not perform a causal analysis to determine whether the reported
    events were, in fact, caused by the ingestion of dietary
    supplements containing ephedrine alkaloids. Our review of these 13
    AERs found numerous problems that raise questions about the causal
    relationship between ingestion of the implicated product and the
    adverse events observed. For example, 30FDA's laboratory analysis
    of the ephedrine alkaloid levels in the 34 AERs showed levels from
    approximately 1 to 50 mg per serving. According to FDA, these
    reports show a pattern of clinically significant adverse events at
    levels approaching and above 10 mg. Given the variability in the
    testing procedure and natural variability in the alkaloid content
    of botanical ingredients, FDA determined a range around 10 mg per
    serving could be expected to deviate by 10 to 20 percent. From
    this, FDA tentatively concluded that an 8 mg limit per serving
    could be associated with a serious adverse event. Page 13
    GAO/HEHS/GGD-99-90 FDA's Proposed Rule on Ephedrine Alkaloids B-
    281581 * three AERs included physician reports that stated the
    cause of the event was not related to a dietary supplement; * one
    consumer-related sample was obtained and tested 2 years after the
    initial event, and possible reformulations of the product might
    have resulted in different levels of ephedrine than the product
    implicated in the reported adverse event; * three individuals
    reporting adverse events had experienced similar problems prior to
    or well after using the dietary supplement; * one individual who
    experienced the event was eating only one meal a day; and * one
    AER contained no medical records. Some of the 13 AERs had more
    than one of these problems. As a result, there are uncertainties
    in FDA's conclusions in setting a specific dosing limit since this
    limit was based on a small number of adverse events-events which
    may or may not have been a result of ingestion of dietary
    supplements containing ephedrine alkaloids. Weak Support for FDA's
    In its proposed rule, FDA also recommended a 7-day limit on the
    use of Proposed Limitation on      dietary supplements containing
    ephedrine alkaloids. FDA used the Duration of Use
    recommendations of some members from its special work group and
    Food Advisory Committee, scientific literature, and data from the
    AERs to support this limit. However, we found several weaknesses
    in FDA's evidence. First, FDA did not present scientific evidence
    specifically pointing to an increase in adverse events beginning
    at 7 days and under normal use conditions. Rather, the scientific
    information FDA used to support a 7-day limit outlined problems
    associated with extended use (months and years) of ephedrine
    alkaloids. The agency also cited support for its 7-day limit from
    studies involving other sympathomimetic agents, such as cocaine
    and methamphetamines, but these studies also involved long-term
    use of the drug. Second, FDA did not demonstrate a causal
    relationship between ingestion of dietary supplements containing
    ephedrine alkaloids and adverse events reported to the agency.
    Since FDA indicated in the proposed rule that 10 to 73 percent of
    the reported adverse events might not be related to consumption of
    dietary supplements containing ephedrine alkaloids, the use of the
    AERs to describe a pattern of response across time Page 14
    GAO/HEHS/GGD-99-90 FDA's Proposed Rule on Ephedrine Alkaloids B-
    281581 is questionable.31 Finally, the agency did not demonstrate
    the relationship between both dose levels and frequency of
    ingestion and duration of use. As a result, there are
    uncertainties in the agency's analysis of the relationship between
    duration of use of dietary supplements containing ephedrine
    alkaloids and the occurrence of adverse events. FDA Based Its Risk
    Reduction    FDA noted in the proposed rule that there were an
    average of 174 adverse Value on Poorly Documented      events
    reported each year between January 1993 and June 1996 and that 81
    Estimates of Serious Adverse    were serious events. FDA divided
    the serious adverse events into five Events
    categories and outlined examples for each type: (1) cardiovascular
    (dysrhythmia or abnormal heart rhythms, severe hypertension,
    cardiac arrest, heart attack, angina, myocardial infarction,
    stroke, and cardiomyopathy or muscle damage to heart); (2) central
    nervous system (psychosis, suicide or attempted suicide, altered
    or loss of consciousness or disorientation or confusion, seizure,
    mania, and severe depression); (3) liver damage or hepatitis; (4)
    death; and (5) other events (vestibular or inner ear disturbances,
    altered serum enzymes, myopathies or muscle disease, genitourinary
    system disturbances or urinary tract problems, prostate problems,
    and dermatological syndromes or rashes). However, FDA did not
    record which AERs it considered to be serious. FDA officials
    stated that a count had been performed by medical doctors of the
    agency on the number of serious events, but there was no record of
    the specific cases that they had identified as serious events. In
    addition, rather than providing AER reviewers a standard
    definition, FDA told them to use their clinical judgment in
    determining what they considered a serious event.32 Also, FDA did
    not perform any assessments of interrater reliability among
    reviewers to ensure that their judgments were consistent. Based on
    the yearly average of serious adverse events reported to FDA from
    January 1993 to June 1996, FDA estimated an annual value of risk
    reduction that would occur as a result of the proposed actions
    outlined in the rule. However, because FDA did not record which
    AERs were deemed 31FDA pointed out in the proposed rule that the
    relationship of the reported adverse event to the consumption of
    dietary supplements categorized as containing ephedrine alkaloids
    had been corroborated in about 27 percent of cases where a
    consumer stopped taking a product and the symptoms improved. FDA
    also noted that a certain number of false reports might also be
    expected. Thus, they believed that the actual percentage of cases
    related to consumption of ephedrine alkaloids was between 27
    percent and 90 percent. FDA's professional judgment was that 80
    percent of the reported cases were actually related to consumption
    of dietary supplements containing ephedrine alkaloids. 32FDA noted
    that the medical doctors were well qualified-by training and
    experience-to make decisions regarding the clinical significance
    of the event. Page 15                     GAO/HEHS/GGD-99-90 FDA's
    Proposed Rule on Ephedrine Alkaloids B-281581 serious, we could
    not determine the accuracy of FDA's conclusions regarding the
    actual benefits that would arise from the proposed rule. FDA's
    Analysis of                 FDA generally complied with executive
    order and statutory requirements Impacts Generally
    for rulemaking. It prepared a cost-benefit analysis containing the
    basic elements expected for an economically significant rule under
    E.O. 12866. Adhered to                        It also prepared an
    initial regulatory flexibility analysis and provided Rulemaking
    opportunities for small businesses to participate in the
    rulemaking process, as required by RFA. FDA's analysis of impacts
    would also likely Requirements but Was satisfy most, if not all,
    of the analytical requirements of UMRA, although its Not Fully
    Transparent             proposed rule does not appear to trigger
    these requirements. However, OIRA's guidance on preparing cost-
    benefit analyses under E.O. 12866 states that an agency's analysis
    must be "transparent"-that is, disclose how the analyses were
    conducted and what assumptions were used. FDA's analysis was only
    partially transparent. Although FDA described its key assumptions
    and identified substantial uncertainties regarding the data
    supporting its proposed rule, it did not fully disclose why
    certain key assumptions were made, the degree of uncertainty
    involved in those assumptions, or that alternative assumptions
    would have had a dramatic effect on the agency's estimates of the
    costs and benefits of the proposed regulatory actions. Because of
    these uncertainties and limitations, the results of FDA's analysis
    are still open to question. FDA's Analysis Addressed
    Under E.O. 12866, agencies are required to conduct cost-benefit
    analyses the Basic Cost-Benefit            for all regulatory
    actions that are likely to result in a $100 million annual
    Elements but Was Only             effect on the economy or are
    otherwise economically significant. OIRA's Partially Transparent
    "best practices" guidance states that a federal agency's cost-
    benefit analysis should contain three basic elements: (1) a
    statement of the need for the proposed action, (2) an examination
    of alternative approaches, and (3) an analysis of benefits and
    costs. Although the guidance provides for flexibility and
    professional judgment in conducting the analysis, it establishes a
    clear expectation that the analysis of the risks, benefits, and
    costs associated with the regulation "must be guided by the
    principles of full disclosure and transparency." For example, the
    guidance says the analysis should identify and explain the data or
    studies on which the estimates of benefits and costs are based
    "with enough detail to permit independent assessment and
    verification of the results." The guidance also states "where
    benefit or cost estimates are heavily dependent on certain
    assumptions, it is essential to make those assumptions explicit
    Page 16              GAO/HEHS/GGD-99-90 FDA's Proposed Rule on
    Ephedrine Alkaloids B-281581 and, where alternative assumptions
    are plausible, to carry out sensitivity analyses based on the
    alternative assumptions."33 (See app. IV for a more detailed
    discussion of the analytical requirements under E.O. 12866 and
    related guidance, as well as our assessment of FDA's cost-benefit
    analysis in terms of that guidance.) FDA's cost-benefit analysis
    contained the three basic elements stipulated in the guidance, but
    certain other specific elements in the guidance appear to be
    lacking: * With regard to the first element, FDA stated that the
    proposed rule was needed because of a significant market failure-
    specifically, inadequate consumer information on the health risks
    associated with dietary supplements containing ephedrine
    alkaloids. However, the OIRA guidance indicates that even where a
    market failure exists, agencies should also discuss the
    appropriateness of alternatives to federal regulation, such as
    state or local regulation. FDA's analysis did not recognize
    efforts in several states to regulate these products or say why
    federal regulation was a better approach. * For the second
    element, FDA discussed seven regulatory options, including (1) a
    baseline alternative of taking no action, (2) taking no action but
    generating additional information, (3) taking the proposed action,
    (4) taking the proposed action but allowing a higher potency limit
    for the supplements, (5) banning dietary supplements that contain
    ephedrine alkaloids, (6) taking the proposed action but not
    requiring the warning statement, and (7) requiring only the
    warning statement. FDA also generally discussed the agency's
    reasoning for selecting the proposed regulation over other
    alternatives. However, FDA's options primarily focused on
    alternative levels of stringency and informational measures-not on
    other basic types of regulatory alternatives suggested in the
    guidance. Also, while FDA included some contextual information in
    the proposed rule, it did not provide a complete picture of the
    baseline risks that may be associated with supplements containing
    ephedrine alkaloids. * For the actual analysis of potential
    benefits and costs-the third element-FDA generally adhered to the
    principles and specific recommendations in OIRA's guidance. For
    example, FDA discussed the distributional effects of the proposed
    rule and alternatives in terms of lost sales to the dietary
    supplement industry. FDA also identified nonmonetized costs
    associated with the proposed rule. However, FDA did not provide
    monetized or quantified benefit or cost estimates for all of its
    regulatory 33To conduct a sensitivity analysis, the analyst
    calculates the costs and benefits of a proposed action using
    different assumptions. Page 17                    GAO/HEHS/GGD-99-
    90 FDA's Proposed Rule on Ephedrine Alkaloids B-281581
    alternatives, and the agency's discussion of those alternatives'
    benefits and costs were compared to the proposed rule-not the
    baseline condition of no regulation as recommended by the
    guidance. The OIRA guidance also stresses the importance of full
    disclosure and transparency in agencies' cost-benefit analyses. We
    assessed the transparency of FDA's cost-benefit analysis of the
    proposed rule using three criteria suggested by the guidance: *
    first, whether it identified the data, models, inferences, and
    assumptions used to calculate the estimates of benefits and costs;
* second, whether it disclosed the reasons why those data, models,
    inferences, and assumptions were selected; and * third, whether it
    assessed the effects of plausible alternative assumptions and
    choices on the results of the analysis-what is often referred to
    as a "sensitivity analysis." Overall, we concluded that FDA's
    analysis was only partially transparent. Using the first
    criterion, FDA's analysis was very transparent. The agency
    provided a clear and lengthy description of the data, assumptions,
    and methodology that it used to calculate the benefits and costs
    of the proposed rule. Against the second criterion, FDA's analysis
    was only partially transparent. For most elements of the analysis,
    FDA identified the underlying data sources used and the rationale
    for the assumptions and conclusions reached by FDA's analysts and
    experts. However, FDA did not fully disclose the underpinnings of
    all of its assumptions and choices. For example, FDA said that
    between 27 percent and 90 percent of adverse event reports were
    "probably" related to the consumption of dietary supplements
    suspected of containing ephedrine alkaloids and that it assumed
    the value was 80 percent, but it did not indicate why it made this
    point-estimate assumption. In response to our questions, FDA
    officials acknowledged that their analysis of impacts was not as
    transparent as it should have been in explaining how the agency
    arrived at some of the assumptions regarding its treatment of
    uncertainty in the underlying data. Against the third criterion,
    FDA's cost-benefit analysis was also only somewhat transparent.
    For example, FDA estimated that the benefits of the proposed rule
    would be between $240 million and $670 million per year. That
    estimate was driven by three factors: (1) FDA's estimate of the
    actual number of adverse events each year (1,110), (2) FDA's
    estimate of the degree to which the proposed rule would reduce
    these events (35 percent to 100 percent across all types of
    proposed actions), and (3) the values FDA Page 18
    GAO/HEHS/GGD-99-90 FDA's Proposed Rule on Ephedrine Alkaloids B-
    281581 assigned to the estimated risk reduction per case (for
    example, $5 million per death avoided).34 Changes in any of these
    values could have dramatically changed FDA's estimates of the
    proposed rule's benefits. For example, FDA's estimate of 1,110
    adverse events each year was based on an average of 174 adverse
    events reported per year and three assumptions: (1) that 80
    percent of the adverse event reports involved consumption of
    dietary supplements suspected of containing ephedrine alkaloids,
    (2) that 80 percent of the supplements actually contained
    ephedrine alkaloids, and (3) that only 10 percent of all adverse
    events related to supplements containing ephedrine alkaloids were
    reported. Using these point estimates (instead of ranges) does not
    reflect the uncertainty FDA indicated was possible regarding these
    values and, therefore, the uncertainty associated with the
    agency's final benefit estimates. Had FDA used its own initial
    estimates of the possible ranges (instead of the 80-percent point
    estimates) for the first two assumptions (27 to 90 percent for the
    first assumption and 25 to 90 percent for the second assumption),
    its estimate of the number of adverse events each year would have
    been 117 to 1,409 instead of a single estimate of 1,110. (See
    table 2.) If FDA's projected effects for the proposed actions were
    applied to this range, the expected reduction in annual adverse
    events would be between 41 and 1,409 cases per year (not the 390
    to 1,110 in the published analysis). This ultimately results in a
    range of potential benefits of roughly $25 million to $850 million
    per year-a much wider range of possible benefits than the $240
    million to $670 million per year estimates that FDA included in
    the proposed rule. 34In its proposed rule, FDA did not report or
    round its estimated values consistently, so slightly different
    numbers for the same element appear within the proposed rule. For
    this report, we are using the values reported in table 6 of FDA's
    proposed rule, such as 1,110 for the estimated annual number of
    adverse events. Page 19                    GAO/HEHS/GGD-99-90
    FDA's Proposed Rule on Ephedrine Alkaloids B-281581 Table 2: FDA's
    Estimate of Expected Benefits Is Sensitive to Changes in Key
    Assumptions Assumptions made regarding uncertainty in the actual
    number of adverse event cases A            B                     C
    D                 E (AxBxCxD)           F               G
    H (FxG) Average        Proportion of               Proportion
    Multiplier            Estimated      Estimated        Value of
    Total annual                   cases           in which the
    to reflect                total     reduction  estimated
    dollar value number of           actually            supplements
    underreporting                      number            in the
    risk    (in millions) reported             related
    actually                      of       of adverse      number of
    reduction             of estimated adverse            to dietary
    contained              adverse                  event
    annual               per                  risk events
    supplements        ephedrine alkaloids              eventsa
    cases           casesb           case          reduction Scenario
    1: FDA's reported results, using a point estimate for total number
    of adverse events.c 174                         0.8
    0.8                      10             1,110      390-1,110
    d        $240-670 Scenario 2: Results using the full range of
    FDA's assumptions regarding uncertainty on number of adverse
    events. 174                  0.27-0.9                  0.25-0.9
    10         117-1,409        41-1,409                 d
    25-850 Scenario 3: Results using assumption that 20 percent of
    cases are reported. 174                         0.8
    0.8                       5              557         195-557
    d          118-336 Scenario 4: Results using assumption that 5
    percent of cases are reported. 174                         0.8
    0.8                      20             2,227      779-2,227
    d        470-1,343 aFor example, if the assumption is that 10
    percent of events are reported, the number of reported cases is
    multiplied by 10. bThe combined effects of all proposed actions
    are assumed to reduce adverse events by between 35 percent and 100
    percent. cFDA's table in the proposed rule on the estimation of
    benefits displayed rounded numbers. The actual calculated values
    may vary from the published figures used here. For example, if the
    figure for estimated total annual adverse events were not rounded,
    it would equal about 1,114 rather than 1,110. dThe value of risk
    reduction varies by the type of event avoided, such as $5 million
    per death avoided and $837,000 for each serious cardiovascular
    system event avoided. FDA converted the individual values and
    estimated totals into 1996 dollars. Calculations for scenarios 2,
    3, and 4 assume the same proportions of types of adverse events as
    FDA used for its estimate. Source: Scenario 1 data are taken from
    table 6 of FDA's proposed rule. Data for other scenarios were GAO
    calculations using information in FDA's proposal. Table 2 also
    shows that changes in the third assumption yield similarly
    dramatic changes in the estimated benefits. FDA indicated in its
    proposed rule that reporting rates might be higher than 10 percent
    if, for example, the potential health risks were widely
    publicized, or lower than 10 percent if consumers and physicians
    assumed that dietary supplements are incapable of producing
    adverse events. If, keeping all other factors constant, FDA had
    assumed that 20 percent of serious adverse events were reported,
    FDA's benefits estimate would have been reduced by about half.
    Page 20                       GAO/HEHS/GGD-99-90 FDA's Proposed
    Rule on Ephedrine Alkaloids B-281581 Conversely, if FDA had
    assumed that only 5 percent of events were being reported, the
    benefits estimate would have doubled. FDA Met RFA
    RFA requires agencies to consider the effects of their rules on
    small entities Requirements, Although    and to take certain
    actions during the rulemaking process. For example, Questions
    Remain          before publishing a proposed rule for which a
    notice of proposed Concerning Impacts on     rulemaking is
    required, sections 603 and 605(b) of RFA require a federal
    Industry                  agency to prepare and make available for
    public comment an initial regulatory flexibility analysis that
    describes the anticipated effects of the proposed rules on small
    entities, unless the head of the agency certifies that the rule
    will not have a "significant economic impact on a substantial
    number of small entities." (See app. V for a more detailed
    discussion of RFA requirements and our analysis of the actions FDA
    took to comply with RFA.) FDA determined that its proposed rule on
    ephedrine alkaloids would have a significant economic impact on a
    substantial number of small entities and prepared an initial
    regulatory flexibility analysis to identify those impacts. FDA's
    proposed rule addressed the basic elements that RFA requires
    agencies to include in an initial regulatory flexibility analysis.
    For example, the rule describes the reasons the agency was
    considering the action, states the purpose and legal basis of the
    rule, describes and provides an estimate of the number of small
    entities to which FDA believed the rule would apply, and describes
    the compliance requirements. In addition to describing direct
    compliance costs of between $3 million and $80 million, FDA said
    that the proposed rule could have significant distributive effects
    in the form of reduced sales of as much as $230 million a year.
    FDA explicitly stated that costs and sales reductions of this
    magnitude might threaten the viability of many firms. FDA also
    discussed significant regulatory alternatives in the rule's
    regulatory flexibility analysis section, noting that most of the
    regulatory alternatives discussed in the cost-benefit analysis
    section would reduce the impact of the rule on small businesses.
    When a rule is promulgated that will have a significant economic
    impact on a substantial number of small entities, section 609(a)
    of RFA further requires agencies to ensure that small entities
    have been given an opportunity to participate in the rulemaking
    process through the "reasonable use" of outreach efforts. Our
    review of the regulatory docket for this rulemaking, as well as
    information obtained during interviews with dietary supplement
    industry representatives, indicated that FDA provided Page 21
    GAO/HEHS/GGD-99-90 FDA's Proposed Rule on Ephedrine Alkaloids B-
    281581 opportunities for small business participation during this
    rulemaking process. For example, in addition to the proposed rule
    itself, FDA published public notices; held public meetings, during
    which industry representatives participated and provided
    testimony; and collected written comments on the issue.
    Representatives of industry trade associations also told us that
    FDA had placed notices about the proposed rule in trade
    literature. However, SBA's Office of Advocacy criticized the
    quality of FDA's regulatory flexibility analysis. Along with
    generally criticizing FDA's scientific analysis for the proposed
    rule, the office contended that FDA did not consider the large
    numbers of independent distributors of dietary supplements in its
    analysis and, as a result, underestimated the number of affected
    small businesses and the impacts of the proposed actions. In
    response to SBA's criticism, FDA officials told us that the agency
    did not need to consider the impact of its proposed rule on
    distributors of ephedrine alkaloids. FDA officials cited several
    court decisions that support the proposition that, under RFA, an
    agency is under no obligation to conduct a small entity impact
    analysis of effects on entities that the agency does not regulate-
    that is, distributors.35 None of the major parties-FDA, SBA's
    Office of Advocacy, or industry associations-have information on
    the actual number of entities involved in the market for dietary
    supplements containing ephedrine alkaloids. All three parties
    acknowledge this as a limitation in attempts to analyze the
    effects of the proposed rule. According to FDA, it has no
    authority to require companies in the dietary supplement industry
    either to register with or contact the agency unless they are
    seeking approval to make a claim in product labeling. However, the
    industry representatives we met with confirmed that manufacturing
    and labeling of these products are generally limited to a
    relatively small number of manufacturing firms. Such firms would
    bear almost all of the direct expenditures required for compliance
    with the proposed rule. SBA's Office of Advocacy recommended that
    FDA develop an outreach strategy to obtain more reliable industry
    data, and FDA officials said that they have contracted for a
    marketing study of the dietary supplement industry that should
    provide better information for future FDA analyses. 35For example,
    United Distribution Cos. v. FERC, 88 F.3d 1105, 1170 (D.C. Cir.
    1996). Page 22                    GAO/HEHS/GGD-99-90 FDA's
    Proposed Rule on Ephedrine Alkaloids B-281581 Proposed Rule Does
    Not    UMRA generally requires covered agencies to prepare
    specific types of Appear to Trigger UMRA    analyses for certain
    rules that include a federal mandate and that may Requirements
    result in the expenditure in any 1 year of $100 million or more by
    the private sector.36 FDA did not explicitly address UMRA in its
    proposed rule on ephedrine alkaloids because its economic analysis
    indicated that direct expenditures imposed on the private sector
    would not rise to the level requiring additional analysis under
    UMRA. FDA estimated that the total compliance costs for the
    proposed action would be between $3 million and $80 million, with,
    at most, $70 million of those costs in the first year of
    implementation. Although FDA also estimated that the rule might
    result in lost sales for the dietary supplement industry of as
    much as $230 million per year, lost sales cannot be considered
    direct expenditures by the private sector and, therefore, cannot
    be used to trigger UMRA's analytical requirements. (See app. VI
    for a more detailed discussion of UMRA requirements and our
    analysis of FDA's compliance.) However, SBA's Office of Advocacy
    contended that FDA overlooked the UMRA requirements and commented
    that if the agency had properly estimated the number of affected
    businesses and the costs that would be imposed on those entities,
    "it would have been apparent that the economic impact of the
    instant rule would impose in excess of $100 million in costs to
    the industry," triggering UMRA requirements. However, the Office
    of Advocacy does not have data on the number of entities involved
    in the market for dietary supplements containing ephedrine
    alkaloids that it believes should have been included in FDA's
    analysis. Therefore, the Office of Advocacy could not demonstrate
    that FDA had underestimated the number of manufacturers that would
    be affected by the relabeling and reformulation requirements in
    the rule or that FDA's cost estimates for those manufacturers were
    in error. Furthermore, even if the expenditures associated with
    the proposed rule had triggered UMRA's analytical requirements,
    FDA appears to have satisfied most, if not all, of those
    requirements. For example, FDA quantitatively and qualitatively
    assessed the anticipated costs and benefits of the rule. FDA also
    identified and considered a number of regulatory alternatives,
    indicating that the other alternatives would not be as effective
    as the proposed rule. Conclusions               FDA was justified
    in determining that the number of AERs relating to dietary
    supplements containing ephedrine alkaloids warranted their
    attention and consideration of steps to address safety concerns.
    The available scientific information suggests that the use of
    products containing synthetic 36The statute provides for annual
    adjustments for inflation. Page 23
    GAO/HEHS/GGD-99-90 FDA's Proposed Rule on Ephedrine Alkaloids B-
    281581 ephedrine alkaloids can result in adverse experiences for
    some individuals, and over-the-counter products containing
    ephedrine alkaloids have dosing recommendations. Furthermore,
    dietary supplement trade associations have suggested specific
    dosing limits for dietary supplements containing ephedrine
    alkaloids. However, while FDA's conclusions regarding the
    desirability of the proposed actions may be valid, we believe
    these conclusions are open to question because of limitations and
    uncertainties associated with the agency's underlying scientific
    and economic analyses. We have concerns about the strength of the
    information upon which FDA based specific elements of its proposed
    rule. There is no scientific information on the specific dosing
    levels and duration limits proposed. FDA, therefore, relied
    heavily on its AERs to determine a dosing regimen and to outline
    benefits that would accrue from the proposed rule. However, the
    AERs suffer from several problems that weaken the conclusions
    drawn by FDA for their specific dosing regimen; the number of AERs
    used to support the dosing regimen is small, their quality is
    questionable, and FDA did not establish a causal link between the
    ingestion of ephedrine alkaloids and the occurrence of particular
    adverse events. Finally, because FDA did not document which AERs
    it identified as involving serious adverse events, it is
    impossible to verify FDA's calculation of the number of these
    events and the accuracy of the benefits that FDA estimated would
    occur as a result of the proposed rule. FDA's analysis contained
    the basic elements expected in a federal agency's cost-benefit
    analysis, and the proposed rule complied with rulemaking
    requirements under RFA. The proposed rule does not appear to
    trigger the UMRA analytical requirements. However, FDA's cost-
    benefit analysis was not always transparent regarding why certain
    key assumptions were made, the degree of uncertainty involved in
    those assumptions, or the effect that alternative assumptions
    would have had on the agency's estimates of the costs and benefits
    of the proposed action. Recommendations    Given the uncertainties
    in the information upon which FDA based its proposed rule, we
    recommend that the Secretary of Health and Human Services direct
    the Commissioner of FDA to obtain additional information to
    support conclusions regarding the specific requirements in the
    proposed rule for dietary supplements containing ephedrine
    alkaloids before proceeding to final rulemaking. Specifically, FDA
    needs to provide stronger evidence on the relationship between the
    intake of dietary Page 24               GAO/HEHS/GGD-99-90 FDA's
    Proposed Rule on Ephedrine Alkaloids B-281581 supplements
    containing ephedrine alkaloids and the occurrence of adverse
    reactions that support the proposed dosing levels and duration of
    use limits. We also recommend that the Secretary direct the
    Commissioner to improve the transparency of FDA's cost-benefit
    analysis in its final rulemaking. Specifically, FDA should more
    fully explain the bases of its cost-benefit assumptions, the
    degree of uncertainty associated with those assumptions, and the
    implications of plausible alternative assumptions to the proposed
    action and other regulatory alternatives. Agency and Other     We
    sent a draft of this report to the Commissioner of FDA, the
    Reviewer Comments    Administrator of SBA, and the Director of
    OMB. FDA and SBA's Office of Advocacy provided written comments,
    which are reprinted in appendixes VII and VIII; OMB did not have
    comments on the report. In its comments, FDA concurred with our
    recommendation that it obtain additional information to support
    its conclusions on specific requirements relating to dietary
    supplements containing ephedrine alkaloids before proceeding to
    final rulemaking. FDA also concurred with our recommendation to
    improve the transparency of the agency's cost-benefit analysis,
    noting that it intends to take appropriate steps to correct the
    deficiencies of the analysis prior to publication of a final rule.
    However, FDA was concerned that our report (1) did not
    sufficiently highlight the agency's justification in examining
    safety concerns related to dietary supplements containing
    ephedrine alkaloids, (2) implied that FDA relied principally on
    AERs to develop the proposed rule, (3) overemphasized the agency's
    failure to conduct a causal analysis linking dietary supplements
    containing ephedrine alkaloids to the AERs, and (4) was not clear
    on whether it was appropriate to use AERs in rulemaking.
    Throughout our report, we express the clear view that given the
    number and type of AERs, it was reasonable for FDA to investigate
    the safety of the supplements. We also acknowledge that FDA's
    actions were based on information from scientific literature as
    well as AERs. However, although clinical trials suggest that
    adverse events could occur in some individuals using ephedrine
    alkaloids at levels approaching 20 mg per serving, they did not
    provide information relating to adverse events below the 20 mg
    level. Therefore, to conclude that an 8 mg per serving level would
    be appropriate, FDA turned to results of tests it performed on a
    small number of products implicated in 13 AERs. In other words,
    information from these Page 25             GAO/HEHS/GGD-99-90
    FDA's Proposed Rule on Ephedrine Alkaloids B-281581 AERs was the
    sole source of support for the specific dosing level of 8 mg per
    serving that FDA proposed. Finally, we continue to hold the view
    that without a causal link between the AERs and the ingestion of
    products containing ephedrine alkaloids, the exclusive use of AERs
    to support a specific dosing regimen is questionable. FDA also
    asked that we clarify that using AERs to develop a proposed rule
    of this type is neither inappropriate nor unscientific. For this
    report, we examined whether the use of AERs in the proposed rule
    on dietary supplements containing ephedrine alkaloids was
    consistent with the use of AERs in prior rulemaking; however, we
    did not take a position on the appropriateness of the general use
    of AERs in rulemaking. SBA's Office of Advocacy stated that our
    recommendations are reasonable and would result in a more
    rational-and possibly less burdensome- regulation. However, the
    officials questioned how we could conclude that FDA had complied
    with RFA while at the same time suggesting that the agency had
    prepared an "inadequate analysis" by using unreliable data and not
    clearly demonstrating that the benefits of the proposed rule
    exceed the cost. Although our report raises concerns about several
    aspects of FDA's analysis, we did not conclude that FDA prepared
    an inadequate analysis. Rather, we concluded that FDA's proposed
    rule contained the primary elements required under E.O. 12866 for
    a cost-benefit analysis and under RFA for an initial regulatory
    flexibility analysis. Furthermore, RFA contains no standards or
    criteria that define an "adequate" regulatory flexibility
    analysis. As noted in SBA's comments, section 607 of RFA notes
    that agencies may provide "general descriptive statements" to
    comply with the act's analytical requirements if quantification is
    not practicable or reliable. Determining whether a regulatory
    flexibility analysis was in compliance with RFA would require
    analysis and judgments concerning the totality of the
    circumstances relating to the specific regulation in question. We
    have no basis and are not in a position to make such a
    determination regarding the quality of FDA's analysis, nor does
    our report do so. SBA's Office of Advocacy also expressed its
    concern that FDA's analysis did not identify the indirect effects
    of the rule on distributors. Office of Advocacy officials said
    that, although certain court decisions stemming from Mid-Tex
    Electric Cooperative, Inc. v. FERC37 indicated that agencies need
    not prepare regulatory flexibility analyses if the effects of a
    rule on an industry are indirect, they disagreed with those
    interpretations and contended that agencies should be required to
    conduct analyses of indirect effects. They also stated that the
    Office of Advocacy was unaware 37773 F.2d 327 (D.C. Cir. 1985).
    Page 26                     GAO/HEHS/GGD-99-90 FDA's Proposed Rule
    on Ephedrine Alkaloids B-281581 of any comparable case law with
    respect to direct and indirect effects as they apply to E.O.
    12866, so they said FDA should have prepared an analysis of the
    rule's effects on distributors. We disagree with these comments
    for three reasons. First, as the Office of Advocacy officials
    acknowledged, prior case law indicates that agencies need not take
    the indirect effects of their rules into account when conducting a
    regulatory flexibility analysis. That interpretation was recently
    reaffirmed in a May 14, 1999, decision by the U.S. Court of
    Appeals for the District of Columbia Circuit.38 Second, the
    absence of case law comparable to Mid-Tex with respect to direct
    and indirect effects as they apply to E.O. 12866 says nothing
    about FDA's analytical obligations under the order. Third, our
    review shows that FDA did address the potential indirect effects
    of the rule through its estimates of total lost sales for the
    dietary supplement industry. FDA also explicitly acknowledged in
    the proposed rule that such lost sales are "obviously very
    significant to the affected parties" and might threaten the
    viability of many firms in this industry. We also obtained
    comments on a draft of this report from a professor of
    pharmacology with expertise on dietary supplements. He agreed that
    while there is adequate reason for FDA to be concerned about the
    safety of products containing ephedrine alkaloids, he believed
    that there are too few AERs of substantive quality to allow for
    the setting of a maximum safe dose. He also concurred with our
    conclusions and recommendations. FDA provided technical comments,
    which we incorporated as appropriate. 38American Trucking
    Associations, Inc., et al, v. United States Environmental
    Protection Agency, No. 97-1440 U.S. App. LEXIS 9064 (D.C. Cir. May
    14, 1999). Page 27                   GAO/HEHS/GGD-99-90 FDA's
    Proposed Rule on Ephedrine Alkaloids B-281581 As agreed with your
    office, unless you publicly announce its contents earlier, we plan
    no further distribution of this report until 30 days from the date
    of this letter. At that time, we will send copies to the Honorable
    Donna E. Shalala, Secretary of Health and Human Services; the
    Honorable Jane E. Henney, Commissioner of FDA; the Honorable Aida
    Alvarez, Administrator of SBA; the Honorable Jacob J. Lew,
    Director of OMB; and others who are interested. We will also
    provide copies to others upon request. GAO contacts and major
    contributors to this report are listed in appendix IX. William J.
    Scanlon Director of Health Financing and Public Health Issues L.
    Nye Stevens Director of Federal Management and Workforce Issues
    Page 28               GAO/HEHS/GGD-99-90 FDA's Proposed Rule on
    Ephedrine Alkaloids Page 29      GAO/HEHS/GGD-99-90 FDA's Proposed
    Rule on Ephedrine Alkaloids Contents Letter
    1 Appendix I
    32 Scope and Methodology Appendix II
    34 Adverse Event Reporting Appendix III
    37 FDA's Prior Use of AERs in Rulemaking Appendix IV
    42 FDA's Analysis of Benefits and Costs Appendix V
    56 Regulatory Flexibility Act Requirements Appendix VI
    61 Unfunded Mandates Reform Act Requirements Appendix VII
    63 Comments From the Department of Health and Human Services Page
    30      GAO/HEHS/GGD-99-90 FDA's Proposed Rule on Ephedrine
    Alkaloids Contents Appendix VIII
    72 Comments From the Small Business Administration Appendix IX
    79 GAO Contacts and Staff Acknowledgments Tables
    Table 1: Summary of 13 AERs Used for Specific Dosing Level
    13 Table 2: FDA's Estimate of Expected Benefits Is Sensitive to
    20 Changes in Key Assumptions Table III.1: Five Prior FDA
    Rulemaking Cases                                   37 Figure
    Figure IV.1: Estimation of Benefits Is Sensitive to Changes in
    53 Underlying Assumptions Abbreviations AEMS         adverse event
    monitoring system AER          adverse event report CDC
    Centers for Disease Control and Prevention CFSAN        Center for
    Food Safety and Applied Nutrition CPSC         Consumer Product
    Safety Commission EPA          Environmental Protection Agency FDA
    Food and Drug Administration GRAS         generally recognized as
    safe OIRA         Office of Information and Regulatory Affairs OMB
    Office of Management and Budget OSN          Office of Special
    Nutritionals RFA          Regulatory Flexibility Act SBA
    Small Business Administration UMRA         Unfunded Mandates
    Reform Act Page 31              GAO/HEHS/GGD-99-90 FDA's Proposed
    Rule on Ephedrine Alkaloids Appendix I Scope and Methodology To
    determine whether the reported cardiovascular and nervous system
    effects outlined in the proposed rule were relevant for dietary
    supplements containing ephedrine alkaloids and whether there was
    evidence that these ingredients could cause adverse experiences in
    some individuals, we examined the scientific literature for case
    reports of adverse events from ingestion of products containing
    ephedrine alkaloids and reviewed findings from clinical trials
    using ephedrine alkaloids to treat obesity. To determine whether
    the Food and Drug Administration (FDA) had any internal guidance
    on the use of adverse events reports (AER) in rulemaking and
    whether these reports had been used in prior rulemaking, we sought
    information from the agency. We examined five prior cases of
    rulemaking that FDA provided in which the Center for Food Safety
    and Applied Nutrition (CFSAN) had used AERs as part of the basis
    for conclusions that were drawn in a proposed rule to ascertain
    how the agency had previously used AERs in rulemaking. In
    examining the basis for FDA's rule, we evaluated information
    contained in a sample of AERs and FDA's reliance on them to
    support a dosing regimen and to estimate benefits from the
    proposed rule. To identify criteria for the quality of the AERs,
    we obtained information on the use of passive surveillance
    systems, including their methodological strengths and weaknesses.
    We also examined the scientific literature for studies on adverse
    event reporting and outcomes from such reporting systems. We
    performed a content analysis of a random sample of 92 out of 864
    AERs filed prior to the proposed rule to determine what
    information was available in the AERs, such as dose, frequency of
    ingestion, duration of use, type of product, the adverse effect
    reported, and whether a medical record was included in the report.
    We examined the list of 13 AERs used by FDA to set the dosing
    regimen outlined in the proposed rule. We compared the analytical
    results of tests performed on the products implicated in these
    AERs to determine the dosing level ingested by the patient
    describing the adverse event. We examined the results of these
    analytical tests to determine the variability in the amount of
    ephedrine alkaloids that could be found in individual bottles and,
    thus, the possible range in the amount of ephedrine alkaloids
    ingested by a complainant. We sought documentation from FDA
    officials on the AERs that were classed as serious, the numbers
    and types of each category of serious event, and how those cases
    had been identified. We reviewed the extent to which FDA's cost-
    benefit analysis for the proposed rule contained the elements
    expected of a federal agency Page 32             GAO/HEHS/GGD-99-
    90 FDA's Proposed Rule on Ephedrine Alkaloids Appendix I Scope and
    Methodology analysis. To identify criteria for this objective, we
    reviewed the cost-benefit requirements in E.O. 12866, "Regulatory
    Planning and Review," and a guidance document issued by the
    Administrator of the Office of Information and Regulatory Affairs
    (OIRA) entitled "Economic Analysis of Federal Regulations Under
    Executive Order 12866." OIRA's guidance was intended to describe
    "best practices" for agencies in preparing cost-benefit analyses
    of regulatory actions under the executive order. Using these best
    practices, we reviewed FDA's cost-benefit analysis in the
    "analysis of impacts" section of the agency's June 1997 proposed
    rule. To obtain additional information, we interviewed FDA
    officials involved in the development of the cost-benefit analysis
    and the proposed rule. We also reviewed supporting documents
    referenced in the proposed rule or included in the electronic
    docket for this rulemaking. As part of that task, we examined each
    comment related to this rulemaking in the electronic docket to
    identify substantive remarks on elements of FDA's analysis. We
    also interviewed officials at OIRA, the Small Business
    Administration (SBA), and trade associations representing the
    dietary supplement industry to obtain their views on FDA's cost-
    benefit analysis. We used a similar approach to describe FDA's
    actions to comply with the requirements of the Regulatory
    Flexibility Act (RFA) and the Unfunded Mandates Reform Act (UMRA).
    We identified criteria by reviewing the specific requirements in
    the statutes and then examined the relevant sections of FDA's
    proposed rule to compare the agency's actions with those
    requirements. As with the previous task, we supplemented our
    review of the proposed rule by interviewing officials at FDA, SBA,
    and industry trade associations. We also reviewed public comments
    submitted on the proposed rule, including comments submitted by
    SBA's Office of Advocacy on FDA's compliance with RFA and UMRA. In
    our evaluation of FDA's cost-benefit analysis and initial
    regulatory flexibility analysis, we relied primarily on the
    information FDA presented in the proposed rule or included in the
    regulatory docket. We did not perform an independent assessment of
    the accuracy and reliability of FDA's underlying data and
    estimates. We also did not examine issues relating to restrictions
    outlined in the proposed rule relating to ephedrine alkaloids and
    their use with other stimulants and labeling restrictions. Page 33
    GAO/HEHS/GGD-99-90 FDA's Proposed Rule on Ephedrine Alkaloids
    Appendix II Adverse Event Reporting The clinical research and
    review staff in CFSAN's Office of Special Nutritionals (OSN) is
    responsible for tracking and reviewing AERs on special nutritional
    products.39 Special nutritional products include dietary
    supplements, infant formulas, medical foods, and foods for special
    dietary use. The adverse event monitoring system (AEMS) used by
    OSN is a voluntary surveillance system established to collect
    information on AERs associated with the use of special nutritional
    products. Only AERs associated with an illness or injury are
    included in AEMS; reports of product quality or consumer
    dissatisfaction are not included. AERs may enter this system
    through a variety of mechanisms, including FDA's drug quality
    reporting system, FDA's Medical Products Reporting Program
    (Medwatch), the United States Pharmacopeia, FDA field offices
    (through the Office of Regulatory Affairs' consumer complaint
    system), and from other federal and state public health agencies.
    Written and electronic correspondence and telephone conversations
    are other sources of information about adverse events reported
    with the use of special nutritional products. When an AER is
    received, it is reviewed to evaluate the need for further follow-
    up, including medical record documents, label and labeling, and
    other types of information necessary for evaluation of the AER.
    Follow-up is routinely attempted on all AERs of serious events or
    when the event is deemed to be clinically significant. MedWatch
    provides guidelines for determining if an adverse event is serious
    for reporting purposes. Events deemed serious include those that
    are fatal, life-threatening, or permanently or significantly
    disabling; require or prolong hospitalization; result in
    congenital anomalies; or require intervention to prevent permanent
    impairment or damage.40 The proposed rule on ephedrine alkaloids
    includes as serious cardiovascular events abnormal heart rhythms,
    stroke, heart attack, and cardiomyopathy. Serious nervous system
    events that were listed in the proposed rule included seizures,
    psychosis, mania, severe depression, vestibular (inner ear)
    disturbances, and loss of consciousness. Other reported adverse
    effects that FDA deemed clinically serious or potentially serious
    included elevations of liver function tests or overt hepatitis,
    myopathies (disease of muscle), disturbances of the genitourinary
    system, and dermatologic manifestations. AEMS is considered an
    invaluable tool for identifying potential serious public health
    issues that may be associated with the use of a particular
    39Information in this appendix was obtained from a continuing
    education article from FDA's Center for Drug Evaluation and
    Research, "The Clinical Impact of Adverse Event Reporting" (Oct.
    1996). 40Health professionals can report any adverse event that
    they judge to be clinically significant. Page 34
    GAO/HEHS/GGD-99-90 FDA's Proposed Rule on Ephedrine Alkaloids
    Appendix II Adverse Event Reporting product or types of products.
    Although more formal epidemiological studies, prospective trials,
    and retrospective case-control studies are more rigorous,
    surveillance can be important as an early alert to potential
    problems. As such, even isolated reports can be definitive in
    associating products with an adverse effect if the product and
    effect are temporally related and there is evidence of dechallenge
    (signs and symptoms resolve or improve when a consumer stops using
    a product) and rechallenge (symptoms recur when the consumer
    resumes using the product). However, like all voluntary
    surveillance systems, AEMS has certain weaknesses. These
    limitations include different interpretations in determining an
    adverse event, underreporting, biases, estimation of population
    exposure, and report quality. In regard to differing
    interpretations, it has been well established that AERs are quite
    subjective and imprecise.41 For example, in one study, clinical
    pharmacologists and treating physicians showed complete agreement
    less than half the time when determining whether medication,
    alcohol, or recreational drug use had caused hospitalization.42 It
    is also well known that placebos43 and even no treatment44 can be
    associated with adverse events. In addition, there is almost
    always an underlying background rate for any clinical event in a
    population, regardless of whether there was exposure to a
    particular product. Another major concern of passive surveillance
    systems is underreporting of adverse events. For example, it was
    estimated that under a British spontaneous reporting system, only
    10 percent of serious events and 2 to 4 percent of nonserious
    events are reported.45 Another estimate has shown that FDA may
    receive, by direct reporting, less than 1 percent of suspected
    serious adverse drug reactions.46 Thus, the cases spontaneously
    reported 41J. Kock-Weser and others, "The Ambiguity of Adverse
    Drug Reactions," European Journal of Clinical Pharmacology, Vol.
    11 (1977), pp. 75-78. 42F. E. Karch and others, "Adverse Drug
    Reactions-A Matter of Opinion," Clinical Pharmacology Therapy,
    Vol. 19 (1976), pp. 489-92. 43D. M. Green, "Pre-Existing
    Conditions, Placebo Reactions, and Side Effects," Annals of
    Internal Medicine, Vol. 60 (1964), pp. 255-65. 44M. M. Reidenberg
    and D. T. Lowenthal, "Adverse Nondrug Reactions," New England
    Journal of Medicine, Vol. 279 (1968), pp. 678-79. 45M. D. Rawlins,
    "Pharmacovigilance: Paradise Lost, Regained or Postponed?," The
    William Withering Lecture, 1994, Vol. 29 (London: J. R. Coll.
    Physicians, 1995), pp. 41-49. 46H. D. Scott and others, "Rhode
    Island Physicians' Recognition and Reporting of Adverse Drug
    Reactions," Rhode Island Medical Journal, Vol. 70 (1987), pp. 311-
    16. Page 35                     GAO/HEHS/GGD-99-90 FDA's Proposed
    Rule on Ephedrine Alkaloids Appendix II Adverse Event Reporting to
    any surveillance program generally represent only a small portion
    of the number that have actually occurred. Biases also limit the
    usefulness of spontaneous reporting systems. Unlike clinical trial
    data, which are obtained under strictly controlled conditions,
    spontaneously reported information is uncontrolled and, therefore,
    subject to the possible influence of a number of reporting biases
    that are brought about by the length of time a product has been on
    the market, the reporting environment, and the quality of the
    data. Compounding these problems, passive surveillance systems
    lack a denominator for the data, such as user population or drug
    exposure patterns. This is helpful to provide the exact number of
    people exposed to a product and, thus, the risk for the adverse
    event to occur. As a result, incidence rates cannot be computed.
    Finally, the ability to analyze information contained in an AER is
    dependent on the quality of information submitted. Optimally, an
    AER should include product name, demographic data, a clinical
    description of the adverse event (including confirmatory and
    laboratory test results), confounding factors (such as medical
    products and medical history), temporal information, dose and
    frequency of use, biopsy or autopsy reports, dechallenge and
    rechallenge information, and outcome.47 Although spontaneous
    reporting systems have a number of weaknesses, they also have
    advantages over more controlled systems. First, they maintain
    ongoing surveillance of products that are marketed and, second,
    they are relatively inexpensive. In fact, they may be the most
    cost-effective way to detect rare, serious adverse events not
    discovered during clinical trials.48 Their usefulness lies in
    hypothesis generation, to explore cause and effect relationships
    concerning an adverse event, and as a warning signal of a
    potential problem with a product. Useful factors for assessing a
    causal relationship include a chronology of administration of a
    product, including beginning and ending of treatment and adverse
    event onset; dechallenge; rechallenge; laboratory test results;
    and previously known toxicity of an agent or product. 47FDA,
    Center for Drug Evaluation and Research, "The Clinical Impact of
    Adverse Event Reporting" (Oct. 1996), pp. 5-6. 48FDA, Center for
    Drug Evaluation and Research, "The Clinical Impact of Adverse
    Event Reporting," p. 6. Page 36                   GAO/HEHS/GGD-99-
    90 FDA's Proposed Rule on Ephedrine Alkaloids Appendix III FDA's
    Prior Use of AERs in Rulemaking AERs have been used by FDA in
    several rulemaking procedures. In response to a request for prior
    rules that had relied on AERs, FDA provided five cases, including
    one in which the "generally recognized as safe" (GRAS) status of
    an agent (sulfites) was revoked. These cases are outlined in table
    III.1. Table III.1: Five Prior FDA Rulemaking Cases
    Regulation                    Substance                  Action 51
    Fed. Reg. 25021            Sulfiting agent            Revoked GRAS
    status for use on fruits and vegetables intended to be served or
    sold raw to consumers. 51 Fed. Reg. 25012            Sulfiting
    agent            Required food labeling for foods. 49 Fed. Reg.
    13679            Protein supplements        Required warning
    labeling. 62 Fed. Reg. 2218             Supplements and
    Required label warning statements and drugs containing iron unit-
    dose packaging. 44 Fed. Reg. 37212            FD&Ca yellow no. 5
    Required labeling in food and drugs for human use. aFood, Drug,
    and Cosmetic. Sulfiting Agents and                      FDA's
    proposed rule to revoke GRAS status for sulfiting agents resulted
    after GRAS Status                               the agency
    received information on the use of sulfur dioxide, sodium sulfite,
    sodium bisulfate, potassium bisulfate, sodium metabisulfate, and
    potassium metabisulfate (collectively known as sulfiting agents,
    or sulfites) on fruits and vegetables intended to be served raw or
    sold raw to consumers and it concluded that such use was not safe.
    This rule, similar to the ephedrine alkaloid ruling, used AERs,
    advisory panels, clinical studies, and case reports to determine
    whether sulfiting agents should maintain its GRAS status. FDA had
    received over 500 consumer complaints, where individuals
    reportedly suffered from a variety of adverse allergic-type
    responses, including reports of 13 deaths; the proposed rule
    indicated that these responses tended to occur in people with
    asthma.49 Among the 500 AERs, approximately 40 percent mentioned
    the occurrence of the event after eating raw fruits and vegetables
    in restaurants (where sulfiting agents are used), while 15 percent
    specifically mentioned the occurrence of the event after drinking
    wine or beer. The proposed rule delineated the clinical outcomes
    from these adverse reactions, ranging from gastrointestinal 49FDA
    relied on prevalence estimates that indicated there are 10 million
    individuals with asthma in the United States, with up to 10
    percent sulfite-sensitive. See Select Committee on GRAS
    Substances, "The Reexamination of the GRAS Status of Sulfiting
    Agents," Life Sciences Research Office, Federation of American
    Societies for Experimental Biology, prepared under FDA contract
    223-83-2020 (Jan. 28, 1985). Page 37
    GAO/HEHS/GGD-99-90 FDA's Proposed Rule on Ephedrine Alkaloids
    Appendix III FDA's Prior Use of AERs in Rulemaking problems to
    anaphylactic shock (hypersensitivity reactions characterized by
    swelling, erythema, bronchoconstrictions, and hypotension). A
    scientific panel also reviewed the AERs submitted to FDA and
    concluded that the reports indicated an association between
    adverse responses and the ingestion of meals that included foods
    containing sulfiting agents. The panel also examined exposure
    estimates and evidence from clinical experiments during its review
    on this topic. While this rule and the ephedrine alkaloid rule are
    similar-because the AERs used in the rulings simply highlighted
    the number of complaints and provided a breakdown of the
    complaints' origins and clinical complications-the proposed rule
    on sulfiting agents outlined additional information not found in
    the proposed rule for ephedrine alkaloids. This included
    information on exposure estimates, prevalence rates within
    sensitive populations, and clinical experiments involving the
    exposure of sensitive and nonsensitive populations to sulfiting
    agents. In addition, dosing regimens were not set for sulfiting
    agents because the proposed rule revoked their GRAS status.
    Sulfiting Agents and    The proposed rule for the labeling of
    products containing sulfiting agents Food Labeling           was
    designed to clarify the circumstances in which the presence of
    sulfiting agents must be declared on food labels. FDA noted in
    this proposed rule that any detectable amount of a sulfiting agent
    would require that the product be labeled because the agency was
    unaware of any evidence that established a level below which these
    substances would not cause a reaction in sensitive individuals.
    FDA relied on scientific data on human sensitivity to sulfiting
    agents to establish that certain sensitive individuals-in
    particular, those with asthma-could react to ingested sulfiting
    agents. FDA also used estimates of the number of individuals with
    asthma and clinical studies on those who were sensitive to
    sulfiting agents to determine the number who might be adversely
    affected. Last, FDA formed an ad hoc advisory committee on
    hypersensitivity to sulfiting agents in food to review and
    evaluate available data on adverse reactions in humans associated
    with the use of sulfiting agents in food. This rule did not
    describe, in any appreciable detail, adverse events. Labeling for
    Protein    In December 1997, FDA proposed labeling requirements
    for protein Supplements             supplements that were used in
    weight reduction programs. The purpose of the rule was to alert
    consumers to the potential health hazards associated Page 38
    GAO/HEHS/GGD-99-90 FDA's Proposed Rule on Ephedrine Alkaloids
    Appendix III FDA's Prior Use of AERs in Rulemaking with consuming
    protein supplements to control weight and to inform consumers that
    the advice of a physician should be sought before using these
    products for weight control. The agency proposed these
    requirements on the basis of evidence that, without proper medical
    supervision, low-calorie diets consisting primarily of protein may
    cause serious medical problems, including death. The proposed rule
    noted that an ad hoc advisory group had been formed to examine
    several cases of deaths associated with individuals using these
    protein products as their primary source of nourishment. A
    tentative final rule published on December 29, 1978, noted that
    more than 165 reports of adverse reactions attributed to
    consumption of protein products by consumers attempting to lose
    weight had been reported to the FDA. These complaints-described as
    "very diverse"-included nausea, tachycardia (rapid heart rate or
    palpitations), breathlessness, and headaches.50 Review of these
    reports by FDA revealed no consistent pattern of complaints or
    reactions, and the tentative final rule concluded that "a cause-
    and-effect relationship can neither be affirmed nor rejected on
    the basis of the existing information from these reports." The
    rule further stated that "There were sufficient details among the
    reported adverse reactions . . . to warrant concern." FDA further
    asserted that imposing a warning on the labeling of these protein
    products-even with the gaps in their understanding of the basic
    mechanism of how these protein supplements affected the body-was
    particularly appropriate because the consequence of indiscriminate
    use of these products was death. FDA also noted that the protein
    products were not inherently dangerous and that the label warning
    was proposed to curb misuse of these products and because the
    products were being sold directly to consumers who might not be
    under proper medical supervision. Data were collected through a
    telephone survey, conducted by FDA and CDC, to determine the
    extent of use of protein products. This information was used to
    establish a death rate by combining this information with data on
    deaths among liquid protein dieters from the same population
    group. This information was compared with data from the National
    Center for Health Statistics on annual death rates due to cardiac
    abnormalities among a similar population group. FDA concluded that
    the observed death rate was greater in those who used low-calorie
    protein diets. FDA also initiated experimental protocols using
    laboratory rats in order to explore the basic 50After a court
    challenge from the National Nutritional Foods Association, a final
    rule was published on April 4, 1984. Page 39
    GAO/HEHS/GGD-99-90 FDA's Proposed Rule on Ephedrine Alkaloids
    Appendix III FDA's Prior Use of AERs in Rulemaking mechanisms for
    the cause of death. These studies were designed to assess the
    correlation of cardiac arrhythmias (irregular heartbeat) and death
    with various aberrant states of nutrition health and rate of
    weight loss. Supplements and                 In October 1994, FDA
    proposed warning labels and unit-dose packaging 51 Drugs
    Containing Iron requirements  for products taken in solid oral
    form to supplement the dietary intake of iron or to provide iron
    for therapeutic purposes. The labeling was proposed because of
    evidence of acute iron poisoning attributable to accidental
    overdoses of iron-containing products and an upsurge in reported
    accidental pediatric ingestion of iron-containing products, which,
    in some cases, resulted in death.52 FDA relied on data from the
    American Association of Poison Control Centers and the U.S.
    Consumer Product Safety Commission (CPSC) that highlighted
    increases in reported fatalities from accidental iron poisonings
    of children. Over 63,000 reports were taken at poison control
    centers involving ingestion of adult iron-containing products,
    with over 47,000 of these involving children under 6 years of age.
    One hundred and fifty-nine cases were classified as major
    outcomes; that is, they were life threatening or resulted in
    permanent injury. An additional 1,500 cases were classified as
    "moderate outcomes"; that is, the patient had symptoms that
    required some form of treatment. Over 76,000 reports to these
    centers involved ingestion of pediatric iron-containing products
    with children under 6 years of age. Likewise, data from the
    National Electronic Injury Surveillance System-a probability
    sample of hospital emergency rooms in the United States used by
    CPSC to measure the magnitude of injury associated with consumer
    products-found a significant upward trend in the estimated number
    of hospital emergency-room-treated iron-ingestion cases involving
    children under 5 years of age. The proposed rule also outlined
    data collected by CPSC on case reports of pediatric deaths,
    including the number of tablets taken, the potency (dosage), and
    total amount ingested. Additionally, a conference was held by CPSC
    to examine the reason for the increase in iron poisonings among
    children. 51Unit-dose packaging is designed to prevent the
    unintended ingestion by children, not to control the recommended
    dose for adults. 52U.S. Consumer Product Safety Commission,
    "Pediatric Iron Poisonings and Fatalities" (May 1994), p. 3. Page
    40                    GAO/HEHS/GGD-99-90 FDA's Proposed Rule on
    Ephedrine Alkaloids Appendix III FDA's Prior Use of AERs in
    Rulemaking The final rule expanded on the information that was
    used as the basis for warning labels by outlining the results of
    animal toxicity studies, acute toxicity in humans, comparison of
    animal toxicity data to human toxicity data, and physiological
    factors that influence toxicity. Thus, information used by FDA in
    its proposed and final rules on iron-containing products included
    data from CPSC, case reports from adverse reactions, and
    scientific literature. FD&C Yellow No. 5    In its proposed rule
    of February 4, 1977, FDA noted that there was accumulating
    evidence that FD&C yellow no. 5 (tartrazine) caused allergic-type
    responses, especially in aspirin-intolerant individuals.53 Thus,
    FDA was proposing to require a label declaration of FD&C yellow
    no. 5 when used to color foods and ingested drugs and to prohibit
    its use in certain drugs for human use. FDA noted in the proposed
    rule that "the precise incidence of intolerance of FD&C yellow no.
    5 in the total population or even in aspirin-intolerant patients
    is not known." However, FDA did outline a number of case reports
    and studies highlighting that these effects did occur and that
    some quantification regarding subpopulations at risk could be
    identified. Specifically, FDA estimated that approximately 47,000
    to 94,000 people would be intolerant to tartrazine. But this
    proposed rule did not denote how many case reports or adverse drug
    reactions formed the basis for FDA's actions. Instead, the primary
    information used in the proposed and final rule related to
    scientific studies demonstrating individual intolerance to
    tartrazine. 53In the final rule, FDA required a label declaration
    of FD&C yellow no. 5 when used to color foods and drugs for human
    use. Page 41                    GAO/HEHS/GGD-99-90 FDA's Proposed
    Rule on Ephedrine Alkaloids Appendix IV FDA's Analysis of Benefits
    and Costs E.O. 12866 and related guidance from OIRA state that
    cost-benefit analyses should contain three basic elements: (1) a
    statement of the need for the proposed action, (2) an examination
    of alternative approaches, and (3) an analysis of benefits and
    costs. Overall, the guidance indicates that an analysis should be
    "transparent." We concluded that FDA's cost-benefit analysis
    contained, to some degree, all three of the recommended elements
    in such analyses. FDA also disclosed the basic methodology, data,
    and assumptions used in its analysis. However, the cost-benefit
    analysis was not always transparent regarding why certain key
    assumptions were made, the degree of uncertainty involved in those
    assumptions, or the effect that alternative assumptions would have
    had on the agency's estimates of the costs and benefits of the
    proposed action. E.O. 12866 and          Conceptually, a cost-
    benefit analysis is a rigorous and data-intensive Related Guidance
    Set    procedure of weighing the costs and benefits of various
    alternatives to a proposed action, informing decisionmakers about
    the potential Agency                  consequences of each.
    However, the results of the analysis, by themselves,
    Responsibilities for    do not determine whether or how an agency
    will regulate in response to a perceived problem. Analysis of
    Benefits and Costs               E.O. 12866 requires agencies to
    conduct cost-benefit analyses for all regulatory actions that are
    likely to result in a $100 million annual effect on the economy or
    are otherwise economically significant. The executive order also
    makes OIRA responsible for reviewing all significant regulatory
    actions and for providing guidance to agencies on issues covered
    by the order. On January 11, 1996, the Administrator of OIRA
    issued guidance describing "best practices" for preparing cost-
    benefit analyses.54 The guidance makes clear that it is not a
    "mechanistic blueprint" and provides for flexibility and the
    exercise of professional judgment in preparing analyses. However,
    the guidance does establish one clear expectation of all such
    analyses: Analysis of the risks, benefits, and costs associated
    with regulation must be guided by the principles of full
    disclosure and transparency. Data, models, inferences, and
    assumptions should be identified and evaluated explicitly,
    together with adequate justifications of choices made, and
    assessments of the effects of these choices on the analysis. The
    existence of plausible alternative models or assumptions and their
    implications should be identified. In the absence of adequate
    valid data, properly identified assumptions are necessary for
    conducting an assessment. 54The OIRA guidance was developed as a
    result of a 2-year study by an interagency group that included
    representatives of all the major regulatory agencies and was co-
    chaired by a member of the Council of Economic Advisers. Page 42
    GAO/HEHS/GGD-99-90 FDA's Proposed Rule on Ephedrine Alkaloids
    Appendix IV FDA's Analysis of Benefits and Costs Within this broad
    framework, the OIRA guidance states that analyses of economically
    significant rules should contain three basic elements: (1) a
    statement of the need for the proposed action, (2) an examination
    of alternative approaches, and (3) an analysis of benefits and
    costs. Within each of these three elements, the guidance
    recommends that certain items be considered or analytical
    approaches be used in preparing an agency's analysis. For example,
    in relation to the third requirement, the guidance says the
    analysis should identify and explain the data or studies on which
    the estimates of benefits and costs are based "with enough detail
    to permit independent assessment and verification of the results."
    The guidance also states, "where benefit or cost estimates are
    heavily dependent on certain assumptions, it is essential to make
    those assumptions explicit and, where alternative assumptions are
    plausible, to carry out sensitivity analyses based on the
    alternative assumptions." The purpose of such disclosure is to
    allow for a reasoned determination by decisionmakers of the
    appropriate level of regulatory action. FDA's Analysis         FDA
    indicated in the proposed rule that the regulation was
    economically Addressed the          significant, and FDA officials
    told us that they used the OIRA guidance in developing the cost-
    benefit analysis for the rule. We compared FDA's Primary Elements
    cost-benefit analysis to the elements and practices outlined in
    that Suggested by OIRA      guidance and concluded that FDA's
    analysis addressed the primary elements suggested by the OIRA
    guidance. However, certain specific Guidance
    elements appeared to be lacking. Need for Regulation    The OIRA
    guidance states that in establishing the need for the proposed
    action, agencies should discuss (1) whether the problem
    constitutes a significant market failure that compels government
    action and (2) if there is a market failure, the appropriateness
    of alternatives to federal regulation that would resolve the
    problem adequately or better than the proposed rule. Among the
    types of market failures that the guidance says agencies can
    discuss are monopolies and inadequate information available to
    consumers about product characteristics. The guidance states that
    alternatives to federal regulation can include subsidies or fees
    that may be more efficient than rigid mandates or state and local
    regulation that would be more appropriate. FDA stated in the
    proposed rule that the rule was needed because of a significant
    market failure-specifically, inadequate information. FDA said that
    despite the presence of warning labels of various types on many of
    the products, "some consumers may not have sufficient information
    on the Page 43                GAO/HEHS/GGD-99-90 FDA's Proposed
    Rule on Ephedrine Alkaloids Appendix IV FDA's Analysis of Benefits
    and Costs health risks associated with dietary supplements
    containing ephedrine alkaloids to make informed choices concerning
    the consumption of these products." Among other points, FDA said
    that the level of information currently used by consumers might be
    less than optimal because of consumer perceptions that products
    marketed as foods or derived from botanical sources are inherently
    safe. However, FDA did not directly address the second major item
    in the guidance under the need for action section-whether
    alternatives to federal regulation would resolve the problem
    adequately or better than the proposed federal regulation. In
    particular, FDA did not recognize that several states were in the
    process of regulating these products and did not discuss whether
    federal regulation was superior to those state regulations (for
    example, because these products are marketed across state lines).
    Examination of Alternative      The OIRA guidance states that an
    agency's cost-benefit analysis should Approaches
    demonstrate that the agency has considered the most important
    alternative approaches to the underlying problem and should
    provide the agency's reasoning for selecting the proposed
    regulation over such alternatives. The guidance notes that the
    number and choice of alternatives to be selected for a detailed
    cost-benefit analysis is a matter of judgment but says the agency
    should nevertheless "explore modifications of some or all of a
    regulation's attributes or provisions to identify appropriate
    alternatives." The guidance also states that the agency should
    explore a number of different types of regulatory alternatives,
    including * more performance-oriented standards for health,
    safety, and environmental regulations; * different requirements
    for different segments of the regulated population; * alternative
    levels of stringency; * alternative effective dates of compliance;
* alternative methods of ensuring compliance; * informational
    measures (for example, mandatory disclosure requirements, such as
    labeling); * more market-oriented approaches; and * consideration
    of whether the agency should adopt a more stringent standard than
    already established by statutory requirements. In the proposed
    rule, FDA identified seven regulatory "options": (1) take no
    action (the baseline alternative); (2) take no action, but
    generate additional information; (3) take the proposed action; (4)
    take the proposed Page 44                GAO/HEHS/GGD-99-90 FDA's
    Proposed Rule on Ephedrine Alkaloids Appendix IV FDA's Analysis of
    Benefits and Costs action but with a higher potency limit; (5) ban
    dietary supplements that contain ephedrine alkaloids; (6) take the
    proposed action, but do not require the warning statement; and (7)
    require only the warning statement. In general, FDA's options
    focused primarily on the OIRA guidance categories of alternative
    levels of stringency and informational measures. FDA did not
    appear to explore performance-oriented standards, different
    requirements for different segments of the regulated population,
    market-oriented approaches, or alternative effective dates of
    compliance. However, it is unclear whether these types of
    alternatives could legitimately be considered for the proposed
    regulation. For example, it is unclear how FDA could establish
    market approaches, such as additional fees on the dietary
    supplement industry, or performance standards for health effects.
    In comments on the rule, industry representatives and SBA's Office
    of Advocacy identified what they viewed as a weakness related to
    FDA's baseline description of the world absent the proposed
    regulation. FDA presented information on the number of adverse
    events believed to be associated with ephedrine alkaloid products
    and described an increasing trend in reported adverse events.
    However, industry representatives and SBA said FDA did not provide
    sufficient information to put these data in a meaningful context.
    For example, they said there was no information on general
    probabilities or risks of people suffering the types of adverse
    events that FDA attributed to ephedrine alkaloid products. They
    also said that FDA did not provide information to put the
    aggregate number and growth in reported adverse events in the
    context of total consumption of dietary supplements that might
    contain ephedrine alkaloids. SBA's Office of Advocacy and industry
    trade associations commented that consumption of such supplements
    might be in the billions of doses. In that context, they contended
    that adverse events associated with the consumption of dietary
    supplements containing ephedrine alkaloids may occur at very low
    rates, perhaps lower than those for some over-the-counter drugs.
    On the other hand, FDA did provide some contextual information in
    the proposed rule on the incidence of adverse events for dietary
    supplements as a whole. FDA's data indicated that AERs believed to
    be associated with supplements containing ephedrine alkaloids
    accounted for approximately 50 to 60 percent of the total number
    of reports received by FDA when the rulemaking was initiated. FDA
    officials told us that the number of AERs in their system
    suspected of involving ephedrine alkaloids "jumped out" compared
    with the numbers for any other substances, although they have not
    done any detailed analysis or tally to document numbers for other
    Page 45                GAO/HEHS/GGD-99-90 FDA's Proposed Rule on
    Ephedrine Alkaloids Appendix IV FDA's Analysis of Benefits and
    Costs substances. Representatives of industry trade associations
    who we met with acknowledged that the incidence of adverse events
    for supplements containing ephedrine alkaloids is greater than
    would be expected for most other dietary supplements. Still,
    providing only this contextual information in the proposed rule
    does not appear to provide a balanced or complete picture of the
    baseline risks that may be associated with supplements containing
    ephedrine alkaloids. Analysis of Benefits and    The bulk of
    OIRA's guidance is devoted to addressing the third element, Costs
    analysis of benefits and costs. In this section, the guidance
    first presents general principles on (1) measuring the benefits
    and costs of each alternative against a baseline condition of the
    way the world would look absent the proposed regulation, (2)
    evaluation of alternatives, (3) discounting costs and benefits
    that may occur at different points in time, (4) treatment of risk
    and uncertainty, (5) assumptions used in the analysis, (6)
    international trade effects, (7) nonmonetized benefits and costs,
    and (8) distributional effects and equity. These general
    principles are followed in the OIRA document by more detailed
    guidance on the analysis of benefits and costs. For example,
    within the benefits section, the guidance states that reductions
    in illnesses, injuries, and fatalities as a result of government
    action are best monetized according to a "willingness to pay"
    approach.55 The guidance also emphasizes that benefits and costs
    should be "incremental," representing changes from the baseline
    condition of no regulation, and that the analysis should identify
    and explain the data or studies on which the estimates are based
    "with enough detail to permit independent assessment and
    verification of the results." In its analysis of benefits and
    costs, FDA adhered to a number of the recommendations in the OIRA
    guidance. For example, the analysis discussed distributional
    effects of the proposed rule and alternatives in terms of lost
    sales to the dietary supplement industry. FDA also discussed
    nonmonetized costs associated with the proposed rule, and it
    monetized the benefits associated with reductions in various types
    of adverse events using the willingness-to-pay approach
    recommended in the guidance. In addition, the analysis presented
    the monetized costs of the proposed rule against the baseline
    condition of no regulation. However, FDA did not provide monetized
    or quantified benefit or cost estimates for all of the other
    alternatives, and the agency's discussion of those alternatives'
    benefits and costs were compared to the proposed rule-not the
    baseline 55The principle of "willingness to pay" captures the
    notion of opportunity cost by providing an aggregate measure of
    what individuals are willing to forgo to enjoy a particular
    benefit. Page 46                    GAO/HEHS/GGD-99-90 FDA's
    Proposed Rule on Ephedrine Alkaloids Appendix IV FDA's Analysis of
    Benefits and Costs condition of no regulation. For example, in
    relation to the benefits associated with allowing higher potency
    levels (option 4) compared to FDA's proposal (option 3), FDA said
    that the effect would be to "reduce those benefits below those
    generated under option 3." (FDA officials told us during our
    review that the agency attempted to reach a balance in conducting
    the analysis and said the analysts try to quantify the effects of
    alternatives when such information would be of value to
    decisionmakers.) Also, some parts of FDA's analysis of these
    alternatives were either unclear or illogical. For example, in
    relation to the option of banning dietary supplements that contain
    ephedrine alkaloids (option 5), FDA stated that "banning these
    products would decrease access to these products by consumers." It
    is not clear why a total ban in the products would not eliminate-
    not just decrease-access. Similarly, FDA also said "the total
    reduction in the consumption of dietary supplements that now
    contain ephedrine would probably be approximately 33 percent under
    this option." It is not clear why a ban would only reduce
    consumption by 33 percent. Another unclear element of the analysis
    concerns discounting. The guidance recommends that dollar
    estimates should be discounted and reported in same-year dollars.
    Although FDA noted that it converted dollar figures for its
    estimates of monetized benefits to 1996 dollars, it did not
    identify in the proposed rule whether its total cost estimates
    were also expressed in 1996 dollars. As a result of its analysis,
    FDA concluded that the proposed rule would generate benefits of
    between $240 million and $670 million per year, quantifiable costs
    of between $3 million and $70 million in the first year, and
    quantifiable costs of between "a minimal amount" and $500,000 in
    each subsequent year. FDA justified the selection of the proposed
    rule over the other regulatory alternatives in terms of the
    greater net benefits expected of the proposed actions. Analysis
    Did Not Fully The OIRA guidance emphasizes the importance of full
    disclosure and Explain Some Key                 transparency of
    key data and assumptions in cost-benefit analyses. As noted
    previously, the guidance says that "data, models, inferences, and
    Assumptions or                   assumptions should be identified
    and evaluated explicitly, together with Assess Implications of
    adequate justifications of choices made, and assessments of the
    effects of these choices on the analysis." Therefore, we assessed
    the transparency Alternative                      of FDA's cost-
    benefit analysis of the proposed rule in terms of three
    Assumptions                      criteria: Page 47
    GAO/HEHS/GGD-99-90 FDA's Proposed Rule on Ephedrine Alkaloids
    Appendix IV FDA's Analysis of Benefits and Costs * first, whether
    it identified the data, models, inferences, and assumptions used
    to calculate the estimates of benefits and costs; * second,
    whether it disclosed the reasons why those data, models,
    inferences, and assumptions were selected; and * third, whether it
    assessed the effects of plausible alternative assumptions and
    choices on the results of the analysis-what is often referred to
    as a "sensitivity analysis." Identification of Data and      In
    relation to the first criterion, FDA's analysis was very
    transparent. The Assumptions                     agency provided a
    clear and lengthy description of the data, assumptions, and
    methodology that it used to calculate the benefits and costs of
    the proposed rule. For example, in calculating the annual
    monetized benefits of the proposed rule, FDA identified for each
    of six categories of adverse events (for example, deaths, serious
    cardiovascular system events, serious nervous system events, and
    abnormal liver functions) (1) the annual number of reported
    adverse events (based on the average number of AERs received each
    year), (2) the estimated total number of such events each year,
    (3) the estimated number of cases that would be reduced by
    implementation of the proposed rule, (4) the monetized value of
    each reduced case, and (5) the estimated value of the total risk
    reduction (the number of reduced cases times the value of each
    case). FDA also identified many of the underlying assumptions that
    it used in developing these elements. For example, in developing
    its estimate of the total number of annual cases, FDA noted that
    it assumed that (1) 80 percent of the reported adverse events
    suspected of involving supplements containing ephedrine alkaloids
    were actually related to the consumption of dietary supplements,
    (2) 80 percent of these supplements actually contained ephedrine
    alkaloids, and (3) only 10 percent of adverse events related to
    ephedrine alkaloids were actually reported. In similar fashion,
    FDA disclosed how it calculated the estimated compliance costs of
    the rule, including the one-time costs associated with relabeling
    and reformulating the affected supplements. Explanation of
    In relation to the second transparency criterion-explaining the
    reasons Assumptions                     for choices made for key
    assumptions and values-FDA's analysis was only partially
    transparent. For most elements of the analysis, FDA identified the
    underlying data sources used and the rationale for the assumptions
    and conclusions reached by FDA's analysts and experts. The
    proposed rule included a lengthy discussion about the scientific
    data and studies FDA and Page 48                GAO/HEHS/GGD-99-90
    FDA's Proposed Rule on Ephedrine Alkaloids Appendix IV FDA's
    Analysis of Benefits and Costs its advisory committees reviewed in
    developing the various provisions of the rule. In estimating the
    effects of these provisions, FDA also identified the underlying
    data sources and analysis in most cases. For example, in its
    discussion on potential relabeling costs, FDA described how it
    used data from a previous FDA rule on nutrition labeling of
    dietary supplements to derive its estimate of the specific costs
    for this rule. In describing how it estimated the effect of
    restrictions on including other stimulants in supplements
    containing ephedrine alkaloids, FDA discussed the results of a
    clinical study on combinations of ephedrine and caffeine and the
    results of its informal review of adverse event reports. However,
    FDA did not fully disclose the underpinnings of all of its
    assumptions and choices. In particular, although FDA identified
    the values it used to estimate the number and severity of adverse
    events for its calculation of benefits, its explanation of the
    underlying data sources, analyses, and basis for the selected
    values was not complete. In its analysis, FDA identified three
    main areas of uncertainty in its estimation of the actual number
    of adverse events believed to be attributable to use of products
    containing ephedrine alkaloids. These areas were (1) the
    proportion of reported adverse events that were actually related
    to consumption of dietary supplements, (2) the proportion of
    reported adverse events that involved supplements that actually
    contained ephedrine alkaloids, and (3) the proportion of adverse
    events that were actually reported. For each of these elements,
    FDA assumed a particular value that drove up the estimated number
    of actual adverse events and, as a direct consequence, the
    estimated benefits of the proposed rule. However, FDA did not
    always disclose why it selected those values or why it selected a
    single value instead of a range of values that would have more
    clearly represented the degree of uncertainty that FDA believed
    was present. In relation to the first area of uncertainty, FDA
    estimated that between 27 and 90 percent of the reported adverse
    events were "probably" related to the consumption of dietary
    supplements suspected of containing ephedrine alkaloids. However,
    FDA did not clearly disclose the basis for estimating this range.
    Within this range, FDA assumed that 80 percent of the reported
    adverse events were actually related to the consumption of dietary
    supplements. FDA did not disclose how it arrived at the choice of
    80 percent as the "most likely" value. Furthermore, in calculating
    the "range" of estimated benefits for the proposed rule, FDA used
    only this point estimate, not the entire 27 to 90 percent range,
    or a more narrowly focused range of values in calculating the
    expected number of cases. Use Page 49                GAO/HEHS/GGD-
    99-90 FDA's Proposed Rule on Ephedrine Alkaloids Appendix IV FDA's
    Analysis of Benefits and Costs of a range of values for this
    element would have more clearly represented the uncertainty
    regarding the number of AERs actually related to the consumption
    of dietary supplements.56 The second source of uncertainty that
    FDA identified was the extent to which all AERs involved products
    that actually contained ephedrine alkaloids. FDA estimated that
    the proportion of reported adverse events associated with dietary
    supplements containing ephedrine alkaloids was "probably" between
    25 and 90 percent based on (1) the labeling of the products
    involved, (2) FDA's own market study and laboratory analysis of
    125 marketed products, and (3) the similarity of the reported
    adverse events to the known effects of ephedrine alkaloids. Within
    this range, FDA assumed that 80 percent of the reported adverse
    events associated with the consumption of dietary supplements
    involved supplements that contained ephedrine. FDA did not
    disclose why it selected 80 percent as the "most likely" value.
    Finally, as was the case in the first area of uncertainty, FDA
    used only this 80 percent point estimate, not the full range or an
    attenuated range of values, in calculating the expected number of
    annual cases. FDA's third identified source of uncertainty was the
    likelihood that all adverse events related to ephedrine alkaloids
    were probably not being reported. FDA noted in the analysis that
    typical reporting rates for passive reporting systems on adverse
    events associated with pharmaceutical drugs are "generally assumed
    to be on the order of 10 percent." However, FDA noted that
    reporting rates might be higher than usual if, for example, the
    potential health risks are widely publicized, or lower than normal
    if consumers and physicians assume that dietary supplements are
    incapable of producing adverse events. FDA said that "to
    incorporate this uncertainty," it assumed that the reporting rate
    for the adverse events related to ephedrine alkaloids was 10
    percent. It was not clear how a single point estimate of 10
    percent would "incorporate" the uncertainty that FDA identified
    for this value. In response to our questions about these
    uncertainties, FDA officials acknowledged that their analysis of
    impacts was not as transparent as it should have been in
    explaining how the agency arrived at some of the assumptions. The
    Director of FDA's Market Studies Division, which prepared the
    cost-benefit analysis, explained that the estimated 56FDA was
    somewhat inconsistent in its use of point estimates versus ranges.
    FDA used its full estimated range of adverse events in projecting
    the results for other regulatory options, such as banning
    supplements with ephedrine alkaloids. FDA officials said that they
    chose to use the full ranges in discussing other options because
    of the regulatory actions in those options. Page 50
    GAO/HEHS/GGD-99-90 FDA's Proposed Rule on Ephedrine Alkaloids
    Appendix IV FDA's Analysis of Benefits and Costs proportions and
    the selected values represented the best professional judgment of
    FDA's analysts and experts. He noted that the upper-bound estimate
    of 90 percent for the first two areas of uncertainty represented
    what the analysts believed was a reasonable choice for the
    "absolute most it could be." He said that the lower-bound estimate
    of 27 percent for the first category was derived from the cases in
    the AERs in which people who stopped using products suspected of
    containing ephedrine alkaloids no longer experienced the effects
    associated with the adverse event. Similarly, FDA's figure of 25
    percent as the most reasonable low-end estimate for the second
    category was based on FDA's review of samples of products
    mentioned in the AERs. He said that the choice of 80 percent as
    the most likely value for both of these categories was a
    determination made by FDA's scientific experts, given all the
    available data and scientific evidence they had reviewed, not just
    the AERs. The director and other FDA officials indicated that FDA
    had much more confidence in the estimate that only 10 percent of
    actual adverse events were reported. They said that, compared to
    the other two sources of uncertainty, FDA knows more about AEMS
    and is also familiar with studies of other passive reporting
    systems used by FDA and other federal agencies. The officials
    noted that, for other systems, they have done reviews for specific
    sites (for example, going to hospitals to check records and do
    testing) and found that reporting rates were about 10 percent. The
    director said that FDA did not provide a range for this category
    in the proposed rule because FDA was confident in the point
    estimate and because it did not have any way-other than through an
    "artificial exercise"-to come up with a range of possible values.
    Implications of Alternative    FDA's cost-benefit analysis was
    also only somewhat transparent with Assumptions
    regard to the third of our transparency criteria-assessment of the
    effects of plausible alternative assumptions and choices on the
    results of the analysis. The degree of transparency in this area
    and the sensitivity of FDA's results to alternative assumptions
    are most clearly illustrated in the agency's estimate of expected
    benefits. FDA estimated that the benefits of the proposed rule
    would be between $240 million and $670 million per year. That
    estimate was driven by three factors: (1) FDA's estimate of the
    number of adverse events each year (1,110), (2) FDA's estimate of
    the degree to which the proposed rule would reduce these events
    (35 percent to 100 percent across all types of proposed actions),
    and (3) the values FDA assigned to the estimated risk Page 51
    GAO/HEHS/GGD-99-90 FDA's Proposed Rule on Ephedrine Alkaloids
    Appendix IV FDA's Analysis of Benefits and Costs reduction per
    case (for example, $5 million per death avoided).57 Changes in any
    of these values could have dramatically changed FDA's estimates of
    the proposed rule's benefits. By using the 35 to 100 percent range
    for the second factor (and even wider ranges for some of the
    individual actions), FDA illustrated the sensitivity of the
    analysis to changes in particular values. This second factor was
    the only component in FDA's calculations that generated a "range"
    in the estimated reduction in annual adverse events (390 to 1,110)
    and, in turn, the size of the expected benefits ($240 million to
    $670 million).58 On the other hand, FDA's estimate of the 1,110
    estimated adverse events each year (factor 1) was based on an
    average of 174 adverse events reported per year and the three
    assumptions about adverse events-that 80 percent of the AERs
    involved dietary supplements, that 80 percent of the supplements
    contained ephedrine alkaloids, and that only 10 percent of all
    adverse events related to supplements containing ephedrine
    alkaloids were reported. FDA's use of these point estimates
    (instead of ranges) does not reflect the sensitivity of the final
    benefit estimates to these assumptions. As illustrated in figure
    IV.1, the results of FDA's estimation of expected benefits are,
    therefore, very sensitive to changes in the underlying
    assumptions. If FDA had used other plausible assumptions, based on
    the information it presented in the proposed rule, the ranges of
    expected benefits would have varied noticeably from the values FDA
    presented in its proposal. 57In its proposed rule, FDA did not
    report or round its estimated values consistently, so slightly
    different numbers for the same element appear within the proposed
    rule. For this report, we are using the values reported in table 6
    of FDA's proposed rule, such as 1,110 for the estimated annual
    number of adverse events. 58Similarly, FDA's $3 million to $80
    million range in the estimated cost of the proposed rule also
    reflects the sensitivity of the agency's assumptions. Page 52
    GAO/HEHS/GGD-99-90 FDA's Proposed Rule on Ephedrine Alkaloids
    Appendix IV FDA's Analysis of Benefits and Costs Figure IV.1:
    Estimation of Benefits Is Sensitive to Changes in Underlying
    Assumptions                                 Estimated Annual
    Benefits (Dollars in Millions) 1400
    1343 1200 1000 850 800 670 600 470 400
    336 240 200 118 25 0 (1) FDA's        (2) Using Full    (3)
    Assuming    (4) Assuming Reported        Range of FDA       20% of
    Cases    5% of Cases Estimates        Assumptions       Are
    Reported    Are Reported Legend High Bound Low Bound Note: The
    data for scenario 1 are from FDA's reported estimates in table 6
    of the proposed rule. Scenario 2 was calculated using the full
    range of assumptions FDA presented in the narrative of its
    proposed rule regarding key assumptions about the number of
    adverse events each year. Scenarios 3 and 4 were calculated
    changing only FDA's selected assumption regarding the reporting
    rate for adverse events. All other assumptions were held constant.
    Source: Scenario 1 data are taken from table 6 of FDA's proposed
    rule. Data for other scenarios were GAO calculations using
    information in FDA's proposal. Given the uncertainty surrounding
    most of the key data and assumptions used in FDA's analysis,
    similar types of sensitivity analyses could be done using
    alternative choices for other elements in the analysis, such as
    the following: Page 53                       GAO/HEHS/GGD-99-90
    FDA's Proposed Rule on Ephedrine Alkaloids Appendix IV FDA's
    Analysis of Benefits and Costs * FDA used published estimates of
    the value that consumers place on reducing certain types of risk
    to estimate the monetary value benefits associated with the
    proposed rule. However, other published estimates for these values
    could also have been used. For example, FDA assumed in its
    analysis that the value of avoiding a death was $5 million. We
    have reported that the Environmental Protection Agency's (EPA)
    assumptions regarding the value of avoiding a death have ranged
    from $1.6 million to $12 million.59 * FDA's estimates of the
    benefits of the rule were also a function of the distribution of
    the types of adverse events (for example, deaths, cardiovascular
    system events, and neurological system events) reported through
    the underlying AERs. Given that there is sufficient uncertainty
    regarding the reports that different distributions of the numbers
    of events are plausible, FDA's analysis could have reflected these
    alternative distributions and the effect they would have had on
    the benefit estimates. * FDA's estimates of the costs of the
    proposed rule were based, in part, on assumptions regarding the
    number of dietary supplement products sold and the number of
    businesses involved in the dietary supplement market. As discussed
    in appendix V, "Regulatory Flexibility Act Requirements," there is
    considerable uncertainty about the actual number of businesses and
    products involved in the market for dietary supplements containing
    ephedrine alkaloids. As FDA officials noted in our meetings with
    them, there are limits to the amount of effort that can be
    expended on an analysis, and they attempt to reach a balance in
    conducting a reasonable effort. They pointed out that the proposed
    rule was long and that they had struggled to keep the length of
    the narrative under control. However, the OIRA guidance recommends
    that agencies carry out sensitivity analyses over the full range
    of plausible values of key parameters, particularly when there are
    several easily identifiable critical assumptions in the analysis
    and information is inadequate to carry out a more formal
    probabilistic simulation for assessment of risks. In this respect,
    FDA's focus on presenting the agency's "best estimate" does not
    seem to fully reflect the uncertainty in the underlying data and
    analysis or the expectations set by the OIRA guidance. In
    explaining why FDA did not do a sensitivity analysis for this
    proposed rule, the Director of the Market Studies Division said
    that, at this point in time, risk assessors are ahead of cost-
    benefit analysts in doing sensitivity analyses. He did note that,
    while FDA did not do a sensitivity analysis for 59Regulatory
    Reform: Agencies Could Improve Development, Documentation, and
    Clarity of Regulatory Economic Analyses (GAO/RCED-98-142, May 26,
    1998), and Air Pollution: Information Contained in EPA's
    Regulatory Impact Analyses Can Be Made Clearer (GAO/RCED-97-38,
    Apr. 14, 1997). Page 54                  GAO/HEHS/GGD-99-90 FDA's
    Proposed Rule on Ephedrine Alkaloids Appendix IV FDA's Analysis of
    Benefits and Costs this particular proposed rule, it would
    probably do more sensitivity analyses in the near future. Page 55
    GAO/HEHS/GGD-99-90 FDA's Proposed Rule on Ephedrine Alkaloids
    Appendix V Regulatory Flexibility Act Requirements The Regulatory
    Flexibility Act requires agencies to consider the effects of their
    rules on small entities. FDA determined that its proposed rule on
    dietary supplements containing ephedrine alkaloids would have a
    significant economic impact on a substantial number of small
    entities. Therefore, FDA prepared an initial regulatory
    flexibility analysis to identify the potential impacts of the
    proposed rule and alternative actions on affected small
    businesses. Also, as required by RFA, FDA provided opportunities
    for small business participation in the rulemaking process.
    However, while FDA addressed the basic requirements of the act,
    SBA's Office of Advocacy criticized the quality of FDA's
    regulatory flexibility analysis. For example, the office contended
    that FDA underestimated the number of affected small businesses
    and the impacts of the proposed actions by not considering the
    large numbers of independent distributors of dietary supplements
    in its analysis. Regulatory Flexibility          During the past
    20 years, the Congress has enacted a number of statutes Act
    Requires Agencies designed to improve the federal rulemaking
    process, one of which is the Regulatory Flexibility Act of 1980,
    as amended (5 U.S.C. 601-612). to Consider Impact of
    Congress enacted RFA in response to concerns about the effect that
    federal Regulation on Small             regulations can have on
    small entities. According to SBA's Office of Advocacy, the major
    goals of the act are to (1) increase agency awareness Businesses
    and understanding of the impact of their regulations on small
    business, (2) require that agencies communicate and explain their
    findings to the public, and (3) encourage agencies to use
    flexibility and to provide regulatory relief to small entities.
    RFA mandates certain actions on the part of agencies during the
    rulemaking process. For example, before publishing a proposed rule
    for which a notice of proposed rulemaking is required, sections
    603 and 605(b) of the act state that a federal agency must prepare
    and make available for public comment an initial regulatory
    flexibility analysis that describes the anticipated effects of the
    proposed rule on small entities, unless the head of the agency
    certifies that the proposed rule will not have a "significant
    economic impact on a substantial number of small entities."
    Specifically, the act states that an agency's initial analysis
    must contain (1) a description of why the agency is considering
    the action; (2) the objectives and legal basis for the proposed
    rule; (3) a description and, where feasible, an estimate of the
    number of small entities to which the proposed rule will apply;
    (4) a description of the projected reporting, recordkeeping, and
    other compliance requirements of the proposed rule; and (5) a
    description of all federal rules that may duplicate, overlap, or
    Page 56              GAO/HEHS/GGD-99-90 FDA's Proposed Rule on
    Ephedrine Alkaloids Appendix V Regulatory Flexibility Act
    Requirements conflict with the proposed rule. The act also states
    that the initial analysis shall discuss significant alternatives
    for small entities that accomplish the objectives of applicable
    statutes, such as different compliance or reporting requirements,
    clarified or consolidated compliance and reporting requirements,
    and an exemption from coverage by the rule. When a rule is
    promulgated that will have a significant economic impact on a
    substantial number of small entities, section 609(a) of RFA
    requires agencies to ensure that small entities have been given an
    opportunity to participate in the rulemaking process through the
    "reasonable use" of outreach techniques.60 The act delineates a
    number of specific types of outreach techniques agencies could
    use, including (1) publishing the notice of proposed rulemaking in
    publications likely to be obtained by small entities, (2) directly
    notifying interested small entities, and (3) conducting open
    conferences or public hearings concerning the rule. Section 612 of
    the act requires the SBA Chief Counsel for Advocacy to monitor
    agencies' compliance with RFA and authorizes the Chief Counsel to
    appear as amicus curiae ("friend of the court") in court
    proceedings to review rules under the act. FDA's Actions Met
    FDA determined that the proposed rule would have a significant
    economic Basic RFA                 impact on a substantial number
    of small entities. The agency concluded that a total of 80 small
    manufacturers and distributors would be affected Requirements but
    by the proposed rule using (1) information in previous studies
    that Attracted Criticism       indicated that 95 percent of all
    dietary supplement manufacturers were small businesses and (2)
    information from two market surveys that identified 85
    manufacturers and distributors of dietary supplements suspected of
    containing ephedrine alkaloids. This determination triggered the
    requirement in sections 603 and 609 of the act, respectively, that
    FDA conduct an initial regulatory flexibility analysis and conduct
    outreach efforts. Regulatory Flexibility    FDA's proposed rule
    addressed the basic elements that RFA requires Analysis
    Requirements     agencies to include in an initial regulatory
    flexibility analysis. The rule described the reasons the agency
    was considering the action (market failure), stated the purpose
    and legal basis of the rule, described and 60FDA is not subject to
    other RFA requirements regarding small business participation,
    such as the requirement to convene small business advocacy review
    panels that the act, as amended by the Small Business Regulatory
    Enforcement Fairness Act, places on EPA and the Occupational
    Safety and Health Administration. For a discussion of these
    panels, see Regulatory Reform: Implementation of the Small
    Business Advocacy Review Panel Requirements (GAO/GGD-98-36, Mar.
    18, 1998). Page 57                   GAO/HEHS/GGD-99-90 FDA's
    Proposed Rule on Ephedrine Alkaloids Appendix V Regulatory
    Flexibility Act Requirements provided an estimate of the number of
    small entities to which FDA believed the rule would apply, and
    described the compliance requirements. In addition to describing
    direct compliance costs of between $3 million and $80 million, FDA
    also said that the proposed rule could have significant
    distributive effects in the form of reduced sales of as much as
    $230 million a year. FDA explicitly stated that costs and sales
    reductions of this magnitude might threaten the viability of many
    firms in this industry. FDA also discussed significant
    alternatives to the proposed rule in the rule's regulatory
    flexibility analysis section, noting that most of the regulatory
    alternatives discussed in the cost-benefit analysis section would
    reduce the impact of this rule on small businesses. For example,
    FDA noted that taking no action or simply generating additional
    information would reduce the impact on small businesses to zero.
    However, FDA said that raising the proposed potency limit would
    have the same impact as the proposed rule and banning dietary
    supplements would have the greatest negative impact. The industry
    representatives we met with confirmed that manufacturing and
    labeling of these products are generally limited to a relatively
    small number of manufacturing firms. Such firms would bear almost
    all of the expenditures required for compliance with the proposed
    rule. However, the executive director of one of the major trade
    associations we contacted pointed out that the proposed actions
    may also create some compliance costs for downstream distributors
    of the many different brands of products marketed. He said that
    these distributors might have to revise their marketing and
    promotional materials (for example, checking and revising claims
    and other information presented in the materials) to comply with
    the proposed rule. It is not clear whether FDA's estimated costs
    for relabeling also address this cost element. In comments
    submitted on the proposed rule, SBA's Office of Advocacy was
    critical of FDA's initial regulatory flexibility analysis, stating
    that the underlying scientific data and analysis did not
    adequately support the proposed actions and the proposed rule did
    not provide enough information for someone to "meaningfully
    comment" on the proposed rule or alternatives. SBA also criticized
    FDA's analysis of the impacts on small businesses, saying it had
    underestimated the number of entities affected because it focused
    solely on the manufacturers of dietary supplements suspected of
    containing ephedrine alkaloids and did not take independent
    distributors into account. The Office of Advocacy noted that "many
    ephedra products are sold by tens of thousands of home-based
    distributorships that are part of multilevel marketing companies.
    Many of Page 58                GAO/HEHS/GGD-99-90 FDA's Proposed
    Rule on Ephedrine Alkaloids Appendix V Regulatory Flexibility Act
    Requirements these businesses, although part of a larger parent
    company, may nevertheless be independently owned and operated and
    considered to be `small business concerns' under the Small
    Business Act." The office pointed out the importance of good
    information about the regulated industry, stating, In order to
    determine the impact of any regulation, an agency must make a
    reasonable effort to identify the type and number of entities
    likely to be affected by the regulation. This process of learning
    about the regulated industry not only helps the agency determine
    whether to certify a rule for regulatory flexibility purposes, it
    also helps the agency develop an analysis of impacts and choose
    appropriate regulatory alternatives that minimize economic burden.
    In response to SBA's criticism, FDA officials told us that the
    agency need not consider the impact of its proposed rule on
    distributors of ephedrine alkaloids. The FDA officials cited
    several court decisions to support the proposition that, under
    RFA, an agency is under no obligation to conduct a small entity
    impact analysis of effects on entities that the agency does not
    regulate-that is, distributors.61 None of the major parties-FDA,
    SBA's Office of Advocacy, or industry associations-has information
    on the actual number of entities involved in the market for
    dietary supplements containing ephedrine alkaloids. Each
    acknowledge this as a limitation in attempts to analyze the
    effects of the proposed rule. FDA officials told us that the
    agency did not do a true market study for the proposed rule. The
    market review that FDA used in its analysis was done primarily for
    the purposes of informing the agency about supplements that might
    contain ephedrine alkaloids. SBA's Office of Advocacy recommended
    that FDA develop an outreach strategy to obtain more reliable
    industry data, and officials of both the Office of Advocacy and
    FDA noted that they have met on this issue since the comment
    period on the proposed rule ended. In addition, FDA officials said
    that they have contracted for a market study of the dietary
    supplement industry that should provide better information for
    future FDA analyses. Outreach Efforts    With regard to the
    requirement in section 609(a) of the act requiring FDA to engage
    in outreach efforts, the regulatory docket shows that FDA provided
    opportunities during the development of the proposed rule for
    participation by affected businesses. For example, in addition to
    publishing the proposed rule, FDA published public notices and
    held public 61For example, United Distribution Cos. v. FERC, 88
    F.3d 1105, 1170 (D.C. Cir. 1996). Page 59
    GAO/HEHS/GGD-99-90 FDA's Proposed Rule on Ephedrine Alkaloids
    Appendix V Regulatory Flexibility Act Requirements meetings,
    during which industry representatives participated and provided
    testimony, and collected written comments on this issue.
    Representatives of industry trade associations also told us that
    FDA had placed notices about the proposed rule in the trade
    literature. Page 60                GAO/HEHS/GGD-99-90 FDA's
    Proposed Rule on Ephedrine Alkaloids Appendix VI Unfunded Mandates
    Reform Act Requirements The Unfunded Mandates Reform Act of 1995
    imposes specific analytical requirements for certain rules that
    include a federal mandate that may result in the expenditure of
    $100 million or more by the private sector in any 1 year. FDA did
    not explicitly address UMRA in its proposed rule. However, FDA's
    economic analysis indicated that the direct expenditures imposed
    by this rule on the private sector would not rise to the level of
    $100 million and, thus, trigger the required additional analysis.
    Furthermore, even if the proposed rule did trigger UMRA's
    analytical requirements, FDA appears to have already satisfied
    most, if not all, of those requirements. UMRA Requirements
    UMRA requires federal agencies to take certain actions during the
    Apply Under Certain            rulemaking process for certain
    types of rules. For example, section 202 of UMRA requires agencies
    to provide a "written statement," including a Circumstances
    qualitative and quantitative assessment of the anticipated costs
    and benefits for any proposed rule that includes a federal mandate
    that may result in the expenditure of $100 million or more by
    state, local, and tribal governments, in the aggregate, or by the
    private sector, in any 1 year.62 Section 205 of the act states
    that, before promulgating any rule for which a written statement
    is required under section 202, agencies must identify and consider
    a reasonable number of regulatory alternatives and select the one
    that is the least costly, most cost-effective, or least burdensome
    alternative that achieves the objectives of the proposed rule. If
    the agency does not select the least costly, most cost-effective,
    or least burdensome option, and if the requirements of section 205
    are not inconsistent with law, UMRA requires the agency head to
    publish with the final rule an explanation of why the least
    costly, most cost-effective, or least burdensome method was not
    adopted. FDA's Proposed Rule            FDA's proposed rule did
    not mention UMRA. FDA officials said they did not Does Not Appear
    to             include a specific UMRA section because the
    proposed rule was quite lengthy and they wanted to avoid adding
    any additional boilerplate. Our Trigger Further Action previous
    work on federal agency rulemaking indicated that even when Under
    UMRA                     their proposed rules do not trigger
    UMRA's analytical requirements, and although they are not required
    to do so, agencies sometimes include a 62Even if an agency's rule
    required $100 million in expenditures by the public or private
    sectors, UMRA's written statement requirements would only apply if
    the rule also imposed an "enforceable duty" that was not a
    condition of federal financial assistance or that did not arise
    from participation in a voluntary federal program. See Unfunded
    Mandates: Reform Act Has Had Little Effect on Agencies' Rulemaking
    Actions (GAO/GGD-98-30, Feb. 4, 1998). Page 61
    GAO/HEHS/GGD-99-90 FDA's Proposed Rule on Ephedrine Alkaloids
    Appendix VI Unfunded Mandates Reform Act Requirements brief
    statement to that effect explaining why they did not believe those
    requirements were applicable.63 Although FDA did not specifically
    mention UMRA, the agency's analysis of the costs associated with
    the proposed rule indicated that the rule would not result in
    expenditures of $100 million or more in any 1 year by the private
    sector. FDA estimated that the total compliance costs for the
    proposed action would be between $3 million and $80 million, with
    at most $70 million of those costs in the first year of
    implementation. These costs are primarily associated with
    relabeling and reformulating the affected supplements and appear
    to be expenditures covered by UMRA but do not rise to the $100-
    million threshold. Although FDA also estimated that the rule might
    result in lost sales for the dietary supplement industry of as
    much as $230 million per year, lost sales could not be considered
    direct expenditures by the private sector and, therefore, could
    not be used to trigger UMRA's analytical requirements. SBA's
    Office of Advocacy contended that FDA overlooked the UMRA
    requirements. In the written comments submitted on FDA's proposed
    rule, the office stated, "If the agency had performed an adequate
    [cost-benefit] analysis, it would have been apparent that the
    economic impact of the instant rule would impose in excess of $100
    million in costs to the industry." Officials from the Office of
    Advocacy told us that, if FDA properly estimated the number of
    affected businesses and the costs that would be imposed on those
    entities, FDA's estimate of expenditures by the private sector
    might increase to the level that would trigger UMRA's
    requirements. However, the Office of Advocacy could not provide
    any data showing that FDA had underestimated the number of
    manufacturers that would be affected by the relabeling and
    reformulation requirements in the rule or that FDA's cost
    estimates for those manufacturers were in error. Even if the
    expenditures associated with the proposed rule had triggered
    UMRA's analytical requirements, FDA appears to have satisfied most
    if not all of those requirements. For example, FDA identified the
    provision of federal law under which the rule was being
    promulgated, quantitatively and qualitatively assessed the
    anticipated costs and benefits of the rule, and identified and
    considered a number of regulatory alternatives. Although FDA said
    that most of the regulatory alternatives would reduce the impact
    of the rule on businesses, FDA also indicated that these other
    options would not be as effective as the proposed rule. 63GGD-98-
    30 and Federal Rulemaking: Agencies Often Published Final Actions
    Without Proposed Rules (GAO/GGD-98-126, Aug. 31, 1998). Page 62
    GAO/HEHS/GGD-99-90 FDA's Proposed Rule on Ephedrine Alkaloids
    Appendix VII Comments From the Department of Health and Human
    Services Note: GAO comments supplementing those in the report text
    appear at the end of this appendix. Page 63      GAO/HEHS/GGD-99-
    90 FDA's Proposed Rule on Ephedrine Alkaloids Appendix VII
    Comments From the Department of Health and Human Services See
    comment 1. Now on p. 23. Now on pp. 23-24. See comment 2. Now on
    pp. 3 and 8. Now on pp. 8-9. See comment 3. Now on pp. 3, 8, 10,
    and 12. Page 64               GAO/HEHS/GGD-99-90 FDA's Proposed
    Rule on Ephedrine Alkaloids Appendix VII Comments From the
    Department of Health and Human Services See comment 4. Now on pp.
    8, 11, 13, and 24. See comment 5. See comment 6. Page 65
    GAO/HEHS/GGD-99-90 FDA's Proposed Rule on Ephedrine Alkaloids
    Appendix VII Comments From the Department of Health and Human
    Services Page 66               GAO/HEHS/GGD-99-90 FDA's Proposed
    Rule on Ephedrine Alkaloids Appendix VII Comments From the
    Department of Health and Human Services Page 67
    GAO/HEHS/GGD-99-90 FDA's Proposed Rule on Ephedrine Alkaloids
    Appendix VII Comments From the Department of Health and Human
    Services Page 68               GAO/HEHS/GGD-99-90 FDA's Proposed
    Rule on Ephedrine Alkaloids Appendix VII Comments From the
    Department of Health and Human Services Page 69
    GAO/HEHS/GGD-99-90 FDA's Proposed Rule on Ephedrine Alkaloids
    Appendix VII Comments From the Department of Health and Human
    Services GAO Comments    1. We have added language to the Results-
    in-Brief section of the report indicating that the number of AERs
    relating to dietary supplements containing ephedrine alkaloids
    warranted FDA's attention and consideration of steps to address
    safety concerns. 2. Throughout our report, we note that FDA used
    information from scientific literature to conclude that ingestion
    of ephedrine alkaloids can result in serious adverse events for
    some individuals. Our review of the scientific literature found
    case reports that suggest that ephedrine alkaloids could increase
    blood pressure in persons with normal and high blood pressure;
    predispose certain individuals to tachycardia (rapid heart rate);
    and cause cardiomyopathy (disease of the heart muscle), stroke, or
    myocardial necrosis (death of cells in the heart). We also found
    descriptions of adverse events associated with ephedrine alkaloids
    that affected the central nervous system, such as mania, paranoid
    psychoses, and seizures. However, FDA relied on AERs in
    establishing key components of its proposed rule-the serving and
    duration of use limits. We have clarified some language in the
    report to better reflect our concerns. 3. FDA highlighted in the
    proposed rule that clinical trials examining the efficacy of
    ephedrine to treat obesity had shown adverse events occurring at
    levels approaching 20 mg per serving. However, these trials did
    not provide information relating to adverse events below the 20 mg
    level. Thus, FDA did rely solely on results of tests performed on
    a small number of products implicated in 13 AERs to conclude that
    an 8 mg per serving level would be appropriate. 4. We note in our
    report that FDA did not perform an analysis on the AERs to
    determine which, if any, of the reported events were caused by
    dietary supplements containing ephedrine alkaloids. Further, we
    outlined several weaknesses inherent in any passive surveillance
    system, such as adverse event reporting. We continue to hold the
    view that without a causal link between the AERs and the ingestion
    of products containing ephedrine alkaloids, the exclusive use of
    AERs to support a specific dosing regimen is questionable. 5. FDA
    noted in its comments that we suggested that FDA's use of AERs in
    this rulemaking in some respects, such as establishing an
    adulteration level, was unprecedented. For this report, we
    examined whether the use of AERs for the proposed rule on dietary
    supplements containing ephedrine alkaloids was consistent with the
    use of AERs in prior rulemaking by FDA's Center for Food Safety
    and Applied Nutrition. We found that AERs had Page 70
    GAO/HEHS/GGD-99-90 FDA's Proposed Rule on Ephedrine Alkaloids
    Appendix VII Comments From the Department of Health and Human
    Services been used in prior rulemaking, but they were used
    differently in this proposed rule when the agency relied solely on
    these reports to set a specific dosing level. However, we did not
    take a position on the appropriateness of the general use of AERs
    in rulemaking. 6. While we do not disagree with FDA's chronology
    of events detailed in pages 2-5 of its letter, we do not believe
    it is fully reflective of the role of the AERs in FDA's proposed
    action. In addition to using the AERs to signal a potential
    problem with products containing ephedrine alkaloids and to
    identify types of adverse events associated with such products,
    FDA also used the AERs as the key element in establishing the
    proposed dosing regimen. Page 71               GAO/HEHS/GGD-99-90
    FDA's Proposed Rule on Ephedrine Alkaloids Appendix VIII Comments
    From the Small Business Administration Note: GAO comments
    supplementing those in the report text appear at the end of this
    appendix. Page 72      GAO/HEHS/GGD-99-90 FDA's Proposed Rule on
    Ephedrine Alkaloids Appendix VIII Comments From the Small Business
    Administration See comment 1. Page 73              GAO/HEHS/GGD-
    99-90 FDA's Proposed Rule on Ephedrine Alkaloids Appendix VIII
    Comments From the Small Business Administration Page 74
    GAO/HEHS/GGD-99-90 FDA's Proposed Rule on Ephedrine Alkaloids
    Appendix VIII Comments From the Small Business Administration See
    comment 2. Page 75              GAO/HEHS/GGD-99-90 FDA's Proposed
    Rule on Ephedrine Alkaloids Appendix VIII Comments From the Small
    Business Administration Page 76              GAO/HEHS/GGD-99-90
    FDA's Proposed Rule on Ephedrine Alkaloids Appendix VIII Comments
    From the Small Business Administration GAO Comments      1.
    Although we did raise several concerns about specific parts of
    FDA's analysis and recommended certain revisions to any final
    rulemaking by FDA, we did not conclude that FDA failed to prepare
    an adequate analysis. RFA contains no standards or criteria that
    define an "adequate" analysis. The provisions of the act specify
    certain actions that an agency must take and certain elements that
    must be included in a regulatory flexibility analysis. Beyond
    those broad parameters, the act does not establish specific
    measures of quality or adequacy for an agency's analysis. Thus, we
    have no basis and are not in a position to make such a
    determination regarding the quality of FDA's analysis. Such a
    determination might be made if a specific rulemaking is judicially
    reviewed.64 We did recognize, however, that FDA's analysis in the
    proposed rule contained the primary elements required under E.O.
    12866, related OIRA guidance, and RFA. 2. We disagree with the
    Office of Advocacy's comments on direct versus indirect effects
    for three reasons. * First, prior case law indicates that agencies
    need not take the indirect effects of their rules into account
    when conducting a regulatory flexibility analysis. Most recently,
    in a May 14, 1999, decision, the U.S. Court of Appeals for the
    District of Columbia Circuit said, "we have consistently
    interpreted the RFA . . . to impose no obligation upon an agency
    `to conduct a small entity impact analysis of effects on entities
    which it does not regulate.'"65 The court also quoted Mid-Tex in
    noting, "Congress did not intend to require that every agency
    consider every indirect effect that any regulation might have on
    small businesses in any stratum of the national economy."66 The
    court also rejected the petitioners' contention that it must defer
    to the interpretations of SBA's Chief Counsel for Advocacy
    regarding RFA, pointing out that SBA "neither administers nor has
    any policymaking role under RFA; at most its role is advisory . .
    . . Therefore, we do not defer to the SBA's interpretation of the
    RFA."67 * Second, we do not believe that it is logical to conclude
    that the absence of case law comparable to Mid-Tex with respect to
    direct and indirect effects 64The Small Business Regulatory
    Enforcement Fairness Act of 1996 amended RFA to provide for
    judicial review of an agency's compliance with RFA. 65American
    Trucking Associations, Inc., et al, v. United States Environmental
    Protection Agency, No. 97-1440 U.S. App. LEXIS 9064, at *40 (D.C.
    Cir. May 14, 1999), citing Motor & Equip. Mfrs. Ass'n. v. Nichols,
    142 F.3d 449, 467 & n.18 (1988). 66American Trucking, 1999 U.S.
    App. LEXIS 9064, at *41. 67American Trucking, 1999 U.S. App. LEXIS
    9064, at *42. The court referred to 5 U.S.C. sections 601(3),
    602(b), 603(a), 605(b), 609(b)(1), and 612 and Scheduled Airlines
    Traffic Offices, Inc. v. Department of Defense, 87 F.3d 1356, 1361
    (D.C. Cir. 1996) (no deference owed to agency interpretation of
    statute it does not administer). Page 77
    GAO/HEHS/GGD-99-90 FDA's Proposed Rule on Ephedrine Alkaloids
    Appendix VIII Comments From the Small Business Administration as
    they apply to E.O. 12866 means that FDA should have prepared an
    analysis of the rule's impact on distributors. The absence of case
    law says nothing about FDA's analytical obligations under the
    executive order. * Third, we believe that, by estimating total
    lost sales to the industry if the proposed rule or other
    regulatory alternatives were implemented, FDA identified the
    potential magnitude of impacts on distributors in both its cost-
    benefit analysis and regulatory flexibility analysis. FDA
    identified such lost sales as a distributive effect, rather than a
    true social cost, because the loss only represents a transfer of
    resources within society (more for consumers and less for the
    dietary supplement industry). In addition, the total loss to the
    industry is the total loss to the industry; it should not be
    double counted-once for manufacturers and again for distributors-
    in a cost-benefit analysis. Finally, FDA explicitly acknowledged
    in its proposed rule that such effects would be significant to
    affected parties and could threaten the viability of many firms in
    this industry. Page 78              GAO/HEHS/GGD-99-90 FDA's
    Proposed Rule on Ephedrine Alkaloids Appendix IX GAO Contacts and
    Staff Acknowledgments GAO Contacts        William Scanlon, (202)
    512-7114 Nye Stevens, (202) 512-8676 Staff               In
    addition to those named above, Tim Bober, Curtis Copeland, Marcia
    Acknowledgments     Crosse, Carolyn L. Feis, and Kurt Kroemer made
    key contributions to this report. (108392, 410378)    Page 79
    GAO/HEHS/GGD-99-90 FDA's Proposed Rule on Ephedrine Alkaloids
    Ordering Information The first copy of each GAO report and
    testimony is free. Additional copies are $2 each. Orders should be
    sent to the following address, accompanied by a check or money
    order made out to the Superintendent of Documents, when necessary.
    VISA and MasterCard credit cards are accepted, also. Orders for
    100 or more copies to be mailed to a single address are discounted
    25 percent. Orders by mail: U.S. General Accounting Office P.O.
    Box 37050 Washington, DC  20013 or visit: Room 1100 700 4th St. NW
    (corner of 4th and G Sts. NW) U.S. General Accounting Office
    Washington, DC Orders may also be placed by calling (202) 512-6000
    or by using fax number (202) 512-6061, or TDD (202) 512-2537. Each
    day, GAO issues a list of newly available reports and testimony.
    To receive facsimile copies of the daily list or any list from the
    past 30 days, please call (202) 512-6000 using a touchtone phone.
    A recorded menu will provide information on how to obtain these
    lists. For information on how to access GAO reports on the
    INTERNET, send an e-mail message with "info" in the body to:
    [email protected] or visit GAO's World Wide Web Home Page at:
    http://www.gao.gov PRINTED ON     RECYCLED PAPER United States
    General Accounting Office           Bulk Rate Washington, D.C.
    20548-0001     Postage & Fees Paid GAO Permit No. G100 Official
    Business Penalty for Private Use $300 Address Correction Requested

*** End of document. ***