Dietary Supplements: Uncertainties in Analyses Underlying FDA's Proposed
Rule on Ephedrine Alkaloids (Letter Report, 07/02/1999,
GAO/HEHS/GGD-99-90).
In 1997, the Food and Drug Administration (FDA) published a proposed
rule that would establish a dosing regimen, require warning statements,
and affect other aspects of product labeling for dietary supplements
containing ephedrine alkaloids, which are promoted as helping
individuals lose weight and increase energy levels. GAO is concerned
that the proposed dosing level was based on information associated with
only 13 adverse event reports and that the proposed duration-of-use
limits were based on scientific studies showing problems with extended
use well beyond the proposal's seven-day limit. FDA did not establish a
causal link between the ingestion of ephedrine alkaloids and adverse
events for either of these two aspects of its rule. FDA did not document
the basis for its estimate of benefits from the rule sufficiently to
allow GAO to determine the estimate's accuracy. FDA has no internal
guidance on using adverse events reports for rulemaking related to
dietary supplements, and its use of reports for this rule was different
from its use in earlier rulemaking. FDA generally complied with
statutory and executive order requirements for rulemaking but did not
disclose why it made key assumptions in its cost-benefit analysis, the
degree of their uncertainty, or alternative assumptions that would have
dramatically affected its estimate of benefits.
--------------------------- Indexing Terms -----------------------------
REPORTNUM: HEHS/GGD-99-90
TITLE: Dietary Supplements: Uncertainties in Analyses Underlying
FDA's Proposed Rule on Ephedrine Alkaloids
DATE: 07/02/1999
SUBJECT: Nutrition research
Health statistics
Agency proceedings
Food and drug legislation
Proposed legislation
Cost effectiveness analysis
Health hazards
Safety standards
Product safety
Safety regulation
IDENTIFIER: CPSC National Electronic Injury Surveillance System
******************************************************************
** This file contains an ASCII representation of the text of a **
** GAO report. This text was extracted from a PDF file. **
** Delineations within the text indicating chapter titles, **
** headings, and bullets have not been preserved, and in some **
** cases heading text has been incorrectly merged into **
** body text in the adjacent column. Graphic images have **
** not been reproduced, but figure captions are included. **
** Tables are included, but column deliniations have not been **
** preserved. **
** **
** Please see the PDF (Portable Document Format) file, when **
** available, for a complete electronic file of the printed **
** document's contents. **
** **
** A printed copy of this report may be obtained from the GAO **
** Document Distribution Center. For further details, please **
** send an e-mail message to: **
** **
** **
** **
** with the message 'info' in the body. **
******************************************************************
United States General Accounting Office GAO
Report to the Chairman and Ranking Minority Member, Committee on
Science, House of Representatives July 1999 DIETARY
SUPPLEMENTS Uncertainties in Analyses Underlying FDA's Proposed
Rule on Ephedrine Alkaloids GAO/HEHS/GGD-99-90 GAO United
States General Accounting Office Washington, D.C. 20548 Health,
Education, and Human Services Division B-281581 July 2, 1999 The
Honorable James Sensenbrenner, Jr. Chairman The Honorable George
E. Brown, Jr. Ranking Minority Member Committee on Science House
of Representatives The dietary supplement industry estimates that
as many as 2 billion doses of dietary supplements containing
botanical ephedrine alkaloids are consumed each year in the United
States. The principal source of botanical ephedrine alkaloids is
the Chinese herb ma huang. Traditionally, this substance was used
by Chinese physicians to treat conditions such as hay fever and
asthma. In Western medicine, synthetic ephedrine alkaloids have
been used in products such as over-the-counter asthma and cold
medicines, while botanical ephedrine alkaloids have been used in
dietary supplements that are promoted to provide health benefits,
including helping individuals lose weight and increase energy
levels. In June 1997, the Food and Drug Administration (FDA)
published a proposed rule that would establish a dosing regimen,
require warning statements, and affect other aspects of product
labeling for dietary supplements containing ephedrine alkaloids.
FDA developed its proposed rule in response to what it termed
"serious illnesses and injuries, including multiple deaths,
associated with the use of dietary supplement products that
contain ephedrine alkaloids." FDA based its rule, in part, on a
number of adverse events reports (AER) it received that indicated
that some health problems could have been associated with use of
dietary supplements containing ephedrine alkaloids. FDA determined
that its proposed rule on dietary supplements containing ephedrine
alkaloids was an economically significant rule and, therefore, was
required to conduct a cost-benefit analysis pursuant to the
requirements in Executive Order 12866.1 Other regulatory analysis
requirements for federal rulemaking are those under the Regulatory
Flexibility Act (RFA), which directs agencies to consider the
potential impact of regulation on small businesses and other small
entities,2 and the Unfunded Mandates Reform Act (UMRA), which
generally requires agencies 1E.O. 12866 was issued on September
30, 1993, and covers all agencies except independent regulatory
agencies. 25 U.S.C. 601-612. Page 1
GAO/HEHS/GGD-99-90 FDA's Proposed Rule on Ephedrine Alkaloids B-
281581 to prepare cost-benefit analyses for any proposed
regulation that would impose mandates likely to result in
expenditures of $100 million or more in any 1 year.3 Industry
groups and the Small Business Administration's (SBA) Office of
Advocacy have challenged FDA's proposed rule, claiming that the
scientific information and AERs supporting the rule were poor and
unreliable. Further, industry groups and the Office of Advocacy
claimed that the cost-benefit analysis performed by FDA
understated the costs of the regulation and overestimated the
benefits. They also had concerns about compliance with RFA's and
UMRA's procedural requirements. In light of the concerns expressed
by SBA and industry groups, you asked us to examine (1) the
scientific basis for FDA's proposed rule and (2) the agency's
adherence to the regulatory analysis requirements for federal
rulemaking. In examining the basis for the rule, you asked that we
examine scientific evidence, FDA's past use of and internal
guidance on AERs in rulemaking, and information contained in the
AERs for determining dosing regimens and benefits that would arise
from the proposed rule. Regarding the rulemaking requirements, you
asked that we determine the extent to which FDA's analysis
contained elements expected in a federal agency's cost-benefit
analysis and to analyze FDA's compliance with the requirements in
RFA and UMRA. To meet these objectives, we interviewed
representatives of and obtained documents from FDA, SBA's Office
of Advocacy, the Office of Information and Regulatory Affairs
(OIRA) in the Office of Management and Budget (OMB), and the
dietary supplement industry. We also examined the public docket of
the proposed rule, performed a content analysis of a random sample
of AERs, and reviewed scientific literature and case reports of
adverse events from products containing ephedrine alkaloids. We
conducted our review from September 1998 to May 1999 in accordance
with generally accepted government auditing standards. (See app. I
for further information on our scope and methodology.) Results in
Brief FDA based its proposed rule on numerous reports of
adverse events associated with products thought to contain
ephedrine alkaloids; it also used evidence from scientific
literature indicating that ingestion of ephedrine alkaloids
adversely affects some individuals. The number and type of AERs
warranted FDA's consideration of steps to address safety 32 U.S.C.
638 and 1532. Page 2 GAO/HEHS/GGD-99-90 FDA's
Proposed Rule on Ephedrine Alkaloids B-281581 issues. However, we
have concerns about the strength of some of the information FDA
used to support two aspects of the proposed rule: the dosing level
and duration of use limits. While there was scientific evidence
showing adverse events at levels above 20 mg per serving, FDA's
dosing level proposal of 8 mg per serving was based on information
associated with only 13 AERs-the quality of which is questionable.
For the duration of use limits, FDA relied on scientific studies
that showed problems associated with extended use, well beyond the
7-day limit proposed. Moreover, FDA did not establish a causal
link between the ingestion of ephedrine alkaloids and the
occurrence of adverse events for either its proposed dosing level
or duration of use. FDA also based its estimate of the benefits of
the proposed rule on the annual number of adverse events reported
to FDA. However, because FDA did not document which AERs it
identified as containing "serious" events, we could not determine
the accuracy of FDA's estimated benefits. In addition, FDA has no
internal guidance on the use of AERs for rulemaking related to
foods and dietary supplements, and the AERs were used differently
in this proposed rule than in prior rulemaking. The agency
generally complied with the statutory and executive order
requirements for rulemaking, but the cost-benefit analysis that
accompanied the rule does not reflect the full range of
uncertainty associated with the proposed rule. FDA's cost-benefit
analysis and other analyses included the primary elements required
under E.O. 12866 and related "best practices" guidance and RFA.
UMRA's requirements did not appear to apply to the rule. Although
FDA disclosed the basic methodology, data, and assumptions used in
its cost-benefit analysis, the agency did not always disclose why
certain key assumptions were made or the degree of uncertainty
involved in those assumptions. It also did not disclose that
alternative assumptions would have had a dramatic effect on the
agency's estimate of the benefits of the proposed actions. While
FDA's conclusions regarding the desirability of the proposed
action may be valid, we believe these conclusions are open to
question because of limitations and uncertainties associated with
the agency's scientific and economic analyses. Given these
uncertainties, we recommend that FDA obtain additional information
to support the proposed dosing levels and duration of use limits
and improve the transparency of its cost-benefit analysis before
proceeding to final rulemaking. Page 3 GAO/HEHS/GGD-
99-90 FDA's Proposed Rule on Ephedrine Alkaloids B-281581
Background FDA is responsible for overseeing the federal
government's regulation of drugs, medical devices, food safety,
veterinary medicine, and biological products. Unlike many of these
products, dietary supplements do not have to undergo preapproval
by FDA to determine their safety or efficacy. The Dietary
Supplement Health and Education Act of 1994 created a new
framework for FDA's regulation of dietary supplements as part of
its oversight of food safety.4 The act allows dietary supplement
products to bear a statement describing how consumption of the
supplement can affect humans,5 but manufacturers of dietary
supplements cannot make a drug claim for the product-that is, a
statement claiming to diagnose, mitigate, treat, cure, or prevent
a specific disease or class of diseases.6 Dietary supplements
containing botanical sources of ephedrine alkaloids are currently
marketed for weight loss and increased energy. These products are
marketed in a variety of forms, including pills, powders, liquid
drops, and teas. Currently, FDA has no requirements that the
packages for these products include dosing regimens and warning
labels. Over-the-counter products containing synthetic ephedrine
alkaloids are considered safe and effective for people ages 12 and
older to temporarily relieve shortness of breath, tightness of
chest, and wheezing due to bronchial asthma, if recommended
dosages are followed. The recommended dosages-which range from
12.5 mg to 25 mg every 4 hours, not to exceed 150 mg per day-are
provided on the label. Labels for over-the-counter drug products
containing ephedrine alkaloids list side effects, such as
nervousness, sleeplessness, nausea, and loss of appetite. The
labels also warn against using the product if, for example, the
potential user has high blood pressure, heart disease, thyroid
disease, or diabetes, or has been hospitalized for asthma or is
taking a prescription drug for asthma, unless directed by a
doctor. Labels also note that if symptoms persist or become worse,
users should discontinue taking the drug and consult a doctor.
After reports of stroke in asthmatics who had taken ephedrine
along with prescription monoamine oxidase inhibitors,7 precautions
against using both drugs concurrently were added to the label.
4P.L. 103-417. 5Such statements may (1) claim a benefit related to
a classical nutritional deficiency disease, (2) describe the role
of a nutrient or dietary ingredient intended to affect the
structure or function in humans, (3) characterize the documented
mechanism by which the ingredient acts to maintain such structure
or function, or (4) describe general well-being from the
consumption of the ingredient. 621 U.S.C. 321 (g)(1). 7Monoamine
oxidase inhibitors are drugs used to treat depression, psychiatric
or emotional conditions, or Parkinson's disease. Page 4
GAO/HEHS/GGD-99-90 FDA's Proposed Rule on Ephedrine Alkaloids B-
281581 FDA has no authority to require submission of reports of
adverse events. FDA must generally rely on consumers or their
friends and family members, physicians or other health care
professionals, product manufacturers, and state health agencies to
voluntarily report adverse events. FDA uses these AERs as a
passive monitoring tool to identify potentially serious public
health issues that may be associated with the use of a particular
product or type of products. Like all passive surveillance
systems, AERs have certain limitations such as underreporting and
poor report quality (see app. II for a further discussion of the
adverse event monitoring system). FDA's Proposed Rule on FDA's
analysis of AERs and information from the scientific literature
relating Ephedrine Alkaloids to ephedrine alkaloids
indicated to the agency that there was cause for concern regarding
a potential public health problem associated with dietary
supplements containing ephedrine alkaloids. At the time of the
proposed rule, FDA had received over 800 AERs for products thought
to contain ephedrine alkaloids-more AERs than the agency had
received for any other dietary supplements. According to agency
officials, FDA found that, unlike most AERs related to foods, the
AERs relating to ephedrine alkaloids commonly involved visits to a
physician or emergency room. Some of these reports were very
serious, including effects such as strokes, and the events were
occurring in a population of young adults in which such serious
events are not expected. FDA officials also stated that the
adverse physiological and pharmacological effects from dietary
supplements believed to contain botanical sources of ephedrine
alkaloids were similar to those reported for drugs containing
synthetic sources of ephedrine alkaloids, but these effects were
being seen at lower doses and potencies for botanicals than in the
drug products. From 1994 through 1997, FDA took a series of steps
to respond to what it perceived was a public health concern
related to these products. In 1994, FDA's Center for Food Safety
and Applied Nutrition (CFSAN), which oversees dietary supplements,
issued a medical bulletin outlining potential adverse reactions
from dietary supplements containing ephedrine alkaloids. The
agency issued a press release warning consumers not to purchase a
particular brand of dietary supplement containing ephedrine
alkaloids that the agency determined could cause severe injury or
death in some people. Page 5 GAO/HEHS/GGD-99-90
FDA's Proposed Rule on Ephedrine Alkaloids B-281581 In October
1995, FDA convened a special work group composed of
pharmacologists, physicians, and industry representatives to
address concerns related to the use of dietary supplements
containing ephedrine alkaloids, but no consensus developed on how
to address the public health concerns outlined by FDA. In August
1996, FDA's Food Advisory Committee was asked to provide opinions
on and rationale for specific ways to address problems associated
with dietary supplements containing ephedrine alkaloids. Over half
the committee members concluded that no safe level for ephedrine
alkaloids could be identified and recommended that these products
be removed from the market. Most of the other members felt that a
fairly low level of ephedrine alkaloids would be "reasonably
safe." The Food Advisory Committee was unable to identify a
benefit for ephedrine alkaloids in terms of supplementing the
diet. In June 1997, FDA published a proposed rule regarding
dietary supplements containing ephedrine alkaloids.8 In two
subsequent notices the agency reopened and extended the comment
period on the proposed rule until December 1997.9 The proposed
rule * defines levels of the amount of ephedrine alkaloids in a
serving of dietary supplements at and above which the product
would be deemed adulterated (8 mg), * places restrictions on the
frequency of use and daily dosages (24 mg or more), * requires
that labels on these supplements contain a statement warning that
the product should not be used for more than 7 days, * prohibits
the use of ephedrine alkaloids with ingredients that have a known
stimulant effect (for example, caffeine), * prohibits certain
labeling claims that require long-term intake of the supplements
to achieve the purported purpose, * requires a warning statement
in conjunction with claims that encourage short-term excessive
intake, and * requires a specific warning statement to appear on
product labels. Federal Regulatory E.O. 12866 establishes
certain rulemaking responsibilities for covered Analysis
Requirements agencies. Among other things, the order states
that, in deciding whether and how to regulate, agencies should
assess all costs and benefits of available regulatory
alternatives, including both quantifiable and qualitative effects.
The order also states that, in choosing among 862 Fed. Reg. 30678.
962 Fed. Reg. 44247 and 62 Fed. Reg. 48968. Page 6
GAO/HEHS/GGD-99-90 FDA's Proposed Rule on Ephedrine Alkaloids B-
281581 alternatives, an agency should select those approaches that
maximize net benefits and "base its decisions on the best
reasonably obtainable scientific, technical, economic, and other
information concerning the need for, and consequences of, the
intended regulation." The order requires agencies to conduct cost-
benefit analyses for all regulatory actions that are likely to
result in a $100 million annual effect on the economy or are
otherwise economically significant. In January 1996, the
Administrator of OIRA issued "best practices" guidance on
preparing cost-benefit analyses under the order. The guidance
indicates that an analysis should contain certain basic elements
and should be "transparent"-disclosing how the analysis was
conducted, what assumptions were used, and the implications of
plausible alternative assumptions. During the past 20 years, the
Congress has attempted to improve the federal rulemaking process
by enacting a number of statutes that impose certain analytical
requirements on agencies issuing proposed regulations. For
example, RFA of 1980 was enacted in response to concerns about the
effect that federal regulations could have on small entities. RFA
directs all agencies to give particular attention to the potential
impact of regulation on small businesses and other small entities
and requires consideration of regulatory alternatives that are
less burdensome to small entities. Under RFA, an agency must
prepare an initial regulatory flexibility analysis at the time
proposed rules are issued unless the head of the agency determines
that the proposed rule would not have a "significant economic
impact upon a substantial number of small entities." The act also
requires agencies to ensure that small entities have an
opportunity to participate in the rulemaking process. Other
statutory rulemaking requirements are set forth in UMRA. UMRA
generally requires agencies (other than independent regulatory
agencies) to prepare cost-benefit and other analyses for any
regulations imposing mandates that are likely to result in
expenditures of $100 million or more in any 1 year either by
state, local, and tribal governments, in the aggregate, or by the
private sector. Although UMRA's scope and requirements differ from
E.O. 12866, the provisions on economic analysis are very similar.
Accordingly, the guidance for implementing the executive order
states that the economic analysis that the agency prepares should
also satisfy the requirements of UMRA. Page 7
GAO/HEHS/GGD-99-90 FDA's Proposed Rule on Ephedrine Alkaloids B-
281581 FDA Analyses Relied To develop its proposed rule,
FDA used a combination of scientific on Poorly
evidence on the effects of ephedrine alkaloids and a set of
reports it received on adverse events associated with dietary
supplements Documented Reports containing ephedrine
alkaloids. While the signs and symptoms described of Adverse
Events in the AERs were consistent with available
scientific evidence and known physiologic and pharmacologic
effects of ephedrine alkaloids, the AERs were poorly documented.
FDA also used the AERs differently than in its past rulemaking.
Specifically, the agency used AERs as the sole source of support
for specific dosing levels, relied on weak information to set
limits on duration of use, and did not perform a causal analysis
to determine whether ingestion of a dietary supplement containing
ephedrine alkaloids caused or contributed to the adverse events.
FDA also used these AERs to determine the number of serious events
that could be attributed to the dietary supplements and the
expected benefits that would arise if the proposed rule were
implemented. However, FDA did not document which AERs it
determined to be serious. Moreover, it did not establish criteria
to determine which events were serious and did not perform any
reliability assessments of its analyses. Scientific Information
Our review of the scientific literature found case reports that
suggested Indicates Ephedrine that ephedrine alkaloids
could increase blood pressure in persons with Alkaloids Can Affect
the normal and high blood pressure;10 predispose certain
individuals to Cardiovascular and tachycardia (rapid
heart rate);11 and cause cardiomyopathy (disease of the 12
13 Nervous Systems heart muscle), stroke, or
myocardial necrosis (death of cells in the heart).14 We also found
descriptions of adverse events associated with 10S. Chua and S.
Benrimoj, "Non-Prescription Sympathomimetic Agents and
Hypertension," Medical Toxicology, Vol. 3 (1988), pp. 387-417.
11D. McCleave and others, "Compartmental Shift of Potassium-A
Result of Sympathomimetic Overdose," Australian and New Zealand
Journal of Medicine, Vol. 8 (1978), pp. 180-83. 12J. Gualtieri and
C. Harris, "Dilated Cardiomyopathy in a Heavy Ephedrine Abuser,"
(abstract) Journal of Toxicology, Clinical Toxicology, Vol. 34
(1996), pp. 581-82. 13A. Bruno and others, "Stroke Associated With
Ephedrine Use," Neurology, Vol. 43 (1993), pp. 1313-16. 14P.
Pentel and others, "Myocardial Necrosis Due to Intraperitoneal
Administration of Phenylpropanolamine in Rats," Fundamental and
Applied Toxicology, Vol. 9 (1987), pp. 167-72. Page 8
GAO/HEHS/GGD-99-90 FDA's Proposed Rule on Ephedrine Alkaloids B-
281581 ephedrine alkaloids that affected the central nervous
system, such as mania,15 paranoid psychoses,16 and seizures.17 Our
review found only one clinical study examining the effects of a
botanical source of ephedrine alkaloids, ma huang, on heart rate
and blood pressure. The study of adults with normal blood pressure
found a statistically-but not clinically-significant increase in
heart rate but variable effects on blood pressure.18 However, "in
combination with other stimulants and at higher doses, effects of
magnification of the heart rate and blood pressure response could
be expected." One report also highlighted a case of mania as a
result of ingestion of a dietary supplement containing ma huang.19
We also reviewed a number of studies using clinical trials that
evaluated the efficacy of ephedrine (a synthetic ephedrine
alkaloid)-alone or in combination with caffeine-for weight loss.
Although these trials were not designed to determine whether
ephedrine is safe to use during weight loss programs, FDA
concluded that this body of literature showed "clinically
significant adverse effects in populations with no known risk
factors with the use of ephedrine and that synergistic adverse
effects can result when ephedrine and caffeine are combined." FDA
further concluded that "The patterns and types of the adverse
effects reported in these trials are consistent with the known
effects of sympathomimetic agents20-that is, they mainly involved
nervous and cardiovascular system effects." AERs Were Used
To develop FDA's proposed rule on dietary supplements containing
Differently for Developing ephedrine alkaloids, CFSAN used AERs
differently than in prior FDA the Proposed Rule Than
rulemaking. FDA officials acknowledge that the agency does not
have any for Prior Rulemaking formal internal guidelines
on the use of AERs for rulemaking and provided five examples of
past rulemaking in which CFSAN used several sources of information
in addition to AERs to support the agency's proposals. For the
15W. Clovis, "Mania and Cough Syrup," Journal of Clinical
Psychiatry, Vol. 54 (1993), p. 200. 16M. T. Lambert, "Paranoid
Psychoses After Abuse of Proprietary Cold Remedies," British
Journal of Psychiatry, Vol. 151 (1987), pp. 548-50. 17S. M.
Mueller and E. B. Solow, "Seizures Associated With a New
Combination Pick-Me-Up Pill," Annals of Neurology, Vol. 11 (1982),
p. 322. 18L. White and others, "Pharmacokinetics and
Cardiovascular Effects of Ma-huang (Ephedra sinica) in
Normotensive Adults," Journal of Clinical Pharmacology, Vol. 37
(1997), pp. 116-22. 19R. R. Capwell, "Ephedrine-Induced Mania From
an Herbal Diet Supplement" (letter), American Journal of
Psychiatry, Vol. 152 (1995), p. 647. 20Such agents produce
physiological responses resembling those that are caused by the
action of the sympathetic nervous system, as in tending to reduce
digestive secretions and speeding up the heart. Page 9
GAO/HEHS/GGD-99-90 FDA's Proposed Rule on Ephedrine Alkaloids B-
281581 proposed rule on dietary supplements containing ephedrine
alkaloids, CFSAN used AERs alone to determine specific dosing
levels.21 In a prior rulemaking, FDA used prevalence estimates
among people with asthma to determine sensitive populations who
might be affected by sulfiting agents22 and convened a scientific
panel to examine exposure estimates and evidence from clinical
experiments.23 For its rulemaking on protein supplements, FDA used
data collected through a telephone survey, conducted jointly with
the Centers for Disease Control and Prevention (CDC), and
information on death rates from the National Center for Health
Statistics. FDA also initiated its own experimental protocols to
examine the basic metabolic mechanisms of protein diets.24 FDA's
proposed rule on supplements and drugs containing iron relied on
data from the American Association of Poison Control Centers and
the U.S. Consumer Product Safety Commission to determine
fatalities from accidental iron poisonings. Data from the National
Electronic Injury Surveillance System were also incorporated to
estimate the number of cases of hospital emergency room treatment
for iron ingestion.25 FDA did not establish dosing levels in any
of these rulemakings. Although AERs were used as early warning
signals in many of these cases, the proposed rule on ephedrine
alkaloids relied more heavily on AERs than did prior rulemaking
efforts. (See app. III for additional information on the use of
AERs in prior rulemaking.) Shortcomings of AERs and The
inherent weaknesses of AERs-along with FDA's heavy reliance on
FDA's Reliance on Them them-lead to uncertainty regarding the
dosing regimen outlined in the Add Uncertainty to FDA's
proposed rule. The AERs used in the rule lacked or had
inconsistent Proposed Rule information relevant to
FDA's analysis, such as the amount of product used, how often it
was used, or for how long it was used. The limitations of a
passive surveillance system such as AERs call into question FDA's
ability to 21FDA also used AERs to show evidence of dechallenge
(signs and symptoms resolve or improve when a consumer stops using
a product) and rechallenge (symptoms recur when the consumer
resumes using the product). 22Sulfiting agents are one type of
chemical preservative that serves to prevent or to delay the
process of browning and deterioration of raw fruits and
vegetables. 2351 Fed. Reg. 25021. 2449 Fed. Reg. 13679. 2562 Fed.
Reg. 2218. Page 10 GAO/HEHS/GGD-99-90 FDA's
Proposed Rule on Ephedrine Alkaloids B-281581 determine a specific
dosing level based solely on these reports.26 FDA did not perform
a causal analysis to determine if, in fact, the 13 AERs it used to
set dosing levels were caused by supplements containing ephedrine
alkaloids. FDA indicated in the proposed rule that 10 to 73
percent of reported adverse events might not be related to
consumption of dietary supplements containing ephedrine alkaloids.
FDA's support for its recommended limit on duration of use was
also weak. FDA's lack of documentation on which adverse events it
classified as serious makes it difficult to determine the validity
of FDA's estimates of the benefits that would arise from the
proposed rule. In addition, FDA did not have specific criteria to
determine which events should be considered serious and did not
perform a reliability assessment to ascertain the validity of its
estimates of serious events. Adverse Event Reports Were Prior
to the proposed rulemaking, FDA received 864 AERs on dietary
Incomplete and Inconsistent supplements containing ephedrine
alkaloids-more AERs than for any other single dietary supplement.
However, the AERs that we examined often lacked important
information, and the information that they did contain was
sometimes inconsistent. These problems suggest that AERs should be
used with caution, and their use can contribute to uncertainty in
FDA's conclusions. In examining a random sample of the AERs (92
out of 864 reports), we found that 39 percent lacked information
on the amount of product consumed, 41 percent lacked information
on the frequency with which the product was consumed, and 28
percent lacked information on the duration for which the product
was consumed. A total of 45 percent of the AERs lacked information
on either dose, frequency, or duration, and 24 percent lacked
information on all three dimensions. Finally, 62 percent of the
AERs in our sample did not contain medical records, which are
important in determining potential underlying conditions that
might have caused the adverse event (rather than assuming
ingestion of dietary supplements containing ephedrine alkaloids
caused the event). We also found cases where the amount of product
consumed or the duration for which it was consumed was listed
differently in multiple locations within an AER. In addition, the
name of the product consumed sometimes varied within an AER. As a
result, it is difficult to identify the 26FDA evaluated the
scientific literature to determine that ephedrine alkaloids could
pose hazards at 20 mg per dose. However, FDA relied solely on AERs
in setting a specific dose level of 8 mg per dose and 24 mg per
day. Page 11 GAO/HEHS/GGD-99-90 FDA's Proposed
Rule on Ephedrine Alkaloids B-281581 correct information that
corresponds to the event being reported and to make inferences
from such reports.27 FDA Relied Solely on AERs for In its
proposed rule, FDA concluded that one possible strategy for
Setting Specific Dosing addressing these adverse events
was to restrict the level of ephedrine Regimens and Did Not
alkaloids in dietary supplements. To determine a possible dosing
level, FDA Determine If These Events reviewed clinical
trials on therapeutic uses for ephedrine alkaloids used Were
Caused by Ephedrine alone and in combination with other
pharmaceutical substances to treat Alkaloids
obesity. These trials indicated that 20 mg ephedrine alkaloids per
dose could cause adverse events to occur in the subjects taking
part in the clinical trial.28 Thus, FDA concluded that 20 mg per
serving of ephedrine alkaloids presented potential risks for some
individuals. However, the studies did not provide information on
risk at levels below 20 mg per serving for use in the general
population, and lean or moderately overweight persons have been
shown to be more sensitive to substances like ephedrine alkaloids
than are obese individuals. To determine if serious adverse events
were occurring at levels below 20 mg per serving, FDA obtained and
tested the dietary supplement products associated with AERs that
were serious, where available.29 FDA then identified the levels of
ephedrine alkaloids in those AERs by performing laboratory
analyses of ephedrine alkaloid levels for dietary supplements
turned over to the agency by the consumers who suffered adverse
events (consumer samples). FDA also performed tests on samples
obtained from the marketplace for situations in which the consumer
experiencing the adverse event no longer possessed the product
(consumer-related samples). In all, from more than 800 AERs
submitted to the agency between 1993 and the time of the proposed
rule, FDA collected and tested 34 samples of consumer or consumer-
related products. From the 34 samples tested, 13 product samples
met two criteria: the results of the analytical tests were valid
and supportable and the products contained less than 20 mg of
ephedrine alkaloids. FDA then set a specific dosing level of 8 mg
per serving. In other words, FDA relied on these 13 AERs where the
products tested yielded ephedrine alkaloid levels below 20 mg per
serving as the sole source of support for the specific dosing
level 27While these types of problems do not occur uniquely among
reports involving ephedrine alkaloids, they reinforce the fact
that there are inherent weaknesses of passive surveillance systems
such as AERs. 28See, for example, A. Astrup and others, "The
Effect and Safety of an Ephedrine/Caffeine Compound Compared to
Ephedrine, Caffeine and Placebo in Obese Subjects on an Energy
Restricted Diet. A Double Blind Trial," International Journal of
Obesity, Vol. 16 (1992), pp. 269-77. 29Testing was necessary
because there are no requirements that product contents be
labeled. Page 12 GAO/HEHS/GGD-99-90 FDA's
Proposed Rule on Ephedrine Alkaloids B-281581 found in the
proposed rule.30 Table 1 provides information on these 13 cases.
Table 1: Summary of 13 AERs Used for Specific Dosing Level Total
ephedrine alkaloids (mg)a AER number Initial Recheck
Type of sample Symptoms as described in AER 10088
1.6 2.0 Consumer Elevated blood
pressure, severe headache, nausea, heavy perspiration, passed out
11233 1.8 Trace Consumer
Hair loss, sleeplessness, chest pain, increased energy 11106
Trace 1.9 Consumer Sweating,
trembling, high blood pressure, menstrual bleeding, suffered
stroke 9754 8.8 8.8 Consumer
Heat stroke, chest and back pain, rapid heart rate, hyperthermia
8475 9.1 9.9 Consumer-related Tremors,
tics, insomnia, hyperactive reflexes 10974 11.0
13.1 Consumer-related Seizure 11144b 10.0, 11.7 9.0,
15.3 Consumer-related Transient amnesia, lost consciousness
10946 10.5 8.3 Consumer
Rash, increased blood pressure 9751 9.2 11.1
Consumer Attempted suicide 10919 11.2
14.4 Consumer Weakness, dizziness, elevated
blood pressure 11134c 14.8 Consumer
Death 11619 15.1 18.5 Consumer
Rapid heart rate, headache, numbness, face droop, dizziness 11298
16.6 20.6 Consumer Blood in urine,
elevated blood pressure aLevels are based on the amount the
consumer reported having used per serving. Where consumer intake
was unknown, the levels were based on the label directions for
use. FDA performed an initial analysis and a recheck analysis for
the samples it obtained. bTwo consumer-related samples were
tested. cRecheck analysis was not conducted because original
analysis was performed by a national or international expert.
While FDA used these 13 AERs to set a dosing level, the agency did
not perform a causal analysis to determine whether the reported
events were, in fact, caused by the ingestion of dietary
supplements containing ephedrine alkaloids. Our review of these 13
AERs found numerous problems that raise questions about the causal
relationship between ingestion of the implicated product and the
adverse events observed. For example, 30FDA's laboratory analysis
of the ephedrine alkaloid levels in the 34 AERs showed levels from
approximately 1 to 50 mg per serving. According to FDA, these
reports show a pattern of clinically significant adverse events at
levels approaching and above 10 mg. Given the variability in the
testing procedure and natural variability in the alkaloid content
of botanical ingredients, FDA determined a range around 10 mg per
serving could be expected to deviate by 10 to 20 percent. From
this, FDA tentatively concluded that an 8 mg limit per serving
could be associated with a serious adverse event. Page 13
GAO/HEHS/GGD-99-90 FDA's Proposed Rule on Ephedrine Alkaloids B-
281581 * three AERs included physician reports that stated the
cause of the event was not related to a dietary supplement; * one
consumer-related sample was obtained and tested 2 years after the
initial event, and possible reformulations of the product might
have resulted in different levels of ephedrine than the product
implicated in the reported adverse event; * three individuals
reporting adverse events had experienced similar problems prior to
or well after using the dietary supplement; * one individual who
experienced the event was eating only one meal a day; and * one
AER contained no medical records. Some of the 13 AERs had more
than one of these problems. As a result, there are uncertainties
in FDA's conclusions in setting a specific dosing limit since this
limit was based on a small number of adverse events-events which
may or may not have been a result of ingestion of dietary
supplements containing ephedrine alkaloids. Weak Support for FDA's
In its proposed rule, FDA also recommended a 7-day limit on the
use of Proposed Limitation on dietary supplements containing
ephedrine alkaloids. FDA used the Duration of Use
recommendations of some members from its special work group and
Food Advisory Committee, scientific literature, and data from the
AERs to support this limit. However, we found several weaknesses
in FDA's evidence. First, FDA did not present scientific evidence
specifically pointing to an increase in adverse events beginning
at 7 days and under normal use conditions. Rather, the scientific
information FDA used to support a 7-day limit outlined problems
associated with extended use (months and years) of ephedrine
alkaloids. The agency also cited support for its 7-day limit from
studies involving other sympathomimetic agents, such as cocaine
and methamphetamines, but these studies also involved long-term
use of the drug. Second, FDA did not demonstrate a causal
relationship between ingestion of dietary supplements containing
ephedrine alkaloids and adverse events reported to the agency.
Since FDA indicated in the proposed rule that 10 to 73 percent of
the reported adverse events might not be related to consumption of
dietary supplements containing ephedrine alkaloids, the use of the
AERs to describe a pattern of response across time Page 14
GAO/HEHS/GGD-99-90 FDA's Proposed Rule on Ephedrine Alkaloids B-
281581 is questionable.31 Finally, the agency did not demonstrate
the relationship between both dose levels and frequency of
ingestion and duration of use. As a result, there are
uncertainties in the agency's analysis of the relationship between
duration of use of dietary supplements containing ephedrine
alkaloids and the occurrence of adverse events. FDA Based Its Risk
Reduction FDA noted in the proposed rule that there were an
average of 174 adverse Value on Poorly Documented events
reported each year between January 1993 and June 1996 and that 81
Estimates of Serious Adverse were serious events. FDA divided
the serious adverse events into five Events
categories and outlined examples for each type: (1) cardiovascular
(dysrhythmia or abnormal heart rhythms, severe hypertension,
cardiac arrest, heart attack, angina, myocardial infarction,
stroke, and cardiomyopathy or muscle damage to heart); (2) central
nervous system (psychosis, suicide or attempted suicide, altered
or loss of consciousness or disorientation or confusion, seizure,
mania, and severe depression); (3) liver damage or hepatitis; (4)
death; and (5) other events (vestibular or inner ear disturbances,
altered serum enzymes, myopathies or muscle disease, genitourinary
system disturbances or urinary tract problems, prostate problems,
and dermatological syndromes or rashes). However, FDA did not
record which AERs it considered to be serious. FDA officials
stated that a count had been performed by medical doctors of the
agency on the number of serious events, but there was no record of
the specific cases that they had identified as serious events. In
addition, rather than providing AER reviewers a standard
definition, FDA told them to use their clinical judgment in
determining what they considered a serious event.32 Also, FDA did
not perform any assessments of interrater reliability among
reviewers to ensure that their judgments were consistent. Based on
the yearly average of serious adverse events reported to FDA from
January 1993 to June 1996, FDA estimated an annual value of risk
reduction that would occur as a result of the proposed actions
outlined in the rule. However, because FDA did not record which
AERs were deemed 31FDA pointed out in the proposed rule that the
relationship of the reported adverse event to the consumption of
dietary supplements categorized as containing ephedrine alkaloids
had been corroborated in about 27 percent of cases where a
consumer stopped taking a product and the symptoms improved. FDA
also noted that a certain number of false reports might also be
expected. Thus, they believed that the actual percentage of cases
related to consumption of ephedrine alkaloids was between 27
percent and 90 percent. FDA's professional judgment was that 80
percent of the reported cases were actually related to consumption
of dietary supplements containing ephedrine alkaloids. 32FDA noted
that the medical doctors were well qualified-by training and
experience-to make decisions regarding the clinical significance
of the event. Page 15 GAO/HEHS/GGD-99-90 FDA's
Proposed Rule on Ephedrine Alkaloids B-281581 serious, we could
not determine the accuracy of FDA's conclusions regarding the
actual benefits that would arise from the proposed rule. FDA's
Analysis of FDA generally complied with executive
order and statutory requirements Impacts Generally
for rulemaking. It prepared a cost-benefit analysis containing the
basic elements expected for an economically significant rule under
E.O. 12866. Adhered to It also prepared an
initial regulatory flexibility analysis and provided Rulemaking
opportunities for small businesses to participate in the
rulemaking process, as required by RFA. FDA's analysis of impacts
would also likely Requirements but Was satisfy most, if not all,
of the analytical requirements of UMRA, although its Not Fully
Transparent proposed rule does not appear to trigger
these requirements. However, OIRA's guidance on preparing cost-
benefit analyses under E.O. 12866 states that an agency's analysis
must be "transparent"-that is, disclose how the analyses were
conducted and what assumptions were used. FDA's analysis was only
partially transparent. Although FDA described its key assumptions
and identified substantial uncertainties regarding the data
supporting its proposed rule, it did not fully disclose why
certain key assumptions were made, the degree of uncertainty
involved in those assumptions, or that alternative assumptions
would have had a dramatic effect on the agency's estimates of the
costs and benefits of the proposed regulatory actions. Because of
these uncertainties and limitations, the results of FDA's analysis
are still open to question. FDA's Analysis Addressed
Under E.O. 12866, agencies are required to conduct cost-benefit
analyses the Basic Cost-Benefit for all regulatory
actions that are likely to result in a $100 million annual
Elements but Was Only effect on the economy or are
otherwise economically significant. OIRA's Partially Transparent
"best practices" guidance states that a federal agency's cost-
benefit analysis should contain three basic elements: (1) a
statement of the need for the proposed action, (2) an examination
of alternative approaches, and (3) an analysis of benefits and
costs. Although the guidance provides for flexibility and
professional judgment in conducting the analysis, it establishes a
clear expectation that the analysis of the risks, benefits, and
costs associated with the regulation "must be guided by the
principles of full disclosure and transparency." For example, the
guidance says the analysis should identify and explain the data or
studies on which the estimates of benefits and costs are based
"with enough detail to permit independent assessment and
verification of the results." The guidance also states "where
benefit or cost estimates are heavily dependent on certain
assumptions, it is essential to make those assumptions explicit
Page 16 GAO/HEHS/GGD-99-90 FDA's Proposed Rule on
Ephedrine Alkaloids B-281581 and, where alternative assumptions
are plausible, to carry out sensitivity analyses based on the
alternative assumptions."33 (See app. IV for a more detailed
discussion of the analytical requirements under E.O. 12866 and
related guidance, as well as our assessment of FDA's cost-benefit
analysis in terms of that guidance.) FDA's cost-benefit analysis
contained the three basic elements stipulated in the guidance, but
certain other specific elements in the guidance appear to be
lacking: * With regard to the first element, FDA stated that the
proposed rule was needed because of a significant market failure-
specifically, inadequate consumer information on the health risks
associated with dietary supplements containing ephedrine
alkaloids. However, the OIRA guidance indicates that even where a
market failure exists, agencies should also discuss the
appropriateness of alternatives to federal regulation, such as
state or local regulation. FDA's analysis did not recognize
efforts in several states to regulate these products or say why
federal regulation was a better approach. * For the second
element, FDA discussed seven regulatory options, including (1) a
baseline alternative of taking no action, (2) taking no action but
generating additional information, (3) taking the proposed action,
(4) taking the proposed action but allowing a higher potency limit
for the supplements, (5) banning dietary supplements that contain
ephedrine alkaloids, (6) taking the proposed action but not
requiring the warning statement, and (7) requiring only the
warning statement. FDA also generally discussed the agency's
reasoning for selecting the proposed regulation over other
alternatives. However, FDA's options primarily focused on
alternative levels of stringency and informational measures-not on
other basic types of regulatory alternatives suggested in the
guidance. Also, while FDA included some contextual information in
the proposed rule, it did not provide a complete picture of the
baseline risks that may be associated with supplements containing
ephedrine alkaloids. * For the actual analysis of potential
benefits and costs-the third element-FDA generally adhered to the
principles and specific recommendations in OIRA's guidance. For
example, FDA discussed the distributional effects of the proposed
rule and alternatives in terms of lost sales to the dietary
supplement industry. FDA also identified nonmonetized costs
associated with the proposed rule. However, FDA did not provide
monetized or quantified benefit or cost estimates for all of its
regulatory 33To conduct a sensitivity analysis, the analyst
calculates the costs and benefits of a proposed action using
different assumptions. Page 17 GAO/HEHS/GGD-99-
90 FDA's Proposed Rule on Ephedrine Alkaloids B-281581
alternatives, and the agency's discussion of those alternatives'
benefits and costs were compared to the proposed rule-not the
baseline condition of no regulation as recommended by the
guidance. The OIRA guidance also stresses the importance of full
disclosure and transparency in agencies' cost-benefit analyses. We
assessed the transparency of FDA's cost-benefit analysis of the
proposed rule using three criteria suggested by the guidance: *
first, whether it identified the data, models, inferences, and
assumptions used to calculate the estimates of benefits and costs;
* second, whether it disclosed the reasons why those data, models,
inferences, and assumptions were selected; and * third, whether it
assessed the effects of plausible alternative assumptions and
choices on the results of the analysis-what is often referred to
as a "sensitivity analysis." Overall, we concluded that FDA's
analysis was only partially transparent. Using the first
criterion, FDA's analysis was very transparent. The agency
provided a clear and lengthy description of the data, assumptions,
and methodology that it used to calculate the benefits and costs
of the proposed rule. Against the second criterion, FDA's analysis
was only partially transparent. For most elements of the analysis,
FDA identified the underlying data sources used and the rationale
for the assumptions and conclusions reached by FDA's analysts and
experts. However, FDA did not fully disclose the underpinnings of
all of its assumptions and choices. For example, FDA said that
between 27 percent and 90 percent of adverse event reports were
"probably" related to the consumption of dietary supplements
suspected of containing ephedrine alkaloids and that it assumed
the value was 80 percent, but it did not indicate why it made this
point-estimate assumption. In response to our questions, FDA
officials acknowledged that their analysis of impacts was not as
transparent as it should have been in explaining how the agency
arrived at some of the assumptions regarding its treatment of
uncertainty in the underlying data. Against the third criterion,
FDA's cost-benefit analysis was also only somewhat transparent.
For example, FDA estimated that the benefits of the proposed rule
would be between $240 million and $670 million per year. That
estimate was driven by three factors: (1) FDA's estimate of the
actual number of adverse events each year (1,110), (2) FDA's
estimate of the degree to which the proposed rule would reduce
these events (35 percent to 100 percent across all types of
proposed actions), and (3) the values FDA Page 18
GAO/HEHS/GGD-99-90 FDA's Proposed Rule on Ephedrine Alkaloids B-
281581 assigned to the estimated risk reduction per case (for
example, $5 million per death avoided).34 Changes in any of these
values could have dramatically changed FDA's estimates of the
proposed rule's benefits. For example, FDA's estimate of 1,110
adverse events each year was based on an average of 174 adverse
events reported per year and three assumptions: (1) that 80
percent of the adverse event reports involved consumption of
dietary supplements suspected of containing ephedrine alkaloids,
(2) that 80 percent of the supplements actually contained
ephedrine alkaloids, and (3) that only 10 percent of all adverse
events related to supplements containing ephedrine alkaloids were
reported. Using these point estimates (instead of ranges) does not
reflect the uncertainty FDA indicated was possible regarding these
values and, therefore, the uncertainty associated with the
agency's final benefit estimates. Had FDA used its own initial
estimates of the possible ranges (instead of the 80-percent point
estimates) for the first two assumptions (27 to 90 percent for the
first assumption and 25 to 90 percent for the second assumption),
its estimate of the number of adverse events each year would have
been 117 to 1,409 instead of a single estimate of 1,110. (See
table 2.) If FDA's projected effects for the proposed actions were
applied to this range, the expected reduction in annual adverse
events would be between 41 and 1,409 cases per year (not the 390
to 1,110 in the published analysis). This ultimately results in a
range of potential benefits of roughly $25 million to $850 million
per year-a much wider range of possible benefits than the $240
million to $670 million per year estimates that FDA included in
the proposed rule. 34In its proposed rule, FDA did not report or
round its estimated values consistently, so slightly different
numbers for the same element appear within the proposed rule. For
this report, we are using the values reported in table 6 of FDA's
proposed rule, such as 1,110 for the estimated annual number of
adverse events. Page 19 GAO/HEHS/GGD-99-90
FDA's Proposed Rule on Ephedrine Alkaloids B-281581 Table 2: FDA's
Estimate of Expected Benefits Is Sensitive to Changes in Key
Assumptions Assumptions made regarding uncertainty in the actual
number of adverse event cases A B C
D E (AxBxCxD) F G
H (FxG) Average Proportion of Proportion
Multiplier Estimated Estimated Value of
Total annual cases in which the
to reflect total reduction estimated
dollar value number of actually supplements
underreporting number in the
risk (in millions) reported related
actually of of adverse number of
reduction of estimated adverse to dietary
contained adverse event
annual per risk events
supplements ephedrine alkaloids eventsa
cases casesb case reduction Scenario
1: FDA's reported results, using a point estimate for total number
of adverse events.c 174 0.8
0.8 10 1,110 390-1,110
d $240-670 Scenario 2: Results using the full range of
FDA's assumptions regarding uncertainty on number of adverse
events. 174 0.27-0.9 0.25-0.9
10 117-1,409 41-1,409 d
25-850 Scenario 3: Results using assumption that 20 percent of
cases are reported. 174 0.8
0.8 5 557 195-557
d 118-336 Scenario 4: Results using assumption that 5
percent of cases are reported. 174 0.8
0.8 20 2,227 779-2,227
d 470-1,343 aFor example, if the assumption is that 10
percent of events are reported, the number of reported cases is
multiplied by 10. bThe combined effects of all proposed actions
are assumed to reduce adverse events by between 35 percent and 100
percent. cFDA's table in the proposed rule on the estimation of
benefits displayed rounded numbers. The actual calculated values
may vary from the published figures used here. For example, if the
figure for estimated total annual adverse events were not rounded,
it would equal about 1,114 rather than 1,110. dThe value of risk
reduction varies by the type of event avoided, such as $5 million
per death avoided and $837,000 for each serious cardiovascular
system event avoided. FDA converted the individual values and
estimated totals into 1996 dollars. Calculations for scenarios 2,
3, and 4 assume the same proportions of types of adverse events as
FDA used for its estimate. Source: Scenario 1 data are taken from
table 6 of FDA's proposed rule. Data for other scenarios were GAO
calculations using information in FDA's proposal. Table 2 also
shows that changes in the third assumption yield similarly
dramatic changes in the estimated benefits. FDA indicated in its
proposed rule that reporting rates might be higher than 10 percent
if, for example, the potential health risks were widely
publicized, or lower than 10 percent if consumers and physicians
assumed that dietary supplements are incapable of producing
adverse events. If, keeping all other factors constant, FDA had
assumed that 20 percent of serious adverse events were reported,
FDA's benefits estimate would have been reduced by about half.
Page 20 GAO/HEHS/GGD-99-90 FDA's Proposed
Rule on Ephedrine Alkaloids B-281581 Conversely, if FDA had
assumed that only 5 percent of events were being reported, the
benefits estimate would have doubled. FDA Met RFA
RFA requires agencies to consider the effects of their rules on
small entities Requirements, Although and to take certain
actions during the rulemaking process. For example, Questions
Remain before publishing a proposed rule for which a
notice of proposed Concerning Impacts on rulemaking is
required, sections 603 and 605(b) of RFA require a federal
Industry agency to prepare and make available for
public comment an initial regulatory flexibility analysis that
describes the anticipated effects of the proposed rules on small
entities, unless the head of the agency certifies that the rule
will not have a "significant economic impact on a substantial
number of small entities." (See app. V for a more detailed
discussion of RFA requirements and our analysis of the actions FDA
took to comply with RFA.) FDA determined that its proposed rule on
ephedrine alkaloids would have a significant economic impact on a
substantial number of small entities and prepared an initial
regulatory flexibility analysis to identify those impacts. FDA's
proposed rule addressed the basic elements that RFA requires
agencies to include in an initial regulatory flexibility analysis.
For example, the rule describes the reasons the agency was
considering the action, states the purpose and legal basis of the
rule, describes and provides an estimate of the number of small
entities to which FDA believed the rule would apply, and describes
the compliance requirements. In addition to describing direct
compliance costs of between $3 million and $80 million, FDA said
that the proposed rule could have significant distributive effects
in the form of reduced sales of as much as $230 million a year.
FDA explicitly stated that costs and sales reductions of this
magnitude might threaten the viability of many firms. FDA also
discussed significant regulatory alternatives in the rule's
regulatory flexibility analysis section, noting that most of the
regulatory alternatives discussed in the cost-benefit analysis
section would reduce the impact of the rule on small businesses.
When a rule is promulgated that will have a significant economic
impact on a substantial number of small entities, section 609(a)
of RFA further requires agencies to ensure that small entities
have been given an opportunity to participate in the rulemaking
process through the "reasonable use" of outreach efforts. Our
review of the regulatory docket for this rulemaking, as well as
information obtained during interviews with dietary supplement
industry representatives, indicated that FDA provided Page 21
GAO/HEHS/GGD-99-90 FDA's Proposed Rule on Ephedrine Alkaloids B-
281581 opportunities for small business participation during this
rulemaking process. For example, in addition to the proposed rule
itself, FDA published public notices; held public meetings, during
which industry representatives participated and provided
testimony; and collected written comments on the issue.
Representatives of industry trade associations also told us that
FDA had placed notices about the proposed rule in trade
literature. However, SBA's Office of Advocacy criticized the
quality of FDA's regulatory flexibility analysis. Along with
generally criticizing FDA's scientific analysis for the proposed
rule, the office contended that FDA did not consider the large
numbers of independent distributors of dietary supplements in its
analysis and, as a result, underestimated the number of affected
small businesses and the impacts of the proposed actions. In
response to SBA's criticism, FDA officials told us that the agency
did not need to consider the impact of its proposed rule on
distributors of ephedrine alkaloids. FDA officials cited several
court decisions that support the proposition that, under RFA, an
agency is under no obligation to conduct a small entity impact
analysis of effects on entities that the agency does not regulate-
that is, distributors.35 None of the major parties-FDA, SBA's
Office of Advocacy, or industry associations-have information on
the actual number of entities involved in the market for dietary
supplements containing ephedrine alkaloids. All three parties
acknowledge this as a limitation in attempts to analyze the
effects of the proposed rule. According to FDA, it has no
authority to require companies in the dietary supplement industry
either to register with or contact the agency unless they are
seeking approval to make a claim in product labeling. However, the
industry representatives we met with confirmed that manufacturing
and labeling of these products are generally limited to a
relatively small number of manufacturing firms. Such firms would
bear almost all of the direct expenditures required for compliance
with the proposed rule. SBA's Office of Advocacy recommended that
FDA develop an outreach strategy to obtain more reliable industry
data, and FDA officials said that they have contracted for a
marketing study of the dietary supplement industry that should
provide better information for future FDA analyses. 35For example,
United Distribution Cos. v. FERC, 88 F.3d 1105, 1170 (D.C. Cir.
1996). Page 22 GAO/HEHS/GGD-99-90 FDA's
Proposed Rule on Ephedrine Alkaloids B-281581 Proposed Rule Does
Not UMRA generally requires covered agencies to prepare
specific types of Appear to Trigger UMRA analyses for certain
rules that include a federal mandate and that may Requirements
result in the expenditure in any 1 year of $100 million or more by
the private sector.36 FDA did not explicitly address UMRA in its
proposed rule on ephedrine alkaloids because its economic analysis
indicated that direct expenditures imposed on the private sector
would not rise to the level requiring additional analysis under
UMRA. FDA estimated that the total compliance costs for the
proposed action would be between $3 million and $80 million, with,
at most, $70 million of those costs in the first year of
implementation. Although FDA also estimated that the rule might
result in lost sales for the dietary supplement industry of as
much as $230 million per year, lost sales cannot be considered
direct expenditures by the private sector and, therefore, cannot
be used to trigger UMRA's analytical requirements. (See app. VI
for a more detailed discussion of UMRA requirements and our
analysis of FDA's compliance.) However, SBA's Office of Advocacy
contended that FDA overlooked the UMRA requirements and commented
that if the agency had properly estimated the number of affected
businesses and the costs that would be imposed on those entities,
"it would have been apparent that the economic impact of the
instant rule would impose in excess of $100 million in costs to
the industry," triggering UMRA requirements. However, the Office
of Advocacy does not have data on the number of entities involved
in the market for dietary supplements containing ephedrine
alkaloids that it believes should have been included in FDA's
analysis. Therefore, the Office of Advocacy could not demonstrate
that FDA had underestimated the number of manufacturers that would
be affected by the relabeling and reformulation requirements in
the rule or that FDA's cost estimates for those manufacturers were
in error. Furthermore, even if the expenditures associated with
the proposed rule had triggered UMRA's analytical requirements,
FDA appears to have satisfied most, if not all, of those
requirements. For example, FDA quantitatively and qualitatively
assessed the anticipated costs and benefits of the rule. FDA also
identified and considered a number of regulatory alternatives,
indicating that the other alternatives would not be as effective
as the proposed rule. Conclusions FDA was justified
in determining that the number of AERs relating to dietary
supplements containing ephedrine alkaloids warranted their
attention and consideration of steps to address safety concerns.
The available scientific information suggests that the use of
products containing synthetic 36The statute provides for annual
adjustments for inflation. Page 23
GAO/HEHS/GGD-99-90 FDA's Proposed Rule on Ephedrine Alkaloids B-
281581 ephedrine alkaloids can result in adverse experiences for
some individuals, and over-the-counter products containing
ephedrine alkaloids have dosing recommendations. Furthermore,
dietary supplement trade associations have suggested specific
dosing limits for dietary supplements containing ephedrine
alkaloids. However, while FDA's conclusions regarding the
desirability of the proposed actions may be valid, we believe
these conclusions are open to question because of limitations and
uncertainties associated with the agency's underlying scientific
and economic analyses. We have concerns about the strength of the
information upon which FDA based specific elements of its proposed
rule. There is no scientific information on the specific dosing
levels and duration limits proposed. FDA, therefore, relied
heavily on its AERs to determine a dosing regimen and to outline
benefits that would accrue from the proposed rule. However, the
AERs suffer from several problems that weaken the conclusions
drawn by FDA for their specific dosing regimen; the number of AERs
used to support the dosing regimen is small, their quality is
questionable, and FDA did not establish a causal link between the
ingestion of ephedrine alkaloids and the occurrence of particular
adverse events. Finally, because FDA did not document which AERs
it identified as involving serious adverse events, it is
impossible to verify FDA's calculation of the number of these
events and the accuracy of the benefits that FDA estimated would
occur as a result of the proposed rule. FDA's analysis contained
the basic elements expected in a federal agency's cost-benefit
analysis, and the proposed rule complied with rulemaking
requirements under RFA. The proposed rule does not appear to
trigger the UMRA analytical requirements. However, FDA's cost-
benefit analysis was not always transparent regarding why certain
key assumptions were made, the degree of uncertainty involved in
those assumptions, or the effect that alternative assumptions
would have had on the agency's estimates of the costs and benefits
of the proposed action. Recommendations Given the uncertainties
in the information upon which FDA based its proposed rule, we
recommend that the Secretary of Health and Human Services direct
the Commissioner of FDA to obtain additional information to
support conclusions regarding the specific requirements in the
proposed rule for dietary supplements containing ephedrine
alkaloids before proceeding to final rulemaking. Specifically, FDA
needs to provide stronger evidence on the relationship between the
intake of dietary Page 24 GAO/HEHS/GGD-99-90 FDA's
Proposed Rule on Ephedrine Alkaloids B-281581 supplements
containing ephedrine alkaloids and the occurrence of adverse
reactions that support the proposed dosing levels and duration of
use limits. We also recommend that the Secretary direct the
Commissioner to improve the transparency of FDA's cost-benefit
analysis in its final rulemaking. Specifically, FDA should more
fully explain the bases of its cost-benefit assumptions, the
degree of uncertainty associated with those assumptions, and the
implications of plausible alternative assumptions to the proposed
action and other regulatory alternatives. Agency and Other We
sent a draft of this report to the Commissioner of FDA, the
Reviewer Comments Administrator of SBA, and the Director of
OMB. FDA and SBA's Office of Advocacy provided written comments,
which are reprinted in appendixes VII and VIII; OMB did not have
comments on the report. In its comments, FDA concurred with our
recommendation that it obtain additional information to support
its conclusions on specific requirements relating to dietary
supplements containing ephedrine alkaloids before proceeding to
final rulemaking. FDA also concurred with our recommendation to
improve the transparency of the agency's cost-benefit analysis,
noting that it intends to take appropriate steps to correct the
deficiencies of the analysis prior to publication of a final rule.
However, FDA was concerned that our report (1) did not
sufficiently highlight the agency's justification in examining
safety concerns related to dietary supplements containing
ephedrine alkaloids, (2) implied that FDA relied principally on
AERs to develop the proposed rule, (3) overemphasized the agency's
failure to conduct a causal analysis linking dietary supplements
containing ephedrine alkaloids to the AERs, and (4) was not clear
on whether it was appropriate to use AERs in rulemaking.
Throughout our report, we express the clear view that given the
number and type of AERs, it was reasonable for FDA to investigate
the safety of the supplements. We also acknowledge that FDA's
actions were based on information from scientific literature as
well as AERs. However, although clinical trials suggest that
adverse events could occur in some individuals using ephedrine
alkaloids at levels approaching 20 mg per serving, they did not
provide information relating to adverse events below the 20 mg
level. Therefore, to conclude that an 8 mg per serving level would
be appropriate, FDA turned to results of tests it performed on a
small number of products implicated in 13 AERs. In other words,
information from these Page 25 GAO/HEHS/GGD-99-90
FDA's Proposed Rule on Ephedrine Alkaloids B-281581 AERs was the
sole source of support for the specific dosing level of 8 mg per
serving that FDA proposed. Finally, we continue to hold the view
that without a causal link between the AERs and the ingestion of
products containing ephedrine alkaloids, the exclusive use of AERs
to support a specific dosing regimen is questionable. FDA also
asked that we clarify that using AERs to develop a proposed rule
of this type is neither inappropriate nor unscientific. For this
report, we examined whether the use of AERs in the proposed rule
on dietary supplements containing ephedrine alkaloids was
consistent with the use of AERs in prior rulemaking; however, we
did not take a position on the appropriateness of the general use
of AERs in rulemaking. SBA's Office of Advocacy stated that our
recommendations are reasonable and would result in a more
rational-and possibly less burdensome- regulation. However, the
officials questioned how we could conclude that FDA had complied
with RFA while at the same time suggesting that the agency had
prepared an "inadequate analysis" by using unreliable data and not
clearly demonstrating that the benefits of the proposed rule
exceed the cost. Although our report raises concerns about several
aspects of FDA's analysis, we did not conclude that FDA prepared
an inadequate analysis. Rather, we concluded that FDA's proposed
rule contained the primary elements required under E.O. 12866 for
a cost-benefit analysis and under RFA for an initial regulatory
flexibility analysis. Furthermore, RFA contains no standards or
criteria that define an "adequate" regulatory flexibility
analysis. As noted in SBA's comments, section 607 of RFA notes
that agencies may provide "general descriptive statements" to
comply with the act's analytical requirements if quantification is
not practicable or reliable. Determining whether a regulatory
flexibility analysis was in compliance with RFA would require
analysis and judgments concerning the totality of the
circumstances relating to the specific regulation in question. We
have no basis and are not in a position to make such a
determination regarding the quality of FDA's analysis, nor does
our report do so. SBA's Office of Advocacy also expressed its
concern that FDA's analysis did not identify the indirect effects
of the rule on distributors. Office of Advocacy officials said
that, although certain court decisions stemming from Mid-Tex
Electric Cooperative, Inc. v. FERC37 indicated that agencies need
not prepare regulatory flexibility analyses if the effects of a
rule on an industry are indirect, they disagreed with those
interpretations and contended that agencies should be required to
conduct analyses of indirect effects. They also stated that the
Office of Advocacy was unaware 37773 F.2d 327 (D.C. Cir. 1985).
Page 26 GAO/HEHS/GGD-99-90 FDA's Proposed Rule
on Ephedrine Alkaloids B-281581 of any comparable case law with
respect to direct and indirect effects as they apply to E.O.
12866, so they said FDA should have prepared an analysis of the
rule's effects on distributors. We disagree with these comments
for three reasons. First, as the Office of Advocacy officials
acknowledged, prior case law indicates that agencies need not take
the indirect effects of their rules into account when conducting a
regulatory flexibility analysis. That interpretation was recently
reaffirmed in a May 14, 1999, decision by the U.S. Court of
Appeals for the District of Columbia Circuit.38 Second, the
absence of case law comparable to Mid-Tex with respect to direct
and indirect effects as they apply to E.O. 12866 says nothing
about FDA's analytical obligations under the order. Third, our
review shows that FDA did address the potential indirect effects
of the rule through its estimates of total lost sales for the
dietary supplement industry. FDA also explicitly acknowledged in
the proposed rule that such lost sales are "obviously very
significant to the affected parties" and might threaten the
viability of many firms in this industry. We also obtained
comments on a draft of this report from a professor of
pharmacology with expertise on dietary supplements. He agreed that
while there is adequate reason for FDA to be concerned about the
safety of products containing ephedrine alkaloids, he believed
that there are too few AERs of substantive quality to allow for
the setting of a maximum safe dose. He also concurred with our
conclusions and recommendations. FDA provided technical comments,
which we incorporated as appropriate. 38American Trucking
Associations, Inc., et al, v. United States Environmental
Protection Agency, No. 97-1440 U.S. App. LEXIS 9064 (D.C. Cir. May
14, 1999). Page 27 GAO/HEHS/GGD-99-90 FDA's
Proposed Rule on Ephedrine Alkaloids B-281581 As agreed with your
office, unless you publicly announce its contents earlier, we plan
no further distribution of this report until 30 days from the date
of this letter. At that time, we will send copies to the Honorable
Donna E. Shalala, Secretary of Health and Human Services; the
Honorable Jane E. Henney, Commissioner of FDA; the Honorable Aida
Alvarez, Administrator of SBA; the Honorable Jacob J. Lew,
Director of OMB; and others who are interested. We will also
provide copies to others upon request. GAO contacts and major
contributors to this report are listed in appendix IX. William J.
Scanlon Director of Health Financing and Public Health Issues L.
Nye Stevens Director of Federal Management and Workforce Issues
Page 28 GAO/HEHS/GGD-99-90 FDA's Proposed Rule on
Ephedrine Alkaloids Page 29 GAO/HEHS/GGD-99-90 FDA's Proposed
Rule on Ephedrine Alkaloids Contents Letter
1 Appendix I
32 Scope and Methodology Appendix II
34 Adverse Event Reporting Appendix III
37 FDA's Prior Use of AERs in Rulemaking Appendix IV
42 FDA's Analysis of Benefits and Costs Appendix V
56 Regulatory Flexibility Act Requirements Appendix VI
61 Unfunded Mandates Reform Act Requirements Appendix VII
63 Comments From the Department of Health and Human Services Page
30 GAO/HEHS/GGD-99-90 FDA's Proposed Rule on Ephedrine
Alkaloids Contents Appendix VIII
72 Comments From the Small Business Administration Appendix IX
79 GAO Contacts and Staff Acknowledgments Tables
Table 1: Summary of 13 AERs Used for Specific Dosing Level
13 Table 2: FDA's Estimate of Expected Benefits Is Sensitive to
20 Changes in Key Assumptions Table III.1: Five Prior FDA
Rulemaking Cases 37 Figure
Figure IV.1: Estimation of Benefits Is Sensitive to Changes in
53 Underlying Assumptions Abbreviations AEMS adverse event
monitoring system AER adverse event report CDC
Centers for Disease Control and Prevention CFSAN Center for
Food Safety and Applied Nutrition CPSC Consumer Product
Safety Commission EPA Environmental Protection Agency FDA
Food and Drug Administration GRAS generally recognized as
safe OIRA Office of Information and Regulatory Affairs OMB
Office of Management and Budget OSN Office of Special
Nutritionals RFA Regulatory Flexibility Act SBA
Small Business Administration UMRA Unfunded Mandates
Reform Act Page 31 GAO/HEHS/GGD-99-90 FDA's Proposed
Rule on Ephedrine Alkaloids Appendix I Scope and Methodology To
determine whether the reported cardiovascular and nervous system
effects outlined in the proposed rule were relevant for dietary
supplements containing ephedrine alkaloids and whether there was
evidence that these ingredients could cause adverse experiences in
some individuals, we examined the scientific literature for case
reports of adverse events from ingestion of products containing
ephedrine alkaloids and reviewed findings from clinical trials
using ephedrine alkaloids to treat obesity. To determine whether
the Food and Drug Administration (FDA) had any internal guidance
on the use of adverse events reports (AER) in rulemaking and
whether these reports had been used in prior rulemaking, we sought
information from the agency. We examined five prior cases of
rulemaking that FDA provided in which the Center for Food Safety
and Applied Nutrition (CFSAN) had used AERs as part of the basis
for conclusions that were drawn in a proposed rule to ascertain
how the agency had previously used AERs in rulemaking. In
examining the basis for FDA's rule, we evaluated information
contained in a sample of AERs and FDA's reliance on them to
support a dosing regimen and to estimate benefits from the
proposed rule. To identify criteria for the quality of the AERs,
we obtained information on the use of passive surveillance
systems, including their methodological strengths and weaknesses.
We also examined the scientific literature for studies on adverse
event reporting and outcomes from such reporting systems. We
performed a content analysis of a random sample of 92 out of 864
AERs filed prior to the proposed rule to determine what
information was available in the AERs, such as dose, frequency of
ingestion, duration of use, type of product, the adverse effect
reported, and whether a medical record was included in the report.
We examined the list of 13 AERs used by FDA to set the dosing
regimen outlined in the proposed rule. We compared the analytical
results of tests performed on the products implicated in these
AERs to determine the dosing level ingested by the patient
describing the adverse event. We examined the results of these
analytical tests to determine the variability in the amount of
ephedrine alkaloids that could be found in individual bottles and,
thus, the possible range in the amount of ephedrine alkaloids
ingested by a complainant. We sought documentation from FDA
officials on the AERs that were classed as serious, the numbers
and types of each category of serious event, and how those cases
had been identified. We reviewed the extent to which FDA's cost-
benefit analysis for the proposed rule contained the elements
expected of a federal agency Page 32 GAO/HEHS/GGD-99-
90 FDA's Proposed Rule on Ephedrine Alkaloids Appendix I Scope and
Methodology analysis. To identify criteria for this objective, we
reviewed the cost-benefit requirements in E.O. 12866, "Regulatory
Planning and Review," and a guidance document issued by the
Administrator of the Office of Information and Regulatory Affairs
(OIRA) entitled "Economic Analysis of Federal Regulations Under
Executive Order 12866." OIRA's guidance was intended to describe
"best practices" for agencies in preparing cost-benefit analyses
of regulatory actions under the executive order. Using these best
practices, we reviewed FDA's cost-benefit analysis in the
"analysis of impacts" section of the agency's June 1997 proposed
rule. To obtain additional information, we interviewed FDA
officials involved in the development of the cost-benefit analysis
and the proposed rule. We also reviewed supporting documents
referenced in the proposed rule or included in the electronic
docket for this rulemaking. As part of that task, we examined each
comment related to this rulemaking in the electronic docket to
identify substantive remarks on elements of FDA's analysis. We
also interviewed officials at OIRA, the Small Business
Administration (SBA), and trade associations representing the
dietary supplement industry to obtain their views on FDA's cost-
benefit analysis. We used a similar approach to describe FDA's
actions to comply with the requirements of the Regulatory
Flexibility Act (RFA) and the Unfunded Mandates Reform Act (UMRA).
We identified criteria by reviewing the specific requirements in
the statutes and then examined the relevant sections of FDA's
proposed rule to compare the agency's actions with those
requirements. As with the previous task, we supplemented our
review of the proposed rule by interviewing officials at FDA, SBA,
and industry trade associations. We also reviewed public comments
submitted on the proposed rule, including comments submitted by
SBA's Office of Advocacy on FDA's compliance with RFA and UMRA. In
our evaluation of FDA's cost-benefit analysis and initial
regulatory flexibility analysis, we relied primarily on the
information FDA presented in the proposed rule or included in the
regulatory docket. We did not perform an independent assessment of
the accuracy and reliability of FDA's underlying data and
estimates. We also did not examine issues relating to restrictions
outlined in the proposed rule relating to ephedrine alkaloids and
their use with other stimulants and labeling restrictions. Page 33
GAO/HEHS/GGD-99-90 FDA's Proposed Rule on Ephedrine Alkaloids
Appendix II Adverse Event Reporting The clinical research and
review staff in CFSAN's Office of Special Nutritionals (OSN) is
responsible for tracking and reviewing AERs on special nutritional
products.39 Special nutritional products include dietary
supplements, infant formulas, medical foods, and foods for special
dietary use. The adverse event monitoring system (AEMS) used by
OSN is a voluntary surveillance system established to collect
information on AERs associated with the use of special nutritional
products. Only AERs associated with an illness or injury are
included in AEMS; reports of product quality or consumer
dissatisfaction are not included. AERs may enter this system
through a variety of mechanisms, including FDA's drug quality
reporting system, FDA's Medical Products Reporting Program
(Medwatch), the United States Pharmacopeia, FDA field offices
(through the Office of Regulatory Affairs' consumer complaint
system), and from other federal and state public health agencies.
Written and electronic correspondence and telephone conversations
are other sources of information about adverse events reported
with the use of special nutritional products. When an AER is
received, it is reviewed to evaluate the need for further follow-
up, including medical record documents, label and labeling, and
other types of information necessary for evaluation of the AER.
Follow-up is routinely attempted on all AERs of serious events or
when the event is deemed to be clinically significant. MedWatch
provides guidelines for determining if an adverse event is serious
for reporting purposes. Events deemed serious include those that
are fatal, life-threatening, or permanently or significantly
disabling; require or prolong hospitalization; result in
congenital anomalies; or require intervention to prevent permanent
impairment or damage.40 The proposed rule on ephedrine alkaloids
includes as serious cardiovascular events abnormal heart rhythms,
stroke, heart attack, and cardiomyopathy. Serious nervous system
events that were listed in the proposed rule included seizures,
psychosis, mania, severe depression, vestibular (inner ear)
disturbances, and loss of consciousness. Other reported adverse
effects that FDA deemed clinically serious or potentially serious
included elevations of liver function tests or overt hepatitis,
myopathies (disease of muscle), disturbances of the genitourinary
system, and dermatologic manifestations. AEMS is considered an
invaluable tool for identifying potential serious public health
issues that may be associated with the use of a particular
39Information in this appendix was obtained from a continuing
education article from FDA's Center for Drug Evaluation and
Research, "The Clinical Impact of Adverse Event Reporting" (Oct.
1996). 40Health professionals can report any adverse event that
they judge to be clinically significant. Page 34
GAO/HEHS/GGD-99-90 FDA's Proposed Rule on Ephedrine Alkaloids
Appendix II Adverse Event Reporting product or types of products.
Although more formal epidemiological studies, prospective trials,
and retrospective case-control studies are more rigorous,
surveillance can be important as an early alert to potential
problems. As such, even isolated reports can be definitive in
associating products with an adverse effect if the product and
effect are temporally related and there is evidence of dechallenge
(signs and symptoms resolve or improve when a consumer stops using
a product) and rechallenge (symptoms recur when the consumer
resumes using the product). However, like all voluntary
surveillance systems, AEMS has certain weaknesses. These
limitations include different interpretations in determining an
adverse event, underreporting, biases, estimation of population
exposure, and report quality. In regard to differing
interpretations, it has been well established that AERs are quite
subjective and imprecise.41 For example, in one study, clinical
pharmacologists and treating physicians showed complete agreement
less than half the time when determining whether medication,
alcohol, or recreational drug use had caused hospitalization.42 It
is also well known that placebos43 and even no treatment44 can be
associated with adverse events. In addition, there is almost
always an underlying background rate for any clinical event in a
population, regardless of whether there was exposure to a
particular product. Another major concern of passive surveillance
systems is underreporting of adverse events. For example, it was
estimated that under a British spontaneous reporting system, only
10 percent of serious events and 2 to 4 percent of nonserious
events are reported.45 Another estimate has shown that FDA may
receive, by direct reporting, less than 1 percent of suspected
serious adverse drug reactions.46 Thus, the cases spontaneously
reported 41J. Kock-Weser and others, "The Ambiguity of Adverse
Drug Reactions," European Journal of Clinical Pharmacology, Vol.
11 (1977), pp. 75-78. 42F. E. Karch and others, "Adverse Drug
Reactions-A Matter of Opinion," Clinical Pharmacology Therapy,
Vol. 19 (1976), pp. 489-92. 43D. M. Green, "Pre-Existing
Conditions, Placebo Reactions, and Side Effects," Annals of
Internal Medicine, Vol. 60 (1964), pp. 255-65. 44M. M. Reidenberg
and D. T. Lowenthal, "Adverse Nondrug Reactions," New England
Journal of Medicine, Vol. 279 (1968), pp. 678-79. 45M. D. Rawlins,
"Pharmacovigilance: Paradise Lost, Regained or Postponed?," The
William Withering Lecture, 1994, Vol. 29 (London: J. R. Coll.
Physicians, 1995), pp. 41-49. 46H. D. Scott and others, "Rhode
Island Physicians' Recognition and Reporting of Adverse Drug
Reactions," Rhode Island Medical Journal, Vol. 70 (1987), pp. 311-
16. Page 35 GAO/HEHS/GGD-99-90 FDA's Proposed
Rule on Ephedrine Alkaloids Appendix II Adverse Event Reporting to
any surveillance program generally represent only a small portion
of the number that have actually occurred. Biases also limit the
usefulness of spontaneous reporting systems. Unlike clinical trial
data, which are obtained under strictly controlled conditions,
spontaneously reported information is uncontrolled and, therefore,
subject to the possible influence of a number of reporting biases
that are brought about by the length of time a product has been on
the market, the reporting environment, and the quality of the
data. Compounding these problems, passive surveillance systems
lack a denominator for the data, such as user population or drug
exposure patterns. This is helpful to provide the exact number of
people exposed to a product and, thus, the risk for the adverse
event to occur. As a result, incidence rates cannot be computed.
Finally, the ability to analyze information contained in an AER is
dependent on the quality of information submitted. Optimally, an
AER should include product name, demographic data, a clinical
description of the adverse event (including confirmatory and
laboratory test results), confounding factors (such as medical
products and medical history), temporal information, dose and
frequency of use, biopsy or autopsy reports, dechallenge and
rechallenge information, and outcome.47 Although spontaneous
reporting systems have a number of weaknesses, they also have
advantages over more controlled systems. First, they maintain
ongoing surveillance of products that are marketed and, second,
they are relatively inexpensive. In fact, they may be the most
cost-effective way to detect rare, serious adverse events not
discovered during clinical trials.48 Their usefulness lies in
hypothesis generation, to explore cause and effect relationships
concerning an adverse event, and as a warning signal of a
potential problem with a product. Useful factors for assessing a
causal relationship include a chronology of administration of a
product, including beginning and ending of treatment and adverse
event onset; dechallenge; rechallenge; laboratory test results;
and previously known toxicity of an agent or product. 47FDA,
Center for Drug Evaluation and Research, "The Clinical Impact of
Adverse Event Reporting" (Oct. 1996), pp. 5-6. 48FDA, Center for
Drug Evaluation and Research, "The Clinical Impact of Adverse
Event Reporting," p. 6. Page 36 GAO/HEHS/GGD-99-
90 FDA's Proposed Rule on Ephedrine Alkaloids Appendix III FDA's
Prior Use of AERs in Rulemaking AERs have been used by FDA in
several rulemaking procedures. In response to a request for prior
rules that had relied on AERs, FDA provided five cases, including
one in which the "generally recognized as safe" (GRAS) status of
an agent (sulfites) was revoked. These cases are outlined in table
III.1. Table III.1: Five Prior FDA Rulemaking Cases
Regulation Substance Action 51
Fed. Reg. 25021 Sulfiting agent Revoked GRAS
status for use on fruits and vegetables intended to be served or
sold raw to consumers. 51 Fed. Reg. 25012 Sulfiting
agent Required food labeling for foods. 49 Fed. Reg.
13679 Protein supplements Required warning
labeling. 62 Fed. Reg. 2218 Supplements and
Required label warning statements and drugs containing iron unit-
dose packaging. 44 Fed. Reg. 37212 FD&Ca yellow no. 5
Required labeling in food and drugs for human use. aFood, Drug,
and Cosmetic. Sulfiting Agents and FDA's
proposed rule to revoke GRAS status for sulfiting agents resulted
after GRAS Status the agency
received information on the use of sulfur dioxide, sodium sulfite,
sodium bisulfate, potassium bisulfate, sodium metabisulfate, and
potassium metabisulfate (collectively known as sulfiting agents,
or sulfites) on fruits and vegetables intended to be served raw or
sold raw to consumers and it concluded that such use was not safe.
This rule, similar to the ephedrine alkaloid ruling, used AERs,
advisory panels, clinical studies, and case reports to determine
whether sulfiting agents should maintain its GRAS status. FDA had
received over 500 consumer complaints, where individuals
reportedly suffered from a variety of adverse allergic-type
responses, including reports of 13 deaths; the proposed rule
indicated that these responses tended to occur in people with
asthma.49 Among the 500 AERs, approximately 40 percent mentioned
the occurrence of the event after eating raw fruits and vegetables
in restaurants (where sulfiting agents are used), while 15 percent
specifically mentioned the occurrence of the event after drinking
wine or beer. The proposed rule delineated the clinical outcomes
from these adverse reactions, ranging from gastrointestinal 49FDA
relied on prevalence estimates that indicated there are 10 million
individuals with asthma in the United States, with up to 10
percent sulfite-sensitive. See Select Committee on GRAS
Substances, "The Reexamination of the GRAS Status of Sulfiting
Agents," Life Sciences Research Office, Federation of American
Societies for Experimental Biology, prepared under FDA contract
223-83-2020 (Jan. 28, 1985). Page 37
GAO/HEHS/GGD-99-90 FDA's Proposed Rule on Ephedrine Alkaloids
Appendix III FDA's Prior Use of AERs in Rulemaking problems to
anaphylactic shock (hypersensitivity reactions characterized by
swelling, erythema, bronchoconstrictions, and hypotension). A
scientific panel also reviewed the AERs submitted to FDA and
concluded that the reports indicated an association between
adverse responses and the ingestion of meals that included foods
containing sulfiting agents. The panel also examined exposure
estimates and evidence from clinical experiments during its review
on this topic. While this rule and the ephedrine alkaloid rule are
similar-because the AERs used in the rulings simply highlighted
the number of complaints and provided a breakdown of the
complaints' origins and clinical complications-the proposed rule
on sulfiting agents outlined additional information not found in
the proposed rule for ephedrine alkaloids. This included
information on exposure estimates, prevalence rates within
sensitive populations, and clinical experiments involving the
exposure of sensitive and nonsensitive populations to sulfiting
agents. In addition, dosing regimens were not set for sulfiting
agents because the proposed rule revoked their GRAS status.
Sulfiting Agents and The proposed rule for the labeling of
products containing sulfiting agents Food Labeling was
designed to clarify the circumstances in which the presence of
sulfiting agents must be declared on food labels. FDA noted in
this proposed rule that any detectable amount of a sulfiting agent
would require that the product be labeled because the agency was
unaware of any evidence that established a level below which these
substances would not cause a reaction in sensitive individuals.
FDA relied on scientific data on human sensitivity to sulfiting
agents to establish that certain sensitive individuals-in
particular, those with asthma-could react to ingested sulfiting
agents. FDA also used estimates of the number of individuals with
asthma and clinical studies on those who were sensitive to
sulfiting agents to determine the number who might be adversely
affected. Last, FDA formed an ad hoc advisory committee on
hypersensitivity to sulfiting agents in food to review and
evaluate available data on adverse reactions in humans associated
with the use of sulfiting agents in food. This rule did not
describe, in any appreciable detail, adverse events. Labeling for
Protein In December 1997, FDA proposed labeling requirements
for protein Supplements supplements that were used in
weight reduction programs. The purpose of the rule was to alert
consumers to the potential health hazards associated Page 38
GAO/HEHS/GGD-99-90 FDA's Proposed Rule on Ephedrine Alkaloids
Appendix III FDA's Prior Use of AERs in Rulemaking with consuming
protein supplements to control weight and to inform consumers that
the advice of a physician should be sought before using these
products for weight control. The agency proposed these
requirements on the basis of evidence that, without proper medical
supervision, low-calorie diets consisting primarily of protein may
cause serious medical problems, including death. The proposed rule
noted that an ad hoc advisory group had been formed to examine
several cases of deaths associated with individuals using these
protein products as their primary source of nourishment. A
tentative final rule published on December 29, 1978, noted that
more than 165 reports of adverse reactions attributed to
consumption of protein products by consumers attempting to lose
weight had been reported to the FDA. These complaints-described as
"very diverse"-included nausea, tachycardia (rapid heart rate or
palpitations), breathlessness, and headaches.50 Review of these
reports by FDA revealed no consistent pattern of complaints or
reactions, and the tentative final rule concluded that "a cause-
and-effect relationship can neither be affirmed nor rejected on
the basis of the existing information from these reports." The
rule further stated that "There were sufficient details among the
reported adverse reactions . . . to warrant concern." FDA further
asserted that imposing a warning on the labeling of these protein
products-even with the gaps in their understanding of the basic
mechanism of how these protein supplements affected the body-was
particularly appropriate because the consequence of indiscriminate
use of these products was death. FDA also noted that the protein
products were not inherently dangerous and that the label warning
was proposed to curb misuse of these products and because the
products were being sold directly to consumers who might not be
under proper medical supervision. Data were collected through a
telephone survey, conducted by FDA and CDC, to determine the
extent of use of protein products. This information was used to
establish a death rate by combining this information with data on
deaths among liquid protein dieters from the same population
group. This information was compared with data from the National
Center for Health Statistics on annual death rates due to cardiac
abnormalities among a similar population group. FDA concluded that
the observed death rate was greater in those who used low-calorie
protein diets. FDA also initiated experimental protocols using
laboratory rats in order to explore the basic 50After a court
challenge from the National Nutritional Foods Association, a final
rule was published on April 4, 1984. Page 39
GAO/HEHS/GGD-99-90 FDA's Proposed Rule on Ephedrine Alkaloids
Appendix III FDA's Prior Use of AERs in Rulemaking mechanisms for
the cause of death. These studies were designed to assess the
correlation of cardiac arrhythmias (irregular heartbeat) and death
with various aberrant states of nutrition health and rate of
weight loss. Supplements and In October 1994, FDA
proposed warning labels and unit-dose packaging 51 Drugs
Containing Iron requirements for products taken in solid oral
form to supplement the dietary intake of iron or to provide iron
for therapeutic purposes. The labeling was proposed because of
evidence of acute iron poisoning attributable to accidental
overdoses of iron-containing products and an upsurge in reported
accidental pediatric ingestion of iron-containing products, which,
in some cases, resulted in death.52 FDA relied on data from the
American Association of Poison Control Centers and the U.S.
Consumer Product Safety Commission (CPSC) that highlighted
increases in reported fatalities from accidental iron poisonings
of children. Over 63,000 reports were taken at poison control
centers involving ingestion of adult iron-containing products,
with over 47,000 of these involving children under 6 years of age.
One hundred and fifty-nine cases were classified as major
outcomes; that is, they were life threatening or resulted in
permanent injury. An additional 1,500 cases were classified as
"moderate outcomes"; that is, the patient had symptoms that
required some form of treatment. Over 76,000 reports to these
centers involved ingestion of pediatric iron-containing products
with children under 6 years of age. Likewise, data from the
National Electronic Injury Surveillance System-a probability
sample of hospital emergency rooms in the United States used by
CPSC to measure the magnitude of injury associated with consumer
products-found a significant upward trend in the estimated number
of hospital emergency-room-treated iron-ingestion cases involving
children under 5 years of age. The proposed rule also outlined
data collected by CPSC on case reports of pediatric deaths,
including the number of tablets taken, the potency (dosage), and
total amount ingested. Additionally, a conference was held by CPSC
to examine the reason for the increase in iron poisonings among
children. 51Unit-dose packaging is designed to prevent the
unintended ingestion by children, not to control the recommended
dose for adults. 52U.S. Consumer Product Safety Commission,
"Pediatric Iron Poisonings and Fatalities" (May 1994), p. 3. Page
40 GAO/HEHS/GGD-99-90 FDA's Proposed Rule on
Ephedrine Alkaloids Appendix III FDA's Prior Use of AERs in
Rulemaking The final rule expanded on the information that was
used as the basis for warning labels by outlining the results of
animal toxicity studies, acute toxicity in humans, comparison of
animal toxicity data to human toxicity data, and physiological
factors that influence toxicity. Thus, information used by FDA in
its proposed and final rules on iron-containing products included
data from CPSC, case reports from adverse reactions, and
scientific literature. FD&C Yellow No. 5 In its proposed rule
of February 4, 1977, FDA noted that there was accumulating
evidence that FD&C yellow no. 5 (tartrazine) caused allergic-type
responses, especially in aspirin-intolerant individuals.53 Thus,
FDA was proposing to require a label declaration of FD&C yellow
no. 5 when used to color foods and ingested drugs and to prohibit
its use in certain drugs for human use. FDA noted in the proposed
rule that "the precise incidence of intolerance of FD&C yellow no.
5 in the total population or even in aspirin-intolerant patients
is not known." However, FDA did outline a number of case reports
and studies highlighting that these effects did occur and that
some quantification regarding subpopulations at risk could be
identified. Specifically, FDA estimated that approximately 47,000
to 94,000 people would be intolerant to tartrazine. But this
proposed rule did not denote how many case reports or adverse drug
reactions formed the basis for FDA's actions. Instead, the primary
information used in the proposed and final rule related to
scientific studies demonstrating individual intolerance to
tartrazine. 53In the final rule, FDA required a label declaration
of FD&C yellow no. 5 when used to color foods and drugs for human
use. Page 41 GAO/HEHS/GGD-99-90 FDA's Proposed
Rule on Ephedrine Alkaloids Appendix IV FDA's Analysis of Benefits
and Costs E.O. 12866 and related guidance from OIRA state that
cost-benefit analyses should contain three basic elements: (1) a
statement of the need for the proposed action, (2) an examination
of alternative approaches, and (3) an analysis of benefits and
costs. Overall, the guidance indicates that an analysis should be
"transparent." We concluded that FDA's cost-benefit analysis
contained, to some degree, all three of the recommended elements
in such analyses. FDA also disclosed the basic methodology, data,
and assumptions used in its analysis. However, the cost-benefit
analysis was not always transparent regarding why certain key
assumptions were made, the degree of uncertainty involved in those
assumptions, or the effect that alternative assumptions would have
had on the agency's estimates of the costs and benefits of the
proposed action. E.O. 12866 and Conceptually, a cost-
benefit analysis is a rigorous and data-intensive Related Guidance
Set procedure of weighing the costs and benefits of various
alternatives to a proposed action, informing decisionmakers about
the potential Agency consequences of each.
However, the results of the analysis, by themselves,
Responsibilities for do not determine whether or how an agency
will regulate in response to a perceived problem. Analysis of
Benefits and Costs E.O. 12866 requires agencies to
conduct cost-benefit analyses for all regulatory actions that are
likely to result in a $100 million annual effect on the economy or
are otherwise economically significant. The executive order also
makes OIRA responsible for reviewing all significant regulatory
actions and for providing guidance to agencies on issues covered
by the order. On January 11, 1996, the Administrator of OIRA
issued guidance describing "best practices" for preparing cost-
benefit analyses.54 The guidance makes clear that it is not a
"mechanistic blueprint" and provides for flexibility and the
exercise of professional judgment in preparing analyses. However,
the guidance does establish one clear expectation of all such
analyses: Analysis of the risks, benefits, and costs associated
with regulation must be guided by the principles of full
disclosure and transparency. Data, models, inferences, and
assumptions should be identified and evaluated explicitly,
together with adequate justifications of choices made, and
assessments of the effects of these choices on the analysis. The
existence of plausible alternative models or assumptions and their
implications should be identified. In the absence of adequate
valid data, properly identified assumptions are necessary for
conducting an assessment. 54The OIRA guidance was developed as a
result of a 2-year study by an interagency group that included
representatives of all the major regulatory agencies and was co-
chaired by a member of the Council of Economic Advisers. Page 42
GAO/HEHS/GGD-99-90 FDA's Proposed Rule on Ephedrine Alkaloids
Appendix IV FDA's Analysis of Benefits and Costs Within this broad
framework, the OIRA guidance states that analyses of economically
significant rules should contain three basic elements: (1) a
statement of the need for the proposed action, (2) an examination
of alternative approaches, and (3) an analysis of benefits and
costs. Within each of these three elements, the guidance
recommends that certain items be considered or analytical
approaches be used in preparing an agency's analysis. For example,
in relation to the third requirement, the guidance says the
analysis should identify and explain the data or studies on which
the estimates of benefits and costs are based "with enough detail
to permit independent assessment and verification of the results."
The guidance also states, "where benefit or cost estimates are
heavily dependent on certain assumptions, it is essential to make
those assumptions explicit and, where alternative assumptions are
plausible, to carry out sensitivity analyses based on the
alternative assumptions." The purpose of such disclosure is to
allow for a reasoned determination by decisionmakers of the
appropriate level of regulatory action. FDA's Analysis FDA
indicated in the proposed rule that the regulation was
economically Addressed the significant, and FDA officials
told us that they used the OIRA guidance in developing the cost-
benefit analysis for the rule. We compared FDA's Primary Elements
cost-benefit analysis to the elements and practices outlined in
that Suggested by OIRA guidance and concluded that FDA's
analysis addressed the primary elements suggested by the OIRA
guidance. However, certain specific Guidance
elements appeared to be lacking. Need for Regulation The OIRA
guidance states that in establishing the need for the proposed
action, agencies should discuss (1) whether the problem
constitutes a significant market failure that compels government
action and (2) if there is a market failure, the appropriateness
of alternatives to federal regulation that would resolve the
problem adequately or better than the proposed rule. Among the
types of market failures that the guidance says agencies can
discuss are monopolies and inadequate information available to
consumers about product characteristics. The guidance states that
alternatives to federal regulation can include subsidies or fees
that may be more efficient than rigid mandates or state and local
regulation that would be more appropriate. FDA stated in the
proposed rule that the rule was needed because of a significant
market failure-specifically, inadequate information. FDA said that
despite the presence of warning labels of various types on many of
the products, "some consumers may not have sufficient information
on the Page 43 GAO/HEHS/GGD-99-90 FDA's Proposed
Rule on Ephedrine Alkaloids Appendix IV FDA's Analysis of Benefits
and Costs health risks associated with dietary supplements
containing ephedrine alkaloids to make informed choices concerning
the consumption of these products." Among other points, FDA said
that the level of information currently used by consumers might be
less than optimal because of consumer perceptions that products
marketed as foods or derived from botanical sources are inherently
safe. However, FDA did not directly address the second major item
in the guidance under the need for action section-whether
alternatives to federal regulation would resolve the problem
adequately or better than the proposed federal regulation. In
particular, FDA did not recognize that several states were in the
process of regulating these products and did not discuss whether
federal regulation was superior to those state regulations (for
example, because these products are marketed across state lines).
Examination of Alternative The OIRA guidance states that an
agency's cost-benefit analysis should Approaches
demonstrate that the agency has considered the most important
alternative approaches to the underlying problem and should
provide the agency's reasoning for selecting the proposed
regulation over such alternatives. The guidance notes that the
number and choice of alternatives to be selected for a detailed
cost-benefit analysis is a matter of judgment but says the agency
should nevertheless "explore modifications of some or all of a
regulation's attributes or provisions to identify appropriate
alternatives." The guidance also states that the agency should
explore a number of different types of regulatory alternatives,
including * more performance-oriented standards for health,
safety, and environmental regulations; * different requirements
for different segments of the regulated population; * alternative
levels of stringency; * alternative effective dates of compliance;
* alternative methods of ensuring compliance; * informational
measures (for example, mandatory disclosure requirements, such as
labeling); * more market-oriented approaches; and * consideration
of whether the agency should adopt a more stringent standard than
already established by statutory requirements. In the proposed
rule, FDA identified seven regulatory "options": (1) take no
action (the baseline alternative); (2) take no action, but
generate additional information; (3) take the proposed action; (4)
take the proposed Page 44 GAO/HEHS/GGD-99-90 FDA's
Proposed Rule on Ephedrine Alkaloids Appendix IV FDA's Analysis of
Benefits and Costs action but with a higher potency limit; (5) ban
dietary supplements that contain ephedrine alkaloids; (6) take the
proposed action, but do not require the warning statement; and (7)
require only the warning statement. In general, FDA's options
focused primarily on the OIRA guidance categories of alternative
levels of stringency and informational measures. FDA did not
appear to explore performance-oriented standards, different
requirements for different segments of the regulated population,
market-oriented approaches, or alternative effective dates of
compliance. However, it is unclear whether these types of
alternatives could legitimately be considered for the proposed
regulation. For example, it is unclear how FDA could establish
market approaches, such as additional fees on the dietary
supplement industry, or performance standards for health effects.
In comments on the rule, industry representatives and SBA's Office
of Advocacy identified what they viewed as a weakness related to
FDA's baseline description of the world absent the proposed
regulation. FDA presented information on the number of adverse
events believed to be associated with ephedrine alkaloid products
and described an increasing trend in reported adverse events.
However, industry representatives and SBA said FDA did not provide
sufficient information to put these data in a meaningful context.
For example, they said there was no information on general
probabilities or risks of people suffering the types of adverse
events that FDA attributed to ephedrine alkaloid products. They
also said that FDA did not provide information to put the
aggregate number and growth in reported adverse events in the
context of total consumption of dietary supplements that might
contain ephedrine alkaloids. SBA's Office of Advocacy and industry
trade associations commented that consumption of such supplements
might be in the billions of doses. In that context, they contended
that adverse events associated with the consumption of dietary
supplements containing ephedrine alkaloids may occur at very low
rates, perhaps lower than those for some over-the-counter drugs.
On the other hand, FDA did provide some contextual information in
the proposed rule on the incidence of adverse events for dietary
supplements as a whole. FDA's data indicated that AERs believed to
be associated with supplements containing ephedrine alkaloids
accounted for approximately 50 to 60 percent of the total number
of reports received by FDA when the rulemaking was initiated. FDA
officials told us that the number of AERs in their system
suspected of involving ephedrine alkaloids "jumped out" compared
with the numbers for any other substances, although they have not
done any detailed analysis or tally to document numbers for other
Page 45 GAO/HEHS/GGD-99-90 FDA's Proposed Rule on
Ephedrine Alkaloids Appendix IV FDA's Analysis of Benefits and
Costs substances. Representatives of industry trade associations
who we met with acknowledged that the incidence of adverse events
for supplements containing ephedrine alkaloids is greater than
would be expected for most other dietary supplements. Still,
providing only this contextual information in the proposed rule
does not appear to provide a balanced or complete picture of the
baseline risks that may be associated with supplements containing
ephedrine alkaloids. Analysis of Benefits and The bulk of
OIRA's guidance is devoted to addressing the third element, Costs
analysis of benefits and costs. In this section, the guidance
first presents general principles on (1) measuring the benefits
and costs of each alternative against a baseline condition of the
way the world would look absent the proposed regulation, (2)
evaluation of alternatives, (3) discounting costs and benefits
that may occur at different points in time, (4) treatment of risk
and uncertainty, (5) assumptions used in the analysis, (6)
international trade effects, (7) nonmonetized benefits and costs,
and (8) distributional effects and equity. These general
principles are followed in the OIRA document by more detailed
guidance on the analysis of benefits and costs. For example,
within the benefits section, the guidance states that reductions
in illnesses, injuries, and fatalities as a result of government
action are best monetized according to a "willingness to pay"
approach.55 The guidance also emphasizes that benefits and costs
should be "incremental," representing changes from the baseline
condition of no regulation, and that the analysis should identify
and explain the data or studies on which the estimates are based
"with enough detail to permit independent assessment and
verification of the results." In its analysis of benefits and
costs, FDA adhered to a number of the recommendations in the OIRA
guidance. For example, the analysis discussed distributional
effects of the proposed rule and alternatives in terms of lost
sales to the dietary supplement industry. FDA also discussed
nonmonetized costs associated with the proposed rule, and it
monetized the benefits associated with reductions in various types
of adverse events using the willingness-to-pay approach
recommended in the guidance. In addition, the analysis presented
the monetized costs of the proposed rule against the baseline
condition of no regulation. However, FDA did not provide monetized
or quantified benefit or cost estimates for all of the other
alternatives, and the agency's discussion of those alternatives'
benefits and costs were compared to the proposed rule-not the
baseline 55The principle of "willingness to pay" captures the
notion of opportunity cost by providing an aggregate measure of
what individuals are willing to forgo to enjoy a particular
benefit. Page 46 GAO/HEHS/GGD-99-90 FDA's
Proposed Rule on Ephedrine Alkaloids Appendix IV FDA's Analysis of
Benefits and Costs condition of no regulation. For example, in
relation to the benefits associated with allowing higher potency
levels (option 4) compared to FDA's proposal (option 3), FDA said
that the effect would be to "reduce those benefits below those
generated under option 3." (FDA officials told us during our
review that the agency attempted to reach a balance in conducting
the analysis and said the analysts try to quantify the effects of
alternatives when such information would be of value to
decisionmakers.) Also, some parts of FDA's analysis of these
alternatives were either unclear or illogical. For example, in
relation to the option of banning dietary supplements that contain
ephedrine alkaloids (option 5), FDA stated that "banning these
products would decrease access to these products by consumers." It
is not clear why a total ban in the products would not eliminate-
not just decrease-access. Similarly, FDA also said "the total
reduction in the consumption of dietary supplements that now
contain ephedrine would probably be approximately 33 percent under
this option." It is not clear why a ban would only reduce
consumption by 33 percent. Another unclear element of the analysis
concerns discounting. The guidance recommends that dollar
estimates should be discounted and reported in same-year dollars.
Although FDA noted that it converted dollar figures for its
estimates of monetized benefits to 1996 dollars, it did not
identify in the proposed rule whether its total cost estimates
were also expressed in 1996 dollars. As a result of its analysis,
FDA concluded that the proposed rule would generate benefits of
between $240 million and $670 million per year, quantifiable costs
of between $3 million and $70 million in the first year, and
quantifiable costs of between "a minimal amount" and $500,000 in
each subsequent year. FDA justified the selection of the proposed
rule over the other regulatory alternatives in terms of the
greater net benefits expected of the proposed actions. Analysis
Did Not Fully The OIRA guidance emphasizes the importance of full
disclosure and Explain Some Key transparency of
key data and assumptions in cost-benefit analyses. As noted
previously, the guidance says that "data, models, inferences, and
Assumptions or assumptions should be identified
and evaluated explicitly, together with Assess Implications of
adequate justifications of choices made, and assessments of the
effects of these choices on the analysis." Therefore, we assessed
the transparency Alternative of FDA's cost-
benefit analysis of the proposed rule in terms of three
Assumptions criteria: Page 47
GAO/HEHS/GGD-99-90 FDA's Proposed Rule on Ephedrine Alkaloids
Appendix IV FDA's Analysis of Benefits and Costs * first, whether
it identified the data, models, inferences, and assumptions used
to calculate the estimates of benefits and costs; * second,
whether it disclosed the reasons why those data, models,
inferences, and assumptions were selected; and * third, whether it
assessed the effects of plausible alternative assumptions and
choices on the results of the analysis-what is often referred to
as a "sensitivity analysis." Identification of Data and In
relation to the first criterion, FDA's analysis was very
transparent. The Assumptions agency provided a
clear and lengthy description of the data, assumptions, and
methodology that it used to calculate the benefits and costs of
the proposed rule. For example, in calculating the annual
monetized benefits of the proposed rule, FDA identified for each
of six categories of adverse events (for example, deaths, serious
cardiovascular system events, serious nervous system events, and
abnormal liver functions) (1) the annual number of reported
adverse events (based on the average number of AERs received each
year), (2) the estimated total number of such events each year,
(3) the estimated number of cases that would be reduced by
implementation of the proposed rule, (4) the monetized value of
each reduced case, and (5) the estimated value of the total risk
reduction (the number of reduced cases times the value of each
case). FDA also identified many of the underlying assumptions that
it used in developing these elements. For example, in developing
its estimate of the total number of annual cases, FDA noted that
it assumed that (1) 80 percent of the reported adverse events
suspected of involving supplements containing ephedrine alkaloids
were actually related to the consumption of dietary supplements,
(2) 80 percent of these supplements actually contained ephedrine
alkaloids, and (3) only 10 percent of adverse events related to
ephedrine alkaloids were actually reported. In similar fashion,
FDA disclosed how it calculated the estimated compliance costs of
the rule, including the one-time costs associated with relabeling
and reformulating the affected supplements. Explanation of
In relation to the second transparency criterion-explaining the
reasons Assumptions for choices made for key
assumptions and values-FDA's analysis was only partially
transparent. For most elements of the analysis, FDA identified the
underlying data sources used and the rationale for the assumptions
and conclusions reached by FDA's analysts and experts. The
proposed rule included a lengthy discussion about the scientific
data and studies FDA and Page 48 GAO/HEHS/GGD-99-90
FDA's Proposed Rule on Ephedrine Alkaloids Appendix IV FDA's
Analysis of Benefits and Costs its advisory committees reviewed in
developing the various provisions of the rule. In estimating the
effects of these provisions, FDA also identified the underlying
data sources and analysis in most cases. For example, in its
discussion on potential relabeling costs, FDA described how it
used data from a previous FDA rule on nutrition labeling of
dietary supplements to derive its estimate of the specific costs
for this rule. In describing how it estimated the effect of
restrictions on including other stimulants in supplements
containing ephedrine alkaloids, FDA discussed the results of a
clinical study on combinations of ephedrine and caffeine and the
results of its informal review of adverse event reports. However,
FDA did not fully disclose the underpinnings of all of its
assumptions and choices. In particular, although FDA identified
the values it used to estimate the number and severity of adverse
events for its calculation of benefits, its explanation of the
underlying data sources, analyses, and basis for the selected
values was not complete. In its analysis, FDA identified three
main areas of uncertainty in its estimation of the actual number
of adverse events believed to be attributable to use of products
containing ephedrine alkaloids. These areas were (1) the
proportion of reported adverse events that were actually related
to consumption of dietary supplements, (2) the proportion of
reported adverse events that involved supplements that actually
contained ephedrine alkaloids, and (3) the proportion of adverse
events that were actually reported. For each of these elements,
FDA assumed a particular value that drove up the estimated number
of actual adverse events and, as a direct consequence, the
estimated benefits of the proposed rule. However, FDA did not
always disclose why it selected those values or why it selected a
single value instead of a range of values that would have more
clearly represented the degree of uncertainty that FDA believed
was present. In relation to the first area of uncertainty, FDA
estimated that between 27 and 90 percent of the reported adverse
events were "probably" related to the consumption of dietary
supplements suspected of containing ephedrine alkaloids. However,
FDA did not clearly disclose the basis for estimating this range.
Within this range, FDA assumed that 80 percent of the reported
adverse events were actually related to the consumption of dietary
supplements. FDA did not disclose how it arrived at the choice of
80 percent as the "most likely" value. Furthermore, in calculating
the "range" of estimated benefits for the proposed rule, FDA used
only this point estimate, not the entire 27 to 90 percent range,
or a more narrowly focused range of values in calculating the
expected number of cases. Use Page 49 GAO/HEHS/GGD-
99-90 FDA's Proposed Rule on Ephedrine Alkaloids Appendix IV FDA's
Analysis of Benefits and Costs of a range of values for this
element would have more clearly represented the uncertainty
regarding the number of AERs actually related to the consumption
of dietary supplements.56 The second source of uncertainty that
FDA identified was the extent to which all AERs involved products
that actually contained ephedrine alkaloids. FDA estimated that
the proportion of reported adverse events associated with dietary
supplements containing ephedrine alkaloids was "probably" between
25 and 90 percent based on (1) the labeling of the products
involved, (2) FDA's own market study and laboratory analysis of
125 marketed products, and (3) the similarity of the reported
adverse events to the known effects of ephedrine alkaloids. Within
this range, FDA assumed that 80 percent of the reported adverse
events associated with the consumption of dietary supplements
involved supplements that contained ephedrine. FDA did not
disclose why it selected 80 percent as the "most likely" value.
Finally, as was the case in the first area of uncertainty, FDA
used only this 80 percent point estimate, not the full range or an
attenuated range of values, in calculating the expected number of
annual cases. FDA's third identified source of uncertainty was the
likelihood that all adverse events related to ephedrine alkaloids
were probably not being reported. FDA noted in the analysis that
typical reporting rates for passive reporting systems on adverse
events associated with pharmaceutical drugs are "generally assumed
to be on the order of 10 percent." However, FDA noted that
reporting rates might be higher than usual if, for example, the
potential health risks are widely publicized, or lower than normal
if consumers and physicians assume that dietary supplements are
incapable of producing adverse events. FDA said that "to
incorporate this uncertainty," it assumed that the reporting rate
for the adverse events related to ephedrine alkaloids was 10
percent. It was not clear how a single point estimate of 10
percent would "incorporate" the uncertainty that FDA identified
for this value. In response to our questions about these
uncertainties, FDA officials acknowledged that their analysis of
impacts was not as transparent as it should have been in
explaining how the agency arrived at some of the assumptions. The
Director of FDA's Market Studies Division, which prepared the
cost-benefit analysis, explained that the estimated 56FDA was
somewhat inconsistent in its use of point estimates versus ranges.
FDA used its full estimated range of adverse events in projecting
the results for other regulatory options, such as banning
supplements with ephedrine alkaloids. FDA officials said that they
chose to use the full ranges in discussing other options because
of the regulatory actions in those options. Page 50
GAO/HEHS/GGD-99-90 FDA's Proposed Rule on Ephedrine Alkaloids
Appendix IV FDA's Analysis of Benefits and Costs proportions and
the selected values represented the best professional judgment of
FDA's analysts and experts. He noted that the upper-bound estimate
of 90 percent for the first two areas of uncertainty represented
what the analysts believed was a reasonable choice for the
"absolute most it could be." He said that the lower-bound estimate
of 27 percent for the first category was derived from the cases in
the AERs in which people who stopped using products suspected of
containing ephedrine alkaloids no longer experienced the effects
associated with the adverse event. Similarly, FDA's figure of 25
percent as the most reasonable low-end estimate for the second
category was based on FDA's review of samples of products
mentioned in the AERs. He said that the choice of 80 percent as
the most likely value for both of these categories was a
determination made by FDA's scientific experts, given all the
available data and scientific evidence they had reviewed, not just
the AERs. The director and other FDA officials indicated that FDA
had much more confidence in the estimate that only 10 percent of
actual adverse events were reported. They said that, compared to
the other two sources of uncertainty, FDA knows more about AEMS
and is also familiar with studies of other passive reporting
systems used by FDA and other federal agencies. The officials
noted that, for other systems, they have done reviews for specific
sites (for example, going to hospitals to check records and do
testing) and found that reporting rates were about 10 percent. The
director said that FDA did not provide a range for this category
in the proposed rule because FDA was confident in the point
estimate and because it did not have any way-other than through an
"artificial exercise"-to come up with a range of possible values.
Implications of Alternative FDA's cost-benefit analysis was
also only somewhat transparent with Assumptions
regard to the third of our transparency criteria-assessment of the
effects of plausible alternative assumptions and choices on the
results of the analysis. The degree of transparency in this area
and the sensitivity of FDA's results to alternative assumptions
are most clearly illustrated in the agency's estimate of expected
benefits. FDA estimated that the benefits of the proposed rule
would be between $240 million and $670 million per year. That
estimate was driven by three factors: (1) FDA's estimate of the
number of adverse events each year (1,110), (2) FDA's estimate of
the degree to which the proposed rule would reduce these events
(35 percent to 100 percent across all types of proposed actions),
and (3) the values FDA assigned to the estimated risk Page 51
GAO/HEHS/GGD-99-90 FDA's Proposed Rule on Ephedrine Alkaloids
Appendix IV FDA's Analysis of Benefits and Costs reduction per
case (for example, $5 million per death avoided).57 Changes in any
of these values could have dramatically changed FDA's estimates of
the proposed rule's benefits. By using the 35 to 100 percent range
for the second factor (and even wider ranges for some of the
individual actions), FDA illustrated the sensitivity of the
analysis to changes in particular values. This second factor was
the only component in FDA's calculations that generated a "range"
in the estimated reduction in annual adverse events (390 to 1,110)
and, in turn, the size of the expected benefits ($240 million to
$670 million).58 On the other hand, FDA's estimate of the 1,110
estimated adverse events each year (factor 1) was based on an
average of 174 adverse events reported per year and the three
assumptions about adverse events-that 80 percent of the AERs
involved dietary supplements, that 80 percent of the supplements
contained ephedrine alkaloids, and that only 10 percent of all
adverse events related to supplements containing ephedrine
alkaloids were reported. FDA's use of these point estimates
(instead of ranges) does not reflect the sensitivity of the final
benefit estimates to these assumptions. As illustrated in figure
IV.1, the results of FDA's estimation of expected benefits are,
therefore, very sensitive to changes in the underlying
assumptions. If FDA had used other plausible assumptions, based on
the information it presented in the proposed rule, the ranges of
expected benefits would have varied noticeably from the values FDA
presented in its proposal. 57In its proposed rule, FDA did not
report or round its estimated values consistently, so slightly
different numbers for the same element appear within the proposed
rule. For this report, we are using the values reported in table 6
of FDA's proposed rule, such as 1,110 for the estimated annual
number of adverse events. 58Similarly, FDA's $3 million to $80
million range in the estimated cost of the proposed rule also
reflects the sensitivity of the agency's assumptions. Page 52
GAO/HEHS/GGD-99-90 FDA's Proposed Rule on Ephedrine Alkaloids
Appendix IV FDA's Analysis of Benefits and Costs Figure IV.1:
Estimation of Benefits Is Sensitive to Changes in Underlying
Assumptions Estimated Annual
Benefits (Dollars in Millions) 1400
1343 1200 1000 850 800 670 600 470 400
336 240 200 118 25 0 (1) FDA's (2) Using Full (3)
Assuming (4) Assuming Reported Range of FDA 20% of
Cases 5% of Cases Estimates Assumptions Are
Reported Are Reported Legend High Bound Low Bound Note: The
data for scenario 1 are from FDA's reported estimates in table 6
of the proposed rule. Scenario 2 was calculated using the full
range of assumptions FDA presented in the narrative of its
proposed rule regarding key assumptions about the number of
adverse events each year. Scenarios 3 and 4 were calculated
changing only FDA's selected assumption regarding the reporting
rate for adverse events. All other assumptions were held constant.
Source: Scenario 1 data are taken from table 6 of FDA's proposed
rule. Data for other scenarios were GAO calculations using
information in FDA's proposal. Given the uncertainty surrounding
most of the key data and assumptions used in FDA's analysis,
similar types of sensitivity analyses could be done using
alternative choices for other elements in the analysis, such as
the following: Page 53 GAO/HEHS/GGD-99-90
FDA's Proposed Rule on Ephedrine Alkaloids Appendix IV FDA's
Analysis of Benefits and Costs * FDA used published estimates of
the value that consumers place on reducing certain types of risk
to estimate the monetary value benefits associated with the
proposed rule. However, other published estimates for these values
could also have been used. For example, FDA assumed in its
analysis that the value of avoiding a death was $5 million. We
have reported that the Environmental Protection Agency's (EPA)
assumptions regarding the value of avoiding a death have ranged
from $1.6 million to $12 million.59 * FDA's estimates of the
benefits of the rule were also a function of the distribution of
the types of adverse events (for example, deaths, cardiovascular
system events, and neurological system events) reported through
the underlying AERs. Given that there is sufficient uncertainty
regarding the reports that different distributions of the numbers
of events are plausible, FDA's analysis could have reflected these
alternative distributions and the effect they would have had on
the benefit estimates. * FDA's estimates of the costs of the
proposed rule were based, in part, on assumptions regarding the
number of dietary supplement products sold and the number of
businesses involved in the dietary supplement market. As discussed
in appendix V, "Regulatory Flexibility Act Requirements," there is
considerable uncertainty about the actual number of businesses and
products involved in the market for dietary supplements containing
ephedrine alkaloids. As FDA officials noted in our meetings with
them, there are limits to the amount of effort that can be
expended on an analysis, and they attempt to reach a balance in
conducting a reasonable effort. They pointed out that the proposed
rule was long and that they had struggled to keep the length of
the narrative under control. However, the OIRA guidance recommends
that agencies carry out sensitivity analyses over the full range
of plausible values of key parameters, particularly when there are
several easily identifiable critical assumptions in the analysis
and information is inadequate to carry out a more formal
probabilistic simulation for assessment of risks. In this respect,
FDA's focus on presenting the agency's "best estimate" does not
seem to fully reflect the uncertainty in the underlying data and
analysis or the expectations set by the OIRA guidance. In
explaining why FDA did not do a sensitivity analysis for this
proposed rule, the Director of the Market Studies Division said
that, at this point in time, risk assessors are ahead of cost-
benefit analysts in doing sensitivity analyses. He did note that,
while FDA did not do a sensitivity analysis for 59Regulatory
Reform: Agencies Could Improve Development, Documentation, and
Clarity of Regulatory Economic Analyses (GAO/RCED-98-142, May 26,
1998), and Air Pollution: Information Contained in EPA's
Regulatory Impact Analyses Can Be Made Clearer (GAO/RCED-97-38,
Apr. 14, 1997). Page 54 GAO/HEHS/GGD-99-90 FDA's
Proposed Rule on Ephedrine Alkaloids Appendix IV FDA's Analysis of
Benefits and Costs this particular proposed rule, it would
probably do more sensitivity analyses in the near future. Page 55
GAO/HEHS/GGD-99-90 FDA's Proposed Rule on Ephedrine Alkaloids
Appendix V Regulatory Flexibility Act Requirements The Regulatory
Flexibility Act requires agencies to consider the effects of their
rules on small entities. FDA determined that its proposed rule on
dietary supplements containing ephedrine alkaloids would have a
significant economic impact on a substantial number of small
entities. Therefore, FDA prepared an initial regulatory
flexibility analysis to identify the potential impacts of the
proposed rule and alternative actions on affected small
businesses. Also, as required by RFA, FDA provided opportunities
for small business participation in the rulemaking process.
However, while FDA addressed the basic requirements of the act,
SBA's Office of Advocacy criticized the quality of FDA's
regulatory flexibility analysis. For example, the office contended
that FDA underestimated the number of affected small businesses
and the impacts of the proposed actions by not considering the
large numbers of independent distributors of dietary supplements
in its analysis. Regulatory Flexibility During the past
20 years, the Congress has enacted a number of statutes Act
Requires Agencies designed to improve the federal rulemaking
process, one of which is the Regulatory Flexibility Act of 1980,
as amended (5 U.S.C. 601-612). to Consider Impact of
Congress enacted RFA in response to concerns about the effect that
federal Regulation on Small regulations can have on
small entities. According to SBA's Office of Advocacy, the major
goals of the act are to (1) increase agency awareness Businesses
and understanding of the impact of their regulations on small
business, (2) require that agencies communicate and explain their
findings to the public, and (3) encourage agencies to use
flexibility and to provide regulatory relief to small entities.
RFA mandates certain actions on the part of agencies during the
rulemaking process. For example, before publishing a proposed rule
for which a notice of proposed rulemaking is required, sections
603 and 605(b) of the act state that a federal agency must prepare
and make available for public comment an initial regulatory
flexibility analysis that describes the anticipated effects of the
proposed rule on small entities, unless the head of the agency
certifies that the proposed rule will not have a "significant
economic impact on a substantial number of small entities."
Specifically, the act states that an agency's initial analysis
must contain (1) a description of why the agency is considering
the action; (2) the objectives and legal basis for the proposed
rule; (3) a description and, where feasible, an estimate of the
number of small entities to which the proposed rule will apply;
(4) a description of the projected reporting, recordkeeping, and
other compliance requirements of the proposed rule; and (5) a
description of all federal rules that may duplicate, overlap, or
Page 56 GAO/HEHS/GGD-99-90 FDA's Proposed Rule on
Ephedrine Alkaloids Appendix V Regulatory Flexibility Act
Requirements conflict with the proposed rule. The act also states
that the initial analysis shall discuss significant alternatives
for small entities that accomplish the objectives of applicable
statutes, such as different compliance or reporting requirements,
clarified or consolidated compliance and reporting requirements,
and an exemption from coverage by the rule. When a rule is
promulgated that will have a significant economic impact on a
substantial number of small entities, section 609(a) of RFA
requires agencies to ensure that small entities have been given an
opportunity to participate in the rulemaking process through the
"reasonable use" of outreach techniques.60 The act delineates a
number of specific types of outreach techniques agencies could
use, including (1) publishing the notice of proposed rulemaking in
publications likely to be obtained by small entities, (2) directly
notifying interested small entities, and (3) conducting open
conferences or public hearings concerning the rule. Section 612 of
the act requires the SBA Chief Counsel for Advocacy to monitor
agencies' compliance with RFA and authorizes the Chief Counsel to
appear as amicus curiae ("friend of the court") in court
proceedings to review rules under the act. FDA's Actions Met
FDA determined that the proposed rule would have a significant
economic Basic RFA impact on a substantial number
of small entities. The agency concluded that a total of 80 small
manufacturers and distributors would be affected Requirements but
by the proposed rule using (1) information in previous studies
that Attracted Criticism indicated that 95 percent of all
dietary supplement manufacturers were small businesses and (2)
information from two market surveys that identified 85
manufacturers and distributors of dietary supplements suspected of
containing ephedrine alkaloids. This determination triggered the
requirement in sections 603 and 609 of the act, respectively, that
FDA conduct an initial regulatory flexibility analysis and conduct
outreach efforts. Regulatory Flexibility FDA's proposed rule
addressed the basic elements that RFA requires Analysis
Requirements agencies to include in an initial regulatory
flexibility analysis. The rule described the reasons the agency
was considering the action (market failure), stated the purpose
and legal basis of the rule, described and 60FDA is not subject to
other RFA requirements regarding small business participation,
such as the requirement to convene small business advocacy review
panels that the act, as amended by the Small Business Regulatory
Enforcement Fairness Act, places on EPA and the Occupational
Safety and Health Administration. For a discussion of these
panels, see Regulatory Reform: Implementation of the Small
Business Advocacy Review Panel Requirements (GAO/GGD-98-36, Mar.
18, 1998). Page 57 GAO/HEHS/GGD-99-90 FDA's
Proposed Rule on Ephedrine Alkaloids Appendix V Regulatory
Flexibility Act Requirements provided an estimate of the number of
small entities to which FDA believed the rule would apply, and
described the compliance requirements. In addition to describing
direct compliance costs of between $3 million and $80 million, FDA
also said that the proposed rule could have significant
distributive effects in the form of reduced sales of as much as
$230 million a year. FDA explicitly stated that costs and sales
reductions of this magnitude might threaten the viability of many
firms in this industry. FDA also discussed significant
alternatives to the proposed rule in the rule's regulatory
flexibility analysis section, noting that most of the regulatory
alternatives discussed in the cost-benefit analysis section would
reduce the impact of this rule on small businesses. For example,
FDA noted that taking no action or simply generating additional
information would reduce the impact on small businesses to zero.
However, FDA said that raising the proposed potency limit would
have the same impact as the proposed rule and banning dietary
supplements would have the greatest negative impact. The industry
representatives we met with confirmed that manufacturing and
labeling of these products are generally limited to a relatively
small number of manufacturing firms. Such firms would bear almost
all of the expenditures required for compliance with the proposed
rule. However, the executive director of one of the major trade
associations we contacted pointed out that the proposed actions
may also create some compliance costs for downstream distributors
of the many different brands of products marketed. He said that
these distributors might have to revise their marketing and
promotional materials (for example, checking and revising claims
and other information presented in the materials) to comply with
the proposed rule. It is not clear whether FDA's estimated costs
for relabeling also address this cost element. In comments
submitted on the proposed rule, SBA's Office of Advocacy was
critical of FDA's initial regulatory flexibility analysis, stating
that the underlying scientific data and analysis did not
adequately support the proposed actions and the proposed rule did
not provide enough information for someone to "meaningfully
comment" on the proposed rule or alternatives. SBA also criticized
FDA's analysis of the impacts on small businesses, saying it had
underestimated the number of entities affected because it focused
solely on the manufacturers of dietary supplements suspected of
containing ephedrine alkaloids and did not take independent
distributors into account. The Office of Advocacy noted that "many
ephedra products are sold by tens of thousands of home-based
distributorships that are part of multilevel marketing companies.
Many of Page 58 GAO/HEHS/GGD-99-90 FDA's Proposed
Rule on Ephedrine Alkaloids Appendix V Regulatory Flexibility Act
Requirements these businesses, although part of a larger parent
company, may nevertheless be independently owned and operated and
considered to be `small business concerns' under the Small
Business Act." The office pointed out the importance of good
information about the regulated industry, stating, In order to
determine the impact of any regulation, an agency must make a
reasonable effort to identify the type and number of entities
likely to be affected by the regulation. This process of learning
about the regulated industry not only helps the agency determine
whether to certify a rule for regulatory flexibility purposes, it
also helps the agency develop an analysis of impacts and choose
appropriate regulatory alternatives that minimize economic burden.
In response to SBA's criticism, FDA officials told us that the
agency need not consider the impact of its proposed rule on
distributors of ephedrine alkaloids. The FDA officials cited
several court decisions to support the proposition that, under
RFA, an agency is under no obligation to conduct a small entity
impact analysis of effects on entities that the agency does not
regulate-that is, distributors.61 None of the major parties-FDA,
SBA's Office of Advocacy, or industry associations-has information
on the actual number of entities involved in the market for
dietary supplements containing ephedrine alkaloids. Each
acknowledge this as a limitation in attempts to analyze the
effects of the proposed rule. FDA officials told us that the
agency did not do a true market study for the proposed rule. The
market review that FDA used in its analysis was done primarily for
the purposes of informing the agency about supplements that might
contain ephedrine alkaloids. SBA's Office of Advocacy recommended
that FDA develop an outreach strategy to obtain more reliable
industry data, and officials of both the Office of Advocacy and
FDA noted that they have met on this issue since the comment
period on the proposed rule ended. In addition, FDA officials said
that they have contracted for a market study of the dietary
supplement industry that should provide better information for
future FDA analyses. Outreach Efforts With regard to the
requirement in section 609(a) of the act requiring FDA to engage
in outreach efforts, the regulatory docket shows that FDA provided
opportunities during the development of the proposed rule for
participation by affected businesses. For example, in addition to
publishing the proposed rule, FDA published public notices and
held public 61For example, United Distribution Cos. v. FERC, 88
F.3d 1105, 1170 (D.C. Cir. 1996). Page 59
GAO/HEHS/GGD-99-90 FDA's Proposed Rule on Ephedrine Alkaloids
Appendix V Regulatory Flexibility Act Requirements meetings,
during which industry representatives participated and provided
testimony, and collected written comments on this issue.
Representatives of industry trade associations also told us that
FDA had placed notices about the proposed rule in the trade
literature. Page 60 GAO/HEHS/GGD-99-90 FDA's
Proposed Rule on Ephedrine Alkaloids Appendix VI Unfunded Mandates
Reform Act Requirements The Unfunded Mandates Reform Act of 1995
imposes specific analytical requirements for certain rules that
include a federal mandate that may result in the expenditure of
$100 million or more by the private sector in any 1 year. FDA did
not explicitly address UMRA in its proposed rule. However, FDA's
economic analysis indicated that the direct expenditures imposed
by this rule on the private sector would not rise to the level of
$100 million and, thus, trigger the required additional analysis.
Furthermore, even if the proposed rule did trigger UMRA's
analytical requirements, FDA appears to have already satisfied
most, if not all, of those requirements. UMRA Requirements
UMRA requires federal agencies to take certain actions during the
Apply Under Certain rulemaking process for certain
types of rules. For example, section 202 of UMRA requires agencies
to provide a "written statement," including a Circumstances
qualitative and quantitative assessment of the anticipated costs
and benefits for any proposed rule that includes a federal mandate
that may result in the expenditure of $100 million or more by
state, local, and tribal governments, in the aggregate, or by the
private sector, in any 1 year.62 Section 205 of the act states
that, before promulgating any rule for which a written statement
is required under section 202, agencies must identify and consider
a reasonable number of regulatory alternatives and select the one
that is the least costly, most cost-effective, or least burdensome
alternative that achieves the objectives of the proposed rule. If
the agency does not select the least costly, most cost-effective,
or least burdensome option, and if the requirements of section 205
are not inconsistent with law, UMRA requires the agency head to
publish with the final rule an explanation of why the least
costly, most cost-effective, or least burdensome method was not
adopted. FDA's Proposed Rule FDA's proposed rule did
not mention UMRA. FDA officials said they did not Does Not Appear
to include a specific UMRA section because the
proposed rule was quite lengthy and they wanted to avoid adding
any additional boilerplate. Our Trigger Further Action previous
work on federal agency rulemaking indicated that even when Under
UMRA their proposed rules do not trigger
UMRA's analytical requirements, and although they are not required
to do so, agencies sometimes include a 62Even if an agency's rule
required $100 million in expenditures by the public or private
sectors, UMRA's written statement requirements would only apply if
the rule also imposed an "enforceable duty" that was not a
condition of federal financial assistance or that did not arise
from participation in a voluntary federal program. See Unfunded
Mandates: Reform Act Has Had Little Effect on Agencies' Rulemaking
Actions (GAO/GGD-98-30, Feb. 4, 1998). Page 61
GAO/HEHS/GGD-99-90 FDA's Proposed Rule on Ephedrine Alkaloids
Appendix VI Unfunded Mandates Reform Act Requirements brief
statement to that effect explaining why they did not believe those
requirements were applicable.63 Although FDA did not specifically
mention UMRA, the agency's analysis of the costs associated with
the proposed rule indicated that the rule would not result in
expenditures of $100 million or more in any 1 year by the private
sector. FDA estimated that the total compliance costs for the
proposed action would be between $3 million and $80 million, with
at most $70 million of those costs in the first year of
implementation. These costs are primarily associated with
relabeling and reformulating the affected supplements and appear
to be expenditures covered by UMRA but do not rise to the $100-
million threshold. Although FDA also estimated that the rule might
result in lost sales for the dietary supplement industry of as
much as $230 million per year, lost sales could not be considered
direct expenditures by the private sector and, therefore, could
not be used to trigger UMRA's analytical requirements. SBA's
Office of Advocacy contended that FDA overlooked the UMRA
requirements. In the written comments submitted on FDA's proposed
rule, the office stated, "If the agency had performed an adequate
[cost-benefit] analysis, it would have been apparent that the
economic impact of the instant rule would impose in excess of $100
million in costs to the industry." Officials from the Office of
Advocacy told us that, if FDA properly estimated the number of
affected businesses and the costs that would be imposed on those
entities, FDA's estimate of expenditures by the private sector
might increase to the level that would trigger UMRA's
requirements. However, the Office of Advocacy could not provide
any data showing that FDA had underestimated the number of
manufacturers that would be affected by the relabeling and
reformulation requirements in the rule or that FDA's cost
estimates for those manufacturers were in error. Even if the
expenditures associated with the proposed rule had triggered
UMRA's analytical requirements, FDA appears to have satisfied most
if not all of those requirements. For example, FDA identified the
provision of federal law under which the rule was being
promulgated, quantitatively and qualitatively assessed the
anticipated costs and benefits of the rule, and identified and
considered a number of regulatory alternatives. Although FDA said
that most of the regulatory alternatives would reduce the impact
of the rule on businesses, FDA also indicated that these other
options would not be as effective as the proposed rule. 63GGD-98-
30 and Federal Rulemaking: Agencies Often Published Final Actions
Without Proposed Rules (GAO/GGD-98-126, Aug. 31, 1998). Page 62
GAO/HEHS/GGD-99-90 FDA's Proposed Rule on Ephedrine Alkaloids
Appendix VII Comments From the Department of Health and Human
Services Note: GAO comments supplementing those in the report text
appear at the end of this appendix. Page 63 GAO/HEHS/GGD-99-
90 FDA's Proposed Rule on Ephedrine Alkaloids Appendix VII
Comments From the Department of Health and Human Services See
comment 1. Now on p. 23. Now on pp. 23-24. See comment 2. Now on
pp. 3 and 8. Now on pp. 8-9. See comment 3. Now on pp. 3, 8, 10,
and 12. Page 64 GAO/HEHS/GGD-99-90 FDA's Proposed
Rule on Ephedrine Alkaloids Appendix VII Comments From the
Department of Health and Human Services See comment 4. Now on pp.
8, 11, 13, and 24. See comment 5. See comment 6. Page 65
GAO/HEHS/GGD-99-90 FDA's Proposed Rule on Ephedrine Alkaloids
Appendix VII Comments From the Department of Health and Human
Services Page 66 GAO/HEHS/GGD-99-90 FDA's Proposed
Rule on Ephedrine Alkaloids Appendix VII Comments From the
Department of Health and Human Services Page 67
GAO/HEHS/GGD-99-90 FDA's Proposed Rule on Ephedrine Alkaloids
Appendix VII Comments From the Department of Health and Human
Services Page 68 GAO/HEHS/GGD-99-90 FDA's Proposed
Rule on Ephedrine Alkaloids Appendix VII Comments From the
Department of Health and Human Services Page 69
GAO/HEHS/GGD-99-90 FDA's Proposed Rule on Ephedrine Alkaloids
Appendix VII Comments From the Department of Health and Human
Services GAO Comments 1. We have added language to the Results-
in-Brief section of the report indicating that the number of AERs
relating to dietary supplements containing ephedrine alkaloids
warranted FDA's attention and consideration of steps to address
safety concerns. 2. Throughout our report, we note that FDA used
information from scientific literature to conclude that ingestion
of ephedrine alkaloids can result in serious adverse events for
some individuals. Our review of the scientific literature found
case reports that suggest that ephedrine alkaloids could increase
blood pressure in persons with normal and high blood pressure;
predispose certain individuals to tachycardia (rapid heart rate);
and cause cardiomyopathy (disease of the heart muscle), stroke, or
myocardial necrosis (death of cells in the heart). We also found
descriptions of adverse events associated with ephedrine alkaloids
that affected the central nervous system, such as mania, paranoid
psychoses, and seizures. However, FDA relied on AERs in
establishing key components of its proposed rule-the serving and
duration of use limits. We have clarified some language in the
report to better reflect our concerns. 3. FDA highlighted in the
proposed rule that clinical trials examining the efficacy of
ephedrine to treat obesity had shown adverse events occurring at
levels approaching 20 mg per serving. However, these trials did
not provide information relating to adverse events below the 20 mg
level. Thus, FDA did rely solely on results of tests performed on
a small number of products implicated in 13 AERs to conclude that
an 8 mg per serving level would be appropriate. 4. We note in our
report that FDA did not perform an analysis on the AERs to
determine which, if any, of the reported events were caused by
dietary supplements containing ephedrine alkaloids. Further, we
outlined several weaknesses inherent in any passive surveillance
system, such as adverse event reporting. We continue to hold the
view that without a causal link between the AERs and the ingestion
of products containing ephedrine alkaloids, the exclusive use of
AERs to support a specific dosing regimen is questionable. 5. FDA
noted in its comments that we suggested that FDA's use of AERs in
this rulemaking in some respects, such as establishing an
adulteration level, was unprecedented. For this report, we
examined whether the use of AERs for the proposed rule on dietary
supplements containing ephedrine alkaloids was consistent with the
use of AERs in prior rulemaking by FDA's Center for Food Safety
and Applied Nutrition. We found that AERs had Page 70
GAO/HEHS/GGD-99-90 FDA's Proposed Rule on Ephedrine Alkaloids
Appendix VII Comments From the Department of Health and Human
Services been used in prior rulemaking, but they were used
differently in this proposed rule when the agency relied solely on
these reports to set a specific dosing level. However, we did not
take a position on the appropriateness of the general use of AERs
in rulemaking. 6. While we do not disagree with FDA's chronology
of events detailed in pages 2-5 of its letter, we do not believe
it is fully reflective of the role of the AERs in FDA's proposed
action. In addition to using the AERs to signal a potential
problem with products containing ephedrine alkaloids and to
identify types of adverse events associated with such products,
FDA also used the AERs as the key element in establishing the
proposed dosing regimen. Page 71 GAO/HEHS/GGD-99-90
FDA's Proposed Rule on Ephedrine Alkaloids Appendix VIII Comments
From the Small Business Administration Note: GAO comments
supplementing those in the report text appear at the end of this
appendix. Page 72 GAO/HEHS/GGD-99-90 FDA's Proposed Rule on
Ephedrine Alkaloids Appendix VIII Comments From the Small Business
Administration See comment 1. Page 73 GAO/HEHS/GGD-
99-90 FDA's Proposed Rule on Ephedrine Alkaloids Appendix VIII
Comments From the Small Business Administration Page 74
GAO/HEHS/GGD-99-90 FDA's Proposed Rule on Ephedrine Alkaloids
Appendix VIII Comments From the Small Business Administration See
comment 2. Page 75 GAO/HEHS/GGD-99-90 FDA's Proposed
Rule on Ephedrine Alkaloids Appendix VIII Comments From the Small
Business Administration Page 76 GAO/HEHS/GGD-99-90
FDA's Proposed Rule on Ephedrine Alkaloids Appendix VIII Comments
From the Small Business Administration GAO Comments 1.
Although we did raise several concerns about specific parts of
FDA's analysis and recommended certain revisions to any final
rulemaking by FDA, we did not conclude that FDA failed to prepare
an adequate analysis. RFA contains no standards or criteria that
define an "adequate" analysis. The provisions of the act specify
certain actions that an agency must take and certain elements that
must be included in a regulatory flexibility analysis. Beyond
those broad parameters, the act does not establish specific
measures of quality or adequacy for an agency's analysis. Thus, we
have no basis and are not in a position to make such a
determination regarding the quality of FDA's analysis. Such a
determination might be made if a specific rulemaking is judicially
reviewed.64 We did recognize, however, that FDA's analysis in the
proposed rule contained the primary elements required under E.O.
12866, related OIRA guidance, and RFA. 2. We disagree with the
Office of Advocacy's comments on direct versus indirect effects
for three reasons. * First, prior case law indicates that agencies
need not take the indirect effects of their rules into account
when conducting a regulatory flexibility analysis. Most recently,
in a May 14, 1999, decision, the U.S. Court of Appeals for the
District of Columbia Circuit said, "we have consistently
interpreted the RFA . . . to impose no obligation upon an agency
`to conduct a small entity impact analysis of effects on entities
which it does not regulate.'"65 The court also quoted Mid-Tex in
noting, "Congress did not intend to require that every agency
consider every indirect effect that any regulation might have on
small businesses in any stratum of the national economy."66 The
court also rejected the petitioners' contention that it must defer
to the interpretations of SBA's Chief Counsel for Advocacy
regarding RFA, pointing out that SBA "neither administers nor has
any policymaking role under RFA; at most its role is advisory . .
. . Therefore, we do not defer to the SBA's interpretation of the
RFA."67 * Second, we do not believe that it is logical to conclude
that the absence of case law comparable to Mid-Tex with respect to
direct and indirect effects 64The Small Business Regulatory
Enforcement Fairness Act of 1996 amended RFA to provide for
judicial review of an agency's compliance with RFA. 65American
Trucking Associations, Inc., et al, v. United States Environmental
Protection Agency, No. 97-1440 U.S. App. LEXIS 9064, at *40 (D.C.
Cir. May 14, 1999), citing Motor & Equip. Mfrs. Ass'n. v. Nichols,
142 F.3d 449, 467 & n.18 (1988). 66American Trucking, 1999 U.S.
App. LEXIS 9064, at *41. 67American Trucking, 1999 U.S. App. LEXIS
9064, at *42. The court referred to 5 U.S.C. sections 601(3),
602(b), 603(a), 605(b), 609(b)(1), and 612 and Scheduled Airlines
Traffic Offices, Inc. v. Department of Defense, 87 F.3d 1356, 1361
(D.C. Cir. 1996) (no deference owed to agency interpretation of
statute it does not administer). Page 77
GAO/HEHS/GGD-99-90 FDA's Proposed Rule on Ephedrine Alkaloids
Appendix VIII Comments From the Small Business Administration as
they apply to E.O. 12866 means that FDA should have prepared an
analysis of the rule's impact on distributors. The absence of case
law says nothing about FDA's analytical obligations under the
executive order. * Third, we believe that, by estimating total
lost sales to the industry if the proposed rule or other
regulatory alternatives were implemented, FDA identified the
potential magnitude of impacts on distributors in both its cost-
benefit analysis and regulatory flexibility analysis. FDA
identified such lost sales as a distributive effect, rather than a
true social cost, because the loss only represents a transfer of
resources within society (more for consumers and less for the
dietary supplement industry). In addition, the total loss to the
industry is the total loss to the industry; it should not be
double counted-once for manufacturers and again for distributors-
in a cost-benefit analysis. Finally, FDA explicitly acknowledged
in its proposed rule that such effects would be significant to
affected parties and could threaten the viability of many firms in
this industry. Page 78 GAO/HEHS/GGD-99-90 FDA's
Proposed Rule on Ephedrine Alkaloids Appendix IX GAO Contacts and
Staff Acknowledgments GAO Contacts William Scanlon, (202)
512-7114 Nye Stevens, (202) 512-8676 Staff In
addition to those named above, Tim Bober, Curtis Copeland, Marcia
Acknowledgments Crosse, Carolyn L. Feis, and Kurt Kroemer made
key contributions to this report. (108392, 410378) Page 79
GAO/HEHS/GGD-99-90 FDA's Proposed Rule on Ephedrine Alkaloids
Ordering Information The first copy of each GAO report and
testimony is free. Additional copies are $2 each. Orders should be
sent to the following address, accompanied by a check or money
order made out to the Superintendent of Documents, when necessary.
VISA and MasterCard credit cards are accepted, also. Orders for
100 or more copies to be mailed to a single address are discounted
25 percent. Orders by mail: U.S. General Accounting Office P.O.
Box 37050 Washington, DC 20013 or visit: Room 1100 700 4th St. NW
(corner of 4th and G Sts. NW) U.S. General Accounting Office
Washington, DC Orders may also be placed by calling (202) 512-6000
or by using fax number (202) 512-6061, or TDD (202) 512-2537. Each
day, GAO issues a list of newly available reports and testimony.
To receive facsimile copies of the daily list or any list from the
past 30 days, please call (202) 512-6000 using a touchtone phone.
A recorded menu will provide information on how to obtain these
lists. For information on how to access GAO reports on the
INTERNET, send an e-mail message with "info" in the body to:
[email protected] or visit GAO's World Wide Web Home Page at:
http://www.gao.gov PRINTED ON RECYCLED PAPER United States
General Accounting Office Bulk Rate Washington, D.C.
20548-0001 Postage & Fees Paid GAO Permit No. G100 Official
Business Penalty for Private Use $300 Address Correction Requested
*** End of document. ***