Adverse Drug Events: The Magnitude of Health Risk Is Uncertain Because of
Limited Incidence Data (Letter Report, 01/18/2000, GAO/HEHS-00-21).

Pursuant to a congressional request, GAO summarized from available
research what is known about adverse drug events (ADE), focusing on the:
(1) different types and causes of ADEs; (2) evidence on the overall
incidence and cost of ADEs in the United States; and (3) measures that
have been proposed to reduce the number and severity of ADEs.

GAO noted that: (1) ADEs arise either from adverse drug reactions (ADR),
which are previously known or newly detected side effects of drugs, or
from medication errors committed by health care professionals or the
patients themselves; (2) two factors that can increase the risk of a
patient's suffering from an ADR are illness severity and intensity of
treatment, including taking several drugs simultaneously; (3) studies of
several hospital populations found that most medication errors did not
cause ADEs, but because so many drug doses were given, an estimated
quarter to half of all ADEs among the hospital patients resulted from
medication errors; (4) other studies found medication errors were most
often the result of physicians' prescribing errors and nurses' errors in
administering drugs; (5) certain classes of drugs have consistently been
associated with medication errors, including analgesics, antibiotics,
psychotropics, and cardiovascular drugs; (6) although it is clear a wide
range of commonly used drugs cause ADEs, little is known about their
frequency; (7) data routinely collected on ADEs during clinical trials
or after drugs are marketed are intended to identify the ADEs that are
associated with particular drugs and do not focus on their frequency;
(8) information on the overall incidence of ADEs from all drugs is
limited to a few research studies that typically examined the experience
of patients in one or two specific institutions leaving the overall
incidence of ADEs in outpatient care largely unexplored; (9) the most
broadly based information on ADE incidence comes from two studies
drawing on statewide samples of hospital patients; (10) these studies
applied a particularly restrictive definition of ADE in finding that
ADEs occurred at a rate of 0.56 for every 100 patients in Colorado and
Utah, 0.72 in New York; (11) although studies have estimated the overall
rate of fatalities from ADEs and the total costs of treating ADEs, their
estimates are open to question because of the limited underlying data on
overall incidence available to support them; (12) greater understanding
of certain factors that affect the likelihood of ADEs has led
researchers and patient safety advocates to suggest a range of measures
to decrease their number and severity; and (13) proposals for reducing
ADRs include improving communication between patients and physicians
about the risks and benefits of medications, and expanding and
accelerating research on the safety of marketed drugs to more quickly
detect previously unknown ADRs and determine the risk factors that
increase their likelihood.

--------------------------- Indexing Terms -----------------------------

 REPORTNUM:  HEHS-00-21
     TITLE:  Adverse Drug Events: The Magnitude of Health Risk Is
	     Uncertain Because of Limited Incidence Data
      DATE:  01/18/2000
   SUBJECT:  Health services administration
	     Medical research
	     Health care services
	     Drugs
	     Safety regulation
	     Health hazards
	     Physicians
IDENTIFIER:  FDA Adverse Drug Reaction Reporting System
	     Utah
	     New York
	     Colorado

******************************************************************
** This file contains an ASCII representation of the text of a  **
** GAO report.  Delineations within the text indicating chapter **
** titles, headings, and bullets are preserved.  Major          **
** divisions and subdivisions of the text, such as Chapters,    **
** Sections, and Appendixes, are identified by double and       **
** single lines.  The numbers on the right end of these lines   **
** indicate the position of each of the subsections in the      **
** document outline.  These numbers do NOT correspond with the  **
** page numbers of the printed product.                         **
**                                                              **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced.  Tables are included, but    **
** may not resemble those in the printed version.               **
**                                                              **
** Please see the PDF (Portable Document Format) file, when     **
** available, for a complete electronic file of the printed     **
** document's contents.                                         **
**                                                              **
** A printed copy of this report may be obtained from the GAO   **
** Document Distribution Center.  For further details, please   **
** send an e-mail message to:                                   **
**                                                              **
**                                            **
**                                                              **
** with the message 'info' in the body.                         **
******************************************************************

Cover
================================================================ COVER

Report to Congressional Requesters

January 2000

ADVERSE DRUG EVENTS - THE
MAGNITUDE OF HEALTH RISK IS
UNCERTAIN BECAUSE OF LIMITED
INCIDENCE DATA

GAO/HEHS-00-21

Adverse Drug Events

(108390)

Abbreviations
=============================================================== ABBREV

  ADE - Adverse drug event
  ADR - Adverse drug reaction
  AERS - Adverse Event Reporting System
  FDA - Food and Drug Administration
  HMPS - Harvard Medical Practice Study
  ICU - Intensive care unit
  NSAID - Nonsteroidal anti-inflammatory drug
  WHO - World Health Organization

Letter
=============================================================== LETTER

B-281822

January 18, 2000

Congressional Requesters

In 1998, about 2.7 billion prescriptions were filled in the United
States.  Prescription drugs have great clinical benefits, but they
also have risks.  Although most health problems associated with the
use of pharmaceuticals are relatively minor, serious adverse drug
events (ADE) that lead to hospitalization, disability, or death do
occur.  Because exposure to prescription drugs is so high, even a
very low ADE rate can lead to a large number of serious injuries and
deaths.  The Food and Drug Administration (FDA), the federal agency
that approves drugs for marketing; the pharmaceutical industry;
physician groups; consumer advocates; and health care researchers all
agree that every medicine has risks as well as benefits.  However,
they disagree substantially about the extent of the risk, how to
categorize ADEs, and what, if anything, should be done to reduce
their number. 

In light of this debate, you asked us to summarize from available
research what is known about ADEs.  In this report, we (1) describe
the different types and causes of ADEs, (2) examine the evidence on
the overall incidence and cost of ADEs in the United States, and (3)
describe measures that have been proposed to reduce the number and
severity of ADEs.  To conduct our work, we reviewed the relevant
scientific literature published since 1980, organized a symposium of
experts, and spoke with experts in government, academia, and
industry.  Our methodology is described in appendix I.  We conducted
our work between November 1998 and December 1999 in accordance with
generally accepted government auditing standards.  At your request,
we plan to examine next the adequacy of the U.S.  system for
monitoring the safety of prescription drugs that FDA has approved for
marketing. 

   RESULTS IN BRIEF
------------------------------------------------------------ Letter :1

Adverse drug events arise either from adverse drug reactions (ADR),
which are previously known or newly detected side effects of drugs,
or from medication errors committed by health care professionals or
the patients themselves.  Many types of drugs can cause ADRs; the
drugs reported as associated with ADRs vary among different studies,
depending on the patients and settings examined.  Two factors that
can increase the risk of a patient's suffering from an ADR are
illness severity and intensity of treatment, including taking several
drugs simultaneously.  Studies of several hospital populations found
that most medication errors did not cause ADEs, but because so many
drug doses were given, an estimated quarter to half of all ADEs among
the hospital patients resulted from medication errors.  Other studies
found that medication errors were most often the result of
physicians' prescribing errors and nurses' errors in administering
drugs.  Certain classes of drugs have consistently been associated
with medication errors, including analgesics, antibiotics,
psychotropics, and cardiovascular drugs. 

Although it is clear that a wide range of commonly used drugs cause
ADEs with potentially serious consequences for patients, relatively
little is known about their frequency.  Data routinely collected on
ADEs during clinical trials or after drugs are marketed are intended
to identify the ADEs that are associated with particular drugs and do
not focus on their frequency.  Information on the overall incidence
of ADEs from all drugs is limited to a few research studies that
typically examined the experience of patients in one or two specific
institutions--generally hospitals or sometimes nursing homes--leaving
the overall incidence of ADEs in outpatient care largely unexplored. 
The most broadly based information on ADE incidence comes from two
studies drawing on statewide samples of hospital patients.  These
studies applied a particularly restrictive definition of ADE in
finding that ADEs occurred at a rate of 0.56 for every 100 patients
admitted in Colorado and Utah, 0.72 in New York.  Other studies that
used broader definitions found a range of 2 to 30 ADEs per 100
hospital admissions.  Two studies of ADEs in individual nursing homes
reported an incidence of 0.44 to 0.71 ADEs per patient month. 
Although studies have estimated the overall rate of fatalities from
ADEs and the total costs of treating ADEs, their estimates are open
to question because of the limited underlying data on overall
incidence available to support them. 

Greater understanding of certain factors that affect the likelihood
of ADEs has led researchers and patient safety advocates to suggest a
range of measures to decrease their number and severity.  Proposals
for reducing ADRs include improving communication between patients
and physicians about the risks and benefits of medications and
expanding and accelerating research on the safety of marketed drugs
to more quickly detect previously unknown ADRs and determine the risk
factors that increase their likelihood.  Suggestions for reducing
medication errors include developing computerized prescribing and
dispensing systems to detect possible errors, avoiding confusing
names and packaging for medications, increasing the role of
pharmacists as advisers to physicians prescribing drugs and in
monitoring drug therapy, and improving health care providers'
pharmaceutical education. 

   BACKGROUND
------------------------------------------------------------ Letter :2

Each year, nearly half of all Americans take prescription drugs,
spending about $100 billion.  For many medical conditions,
pharmaceuticals are the treatment of choice.  Pharmaceuticals have
contributed to lengthening life expectancy by virtually eliminating
the risk from some formerly deadly infectious diseases and, more
recently, by providing tools for more effective management of chronic
conditions such as heart disease, hypertension, diabetes, and asthma. 

Like all medical interventions, pharmaceuticals have risks as well as
benefits.  Serious ADEsthose that cause hospitalization, significant
disability, or deathare relatively rare.  Most ADEs involve
transient and less severe side effects from drugs, such as nausea or
rash.  Many ADEs reflect the inherent risks, both known and unknown,
of drugs that are prescribed and administered correctly; these are
what we call ADRs in this report.  Other ADEs stem from errors in
prescribing, dispensing, or administering prescription drugs.  Common
examples include physicians' prescribing antibiotics to patients who
have documented allergies to them, nurses' not properly diluting
intravenous solutions, and patients' failing to take medications as
directed.\1

FDA within the Department of Health and Human Services decides which
drugs are approved for use by the general public.  It assesses the
information that pharmaceutical companies provide when they seek
approval to market a drug to determine whether the drug is both safe
and effective in treating one or more specified medical conditions. 
In making this assessment, FDA considers safety not in absolute terms
but as a balance of risks and benefits.  For example, a new drug may
have serious adverse effects on some patients but still win FDA's
approval because of its overall effectiveness in treating certain
conditions relative to alternative therapies.  FDA also considers
drugs in relation to the illness they are meant to cure or relieve. 
Patients with life-threatening conditions may be willing to assume
more risk than other patients. 

FDA continues to assess the risks and benefits of drugs after they
are initially approved, primarily on the basis of reports health
professionals and patients make about their experience with them. 
These reports are voluntarily submitted to either FDA itself or, more
commonly, a drug's manufacturer, which is required to pass them on to
FDA.  As FDA and the medical community learn more about a particular
drug, FDA may require manufacturers to change its labeling.  Such
changes may restrict the conditions for which the drug is approved or
require certain safety precautions.  However, new information can
also point to expanded uses for a drug.  For example, it was recently
discovered that the diuretic spironolactone, which has been marketed
for 30 years, may substantially reduce the risk of death in patients
with congestive heart failure (Pitt, 1999).\2 In other cases, new
applications of a drug have revealed previously unsuspected risks. 
After more than 20 years on the market, the drug fenfluramine was
linked to damaged heart valves when used in combination with
phentermine, another weight-loss medication.  This was one of the
rare instances in which FDA requested that the manufacturer withdraw
its drug from the market rather than change the label. 

--------------------
\1 The distinction between medication errors and an ADE associated
with the inherent risks of drugs is sometimes ambiguous.  For
example, some drugs are difficult to use safely because toxic doses
are only slightly larger than those required for treatment, or
appropriate dosing levels may vary over time, requiring frequent
adjustments.  These factors provide abundant opportunities for
medication errors. 

\2 Interlinear bibliographic citations refer to the bibliography at
the end of this report. 

   THE RISK OF ADES HAS MULTIPLE
   FACTORS
------------------------------------------------------------ Letter :3

Some ADRs are the predictable result of a drug's known
pharmacological properties, some become predictable as experience
with using a drug expands, and others are not predictable because
they are caused by individual sensitivities or allergies in
particular patients.  Many types of drugs cause ADRs:  Different
studies vary in the drugs they report as associated with ADRs,
depending on the patients and settings they examine.  Two clinical
factors known to increase the risk of a patient's suffering an ADR
are the severity of illness and intensity of treatment, including
taking several drugs simultaneously (polypharmacy).  Several studies
of hospital patients found that most medication errors did not cause
an ADE but that the few that did were still so numerous that they
accounted for a quarter to half of all ADEs.  Other studies found
that medication errors were most often the result of physicians'
prescribing errors and nurses' administration errors.  Analgesics,
antibiotics, psychotropics, and cardiovascular drugs are among the
drug classes that have been consistently associated with a greater
proportion of medication errors. 

      INHERENT PROPERTIES OF
      MEDICATIONS LEAD TO MANY
      ADES
---------------------------------------------------------- Letter :3.1

Many ADRs are the predictable result of a drug's known
pharmacological properties and are often listed in a medication's
label.  For example, hemorrhaging is the most common ADR for
warfarin, a drug that reduces the risk of heart attack, stroke, and
other conditions by decreasing the clotting ability of blood. 
Nonsteroidal anti-inflammatory drugs (NSAID), prescribed over long
periods for rheumatoid arthritis, suppress enzymes that protect the
lining of the stomach and intestines, which causes serious
gastrointestinal complications in a small percentage of patients. 

Other ADRs, including allergic reactions, are unpredictable, caused
by sensitivities in particular patients who have neither a known risk
factor nor a history of adverse reactions to a specific drug.  An
unpredictable ADR is more likely to cause disability or death than
one that is predictable.  Still other ADRs are related to previously
undetected inherent risks, including drug-drug and drug-food
interactions, that become evident as a drug is used by many types of
patients, having many kinds of comorbidities and taking many other
medications, including over-the-counter drugs and dietary
supplements.  FDA's system for collecting voluntary reports on
adverse experiences with marketed drugs is designed specifically to
uncover these kinds of previously unknown risks. 

Many types of drugs can cause ADRs.  Therefore, the drugs associated
with ADRs in particular studies vary, depending on the patients and
clinical settings studied.  In addition, some drug classes are
associated with a substantial number of ADRs simply because they are
prescribed to many patients.  These include antibiotics, narcotic
analgesics, drugs to control hyperglycemia in type II diabetics,
psychotropic drugs such as antidepressants and tranquilizers, and
NSAIDs.\3 However, some classes of drugs have notably lower ADR rates
despite high rates of use.  In the studies we reviewed,
antihistamines and the statin drugs prescribed to lower cholesterol
levels were rarely associated with serious ADRs. 

--------------------
\3 Wolfe and others (1999) estimated 16,500 NSAID-related deaths
annually among arthritis patients in the United States. 

      SOME PATIENTS HAVE A GREATER
      RISK OF ADRS THAN OTHERS
---------------------------------------------------------- Letter :3.2

Patients who are very ill, including those with several concurrent
diagnoses, have a greater ADR risk than others.  Not only are they
more fragile but their illnesses may require several simultaneous
treatments.  In addition, they may be receiving more aggressive
treatments that are known to entail significant risks.  One study
found that pediatric cancer patients undergoing chemotherapy had
suffered ADRs at a rate 10 times that of other children admitted for
inpatient treatment (Mitchell and others, 1988).  Another study found
that hospital patients in intensive care units (ICU) had 80 percent
more ADRs than patients in general medical wards and that most of
this difference was accounted for by the greater number of
medications given the ICU patients (Bates and others, 1995a). 
(Controlling for the number of medications reduced the adverse event
rate for ICU patients to 20 percent more than that for general
medical patients.)

Some reports have found that elderly persons and women have more ADRs
than younger persons and men.  However, it is possible that age and
gender are merely related to other known risk factors instead of
representing additional, independent risks of ADRs.\4 In some
studies, controlling for the number of medications being taken
substantially diminishes any relationship between age and ADRs
(Gurwitz and Avorn, 1991).  Although other studies have shown that
women both use more drugs and have more ADRs than men overall, these
studies did not control for illness severity and the number of
different medications the patients took. 

--------------------
\4 Changes in the metabolism of drugs with age mean that dosing
requirements for older persons differ from those for younger adults,
complicating the determination of a proper dose.  At the same time,
elderly persons frequently have other concurrent illnesses and, as a
result, usually take several drugs.  One study of hospital admissions
found that elderly patients took an average of 3.5 prescription drugs
before their hospitalization (Grymonpre and others, 1988). 

      MEDICATION ERRORS ARE COMMON
      AND HAVE NUMEROUS CAUSES
---------------------------------------------------------- Letter :3.3

Very few medication errors cause ADEs, either because errors are
caught before the drugs are administered or because specific errors
created no ill effects.\5 Nonetheless, because so many drug doses are
given, an estimated quarter to half of all ADEs among hospital
patients result from medication errors (Bates and others, 1995a;
Classen and others, 1997).\6 For example, a 6-month hospital study
found that the ADE rate was 1 for every 10,000 doses; 700,000 drug
doses were given, and there were 70 injuries.\7 Although estimates of
the actual effect of medication errors vary, they have led to
hospital admissions, emergency department visits, and the death of
hospitalized patients (Nelson and Talbert, 1996; Classen and others,
1991; Schneitman-McIntire and others, 1996). 

Most medication errors in hospitals involve the prescriptions ordered
by physicians and nurses' administration of drugs.  Pharmacists make
relatively few medication errors when they transcribe, verify, and
dispense hospital prescriptions.  Physicians' errors include
overdosing and underdosing, prescribing drugs the patients have
documented allergies to, and prescribing drugs known to interact
adversely with other medications patients take.  Administration
errors include giving drugs other than those prescribed, giving drugs
at the incorrect time, and giving patients the incorrect form of a
drug, such as an injection rather than a tablet. 

Patients' noncompliance is also a major ADE source.  Outside the
hospital and other health care institutions, patients are responsible
for complying with their drug regimen rather than relying on health
care professionals.  They may underuse or overuse drugs, run out of a
medication, or take medications inconsistently.  Their noncompliance
is an important cause of emergency department visits and hospital
admissions.  For example, researchers reported that 58 percent of
ADEs in patients visiting one hospital's emergency department were
caused by noncompliance (Dennehy and others, 1996).  Another study
found that 11 percent of all elderly patients admitted to a hospital
were related to noncompliance (Col and others, 1990). 

Sometimes, consistent patterns of medication error have resulted in
the withdrawal of certain drugs from the market.  Recently, for
example, the manufacturer of bromfenac withdrew the drug when
physicians continued to prescribe it for more than 10 days, even
after FDA had warned against long-term administration.  Similarly,
the antihistamine terfenadine was withdrawn from the market after
warnings and new labels failed to stop its use with certain other
medications that could cause serious heart problems. 

--------------------
\5 Bates and others (1995b) found that 1 percent of medication errors
lead to ADEs.  If missed doses (approximately half of the errors) are
excluded, then 2 percent lead to ADEs. 

\6 A 1994 review by Pearson and others indicated that 30 to 80
percent of ADRs are preventable.  However, the majority of the
studies they cited were published before 1980, and therefore we did
not include them in our review. 

\7 Comments of Lucian Leape, The Safety of Pharmaceuticals: 
Monitoring and Regulation, American Enterprise Institute conference,
Washington, D.C., Mar.  26, 1999. 

      SOME DRUGS LEAD TO
      MEDICATION ERRORS MORE OFTEN
      THAN OTHERS DO
---------------------------------------------------------- Letter :3.4

Analgesics, antibiotics, and cardiovascular and psychotropic drugs
are among the classes of drugs consistently associated with
medication errors.  The number of errors for a drug class is a
function of the error rate for the class and how often drugs in the
class are used.  The error rate for cardiovascular drugs is lower
than that for many other drug classes; the large number of errors for
this class primarily reflects the large number of patients taking
these drugs.  Not only are analgesics and antibiotics used frequently
but their error rates are among the highest (Bates and others, 1998). 

Some drugs have high medication error rates because their
pharmacological properties make them difficult to use, even when
administered in generally recommended doses.  For instance, both the
anticoagulant warfarin and the cardiac stimulant digoxin have narrow
therapeutic indexes, meaning that the dosage levels for therapeutic
effectiveness are close to toxic, and both require careful adjustment
of dosage levels in individual patients.  Known drug interactions
pose additional risks, since some drugs interact in potentially
dangerous ways with many other pharmaceuticals.  For example, the
label for warfarin indicates clinically significant interactions with
approximately a hundred other drugs. 

Drugs with similar names can also lead to medication errors. 
Physicians may confuse names when prescribing drugs and pharmacists
may do the same when dispensing them.  Recently, concern has been
raised about possible confusion between Celebrex, Celexa, and Cerbyx,
names that look and sound alike but that represent very different
drugsa pain medication used to treat arthritis, an antidepressant,
and an antiseizure drug, respectively. 

Certain medications have been identified with a greater incidence of
patient noncompliance, including insulin, phenytoin, and drugs in
metered-dose inhalers (Dennehy and others, 1996; Prince and others,
1992).\8 Each of these medications requires careful monitoring by the
patient or physician to determine when it should be given and in what
dose.  Consequently, the potential for noncompliance is quite high. 

--------------------
\8 Phenytoin is an anticonvulsant used to control seizures in certain
types of epilepsy and other conditions. 

   LITTLE IS KNOWN ABOUT THE
   INCIDENCE AND COST OF ADES
------------------------------------------------------------ Letter :4

Although it is clear that a wide range of commonly used drugs cause
ADEs with potentially serious consequences for patients, relatively
little is known about the frequency of the ADEs.  Data routinely
collected on ADEs before and after drugs are marketed focus more on
identifying which ADEs are associated with which drugs.  Information
on the overall incidence of ADEs from all drugs is limited to a few
research studies that have typically examined the experience of
patients in one or two specific institutionsgenerally hospitals or
sometimes nursing homesleaving the overall incidence of ADEs in
outpatient care largely unexplored.  The most broadly based
information on ADE incidence comes from two studies drawing on
statewide samples of hospital patients.  These studies applied a
particularly restrictive definition of ADE in finding a rate of 0.56
for every 100 patients admitted in Colorado and Utah and 0.72 in New
York.  This compares with a range of 2 to 30 ADEs per 100 admissions
found in other studies with increasingly expansive definitions of
ADE.  Two studies of ADEs in individual nursing homes reported an
incidence of 0.44 to 0.71 per patient month.  Other studies have
estimated the overall rate of fatalities from ADEs and the total
costs of treating them, but both estimates are questionable because
of gaps in the underlying data on ADE incidence rates. 

      ADES ARE OFTEN DIFFICULT TO
      IDENTIFY
---------------------------------------------------------- Letter :4.1

Linking a particular symptom to a specific drug is difficult,
primarily because ADEs are relatively rare for most drugs marketed in
the United States and because drugs are often given to seriously ill
patients whose underlying conditions manifest many symptoms.  The
best chance of identifying ADEs is when they show distinct effects
shortly after a drug is administered. 

Other ADEs can be extraordinarily difficult to detect.  For example,
symptoms that develop with the prolonged use of a drug require
studies with long follow-up periods to determine whether ADEs have
occurred.  Similarly, rare adverse events require studies with very
large numbers of patients to accumulate a sufficient number of
problematic cases, and adverse symptoms that mimic those of a
patient's underlying condition require carefully controlled clinical
trials.\9

--------------------
\9 One well-known example comes from the Cardiac Arrhythmia
Suppression Trial, which found that antiarrhythmia medications
doubled the risk of cardiac arrest and death in heart attack
survivors.  This relationship was not detected in clinical practice
because patients with heart disease regularly have arrhythmias and
heart attacks, providing a ready alternative explanation that masked
the causal role of the drugs.  It has been estimated that these
medications caused up to 50,000 premature deaths (see Echt and
others, 1991). 

      DATA ON ADE INCIDENCE
      COLLECTED ROUTINELY BEFORE
      AND AFTER DRUG APPROVAL ARE
      NOT COMPREHENSIVE
---------------------------------------------------------- Letter :4.2

Safety is a prominent concern throughout drug development, and many
dangerous substances are identified and their testing is halted in
the process.  Nonetheless, by themselves, the results of clinical
trials submitted with an application to FDA to market a drug cannot
provide comprehensive information on possible adverse events (Faich,
1986).  First, the number of patients typically included in
preapproval clinical trials is too small to detect less frequent
adverse events.  According to the pharmaceutical industry, the total
number of patients in such trials averages roughly 4,000 per drug. 
Consequently, adverse events that occur in 1 of 10,000 patients, for
example, often do not appear at all in any clinical trials.  In
addition, the patients who are included in clinical trials are
selected to obtain a clear picture of a drug's safety and efficacy
and are therefore unlikely to reflect the full range of consumers who
will actually use the drug.  For example, participants in clinical
trials usually include few elderly patients, few patients with
serious illnesses other than the one the drug targets, and few
patients taking many other medications.  Clinical trials also usually
last for a relatively short time, so that adverse events that occur
with long-term treatment are not likely to be detected. 

The limitations of the data on adverse events derived from clinical
trials can be especially critical during a drug's initial marketing
period.  When a drug is first available to consumers, it can be
quickly prescribed to hundreds of thousands of patients who are far
more heterogeneous than the patients studied in the clinical trials. 
Further, physicians often prescribe new drugs to patients who have
not responded to older medications; thus, the initial recipients of a
drug are more likely to be especially ill and unlike the patients
studied in the clinical trials. 

FDA's current postmarketing data collection systems for approved
drugs are intended to compensate for the limitations of information
from clinical trials by detecting the existence of previously
unidentified ADEs.  However, because FDA's Adverse Event Reporting
System (AERS) relies on voluntary reports from physicians,
pharmacists, patients, and others, it can uncover instances of
problems but it cannot determine their incidence.\10 The same
intrinsic limitation applies to the incident reporting systems that
many hospitals have established to monitor adverse events, including
ADEs.  All such systems based on spontaneous reporting detect only a
fraction of the total number of adverse events (Cullen and others,
1995).  FDA's AERS includes an estimated 1 to 10 percent of adverse
events (Goldman and others, 1996).  In addition, the adverse events
that are reported are unlikely to be representative of the much
larger number of unreported events.  For example, there is evidence
that ADEs are reported more often to FDA if they involve a newly
released drug or one sold by a company that has a relatively large
postmarketing surveillance program (Baum and others, 1994). 
Consequently, any estimate of ADE incidence based on a spontaneous
reporting system such as AERS would necessarily incorporate the
biases of the data, undercounting some types of adverse events and
overcounting others. 

FDA, recognizing the limitations of its spontaneous reporting system,
augments the data in AERS with information from other sources.  If
signals from AERS reports suggest new adverse events or an
unexpectedly large number of known ADEs, FDA can gather additional
information from several health maintenance organizations that have
cooperative agreements with the agency to use their databases of
member medical and pharmacy records to investigate issues of ADE
causation and incidence.  However, these databases sometimes do not
have enough patients taking a particular drug for a given medical
condition to provide definitive answers to the questions that have
arisen.\11

--------------------
\10 Health care providers and patients are not obligated to report
suspected ADEs to FDA.  However, they are encouraged to report events
either directly to AERS or to the drug's manufacturer, and the
manufacturers are required to forward all adverse event reports they
receive to FDA. 

\11 Without information on the incidence of ADEs, it can be difficult
for FDA to assess the level of risk a drug poses.  For example, in
March 1999, an FDA advisory committee considering the safety of
troglitazone, a drug for type II diabetes, was unable to determine
the number of patients who suffered liver failure while taking the
drug.  Estimates of the number of deaths and liver transplants
presented by an FDA epidemiologist and the drug's manufacturer
differed by tenfold. 

      KNOWLEDGE OF OVERALL ADE
      INCIDENCE IS FRAGMENTARY
---------------------------------------------------------- Letter :4.3

There is relatively little information on ADE frequency overall for
all types of drugs.  The data collected routinely before and after
drug approval and through studies of ADEs associated with specific
medications do not answer this question.\12 Appendix II describes the
relatively few studies that we identified that were designed
specifically to examine the overall incidence of ADEs. 

One potential reason for the paucity of research in this area is the
methodological challenge it presents.  Determining that an adverse
event occurred and that it was caused by a drug and not some other
factor, such as the patient's underlying disease, is necessarily more
complex when the scope of the investigation includes all possible
adverse events and every drug the patient took.  Researchers have to
consider much more information from each patient's medical record on
symptoms, diagnostic tests, and treatments. 

Researchers conducting these studies have typically responded to this
challenge by focusing on a narrowly defined patient population.  For
example, the large majority of the studies deal exclusively with
patients treated in one or two specific institutions.  On a practical
level, this enables researchers to examine the complete medical
record for hundreds of cases without having to go to multiple
institutions to first obtain permission and then copy and ship the
often voluminous patient records from diverse locations. 

The disadvantage of focusing on institutions is that the extent to
which ADEs in different treatment settings are studied is quite
uneven.  Researchers tend to study the type of institutions that have
the most complete and detailed records, which is usually hospitals. 
Consequently, ADEs in other settings are examined either less often
or not at all.  We found only a few studies of ADEs among nursing
home residents and only one small study of ADEs that occurred in the
community and were treated in physicians' offices.\13 The general
lack of information about the incidence of ADEs that occur and are
treated outside hospitals and nursing homes means that our basing our
estimates of overall ADE incidence on current knowledge necessarily
limits us to institutional settings. 

A related problem arises in attempting to extrapolate from the
studies of overall ADE incidence in selected hospitals and nursing
homes to other comparable institutions.  Without evidence that the
studied institutions are representative of others, it is not
appropriate to project the results to patients treated in other
facilities.  The one or two institutions studied may differ
substantially from other institutions of the same type with respect
to the characteristics of the patients served or services provided,
which in turn could affect the overall rate of ADEs. 

--------------------
\12 Much of the substantial literature on ADEs consists of published
studies that focused on a specific drug or class of drugs.  See, for
example, the 239 abstracts of studies conducted by the Boston
Collaborative Drug Surveillance Program listed in Jick (1992).  Such
studies may indicate the incidence of adverse events with a given
medication or drug class, but there is no direct way to aggregate
specific drugs and types of ADEs to arrive at an overall incidence
rate. 

\13 Klein and others (1984) was a study of 299 mostly chronically ill
patients treated in outpatient clinics run by Johns Hopkins
University and is therefore unlikely to reflect community-based care
as a whole.  There are several studies from periods before our 1980
cutoff date and from foreign countries, but this is the only one we
found that examined overall ADE incidence in outpatient care that met
our selection criteriaU.S.  patient data from 1980 or later. 
However, several new studies of ADE incidence in noninstitutionalized
populations are now under way. 

      ADES DIFFER FOR HOSPITAL
      PATIENTS AND NURSING HOME
      RESIDENTS
---------------------------------------------------------- Letter :4.4

With two exceptions, the existing studies of ADEs among hospital
patients each reported data from a different individual hospital (or,
in one case, two hospitals) and they frequently differed
substantially in the way they defined and counted ADEs.  Some studies
examined how many hospital admissions stemmed from ADEs, others
tracked ADEs that occurred during the course of a hospital stay, and
a few did both.  One study focused solely on ADRs (thereby excluding
medication errors), another identified any injury caused by a drug,
and several others counted any adverse experience associated with the
use of a medication.  Some included events that were possibly, but
not definitely, caused by a drug, while others did not.  All these
variations help explain the range in ADE incidence reported by
different studies. 

The two studies with an unusually broad, statewide sample of patients
but a highly restrictive definition of ADEs found rates of 0.56 and
0.72 ADEs for every 100 hospital admissions.  The higher figure
emerged from the Harvard Medical Practice Study (HMPS), which
examined a representative sample of all nonpsychiatric patients
treated in acute care hospitals in New York in 1984 (Brennan and
others, 1991; Leape and others, 1991).  It therefore included a
proportionate mix of patients from teaching and nonteaching, urban
and rural, and large and small hospitals.  More recently, the same
methodology was applied to statewide samples of 1992 hospital
discharges in Colorado and Utah (Thomas and others, forthcoming). 
Even though a sample of patients in one or two states is vulnerable
to certain biasessuch as those deriving from regional variation in
clinical practice patternsthe databases for the studies that used
them are far more diversified and representative than those of the
other studies we examined.  Moreover, the fact that the rates found
in these two studies are relatively close, despite the studies'
variation in time and place, suggests that regional and temporal
variation in ADE incidence may not be very large.  However, by
counting only those ADEs that resulted in disability, prolongation of
a patient's hospital stay, or death, these two studies identified
just a fraction of the patients injured by drugs.\14

Four other studies examined adverse drug events among hospital
inpatients, reporting ADE incidence rates ranging from 2.0 to 30 ADEs
per 100 admissions (see table II.1 in appendix II).\15 The higher
rates came from the studies with more expansive definitions of ADEs. 
There was less variation among the studies in their reported
incidence of moderate to severe ADEs, which ranged from 1.9 to 19 per
100 admissions. 

Compared with hospital patients, nursing home residents are generally
more frail and functionally impaired.  While nursing homes are
designed to provide less intensive care than hospitals, their
residents still receive many medications.  These factors probably
increase the vulnerability of nursing home residents to ADEs. 
Patients also tend to stay longer in nursing homes than in hospitals,
so ADE rates for nursing home residents are often reported per unit
of time, such as patient months, to adjust for risk differences
attributable to longer and shorter stays. 

We found fewer studies of ADE incidence in nursing homes than in
hospitals, and none examined more than one or two institutions.\16 As
with the hospital studies, the definition of what constituted an ADE
varied substantially.  One study with a more narrow definition
reported an incidence of 0.44 ADEs for every month that a patient
spent in that institution, compared with 0.71 ADEs reported in a
second study with a much broader definition (see table II.2 in
appendix II).\17 These rates are roughly comparable with the rates
reported by the one study of hospital ADEs that presented ADE
incidence in terms of time spent in the hospital.\18

--------------------
\14 Bates and others (1995a), conducting a study in two Boston-area
teaching hospitals, applied both the ADE definition that HMPS used
and a broader ADE definition that included any injury related to a
prescribed drug.  The ADE incidence rate was 0.5 percent under the
HMPS definition and 6.5 percent under the broader definition. 

\15 It is not possible to calculate an incidence rate for studies
focusing on ADE-caused admissions to a particular hospital, because
there is no defined population at risk for admission to that
hospital, and only that hospital, if an ADE occurs.  See appendix II. 

\16 A study of 18 nursing homes in eastern and central Massachusetts
is under way. 

\17 The first study (Gerety and others, 1993) basically looked for
ADRs.  The second (Cooper, 1986) looked for a much larger category of
drug-related problems, including any unwanted consequence of using or
not using drug therapy. 

\18 This study (Bates and others, 1995a) reported an overall
incidence of 0.345 ADEs per patient month, ranging from 0.267 in
surgical general care wards to 0.582 in medical intensive care wards. 
The figures were converted from events per 1,000 patient days to
events per patient month. 

      ESTIMATES OF DEATHS ARE
      UNCERTAIN
---------------------------------------------------------- Letter :4.5

Adverse drug events are sometimes so severe that patients die from
them.  There is little certainty about the frequency of fatal ADEs,
because the data on fatalities stemming from ADEs are even more
sparse than the data on overall ADE incidence.  Recently, Lazarou,
Pomeranz, and Corey (1998a) attempted to synthesize available data on
ADR fatalities.\19 To derive their estimate of 106,000 fatal ADRs in
the United States in 1994, they drew on data from 16 ADR studies
published between 1964 and 1995.  The studies cumulatively looked at
78 deaths, but only two of the studies had more than 10 deaths, and
more than 40 percent of the deaths were reported in one 1973 study. 
Consequently, there were too few deaths to arrive at a stable
estimate of total ADR fatalitiesas even a small change in the number
of deaths reported in the studies would lead to substantial changes
in the number of deaths extrapolated to the national population. 

In addition to the small number of deaths on which this estimate was
based, there is a major question about relying on data from studies
more than 20 years old.  Since the 1960s and 1970s, drug therapies
have shifted markedly for many conditions, with a generally more
intensive use of pharmaceuticals now than in the past.  Of the 16
studies that Lazarou and colleagues included in their analysis of ADE
fatalities, only 4 were published after 1976.  Collectively, these 4
studies accounted for a total of 5 deaths, compared with 73 in the 12
earlier studies.  Thus, the projection

Lazarou and his colleagues made for the incidence of fatal ADEs for
1994 was actually based on the experience of patients 20 or more
years earlier.\20

--------------------
\19 Their estimates are for fatal adverse drug reactions; they did
not address fatalities caused by medication errors. 

\20 When Kenneth Fremont-Smith (1998) criticized Lazarou and his
colleagues on this point, they responded (1998b) that they had since
analyzed unpublished data from 32 additional studies of fatal ADEs in
industrialized countries other than the United States.  They stated
that these data, when combined with the U.S.  data, showed no trend
in rates of fatal ADRs, either up or down over time.  They also found
no statistically significant difference between the U.S.  and
non-U.S.  studies.  They maintained that this demonstrated that the
rate of fatal ADRs had not changed in the United States, thereby
validating their original estimate.  They took this approach because,
in their view, the data from recent studies of fatal ADRs among U.S. 
hospital patients were not sufficient to derive a statistically
reliable estimate of the incidence of fatal ADRs in the United
States. 

      DATA ON THE COST OF TREATING
      ADES ARE LIMITED
---------------------------------------------------------- Letter :4.6

The lack of overall incidence data for ADEs in the United States
impedes making a reliable estimate of the nationwide costs of adverse
drug events, although several studies have reported similar costs for
treating ADEs in hospitals.  Researchers have followed different
approaches in attempting to generate information about the direct
costs of treating adverse drug events, but we found only one study
that attempted to calculate indirect costs such as lost income. 

Most studies of ADE costs have focused on one or two individual
institutions.  (See appendix III.) Three of the four studies that
specifically analyzed the average excess hospital costs resulting
from ADEs reported estimates ranging from $1,939 to $2,595 (Bates and
others, 1997; Classen and others, 1997; Evans and others, 1994b). 
The outlier study reported average ADE costs of only $783 (Schneider
and others, 1995).  Two of these studies also extrapolated their
findings on ADE incidence and costs in these particular hospitals to
all hospital patients in the United States, producing estimates of
$1.56 billion and $4 billion in additional hospital costs per year
nationwide from ADEs (Bates and others, 1997; Classen and others,
1997).  While these estimates may help indicate the general scope of
ADE costs, because each is based on just one or two hospitals, their
precision for estimating costs on a national level is limited. 

Three other studies used expert panels to generate ADE cost estimates
(Bootman, Harrison, and Cox, 1997; Johnson and others, 1995, 1997). 
The experts came to a collective judgment as to the likely
probability of specified negative outcomes arising from drug therapy,
which translated into an incidence rate for the patient population as
a whole.\21 The total cost of ADEs was then calculated by multiplying
the estimated number of ADEs by the unit cost of treating them.\22
For one study in 1995, this process produced an estimate for the
costs of drug-related morbidity and mortality in the ambulatory
setting of $76.6 billion annually, primarily because of the resulting
admissions to hospitals, costing $47.4 billion, and long-term-care
facilities, costing $14.4 billion.\23 A later study in 1997 estimated
the cost of drug-related morbidity and mortality in nursing
facilities to be around $7.6 billion annually (Bootman, Harrison, and
Cox, 1997).  The probability statements developed by an expert panel
are inherently subjective and would be likely to change if the
composition of the panel changed.  Since the cost estimates are based
on these estimated incidence rates, the cost estimates are also open
to question. 

Finally, the 1999 study conducted in Colorado and Utah also collected
information about the costs of adverse events, including those that
were drug-related (Thomas and others, 1999).  Distinguishing this
study are its broad-based sample of 14,732 randomly sampled
discharges from hospitals in two states and its inclusion of indirect
as well as direct treatment costs for ADEs.  From data extracted from
the patients' medical records, physicians and malpractice claims
adjusters estimated the patient's degree of disability and likely use
of health care in the future.  Projected inpatient and outpatient
health care costs, lost wages, and lost household production were
then estimated, and the total was reported as an aggregate national
figure.  The $5.2 billion estimate for hospital costs alone exceeded
the costs reported in both of the earlier studies from individual
institutions.  Adding in the estimated cost of outpatient care, lost
income, and lost household production brought the total of direct and
indirect costs to an estimated $12.2 billion in 1996 dollars.  This
result is based on a much broader sample than in the earlier studies,
although still limited to Colorado and Utah.  In extrapolating the
results to an aggregate national estimate, the authors did not
attempt to adjust for the likely variation in hospital costs and
personal incomes in other parts of the country. 

--------------------
\21 The panel members in one of these studies were pharmacists who
were selected because of their extensive clinical practice in
ambulatory settings and recognition as leaders in pharmacy practice
in the United States (Johnson and others, 1995).  In a later study,
the panel members were physicians and consultant pharmacists with
practice experience in nursing facilities and geriatric care
(Bootman, Harrison, and Cox, 1997). 

\22 The expert panels also estimated the proportion of patients with
negative outcomes who would receive various types of treatment in
response, such as additional physician visits, new prescriptions,
emergency department visits, nursing home admissions, and
hospitalizations.  The cost of each scenario was calculated as
monetary valuesobtained from published statistical series and
research reportsfor each type of treatment employed in that
scenario, and then an aggregate cost figure was computed from the
estimated total cost of each separate scenario and multiplied by its
estimated probability across all patients (Johnson and others, 1995). 

\23 With adjustments in the assumptions of the model, the estimates
of total ADE costs ranged from $30.1 billion to $136.8 billion.  (See
Johnson and others, 1995.)

   MEASURES INTENDED TO REDUCE THE
   NUMBER AND SEVERITY OF ADVERSE
   DRUG EVENTS
------------------------------------------------------------ Letter :5

Increased understanding of what makes ADEs likely has led researchers
and patient safety advocates to develop a variety of measures
intended to decrease their number and severity.  The approaches they
have suggested to reduce ADRs include improving communication between
patients and physicians about the risks and benefits of medications,
as well as expanding and accelerating research on the safety of
marketed drugs to reduce the time it takes to detect previously
unknown ADRs and determine the risk factors that identify the
patients who are most likely to experience them.  Measures designed
to reduce the number of medication errors include developing
computerized prescribing and dispensing systems to detect errors,
avoiding confusing names and packaging, increasing the role of
pharmacists as advisers to physicians in prescribing drugs and in
monitoring drug therapy, and improving physicians' pharmaceutical
education. 

      DRUG DEVELOPMENT AND BETTER
      INFORMATION TO REDUCE ADRS
---------------------------------------------------------- Letter :5.1

Some have suggested that the process of drug research and development
could help reduce ADRs as pharmaceutical companies respond to market
incentives by developing new medications with fewer risks than the
ones they replace.  For example, a new generation of NSAIDs called
cox-2 inhibitors has recently reached the market; these drugs were
designed specifically to lower the risk of gastrointestinal injury
compared with traditional anti-inflammatory drugs such as aspirin or
ibuprofen.  Similarly, while the first in a new class of diabetes
drugs, troglitazone, approved for marketing in 1997, has been
associated with rare cases of deadly liver failure, FDA has since
approved other drugs with similar clinical benefits but less liver
toxicity (although they may have other serious side effects). 

Others have suggested that physician's educationand communication
between physicians and patientsabout the benefits and risks of
particular drugs be improved in order to promote informed
decisionmaking about pharmaceuticals and thereby help reduce the
incidence of ADRs.  Given the large number of drugs on the market and
the voluminous information about each one, some observers have
suggested that computerized systems could be designed to help remind
physicians when they submit prescriptions about important therapeutic
considerations, including comparative benefits, risks, and
contraindications for several similar drugs. 

Expanded surveillance programs to gather information about marketed
drugs might also help prevent ADRs by more quickly accumulating
information about them for particular drugs.  For example, FDA has
proposed for discussion several methods of rapidly gathering
information during the crucial period following a drug's approval for
marketing.  These ideas include establishing a network of health care
facilities to serve as sentinel sites for closely monitoring the
experiences of the first patients to take a new drug and slowing down
the introduction of new drugs until sufficient information has been
collected about their risks once they are on the market.\24
Quantified risk data about new drugs could assist physicians and
patients to make more informed treatment choices. 

Data collection efforts that document patient risk factors for
adverse events have the effect of moving some ADRs into the category
of preventable medication errors.  For example, reports of
cardiovascular complications from using sildenafil surfaced after its
approval in 1998, ultimately causing a change in the product's label
to warn physicians about its dangers for patients with certain
preexisting conditions.  Patients with these conditions who took
sildenafil and suffered an adverse cardiovascular event when the drug
was first marketed might have been then classified as suffering an
ADR but today would probably be considered victims of medication
error. 

--------------------
\24 The benefits of slowing down the marketing of new drugs must be
weighed against the health costs of potentially restricting patients'
access to them. 

      MEASURES TO REDUCE
      MEDICATION ERRORS
---------------------------------------------------------- Letter :5.2

Numerous measures to reduce medication errors have been taken, and
experts have proposed others for drug manufacturers and health care
providersmany aiming to make errors more difficult across a range of
specific circumstances.  For instance, computer systems can screen
prescriptions to detect errors in dosage levels or known allergies. 
Some proposed measures, such as eliminating look-alike packaging,
would make it physically harder to dispense or administer the wrong
drug.  Table 1, which is meant to be illustrative rather than
comprehensive or evaluative, describes a number of general
approaches. 

                                         Table 1
                         
                            Current and Proposed Approaches to
                                 Reduce Medication Errors

Approach                      Intended benefit
----------------------------  -----------------------------------------------------------
Dispensing change

Physicians' direct computer   Physicians' entering prescriptions on a computer rather
entry of prescriptions        than writing them reduces transcription errors and
                              indicates potentially problematic prescriptions. For
                              instance, it can indicate an improper dose that is being
                              prescribed or a drug that might interact with another
                              medication the patient is taking.

Unit dosing                   Dispensing drugs from the pharmacy in single-unit or unit-
                              dose packages (for instance, blister packs) makes them
                              ready to administer.

Automated hospital            Such systems notify nurses when a drug is to be
dispensing systems            administered and allow access only to it. The systems also
                              record what has been given and when as well as reducing
                              delays in giving patients their medications and decreasing
                              other administration errors.

Bar coding hospital           Machine-readable labels can facilitate matching patients
medications                   with their prescribed medications and documenting drug
                              dispensing and administration.

Focus on high-alert drugs     Specific systems and educational initiatives minimize
                              errors with the drugs that have the greatest potential to
                              cause serious harm when used incorrectly, such as insulin,
                              opiates and narcotics, potassium chloride concentrate, and
                              intravenous anticoagulants.

Packaging and physical
change

Differentiated drug names     Giving drugs whose names sound alike (for example,
                              Celebrex, Celexa, and Cerbyx) different names could reduce
                              the likelihood of their being confused.

Differentiated packaging      Packaging different drugs differently would make them
                              easily distinguished.

Standardized packaging        Uniform labels with standards for print size and color
                              would help practitioners and patients know where to look
                              for particular information.

One name and one look for     Drugs would be less easily confused if each one had only
each drug                     one name, not a generic and a brand name, and two or more
                              manufacturers who made the same drug gave the pills the
                              same design, packaging, and labeling.

Change in sensitivity to
ADEs

Physicians' education         Educating physicians about pharmaceuticals more, both
                              during and after medical school, would improve their
                              prescribing practices.

Greater pharmacist            Including pharmacists in hospital rounds helps physicians
involvement                   make prescribing decisions, and increasing the role of
                              community pharmacists in monitoring drug therapies improves
                              patients' compliance.

Timely communication          Timely feed back on ongoing ADEs could help physicians in
                              hospitals prevent the progression of ADEs to more severe
                              forms.

Computerized ADE monitoring   Computer programs designed to screen for potential ADEs,
                              using data from electronic inpatient or outpatient medical
                              records, such as orders for known antidotes or specific
                              laboratory test abnormalities, cut their number and
                              frequency.

Culture change

Encouraged reporting          Changing an institution's culture so that errors are seen
                              as an indication of where systemic improvements are needed
                              rather than simply assigning blame to individuals would
                              make it more likely that mistakes would be reported.
-----------------------------------------------------------------------------------------

   AGENCY COMMENTS
------------------------------------------------------------ Letter :6

We provided a draft of this report to the Commissioner of FDA and
five outside experts, including physicians, pharmacists, and
epidemiologists who are actively involved in analyzing ADEs.  FDA
responded that the report accurately describes the current status of
adverse event reporting.  The agency also provided technical comments
that we incorporated as appropriate.  The outside experts generally
found that our characterization of currently available information on
ADEs is accurate and thorough.  However, several of them expressed
concern that our critical assessment of existing studies might create
the misperception that there is little evidence that ADEs pose a
substantial health risk to patients.  We revised sections of the
report and its title to make clear that while the magnitude of the
health risk is uncertain its existence is not.  Other comments from
the experts led us to make additional corrections and clarifications
to the text. 

---------------------------------------------------------- Letter :6.1

As we arranged with your office, unless you publicly announce the
contents of this report earlier, we plan no further distribution of
it until 30 days after its issue date.  We will then send copies to
the Secretary of the Department of Health and Human Services, the
Commissioner of FDA, and others who are interested.  We will also
make copies of the report available to others on request. 

If you or your staff have any questions, please contact me at (202)
512-7114.  Robert M.  Copeland, Martin T.  Gahart, Michele Orza, Eric
A.  Peterson, and Helene F.  Toiv were the major contributors to this
report. 

Janet Heinrich
Associate Director, Health Financing
 and Public Health Issues

List of Requesters

The Honorable James M.  Jeffords
Chairman
The Honorable Edward M.  Kennedy
Ranking Minority Member
Committee on Health, Education, Labor, and Pensions
United States Senate

The Honorable Bill Frist
Chairman
Subcommittee on Public Health
Committee on Health, Education, Labor, and Pensions
United States Senate

The Honorable Richard J.  Durbin
The Honorable Jack Reed
United States Senate

The Honorable Thomas J.  Bliley, Jr.
Chairman
The Honorable John D.  Dingell
Ranking Minority Member
Committee on Commerce
House of Representatives

The Honorable Henry A.  Waxman
Ranking Minority Member
Committee on Government Reform
House of Representatives

OBJECTIVES, SCOPE, AND METHODOLOGY
=========================================================== Appendix I

The objectives of this report were to (1) describe the different
types and causes of adverse drug events (ADE), (2) examine the
evidence on the overall incidence and cost of ADEs in the United
States, and (3) describe measures that have been proposed to reduce
the number and severity of ADEs. 

This study concerns only prescription medicines.  We did not examine
vaccines or other biologics, medical devices, procedures, or
nonprescription or illicit drugs.  The study encompasses both adverse
events that result from the intrinsic pharmacological characteristics
of drugs and those that stem from mistakes that physicians, nurses,
pharmacists, other health professionals, and patients make in using
pharmaceuticals.  We focused on obtaining information about the use
of prescription drugs in the United States since 1980 across the full
range of treatment settings, including hospitals, long-term care
institutions, and outpatient facilities.  We chose 1980 as the cutoff
because many drugs in use now were not available before then and, for
many major clinical conditions, the main classes of drugs relied on
now did not exist before 1980.  The results obtained in studies
conducted before 1980 may have little applicability for current
clinical practice.  In describing proposed measures to reduce ADEs,
we did not attempt to evaluate their potential effectiveness. 

To identify published studies that were relevant to our study
questions, we pursued three primary strategies.  First, we searched
computerized bibliographic databases, including Medline and Embase,
for citations related to ADEs or reactions and medication error. 
Second, we consulted with academic researchers specializing in this
area and obtained their recommendations for useful studies (see
appendix IV for a list of the outside experts we consulted).  Third,
we examined the footnotes of the studies thus identified for leads to
additional relevant studies.  We then selected studies for more
intensive analysis.  They consist of the studies that presented
primary data on the incidence of and factors related to ADEs in the
United States after 1980. 

We used a standardized data abstraction form to collect categories of
information:  how ADE (or some other corresponding term) was defined
in the study, the types and severity of events observed; the relative
frequency of events for different classes of drugs; the demographic
and clinical characteristics of the patients experiencing the ADEs;
the characteristics of the overall study population, including the
number of subjects and institutions (for example, hospitals or
nursing homes); how the subjects and institutions were selected; data
sources; completeness of the data; and the procedures used to
attribute an adverse event to a drug and to identify medication
errors.  We selected these data elements to address two questions. 
First, what did the study have to say about the frequency and
characteristics of ADEs?  Second, what limitations applied to that
information in terms of the population groups and institution types
to which its results applied, as well as any uncertainties about the
results that could derive from the way the study was conducted? 
Having made these assessments, we drew on the most appropriate
studies to address each of our three objectives. 

We obtained additional data for this study in meetings and interviews
with Food and Drug Administration (FDA) officials and academic,
industry-based, and other experts in the field.  We initiated this
process with a public forum on postmarketing surveillance, which we
organized in conjunction with the Drug Information Association's
Conference on Adverse Event Reporting in Washington, D.C., on
February 24, 1999.  Invited participants included ADE experts with a
range of institutional affiliations (see appendix IV).  The
conference and interviews provided us with background information on
the drug approval and postmarketing surveillance processes,
information on proposals to reduce the incidence of and research on
ADEs that had not yet been published, and clarification on several
methodological issues. 

ESTIMATING THE OVERALL INCIDENCE
OF ADVERSE DRUG EVENTS
========================================================== Appendix II

This appendix examines the strengths and limitations of available
data on the overall incidence of ADEs.  It explains why much of the
information currently collected on ADEs does not pertain to the
question of overall incidence.  It describes the few research studies
that have addressed ADE incidence in various treatment settings: 
hospitals, nursing homes, and emergency departments.  The
characteristics of these studies, in particular their nearly
universal focus on one or two individual institutions, determine the
limitations of current knowledge on the overall frequency of ADEs. 

   THE FEW STUDIES THAT ESTIMATE
   ADE INCIDENCE MEASURE IT IN
   SPECIFIC SETTINGS
-------------------------------------------------------- Appendix II:1

Most of the studies on ADEs assess the risks associated with
individual drugs or classes of drugsthey do not estimate overall
incidence.  In particular, a large number of studies have involved
checking whether specific events such as liver failure or allergic
reactions that appeared in case reports about a drug are in fact
linked to the use of that drug among larger numbers of patients. 
Much of the work sponsored by pharmaceutical companies and FDA falls
into this category. 

Other studies that have examined ADEs more broadly have used data
that are not appropriate for incidence calculations.  For example,
some researchers have examined ADEs by using data from the incident
reporting systems that virtually all hospitals maintain.  These
systems compile reports of adverse events submitted by hospital staff
members, including those involved in drug therapy.  The key
limitation of all such spontaneous reporting systems with regard to
estimating ADE incidence is that the cases that any one person
chooses to report probably differ substantially from the much larger
number of unreported events. 

This leaves a relative handful of studies that attempted to collect
primary data on the full range of ADEs experienced by a defined
patient population.  Their small number reflects the complexity of
the task.  It can often be difficult to distinguish adverse events
caused by a drug from those caused by the medical conditions that the
drugs are intended to treat.  To do this requires either a careful
review of the medical record for each case included in the study or,
preferably, a means of monitoring a patient's care as it is provided
in order to identify and verify ADEs when they occur.  As a practical
matter, it is most feasible to do this in specific institutional
settings.  Therefore, the studies that have produced data on the
incidence of ADEs are restricted by having focused on a particular
treatment settingfor example, hospitals and nursing homesand in
most cases to one or two specific providers. 

   STUDIES OF HOSPITALS VARY
   SUBSTANTIALLY
-------------------------------------------------------- Appendix II:2

The six studies we found that systematically collected primary data
on ADEs occurring during a hospital stay vary substantially along
multiple dimensions.\25 (See table II.1.) First, they define ADEs
very differently.  The Harvard Medical Practice Study (Brennan and
others, 1991) and its replication in Colorado and Utah (Thomas and
others, forthcoming) counted only quite severe adverse events,
whereas several others had no severity threshold.  The Brennan and
Thomas studies and, to a lesser extent, Classen (Classen and others,
1991) included some cases in which an ADE was present on admission
rather than just cases that developed during the hospital stay under
study.  Classen used the World Health Organization (WHO) definition
of an adverse drug reaction (ADR), which excludes adverse effects
produced by doses outside the normal therapeutic range.\26 The other
studies focused on events caused by drug therapy, whatever the
dosage. 

                                        Table II.1
                         
                          Six Studies of Hospital Inpatient ADEs

                                      ADEs per 100
ADE definition          ADE type      admissions       Study size  Site studied
----------------------  ------------  ------------  -------------  ----------------------
Thomas and others, forthcoming
-----------------------------------------------------------------------------------------
Injury caused by        Inpatient     0.56                 14,700  All wards except
medical management      and                                        psychiatric in a
resulting in prolonged  admission                                  stratified sample of
hospitalization or                                                 acute care hospitals
disability at                                                      in Colorado and Utah
discharge\a

Brennan and others, 1991
-----------------------------------------------------------------------------------------
Unintended injury       Inpatient     0.72                 31,429  All wards except
caused by medical       and                                        psychiatric in a
management resulting    admission                                  representative sample
in measurable                                                      of New York State
disability or                                                      acute care hospitals
prolonged
hospitalization\a

Classen and others, 1991
-----------------------------------------------------------------------------------------
Any noxious,            Inpatient\b   1.99                 36,653  All wards in one acute
unintended, and                       (1.91                        care referral center
undesired effect of a                 moderate or                  and teaching hospital
drug that occurs at                   severe)                      in Utah
doses used in humans
for prophylaxis,
diagnosis, or therapy

Bates and others, 1995a
-----------------------------------------------------------------------------------------
An injury resulting     Inpatient     6.5                   4,031  Medical, surgical, and
from administering a                  (2.8                         intensive care units
drug                                  serious,                     (ICU) in two tertiary
                                      life-                        care hospitals in the
                                      threatening,                 Boston area
                                      or fatal)

Steel and others, 1981
-----------------------------------------------------------------------------------------
Any illness resulting   Inpatient     25.5                    815  Medical, medical ICU,
from a diagnostic                     (4.91 life-                  and coronary care unit
procedure or from any                 threatening                  at one university
form of therapy\a                     or fatal)                    teaching hospital in
                                                                   Boston

Bowman and others, 1994
-----------------------------------------------------------------------------------------
Any adverse experience  Inpatient     29.7                  1,024  Internal medicine and
associated with the                   (18.7                        ICU at one county
use of a drug,                        moderate or                  general hospital in
including experiences                 severe)                      Indiana
rated possible on
the Naranjo scale
-----------------------------------------------------------------------------------------
\a We extracted from this definition the injuries that drugs caused. 

\b Seven percent of reported ADEs were present at admission. 

The studies also differed in their approach and intensity of data
collection.  The Brennan and Thomas studies were strictly
retrospective reviews of medical records.  The four other studies in
table II.1 all collected data prospectively (while patients were
undergoing treatment), although Classen first screened cases by
applying computer algorithms to patients' records in order to
identify the cases most likely to experience an ADE.  Prospective
data collection meant that information from a medical record could be
supplemented with interviews with the staff who cared for a
particular patient. 

Differences in the studies' definitions affected the ADE rates they
found.  The two studies with the apparently least restrictive
definitions, Steel and Bowman, reported the highest rates. 
Similarly, the differences in the studies narrow substantially when
the rates from the Brennan and Thomas studies are compared with those
for the more severe cases in the other studies. 

In contrast, there is no apparent relationship between type of
hospital (university or teaching versus community) and rates of
observed ADEs.  The three studies of individual university hospitals
varied from one another by a wide margin.  Meanwhile, the single
available study of a community hospital (Bowman) had the highest
reported rate, while the Brennan and Thomas studies, which combined
both teaching and nonteaching hospitals (but did not break them out
separately for ADEs), had the lowest rates.  These variations may
largely reflect differences on other dimensions (for example, the
definition of ADE) as well as potentially large variations across
individual institutions.  Whether teaching or community hospitals are
likely to have higher ADE rates remains an open question. 

--------------------
\25 These studies attempted, within their given institutional focus,
to capture the experience of a broad range of patients in terms of
demographics and clinical conditions.  Numerous other studies focus
exclusively on various patient subgroupssuch as children, the
elderly, patients with psychiatric conditions, and patients receiving
specific drugs or experiencing specific types of ADEsthat we judged
to be less useful for assessing the overall incidence of ADEs. 

\26 WHO defines ADR as the noxious and unintended effects of a drug
that occur at doses used in humans for prophylaxis, diagnosis, or
therapy.  This definition excludes some medication errors (such as
accidental overdoses) but includes others (such as prescribing the
wrong drug). 

   STUDIES OF NURSING HOMES
   SUGGEST THAT ADES ARE COMMON
   AMONG RESIDENTS
-------------------------------------------------------- Appendix II:3

Nursing home residents, like hospital patients, are typically sick or
frail and receive many medications, circumstances that would tend to
increase their vulnerability to ADEs.  Indeed, the three small-scale
studies we identified that sought to measure the incidence of ADEs in
nursing homes were all quite consistent in finding a high prevalence
of ADEs.  (See table II.2.)

                                        Table II.2
                         
                         Three Studies of ADEs in Long-Term-Care
                                        Facilities

                               ADEs per
ADE definition                 patient-month     Study size  Site studied
-----------------------------  --------------  ------------  ----------------------------
Gerety and others, 1993
-----------------------------------------------------------------------------------------
Known reaction to drug,        0.44                     175  One university-affiliated
appropriate temporal           (115 per 100                  Department of Veterans'
sequence, no alternative       admissions)                   Affairs nursing home
explanation (Naranjo
algorithm)

Cooper, 1996
-----------------------------------------------------------------------------------------
Any noxious and unintended     134 per 100              332  Two rural nursing homes
effect of a drug that occurs   admissions\a
at doses used in humans for
prophylaxis, diagnosis, or
therapy judged at least
probable on the Naranjo
scale

Cooper, 1986
-----------------------------------------------------------------------------------------
Drug-related problem, any      0.71                     102  One long-term-care facility
unwanted consequence from      (1,200 per 100                attached to a small
using or not using drug        admissions)                   community hospital
therapy
-----------------------------------------------------------------------------------------
\a ADEs not reported per month. 

As in the studies of inpatient ADEs, the definition of ADE varied
widely.  It could include not only the kinds of events considered to
be an ADEfor example, Cooper (1986) counted an inability to pay for
prescribed medicationsbut also the degree of certainty required that
a drug had caused the event in question.  The two other studies, with
narrower definitions, reported markedly lower rates, but even they
found that a majority of nursing home residents experienced ADEs.\27

--------------------
\27 The substantial variation in both the ADE rates reported and the
definitions employed, combined with the small number of institutions
and patients studied, leaves considerable uncertainty about the
specific rate of ADEs among nursing home residents.  A more precise
estimate would require one or more larger-scale studies that assessed
the frequency of ADEs across multiple facilities.  One such
assessment is under way in 18 nursing homes in central and eastern
Massachusetts. 

   THE DATA ON EMERGENCY
   DEPARTMENT VISITS AND HOSPITAL
   ADMISSIONS ARE INSUFFICIENT FOR
   ESTIMATING OVERALL ADE
   FREQUENCY
-------------------------------------------------------- Appendix II:4

Unlike ADE rates for inpatient and long-term care, which apply to
events in the specific institutions where the data are collected, ADE
rates reported for emergency department visits and hospital
admissions relate to care that previous health care providers have
already given to patients.  The rates reported for inpatient ADEs
have quite a different meaning from those presented in either studies
of emergency department visits or hospital admissions.  The former
report on the probability that an individual patient will experience
an ADE, while the latter focus on the proportion of emergency
department and hospital patient volume brought about by ADEs. 
Studies of emergency department visits and hospital admissions differ
primarily in level of severity.  The first examine ADEs serious
enough to motivate an emergency department visit, while the second
are about patients whose ADEs required inpatient care. 

To use data from studies of emergency department visits and hospital
admissions to estimate an overall rate of ADEs for a given
population, two requirements must be satisfied.  First, the data must
be from all the hospitals serving the targeted population or at least
from a representative sample.  Second, additional data have to be
collected on ADEs that took place during a hospital episode and were
treated in the hospital where they occurred.  Patients transferred to
another hospital to treat an ADE would normally appear in the
admissions data for the receiving hospital, but those treated for
ADEs in the hospital where they occurred would not be associated with
any admission.\28

Each study that we found on emergency department visits and hospital
admissions for ADEs was limited to a single institution.  (See tables
II.3 and II.4.) The rates these studies reported, therefore, depend
on the particular mix of patients each hospital attracted relative to
the alternative providers available in its geographic area.  Without
information about ADE admissions to other hospitals in these areas,
there is no way of knowing how similar the rates the studies reported
are to those prevailing among other providers. 

                                        Table II.3
                         
                           Four Studies of Emergency Department
                                     Visits for ADEs

                                 ADEs per 100
ADE definition                         visits    Study size  Site studied
-------------------------------  ------------  ------------  ----------------------------
Schneitman-McIntire and others, 1996
-----------------------------------------------------------------------------------------
Unfavorable effect of drug use          1.73        62,216  The emergency department of
 plus poor compliance,                                        a health maintenance
 inappropriate self-                                          organization medical center
 medication, inappropriate                                    in Walnut Creek, Calif.
 prescribing, and drug                   1.23
 interactions
                                        0.244
 ADR: undesired adverse
 effects + allergy + drug
 interactions

 Hospital admission for ADE

Prince and others, 1992
-----------------------------------------------------------------------------------------
Drug-related illness                    2.88        10,184  One tertiary care hospital
                                                              in Pittsburgh, Penna.
 ADR: undesirable event                0.815
 reasonably and temporally
 associated with the use of a
 drug at normal doses + drug
 interactions                           0.697

 Hospital admission for drug-
 related illness

Dennehy and others, 1996
-----------------------------------------------------------------------------------------
Drug-related illness                    3.97         1,260  One university teaching
                                                              hospital in San Francisco,
 ADR: noxious and unintended            1.27                 Calif.
 effect or the result of drug
 therapy
                                        0.635
 Hospital admission for drug-
 related illness

Smith and others. 1997
-----------------------------------------------------------------------------------------
Drug-related problem                    4.24         5,757  One university teaching
                                                              hospital in Lexington, Ky.
 ADR: any undesirable or               0.469
 unexpected drug-related event
 + drug interactions
                                        0.625
 Hospital admission for drug-
 related problem
-----------------------------------------------------------------------------------------

                                        Table II.4
                         
                         Four Studies of Hospital Admissions for
                                           ADEs

                                     ADE rate
                                        among
                                     hospital     Study
ADE definition                     admissions      size  Site studied
-------------------------------  ------------  --------  --------------------------------
Lakshmanan and others, 1986
-----------------------------------------------------------------------------------------
Admission for adverse reactions          4.2%       834  Medical, ICU, and oncology wards
 to medical therapy, surgery,                             at one public teaching hospital
 or diagnostic procedures\a                               in Cleveland, Ohio

Nelson and Talbert, 1996
-----------------------------------------------------------------------------------------
Known reaction to drug;                   5.3       452  Medical, cardiac, and ICU wards
 appropriate temporal sequence;                           at one university-affiliated
 no alternative explanation                               county hospital in Bexar
 (Naranjo and Hallas                                      County, Tex.
 algorithms)

Bigby and others, 1987
-----------------------------------------------------------------------------------------
Hospital admission from                   6.9       686  One teaching hospital in
 emergency department with ADE,                           Cambridge, Mass.
 ADE not defined

Colt and Shapiro, 1989
-----------------------------------------------------------------------------------------
Undesired or unintended                   9.4       244  Medical ward at one community
 response to medication at                                teaching hospital in
 appropriate dosage for                                   Pittsburgh, Penna.
 prophylaxis, diagnosis, or
 therapy
-----------------------------------------------------------------------------------------
\a We extracted from this definition the injuries that drugs caused. 

Whatever the differences in patient populations examined in these
studies, they reported relatively consistent ADE rates.  This may
stem in part from the fairly uniform definition of ADE they applied. 
For emergency department visits, the studies stated that around 2 to
4 percent involved a rather broad category of drug-related illness or
problems, while approximately 1 percent related to ADRs more narrowly
defined.  The range in studies of hospital admissions is somewhat
wider and higherroughly 4 to 9 percent.  The fairly diverse
hospitals, both geographically (California, Pennsylvania, and Texas)
and in institutional affiliation (public, private community,
university, and health maintenance organization), yielded relatively
consistent results. 

Nevertheless, none of these studies provides information as complete
on the full range of ADEs as do the Brennan and Thomas studies.  They
alone both build on a sample representative of a broad-based
population (if still short of nationwide) and have a data collection
approach that encompasses ADEs wherever they occurduring a hospital
stay or before admission, either in another hospital or in an
outpatient setting.  The only ADEs missed in the Brennan and Thomas
studies are those that were not treated in a hospital.  They would
also be missed in all the other studies, except by studies of
patients with ADEs who made emergency department visits. 

--------------------
\28 The results of the Harvard Medical Practice Study suggest that a
large proportion of serious ADEs occur and are treated during the
same hospitalization.  Although separate figures for drug-related
adverse events were not reported, for adverse events overall, 49
percent were in this category.  Most of the remaining adverse events
would appear as hospital admissions, including 31 percent that
occurred during a previous hospitalization and 15 percent that
occurred in an outpatient setting (Brennan and others, 1991, p. 
373). 

SIXTEEN STUDIES ESTIMATING ADE
COSTS IN THE UNITED STATES
========================================================= Appendix III

                               Eligible
Study         Site             patients          Method            Cost estimate
------------  ---------------  ----------------  ----------------  ----------------------
Bates and     Brigham and      All 4,108         Compare           An increased hospital
others, 1997  Women's          patients          comparable        cost of $2,595 per ADE
              Hospital and     admitted to a     patients with     and $4685 per
              Massachusetts    stratified        ADEs to those     preventable ADE; $5.6
              General          random sample of  without           million per year in
              Hospital,        medical and                         additional costs with
              Boston, Mass.    surgical units                      $2.8 million
                               and intensive                       attributable to
                               care and                            preventable ADEs
                               nonintensive
                               care units;
                               excluded
                               obstetric units

Bootman,      Nursing          No patients       Expert panel      $7.6 billion in costs
Harrison,     facilities (no   studied           estimated ADE     to third-party payer
and Cox,      specific sites                     rates and         from drug-related
1997          studied)                           treatments        morbidity and
                                                 needed            mortality in nursing
                                                                   homes

Classen and   LDS Hospital,    All 91,574        Compare           Hospital cost of
others, 1997  Salt Lake City,  hospital          comparable        $2,013 per ADE; 4-
              Utah             admissions        patients with     year cost of
                                                 ADEs to those     $4,482,951 (excludes
                                                 without           liability costs and
                                                                   the cost of injury to
                                                                   patients)

Col and       Unspecified      315               Assess ADRs in    Hospital cost of
others, 1990  community        consecutively     patients who      $2,147 per patient
              teaching         admitted elderly  experienced       admitted with ADE
              hospital         patients          them; no          related to
                                                 comparison group  noncompliance (total
                                                                   cost of $77,289 for 3
                                                                   months); cost of
                                                                   $4,237 per patient
                                                                   admitted with ADR
                                                                   (total cost of
                                                                   $224,542 for 3 months)

Cooper, 1987  Unspecified      All residents;    Assess ADRs in    $3,749 per episode
              nursing home     sample of 6       patients who      that resulted in
                               cases of ADEs to  experienced       hospitalization; up to
                               generate cost     them; no          $340,942 in costs
                               estimate          comparison group  could have been
                                                                   avoided over a 2-year
                                                                   period.

Cullen and    Brigham and      All 4,031         Comparisons       Costs after ADE of
others, 1997  Women's          patients          between patients  $9,192 per ADE in
              Hospital and     admitted to       who had first     surgical general care
              Massachusetts    sample of         ADE in the ICU    unit, $17,437 in
              General          intensive care    and the           medical general care
              Hospital,        and general care  remaining ICU     unit, $17,577 in
              Boston, Mass.    units             patients and      medical intensive care
                                                 between medical   unit, and $20,959 in
                                                 and surgical      surgical intensive
                                                 patients with an  care unit; total cost
                                                 ADE               $1,366,840 over 6
                                                                   months

Dennehy and   University of    1,260 emergency   Assess ADEs in    Hospital costs of $283
others, 1996  California, San  department        patients who      for patients with ADE
              Francisco        patients (68      experienced       who were not
                               percent of all    them; no          hospitalized, $2,815
                               emergency         comparison group  for those who were;
                               department                          $308 for patients with
                               patients)                           preventable ADEs not
                                                                   hospitalized, $2,752
                                                                   for those who were;
                                                                   total annual cost
                                                                   $602,597, with
                                                                   $391,342 coming from
                                                                   avoidable ADEs

Evans and     LDS Hospital,    All hospitalized  Compare patients  Average cost of
others, 1993  Salt Lake City,  patients          with ADEs to      hospitalization for
              Utah                               those without     patients with ADEs
                                                                   from allergic or
                                                                   idiosyncratic
                                                                   reactions was $30,617,
                                                                   from known toxicities
                                                                   $23,256; average cost
                                                                   for patients without
                                                                   ADEs was $6,320

Evans and     LDS Hospital,    All 60,836        Compare           $1,939 higher cost for
others,       Salt Lake City,  hospitalized      comparable        patients with an ADE
1994b         Utah             patients          patients with     than for those
                                                 ADEs to those     without; total annual
                                                 without           cost of $1,103,291

Johnson and   Unspecified      No patients       Expert panel      $76.6 billion in
others, 1995  ambulatory       studied           estimated ADE     annual costs to third-
              settings                           rates and         party payers
                                                 treatments        associated with
                                                 needed            management of drug-
                                                                   related mortality and
                                                                   morbidity

Johnson and   Unspecified      No patients       Expert panel      $45.6 billion savings
others, 1997  ambulatory       studied           estimated ADE     for third-party payers
              settings                           rates and         if pharmaceutical care
                                                 treatments        was instituted
                                                 needed            nationwide, a 59.6
                                                                   percent reduction from
                                                                   $76.6 billion

Prince and    Mercy Hospital,  All 10,184        Assess ADEs in    Average hospital
others, 1992  Pittsburgh,      emergency room    patients who      charge per admission
              Penna.           visits            experienced       for a patient with an
                                                 them; no          ADE was $8,888; total
                                                 comparison group  annual cost of
                                                                   $631,048

Schneider     Ohio State       109 patient       Assess ADEs in    Average institutional
and others,   University       charts reviewed;  patients who      cost of $783 per ADE;
1995          Medical Center,  selection         experienced       total annual cost of
              Columbus, Ohio   criteria not      them; no          $1,497,148
                               stated            comparison
                                                 group

Stoukides,    Roger Williams   All 13,703        Assess ADEs in    Average cost of
D'Agostino,   Medical Center,  emergency room    patients who      $333.81 per emergency
and Kaufman,  Brown            visits            experienced       department visit;
1993          University,                        them; no          total cost of
              Providence,                        comparison group  $39,389.58 for 6
              R.I.                                                 months

Sullivan,     Unspecified      2,942 hospital    Metaanalysis      $8.5 billion spent on
Kreling, and  hospitals        admissions from                     hospitalizations in
Hazlet, 1990  (based on other  seven studies                       1986 because of
              studies)         reviewed                            noncompliance

Thomas and    28 hospitals in  14,732 randomly   Physicians and    $5.2 billion hospital
others, 1999  Colorado and     selected          malpractice       costs to treat ADEs
              Utah             hospital          claims adjusters  nationwide; $12.2
                               discharges        estimated the     billion, including
                                                 extent of         outpatient care, lost
                                                 disability and    wages, and lost
                                                 future health     household production
                                                 care use from
                                                 data from
                                                 medical records
                                                 of patients with
                                                 ADEs
-----------------------------------------------------------------------------------------

EXPERTS WE CONSULTED
========================================================== Appendix IV

David Bates, M.D., Chief, Division of General Medicine, Brigham &
Women's Hospital, Boston, Mass.\29

J.  Lyle Bootman, Ph.D., Executive Director, Center for Health
Outcomes and PharmacoEconomic Research, College of Pharmacy,
University of Arizona, Tucson, Ariz. 

Michael R.  Cohen, M.S., President, Institute for Safe Medication
Practices, Huntingdon Valley, Penna. 

Nancy A.  Dreyer, Ph.D., Chief Executive Officer, Epidemiology
Resources Inc., Newton Lower Falls, Mass. 

David I.  Goldsmith, M.D., Senior Medical Director, Safety
Surveillance, Sanofi Pharmaceuticals, New York, N.Y.\29

Hershel Jick, M.D., Director, Boston Collaborative Drug Surveillance
Program, Lexington, Mass. 

Judith K.  Jones, M.D., Ph.D., President and Chief Executive Officer,
The Degge Group, Arlington, Va.\29

Lucian Leape, Ph.D., Professor, Harvard School of Public Health,
Boston, Mass. 

Murray M.  Lumpkin, M.D., Deputy Center Director, Center for Drug
Evaluation and Research, U.S.  Food and Drug Administration,
Rockville, Md.\29

Henri R.  Manasse, Jr., Ph.D., Sc.D., Executive Vice President and
Chief Executive Officer, American Society of Health-System
Pharmacists, Bethesda, Md.\29

Thomas J.  Moore, Fellow, Center for Health Policy Research, School
of Public Health and Health Services, George Washington University,
Washington, D.C.\29

Robert C.  Nelson, Ph.D., RCN Associates, Annapolis, Md., and Chair,
Committee on Quality Data for Risk Assessment of Drugs, International
Society for Pharmacoepidemiology, Washington, D.C.\29

Richard Platt, M.D., Director of Research, Harvard Pilgrim
Healthcare, Boston, Mass. 

Larry D.  Sasich, Pharm.D., M.P.H., Research Analyst, Public
Citizen's Health Research Group, Washington, D.C.\30

Judith M.  Sills, Pharm.D., Senior Director, U.S.  Regulatory Affairs
and Global Product Safety, Warner-Lambert Consumer Healthcare, Morris
Plains, N.J.\30

Bert Spilker, M.D., Ph.D., Senior Vice President for Scientific and
Regulatory Affairs, Pharmaceutical Research and Manufacturers of
America, Washington, D.C.\30

Brian Strom, M.D., M.P.H., Director, Center for Clinical Epidemiology
and Biostatistics, University of Pennsylvania, Philadelphia,
Penna.\30

Eleanor M.  Vogt, R.Ph., Ph.D., Senior Fellow, National Patient
Safety Foundation, American Medical Association, Washington, D.C.\30

Ray Woosley, M.D., Ph.D., Chairman, Department of Pharmacology,
Georgetown University Medical Center, Washington, D.C.\30

--------------------
\29 Invited participant at the joint General Accounting Office and
Drug Information Association session, Postmarketing Surveillance: 
Considerations for Policymakers, conference on Adverse Event
Reporting:  From Theory to Practice, Washington, D.C., Feb.  24,
1999. 

\30 Invited participant at the joint General Accounting Office and
Drug Information Association session, Postmarketing Surveillance: 
Considerations for Policymakers, conference on Adverse Event
Reporting:  From Theory to Practice, Washington, D.C., Feb.  24,
1999. 

BIBLIOGRAPHY
============================================================ Chapter 0

Allan, E.L., and others.  Dispensing Errors and Counseling in
Community Practice. American Pharmacy, Vol.  NS35 (1995), pp. 
25-33. 

Barker, K.N., and others.  Consultation Evaluation of a Hospital
Medication System:  Analysis of the Existing System. American
Journal of Hospital Pharmacy, Vol.  41 (1984), pp.  2009-16. 

Bates, D.W., and others.  "Incidence and Preventability of Adverse
Drug Events in Hospitalized Adults." Journal of General Internal
Medicine, Vol.  8 (1993), pp.  289-94. 

Bates, D.W., and others.  "Incidence of Adverse Drug Events and
Potential Adverse Drug Events:  Implications for Prevention." Journal
of the American Medical Association, Vol.  274, No.  1 (1995a), pp. 
29-34. 

Bates, D.W., and others.  Relationship Between Medication Errors and
Adverse Drug Events. Journal of General Internal Medicine, Vol.  10
(1995b), pp.  199-205. 

Bates, D.W., and others.  "The Costs of Adverse Drug Events in
Hospitalized Patients." Journal of the American Medical Association,
Vol.  277, No.  4 (1997), pp.  307-11. 

Bates, D.W., and others.  "Effect of Computerized Physician Order
Entry and a Team Intervention on Prevention of Serious Medication
Errors." Journal of the American Medical Association, Vol.  280, No. 
15 (1998), pp.  1311-16. 

Baum, C., and others.  The Spontaneous Reporting System in the
United States. In B.L.  Strom, ed.  Pharmacoepidemiology, 2\nd ed. 
New York:  John Wiley and Sons, 1994. 

Beers, M.H., M.  Storrie, and G.  Lee.  Potential Adverse Drug
Interactions in the Emergency Room:  An Issue in the Quality of
Care. Annals of Internal Medicine, Vol.  112 (1990), pp.  61-64. 

Beers, M.H., and others.  Inappropriate Medication Prescribing in
Skilled-Nursing Facilities. Annals of Internal Medicine, Vol.  117
(1992), pp.  684-89. 

Bero, L.A., H.L.  Lipton, and J.A.  Bird.  Characterization of
Geriatric Drug-Related Hospital Readmissions. Medical Care, Vol. 
29, No.  10 (Oct.  1991), pp.  989-1003. 

Bigby, J., and others.  Assessing the Preventability of Emergency
Hospital Admissions:  A Method for Evaluating the Quality of Medical
Care in a Primary Care Facility. The American Journal of Medicine,
Vol.  83 (1987), pp.  1031-36. 

Bootman, J.L., D.L.  Harrison, and E.  Cox.  The Health Care Cost of
Drug-Related Morbidity and Mortality in Nursing Facilities. Archives
of Internal Medicine, Vol.  157 (1997), pp.  2089-96. 

Bowman, L., and others.  Incidence of Adverse Drug Reactions in
Adult Medical Inpatients. Canadian Journal of Hospital Pharmacy,
Vol.  47 (1994), pp.  209-16. 

Brennan, T.A., and others.  Incidence of Adverse Events and
Negligence in Hospitalized Patients:  Results of the Harvard Medical
Practice Study I. New England Journal of Medicine, Vol.  324, No.  6
(1991), pp.  370-76. 

Classen, D.C., and others.  Computerized Surveillance of Adverse
Drug Events in Hospital Patients. Journal of the American Medical
Association, Vol.  266 (1991), pp.  2847-51. 

Classen, D.C., and others.  "Adverse Drug Events in Hospitalized
Patients:  Excess Length of Stay, Extra Costs, and Attributable
Mortality." Journal of the American Medical Association, Vol.  277,
No.  4 (1997), pp.  301-06. 

Col, N., and others.  The Role of Medication Noncompliance and
Adverse Drug Reactions in Hospitalizations of the Elderly. Archives
of Internal Medicine, Vol.  150 (1990), pp.  841-45. 

Colt, H.G., and A.P.  Shapiro.  Drug-Induced Illness as a Cause for
Admission to a Community Hospital. Journal of the American
Geriatrics Society, Vol.  37 (1989), pp.  323-26. 

Cooper, J.W.  Drug-Related Problems in a Geriatric Long Term Care
Facility. Journal of Geriatric Drug Therapy, Vol.  1 (1986), pp. 
47-68. 

Cooper, J.W.  Adverse Drug Reactions and Interactions in a Nursing
Home. Nursing Homes and Senior Citizen Care, Vol.  36 (1987), pp. 
7-11. 

Cooper, J.W.  Probable Adverse Drug Reactions in a Rural Geriatric
Nursing Home Population:  A Four-Year Study. Journal of the American
Geriatrics Society, Vol.  44 (1996), pp.  194-97. 

Cooper, J.W.  Adverse Drug Reaction-Related Hospitalizations of
Nursing Facility Patients:  A 4-Year Study. Southern Medical
Journal, Vol.  92 (1999), pp.  485-90. 

Corr, K., and R.  Stoller.  "Adverse Drug Event Reporting at Veterans
Affairs Facilities (Letter). American Journal of Health-System
Pharmacy, Vol.  53 (1996), pp.  314-15. 

Cullen, D.J., and others.  The Incident Reporting System Does Not
Detect Adverse Drug Events:  A Problem for Quality Improvement.
Joint Commission Journal on Quality Improvement, Vol.  21, No.  10
(Oct.  1995), pp.  541-48. 

Cullen, D.J., and others.  Preventable Adverse Drug Events in
Hospitalized Patients:  A Comparative Study of Intensive Care and
General Care Units. Critical Care Medicine, Vol.  25, No.  8 (1997),
pp.  1289-97. 

Dahlgren, A.F.  "Adverse Drug Reactions in Home Care Patients
Receiving Nafcillin or Oxacillin. American Journal of Health-System
Pharmacy, Vol.  54 (1997), pp.  1176-79. 

Dalton-Bunnow, M.F., and F.J.  Halvachs.  Computer-Assisted Use of
Tracer Antidote Drugs to Increase Detection of Adverse Drug
Reactions:  A Retrospective and Concurrent Trial. Hospital Pharmacy,
Vol.  28 (1993), pp.  746-55. 

Dennehy, C.E., and others.  "Drug-Related Illness in Emergency
Department Patients. American Journal of Health-System Pharmacy,
Vol.  53 (1996), pp.  1422-26. 

Echt, D.S., and others.  Mortality and Morbidity in Patients
Receiving Encainide, Flecainide, or Placebo. New England Journal of
Medicine, Vol.  324, No.  12 (1991), pp.  781-88. 

Einarson, Thomas R.  Drug-related Hospital Admissions. Annals of
Pharmacotherapy, Vol.  27 (1993), pp.  832-40. 

Evans, R.S., and others.  Prevention of Adverse Drug Events through
Computerized Surveillance. Sixteenth Annual Symposium on Computer
Applications in Medical Care.  New York:  McGraw-Hill, 1993.  Pp. 
437-41. 

Evans, R.S., and others.  Preventing Adverse Drug Events in
Hospitalized Patients. The Annals of Pharmacotherapy, Vol.  28
(1994a), pp, 523-27. 

Evans, R.S., and others.  Using a Hospital Information System to
Assess the Effects of Adverse Drug Events. Seventeenth Annual
Symposium on Computer Applications in Medical Care.  New York: 
McGraw-Hill, 1994b.  Pp.  161-65. 

Faich, G.A.  Adverse-Drug-Reaction Monitoring. New England Journal
of Medicine, Vol.  314, No.  24 (1986), pp.  1589-92. 

Flynn, E.A., R.E.  Pearson, and K.N.  Barker.  Observational Study
of Accuracy in Compounding I.V.  Admixtures at Five Hospitals.
American Journal of Health-System Pharmacy, Vol.  54 (1997), pp. 
904-12. 

Folli, H.L., and others.  Medication Error Prevention by Clinical
Pharmacists in Two Children's Hospitals. Pediatrics, Vol.  79, No. 
5 (May 1987), pp.  718-22. 

Fremont-Smith, K.  Adverse Drug Reactions in Hospitalized Patients
(Letter). Journal of the American Medical Association, Vol.  280,
No.  20 (1998), p.  1741. 

Gerety, M.B., and others.  Adverse Events Related to Drugs and Drug
Withdrawal in Nursing Home Residents. Journal of the American
Geriatric Society, Vol.  41 (1993), pp.  1326-32. 

Goldman, S.A., and others.  The Clinical Impact of Adverse Event
Reporting.  Washington, D.C.:  FDA, Center for Drug Evaluation and
Research, Oct.  1996. 

Gray, S.L., and others.  "Adverse Drug Events in Hospitalized
Elderly." Journal of Gerontology:  Medical Sciences, Vol.  53A, No. 
1 (1998), pp.  M59-M63. 

Grymonpre, R.E., and others.  Drug-Associated Hospital Admissions in
Older Medical Patients. Journal of the American Geriatrics Society,
Vol.  36 (1988), pp.  1092-98. 

Gurwitz, J.H., and J.  Avorn.  The Ambiguous Relation Between Aging
and Adverse Drug Reactions. Annals of Internal Medicine, Vol.  114,
No.  11 (1991), pp.  956-66. 

Ives, T.J., and others.  Drug-Related Admissions to a Family
Medicine Inpatient Service. Archives of Internal Medicine, Vol.  147
(1987), pp.  1117-20. 

Jick, H.  Boston Collaborative Drug Surveillance Program. Hospital
Pharmacy, Vol.  27, No.  4S (Apr.  1992), pp.  S3-S55. 

Johnson, J.A., and others.  "Drug-Related Morbidity and Mortality:  A
Cost-of-Illness Model. Archives of Internal Medicine, Vol.  155
(1995), pp.  1949-56. 

Johnson, J.A., and others.  "Drug-Related Morbidity and Mortality and
the Economic Impact of Pharmaceutical Care. American Journal of
Health-System Pharmacy, Vol.  54 (1997), pp.  554-58. 

Kimmel, S.E., and others.  Adverse Events After Protamine
Administration in Patients Undergoing Cardiopulmonary Bypass:  Risks
and Predictors of Under-Reporting. Journal of Clinical Epidemiology,
Vol.  51, No.  1 (1998), pp.  1-10. 

Klein, L.E., and others.  Medication Problems Among Outpatients:  A
Study with Emphasis on the Elderly. Archives of Internal Medicine,
Vol.  144 (June 1984), pp.  1185-88. 

Lakshmanan, M.C., and others.  Hospital Admissions Caused by
Iatrogenic Disease. Archives of Internal Medicine, Vol.  146 (1986),
pp.  1931-34. 

Lazarou, J., B.  Pomeranz, B., and P.  Corey.  Incidence of Adverse
Drug Reactions in Hospitalized Patients:  A Meta-analysis of
Prospective Studies. Journal of the American Medical Association,
Vol.  279, No.  15 (1998a), pp.  1200-05. 

Lazarou, J., B.  Pomeranz, and P.  Corey.  Adverse Drug Reactions in
Hospitalized Patients (Response). Journal of the American Medical
Association, Vol.  280, No.  20 (1998b), p.  1743. 

Leape, L.L.  "Error in Medicine." Journal of the American Medical
Association, Vol.  272, No.  23 (1994), pp.  1851-57. 

Leape, L.L., and others.  The Nature of Adverse Events in
Hospitalized Patients:  Results of the Harvard Medical Practice Study
II. New England Journal of Medicine, Vol.  324, No.  6 (1991), pp. 
377-84. 

Leape, L.L., and others.  Preventing Medical Injury. Quality Review
Bulletin, Vol.  19 (1993), pp.  144-49. 

Leape, L.L., and others.  "Systems Analysis of Adverse Drug Events."
Journal of the American Medical Association, Vol.  274, No.  1
(1995), pp.  35-43. 

Lesar, T.S., and others.  Medication Prescribing Errors in a
Teaching Hospital. Journal of the American Medical Association, Vol. 
263 (1990), pp.  2329-34. 

Lesar, T.S., and others.  "Factors Related to Errors in Medication
Prescribing." Journal of the American Medical Association, Vol.  277,
No.  4 (1997), pp.  312-17. 

Martin, R.M., and others.  Age and Sex Distribution of Suspected
Adverse Drug Reactions to Newly Marketed Drugs in General Practice in
England:  Analysis of 48 Cohort Studies. British Journal of Clinical
Pharmacology, Vol.  46 (1998), pp.  505-11. 

Mitchell, A.A., and others.  Adverse Drug Reactions in Children
Leading to Hospital Admission. Pediatrics, Vol.  82 (1988), pp. 
24-29. 

Nelson, K.M., and R.L.  Talbert.  Drug-Related Hospital Admissions.
Pharmacotherapy, Vol.  16 (1996), pp.  701-07. 

Orsini, M.J., and others.  An ADR Surveillance Program:  Increasing
Quality, Number of Incidence Reports. Formulary, Vol.  30 (1995),
pp.  454-61. 

Pearson, T.F., and others.  Factors Associated with Preventable
Adverse Drug Reactions. American Journal of Hospital Pharmacy, Vol. 
51 (1994), pp.  2268-72. 

Phillips, D.P., N.  Christenfeld, and L.M.  Glynn.  Increase in U.S. 
Medication-Error Deaths Between 1983 and 1993. The Lancet, Vol.  351
(1998), pp.  643-44. 

Pitt, B., and others.  The Effect of Spironolactone on Morbidity and
Mortality in Patients with Severe Heart Failure. New England Journal
of Medicine, Vol.  341, No.  10 (1999), pp.  709-17. 

Prince, B.S., and others.  Drug-Related Emergency Department Visits
and Hospital Admissions. American Journal of Hospital Pharmacy, Vol. 
49 (1992), pp.  1696-1700. 

Raju, T.N.K., and others.  "Medication Errors in Neonatal and
Paediatric Intensive-Care Units. The Lancet, Vol.  2 (1989), pp. 
374-76. 

Raschke, R.A., and others.  "A Computer Alert System to Prevent
Injury From Adverse Drug Events. Journal of the American Medical
Association, Vol.  280, No.  15 (1998), pp.  1317-20. 

Rupp, M.T., M.  DeYoung, and S.W.  Schondelmeyer.  Prescribing
Problems and Pharmacist Interventions in Community Practice. Medical
Care, Vol.  30 (1992), pp.  926-40. 

Salem, R.B., T.M.  Keane, and J.G.  Williams.  Drug-Related
Admissions to a Veterans' Administration Psychiatric Unit. Drug
Intelligence and Clinical Pharmacy, Vol.  18 (1984), pp.  74-76. 

Saltiel, E., E.  Johnson, and R.  Shane.  A Team Approach to Adverse
Drug Reaction Surveillance:  Success at a Tertiary Care Hospital.
Hospital Formulary, Vol.  30 (1995), pp.  226-32. 

Schneider, J.K., L.C.  Mion, and J.D.  Frengley.  Adverse Drug
Reactions in an Elderly Outpatient Population. American Journal of
Hospital Pharmacy, Vol.  49 (1992), pp.  90-96. 

Schneider, P.J., and others.  Cost of Medication-Related Problems at
a University Hospital. American Journal of Health-System Pharmacy,
Vol.  52 (1995), pp.  2415-18. 

Schernitski, P., and others.  Demographic Characteristics of Elderly
Drug Overdose Patients Admitted to a Hospital Emergency Department.
Journal of the American Geriatrics Society, Vol.  28 (1980), pp. 
544-46. 

Schneitman-McIntire, O., and others.  "Medication Misadventures
Resulting in Emergency Department Visits at an HMO Medical Center.
American Journal of Health-System Pharmacy, Vol.  53 (1996), pp. 
1416-22. 

Seeger, J.D., and others.  "Characteristics Associated with Ability
to Prevent Adverse Drug Reactions in Hospitalized Patients."
Pharmacotherapy, Vol.  18, No.  6 (1998), pp.  1284-89. 

Shaughnessy, A.F., and R.O.  Nickel.  Prescription-Writing Patterns
and Errors in a Family Medicine Residency Program. The Journal of
Family Practice, Vol.  29 (1989), pp.  290-95. 

Smith, K.M., and others.  "Drug-Related Problems in Emergency
Department Patients. American Journal of Health-System Pharmacy,
Vol.  54 (1997), pp.  295-98. 

Steel, K., and others.  Iatrogenic Illness on a General Medical
Service at a University Hospital. New England Journal of Medicine,
Vol.  304 (1981), pp.  638-42. 

Stewart, R.B., and others.  Drug-related Admissions to an Inpatient
Psychiatric Unit. American Journal of Psychiatry, Vol.  137 (1980),
pp.  1093-95. 

Stoukides, C.A., P.R.  D'Agostino, and M.B.  Kaufman.  Adverse Drug
Reaction Surveillance in an Emergency Room. American Society of
Hospital Pharmacy, Vol.  50 (1993), pp.  712-14. 

Sullivan, S.D., D.H.  Kreling, and T.K.  Hazlet.  Noncompliance with
Medication Regimens and Subsequent Hospitalizations:  A Literature
Analysis and Cost of Hospitalization Estimate. Journal of Research
in Pharmaceutical Economics, Vol.  2 (1990), pp.  19-33. 

Thomas, E.J., and others.  Costs of Medical Injuries in Utah and
Colorado. Inquiry, Vol.  36 (Fall 1999), pp.  255-64. 

Thomas, E.J., and others.  Incidence and Types of Adverse Events and
Negligent Care in Utah and Colorado. Medical Care, forthcoming. 

Warren, J.L., and others.  Hospitalizations With Adverse Events
Caused by Digitalis Therapy Among Elderly Medicare Beneficiaries.
Archives of Internal Medicine, Vol.  154 (1994), pp.  1482-87. 

Wolfe, M.M., and others.  Gastrointestinal Toxicity of Nonsteroidal
Antiinflammatory Drugs. New England Journal of Medicine, Vol.  340,
No.  24 (1999), pp.  1888-99. 

*** End of document. ***