Adverse Events: Surveillance Systems for Adverse Events and Medical
Errors (Testimony, 02/09/2000, GAO/T-HEHS-00-61).

Pursuant to a congressional request, GAO discussed its report on issues
related to adverse medical events in the nations health care system.

GAO noted that: (1) the results of GAO's work have important
implications for addressing adverse medical events including the design
of surveillance systems to detect adverse events and medical errors; (2)
while adverse events have been recognized as a serious problem, the full
magnitude of their threat to the health of the American public is
unknown; (3) gathering valid and useful information about adverse events
is extremely difficult; (4) all systems that rely on health care
providers to take the initiative to make a report--known as passive or
spontaneous reporting systems--have serious limitations; (5) this is
true whether or not providers are legally required to report adverse
events; (6) both mandatory and voluntary spontaneous reporting systems
share this limitation; (7) many of the injuries patients suffer as a
result of medical treatment do not stem from errors but reflect the
inherent risks of treatments that are administered correctly; and (8) it
can be difficult both to identify these adverse reactions and
distinguish them from medical errors or from the course of a patient's
underlying illnesses.

--------------------------- Indexing Terms -----------------------------

 REPORTNUM:  T-HEHS-00-61
     TITLE:  Adverse Events: Surveillance Systems for Adverse Events
	     and Medical Errors
      DATE:  02/09/2000
   SUBJECT:  Reporting requirements
	     Medical information systems
	     Consumer protection
	     Pharmacological research
	     Data collection
	     Medical research
	     Drugs
	     Health care services
	     Health hazards
IDENTIFIER:  New York
	     Colorado
	     Utah
	     CDC/FDA Vaccine Adverse Events Reporting System
	     California

******************************************************************
** This file contains an ASCII representation of the text of a  **
** GAO Testimony.                                               **
**                                                              **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced.  Tables are included, but    **
** may not resemble those in the printed version.               **
**                                                              **
** Please see the PDF (Portable Document Format) file, when     **
** available, for a complete electronic file of the printed     **
** document's contents.                                         **
**                                                              **
******************************************************************

   * For Release on Delivery
     Expected at 10:30 a.m.

Wednesday,

February 9, 2000

GAO/T-HEHS-00-61

ADVERSE EVENTS

Surveillance Systems for Adverse Events and Medical Errors

        Statement of Janet Heinrich, Associate Director

Health Financing and Public Health Issues

Health, Education, and Human Services Division

Testimony

Before the Subcommittees on Health and Environment, and Oversight and
Investigations, Committee on Commerce, and Subcommittee on Health, Committee
on Veterans' Affairs, House of Representatives

United States General Accounting Office

GAO

ADVERSE EVENTS: Surveillance Systems for Adverse Events and Medical Errors

Mr. Chairman and Committee Members:

I am pleased to have the opportunity to testify today as you consider issues
related to adverse medical events in the nation's health care system.
Adverse events are receiving considerable attention now as a result of the
recent Institute of Medicine report on medical errors. Adverse events are
injuries to patients caused by medical treatment; medical errors are
mistakes in medical care that may or may not lead to harm. Efforts to
identify adverse events and evaluate their causes are important components
of strategies to reduce harm to patients. Several of our recent reports have
considered surveillance systems for medical products, particularly drugs and
medical devices. For example, last week we released a report that
synthesizes current research on adverse drug events (ADE). We have also
evaluated the Food and Drug Administration's (FDA) system for monitoring
problems with medical devices.

In summary, I believe that the results of our work have important
implications for addressing adverse medical events including the design of
surveillance systems to detect adverse events and medical errors. First,
while adverse events have been recognized as a serious problem, the full
magnitude of their threat to the health of the American public is unknown.
Second, gathering valid and useful information about adverse events is
extremely difficult. For example, all systems that rely on health care
providers to take the initiative to make a report-known as passive or
spontaneous reporting systems-have serious limitations. This is true whether
or not providers are legally required to report adverse events; that is,
both mandatory and voluntary spontaneous reporting systems share this
limitation. Furthermore, many of the injuries patients suffer as a result of
medical treatment do not stem from errors but reflect the inherent risks of
treatments that are administered correctly. It can be difficult both to
identify these adverse reactions and distinguish them from medical errors or
from the course of a patient's underlying illnesses.

Little Is Known About The Incidence Of Adverse Events

Relatively little information exists on the incidence of adverse events of
all types, including, for example, those caused by drugs, medical device
malfunctions, and diagnostic mistakes. Aside from small studies of
individual institutions, the best available information comes from two
studies of statewide samples of hospitalized patients. The first assessed
adverse events in New York in 1984, and the second employed a comparable
approach to examine the incidence of adverse events in Utah and Colorado in
1992. The widely cited estimate that 44,000 to 98,000 deaths per year are
attributable just to medical errors comes from an extrapolation of the
results of these two studies to the United States population as a whole.
Although these studies are the best available, national estimates based on
them have not taken into account regional variations in clinical practice
patterns and patient characteristics.

The largest category of adverse events caused by medical treatment, about
one-fifth of the total, consists of those brought about by drugs. Although
it is clear that a wide range of commonly used drugs cause adverse drug
events with potentially serious consequences for patients, relatively little
is known about the frequency of ADEs. In part, this reflects the reality,
which we discuss later, that identifying a medication as the cause of an
adverse event can often be difficult and uncertain. Consequently, the
available information on ADE incidence tends to be fragmentary and
inconsistent. Data routinely collected on ADEs during clinical trials or
after drugs have been marketed are intended to identify which ADEs are
associated with particular drugs and do not focus on how often ADEs take
place. Information on the overall incidence of ADEs from all drugs is
limited to a few research studies that typically examine the experience of
patients in one or two specific institutions-generally hospitals or
sometimes nursing homes-leaving the overall incidence of ADEs in outpatient
care largely unexplored.

The most broadly based information on ADE incidence comes from the two
studies that I mentioned earlier. These studies applied a particularly
restrictive definition of ADEs in finding that they occurred at a rate of
0.56 for every 100 patients admitted in Colorado and Utah and 0.72 per 100
admissions in New York. The studies counted only ADEs that resulted in
disability, prolongation of a patient's hospital stay, or death, meaning
that a significant fraction of the patients less seriously injured by drugs
was omitted. Other studies that used broader definitions, but applied them
in the context of specific institutions, found a range of 2 to 30 ADEs per
100 hospital admissions. There are still fewer published studies examining
ADEs in nursing homes, and all are limited to one or two individual
providers. Two of these studies reported an incidence of 0.44 to 0.71 ADEs
per patient month, rates roughly comparable to the rate reported in one
study of hospital ADEs that presented ADE incidence in terms of time spent
in the hospital.

Useful And Valid Information About Adverse Events Is Difficult To Obtain

Limitations of Spontaneous Reporting Systems

In our recent review of the research on adverse drug events, we learned what
is known about the strengths and limitations of adverse event reporting
systems of both the mandatory and voluntary variety. It is well known that
all spontaneous reporting systems experience a high level of underreporting.
For example, FDA believes that its system for gathering information about
ADEs, the Adverse Event Reporting System (AERS), receives reports for only
about 1 to 10 percent of all ADEs. Indeed, FDA relies on AERS primarily to
generate "signals" of new adverse drug events that the agency can then
investigate through other data sources.

Even mandatory systems can manifest extensive underreporting. For example,
the Institute of Medicine collected detailed information on mandatory
adverse event reporting programs in 13 states. According to these data, the
state programs receive highly variable numbers of reports. For example,
between 15,000 and 20,000 reports are submitted annually in New York,
compared with approximately 4,300 in California. The Institute of Medicine
did not cite any studies assessing the extent of underreporting in the
various state programs, but it noted the general presumption that to varying
degrees all are affected by it. Thus, no one knows at this point what
proportion of reportable cases is actually reported to any of the state
systems.

There are many possible reasons for underreporting. Among those commonly
cited are the fear of being blamed, the potential for legal liability, and
an expectation that reports will not have any effect. In addition, depending
on the definition of adverse events, and how that definition is interpreted,
there may be considerable variability among health care providers and
institutions about the kinds of events that are reported. Some of the
examples of serious adverse events to be covered by the Institute's proposed
mandatory reporting program are relatively unambiguous-a maternal death, for
instance. But others, such as "serious injuries associated with the use of a
new device, operation, or medication," are not as clear because they are
based on judgments of the causes of patient injury, not an easily observed
clinical outcome.

Various measures can be taken to address some of these disincentives to
reporting and thereby increase the number of reports submitted. These
include protecting the confidentiality of reporters and making it easier to
file reports. Both were part of a pilot study FDA sponsored of a new system
for collecting reports about adverse events for medical devices. That study
received adverse event reports at a rate ten times greater than in the
current medical device surveillance system, even though the current system
mandates the reporting of the same types of events. However, because the
reporters may be unknown in a confidential reporting system, it is much
harder to follow up reports in order to clarify important information that
may be ambiguous or missing. A truly confidential reporting system places a
significant burden on adverse event reports to contain all the information
that a regulatory agency, or a product's manufacturer, needs or will need in
the future to understand the potential public health risk.

Moreover, underreporting is only part of the problem. The bigger difficulty
is that the subset of adverse events that are reported does not accurately
reflect the universe of all adverse events. The available studies indicate
that there is substantial bias in reporting. In the area of drug-related
events, we found that a wide variety of factors could affect the likelihood
of reporting. For example, more reports are received during a drug's first
few years on the market than later, and drug manufacturers with extensive
postmarketing surveillance efforts gather more reports than other companies
do. Therefore, it is not legitimate to infer that patterns or trends that
emerge in reported events reflect what is happening with adverse events
overall.

To get valid information on the incidence of adverse events, we need data
that do not come from a spontaneous reporting system. This generally
involves a proactive examination of a random sample of patient records, as
was done in both the New York and Utah and Colorado studies that I mentioned
earlier. In fact, the Institute of Medicine report supports having a new
organization, a Center for Patient Safety, collect data on the incidence of
adverse events through studies of this type. More such studies are needed if
we are to have accurate data on the magnitude of the problem that adverse
events represents.

However, studies based on large, representative samples of patient records
tend to be expensive and time consuming to complete. Therefore, there will
always be the temptation to draw implicit inferences from the more readily
available data from the existing adverse event reporting systems about where
medical errors are most likely to occur and how much progress, if any, has
been made in reducing them. The Institute of Medicine's recommendation to
implement standard definitions and formats for the mandatory reporting of
serious adverse events is likely to encourage greater reliance and use of
those reports. Standardizing definitions cannot overcome the
nonrepresentative quality of reported adverse events. Standardized
definitions and formats will, however, enhance the utility of adverse
reports for other types of analyses that are not concerned with incidence.
For example, they will facilitate analyses of multiple instances of a
particular type of adverse event. Such analyses can help identify the key
underlying factors that explain why these adverse events occur.

Even with the limitations of mandatory and voluntary reporting systems, the
information they generate can help in reducing medical errors and associated
adverse events. In some cases, the fact that a particular kind of adverse
event occurred one or more times and has been reported is sufficient to
motivate action and dictate its direction. In those cases, incident
reporting systems can function effectively and may have substantial
advantages. However, it is often important to understand the frequency of a
particular type of error and whether that has changed over time. In these
cases, the incomplete data coming from reporting systems may not be
sufficient. It is better to rely then, if possible, on data that derive from
an examination of a sample of patient records.

Many Adverse Events Are Not Caused by Medical Errors

Efforts to reduce adverse events should not focus exclusively on those
caused by errors. The available studies indicate that just over half of
adverse events of all types are caused by errors in treatment. The study of
New York hospital discharges found that 58 percent of adverse events were
preventable, compared with 53 percent in the corresponding study of Utah and
Colorado hospital patients. This means that nearly as many adverse events
result from appropriate medical treatment as from errors.

The proportion of adverse events involving drugs that is due to medical
error is even lower. Available data suggest that one-half to two-thirds of
ADEs occur when drugs have been used appropriately. Many of these ADEs are
the result of a drug's known pharmacological properties and are often listed
on the medication's label. For example, hemorrhaging is the most common
adverse reaction for warfarin, a drug that reduces the risk of heart attack,
stroke, and other conditions by decreasing the clotting ability of blood.
Other adverse reactions, including allergic reactions, are less predictable,
caused by sensitivities in individual patients who have no history of
adverse reactions to a specific drug. Still other adverse reactions are
related to previously undetected risks. These include drug-drug and
drug-food interactions that become evident as a drug is used by many types
of patients, having many kinds of concurrent illnesses, and taking many
other medications, as well as over-the-counter drugs and dietary
supplements. FDA's system for collecting voluntary reports on adverse
experiences with marketed drugs is designed specifically to uncover these
kinds of previously unknown risks.

Many types of drugs can cause adverse reactions. Some drug classes are
associated with a substantial number of adverse reactions mainly because
they are prescribed to many patients. These include antibiotics, narcotic
analgesics, drugs to control hyperglycemia in type II diabetics,
psychotropic drugs such as antidepressants and tranquilizers, and
nonsteroidal anti-inflammatory drugs (NSAIDs). However, some classes of
drugs have notably lower rates of adverse reactions despite high rates of
use. Antihistamines and the statin drugs prescribed to lower cholesterol
levels are rarely associated with serious adverse reactions.

Patients who are very ill, including those with several concurrent
diagnoses, have a greater risk of adverse reaction than others do. Not only
are they more fragile but their illnesses may require several simultaneous
treatments. In addition, they may be receiving more aggressive treatments
that are known to entail significant risks. Some reports have found that
elderly persons and women have more adverse reactions than younger persons
and men. However, it is possible that age and gender are merely related to
other risk factors instead of independently increasing the likelihood of an
adverse reaction. In some studies, controlling for the number of medications
being taken substantially diminishes any relationship between age and
adverse reactions.

As with medical errors, passive surveillance systems are inadequate for
measuring the frequency or rate of adverse drug reactions. Other kinds of
studies are required to develop this information. Thus, adverse reactions
that develop after the prolonged use of a drug require studies with long
follow-up periods to determine whether the adverse events are related to the
drug. Similarly, rare adverse reactions require studies with very large
numbers of patients to accumulate a sufficient number of problematic cases,
and adverse symptoms that mimic those of a patient's underlying condition
require carefully controlled clinical trials. For example, the Cardiac
Arrhythmia Suppression Trial found that antiarrhythmia medications doubled
the risk of cardiac arrest and death in heart attack survivors. This was not
detected in clinical practice (nor fully captured in spontaneous reporting
systems) because patients with heart disease regularly have arrhythmias and
heart attacks, providing a ready alternative explanation that masked the
causal role of the drugs. It has been estimated that these medications
caused up to 50,000 premature deaths.

In conclusion, surveillance systems that uncover and document adverse events
can collect valuable data, but they are not sufficient, by themselves, to
improve medical care. The data need to be analyzed and interpreted to create
a better understanding of the reasons for adverse events. Sometimes one
adverse event, if carefully examined, can provide insights of this sort. At
other times, analysts need to assess multiple examples of a particular type
of event to discern the critical causal factors. However, for both types of
analysis, the quality of the data that are collected is critical. Accurate
information on the process of care provided and the patient's response to
that care is required to determine the key factors that led to an adverse
event. Thoughtful analyses can then use these data to identify specific
changes in health care systems and processes that can reduce the likelihood
of adverse events caused by both medical errors and the normal risks of
adverse outcomes inherent in all medical interventions.

GAO Contact And Acknowledgments

For future contacts regarding this testimony, please call Janet Heinrich at
(202) 512-7119. Key contributors include Martin T. Gahart and Eric A.
Peterson.

(201031)
  
*** End of document. ***