FDA Drug Approval: Review Time Has Decreased in Recent Years (Letter
Report, 10/20/95, GAO/PEMD-96-1).
Pursuant to a congressional request, GAO provided data on the Food and
Drug Administration's (FDA) new drug application (NDA) process, focusing
on: (1) whether the timeliness of the review and approval process for
new drugs changed in recent years; (2) the factors that distinguish NDA
that are approved quickly from those that take longer to approve; (3)
what distinguishes NDA that are approved from those that are not; and
(4) how FDA drug approval process compares with the approval process in
the United Kingdom.
GAO found that: (1) the average number of months for NDA to be approved
by FDA decreased from 33 months in 1987 to 19 months in 1992; (2) the
overall decrease in approval times was achieved through gradual
reductions in the submission of all NDA from 1987 to 1992; (3) the
priority FDA assigns to an NDA and the experience of its sponsor
determine the timeliness and likelihood of the approval process; and (4)
although comparable data is limited, the review times for FDA and its
counterpart agency in the United Kingdom are similar.
--------------------------- Indexing Terms -----------------------------
REPORTNUM: PEMD-96-1
TITLE: FDA Drug Approval: Review Time Has Decreased in Recent Years
DATE: 10/20/95
SUBJECT: Pharmaceutical industry
Pharmacological research
Drugs
Regulatory agencies
Food and drug law
Comparative analysis
Safety regulation
Consumer protection
Product safety
Foreign governments
IDENTIFIER: United Kingdom
**************************************************************************
* This file contains an ASCII representation of the text of a GAO *
* report. Delineations within the text indicating chapter titles, *
* headings, and bullets are preserved. Major divisions and subdivisions *
* of the text, such as Chapters, Sections, and Appendixes, are *
* identified by double and single lines. The numbers on the right end *
* of these lines indicate the position of each of the subsections in the *
* document outline. These numbers do NOT correspond with the page *
* numbers of the printed product. *
* *
* No attempt has been made to display graphic images, although figure *
* captions are reproduced. Tables are included, but may not resemble *
* those in the printed version. *
* *
* A printed copy of this report may be obtained from the GAO Document *
* Distribution Facility by calling (202) 512-6000, by faxing your *
* request to (301) 258-4066, or by writing to P.O. Box 6015, *
* Gaithersburg, MD 20884-6015. We are unable to accept electronic orders *
* for printed documents at this time. *
**************************************************************************
Cover
================================================================ COVER
Report to Congressional Requesters
October 1995
FDA DRUG APPROVAL - REVIEW TIME
HAS DECREASED IN RECENT YEARS
GAO/PEMD-96-1
FDA Review Time for Drugs
(973799)
Abbreviations
=============================================================== ABBREV
CSM - Committee on the Safety of Medicines
FDA - Food and Drug Administration
GAO - U.S. General Accounting Office
MCA - Medicines Control Agency
NDA - New drug application
NME - New molecular entity
Letter
=============================================================== LETTER
B-266023
October 20, 1995
The Honorable Nancy Landon Kassebaum
Chairman
The Honorable Edward M. Kennedy
Ranking Minority Member
Committee on Labor and Human Resources
United States Senate
New drugs marketed in the United States must be approved first by the
Food and Drug Administration (FDA).\1 Approval comes after FDA has
determined from data submitted by a drug's sponsor that the drug is
safe and effective for use as indicated on its label and that the
manufacturer can ensure its quality. Various parties calling for the
legislative reform of FDA in recent months are concerned with the
length of the approval process. Advocates of reform argue that
shortening the time it takes to get new drugs approved will
contribute both to public health, by making effective therapies
available sooner to people who need them, and to the economic health
of the pharmaceutical industry, by allowing drug manufacturers to
sell their products sooner. Opposed to major reform, FDA claims that
in recent years review time has been reduced considerably.
--------------------
\1 See 21 U.S.C. 355 (1988).
PURPOSE
------------------------------------------------------------ Letter :1
Time--specifically the period that begins with the submission to FDA
of a new drug application (NDA) and that ends when a final decision
is made on that application (the period known as the NDA review phase
of drug development)--is the focus of this report. At your request,
we have assembled data on all new drug applications submitted to FDA
in 1987-94 to answer three questions:
Has the timeliness of the review and approval process for new drugs
changed in recent years?
What factors distinguish NDAs that are approved relatively quickly
from those that take longer to be approved?
What distinguishes NDAs that are approved from those that are not?
Additionally, as you asked, we obtained the most recently available
data on how long it takes for drugs to be approved in the United
Kingdom and compared them with approval times in the United States.
Because GAO has access to all applications, both those that have been
approved and those that have not, our report is the first to present
comprehensive data on review time for all NDAs submitted to FDA.\2
--------------------
\2 Much of the information in an NDA (and even its existence) remains
proprietary until FDA approves it. This means that information on
NDAs that are not approved has not been publicly available, so that
previous studies of review and approval times have been unable to
include it.
BACKGROUND
------------------------------------------------------------ Letter :2
The process of bringing a drug to market is lengthy and complex and
begins with laboratory investigations of the drug's potential. For
drugs that seem to hold promise, preclinical animal studies are
typically conducted to see how a drug affects living systems. If the
animal studies are successful, the sponsoring pharmaceutical firm
designs and initiates clinical studies in which the drug is given to
humans. At this point, FDA becomes directly involved for the first
time.
Before any new drug can be tested on humans, the drug's sponsor must
submit an investigational new drug application to FDA that summarizes
the preclinical work, lays out a plan for how the drug will be tested
on humans, and provides assurances that appropriate measures will be
taken to protect them. Unless FDA decides that the proposed study is
unsafe, clinical testing may begin 31 days after this application is
submitted to FDA. While clinical trials progress through several
phases aimed at establishing safety and efficacy, the manufacturer
develops the processes necessary to produce large quantities of the
drug that meet the quality standards for commercial marketing.
When all this has been done, the pharmaceutical firm submits an NDA
that includes the information FDA needs to determine whether the drug
is safe and effective for its intended use and whether the
manufacturing process can ensure its quality. The first decision FDA
must make is whether to accept the NDA or to refuse to file it
because it does not meet minimum requirements. Once FDA has accepted
an NDA, it decides whether to approve the drug on the basis of the
information in the application and any supplemental information FDA
has requested. FDA can approve the drug for marketing (in an
"approval letter") or it may indicate (in an "approvable letter")
that it can approve the drug if the sponsor resolves certain issues.
Alternatively, FDA may withhold approval (through a "nonapprovable
letter" that specifies the reasons). Throughout the process, the
sponsor remains an active participant by responding to FDA's
inquiries and concerns. The sponsor has the option, moreover, of
withdrawing the application at any time.
METHOD
------------------------------------------------------------ Letter :3
For each NDA submitted between 1987 and 1994, we obtained from FDA
information on the dates of its significant events between initial
submission and final decision as well as the last reported status of
the application as of May 1995. To ensure that the data were valid,
we independently checked them against values in published reports and
other sources. (The variables that we used in our analysis and the
procedures that we used to validate the data can be found in appendix
I.)
We computed time by measuring the interval between all significant
events. Results using other ways to calculate review time are
compared to ours in appendix II. We used regression analysis to
determine the factors that were significantly related to time and to
determine which factors were significantly related to approval. (The
results of the regression analyses on time are in appendix IV, on
approval in appendix V.\3 )
Some of our analyses include all the NDAs, while others focus on
specific subgroups. Most notably, we restricted analyses of overall
time to NDAs that had been submitted by the end of 1992 to avoid the
bias introduced by including applications that have had an
insufficient time to "mature." (Appendix VI describes the
implications of this decision for our results.) Because our analyses
of final decisions concentrate on NDAs submitted through the end of
1992, the data we present do not address the consequences of the full
implementation of the Prescription Drug User Fee Act of 1992.\4 Our
findings pertain only to FDA's Center for Drug Evaluation and
Research and do not reflect the activities of the agency's five other
centers.\5
We focused only on the NDA review phase--the final critical step of
bringing a drug to market. We did not address the lengthier process
of initial exploration and clinical testing, which together with the
NDA phase average more than a decade, nor did we study the phase that
follows a drug's approval, during which additional studies can be
conducted and attention paid to potential adverse events associated
with its widespread use in the general population.
--------------------
\3 In appendix III, we discuss intermediate outcomes from FDA's
review process.
\4 The Congress passed the act (Public Law 102-571) in October 1992
to provide FDA with additional resources to expedite drug review and
approval. Because it takes time to hire and train reviewers and for
fees to accrue, the effects of full implementation may not be evident
for several years. The act is due for reauthorization after 1997, by
which time FDA has agreed to meet the act's goals for improved
performance.
\5 The other centers are the Center for Biologics Evaluation and
Research, the Center for Devices and Radiological Health, the Center
for Food Safety and Applied Nutrition, the Center for Veterinary
Medicine, and the National Center for Toxicological Research. Even
within the Center for Drug Evaluation and Research, our findings
pertain only to the review and approval process for NDAs and not to
other functions such as the investigational new drug phase or the
regulation of generic drugs.
RESULTS IN BRIEF
------------------------------------------------------------ Letter :4
We found a considerable reduction in approval time for NDAs submitted
between 1987 and 1992. It took an average of 33 months for NDAs
submitted in 1987 to be approved but only 19 months on average to
approve NDAs submitted in 1992. Further, the reduction in time was
observed for all NDAs and not just for those that had been approved.
As figure 1 shows, the overall decrease in approval times was
achieved through gradual reductions in time for applications
submitted in each successive year.
Figure 1: Months Between
Submission and Approval for
NDAs Submitted 1987-92\a
(See figure in printed
edition.)
\a Number of approved NDAs: 1987, 80; 1988, 75; 1989, 65; 1990, 53;
1991, 64; 1992, 53.
The priority FDA assigns to an NDA and the experience of its sponsor
are the two factors that significantly affect the likelihood that the
NDA will be decided on quickly. FDA assigns priority status to
applications for drugs that are expected to provide therapeutic
benefit to consumers beyond that of drugs already marketed. These
NDAs take an average of 10 months less to be approved than do
standard applications (those for which there is no perceived
therapeutic benefit beyond that for available drugs). Applications
from the most experienced sponsors take an average of 4 months less
time to be approved than those from less experienced sponsors.
Priority status and sponsor experience are also the two factors that
predict the likelihood that an NDA will be approved. Priority NDAs
are four times more likely to be approved than standard NDAs.
Applications submitted by the most experienced sponsors are three
times more likely to be approved than those submitted by the least
experienced companies.
Finally, the limited comparable data available on review time for FDA
and the counterpart agency in the United Kingdom paint a more
ambiguous picture than that presented in many recent reports. In
fact, the latest data published by the regulatory agency in the
United Kingdom show that it does not have faster approval times than
FDA.
OUR ANALYSIS
------------------------------------------------------------ Letter :5
FDA received 905 NDAs in 1987-94. The total number of NDAs fell from
1987 but remained relatively stable in the ensuing years through 1994
(with the exception of the uncharacteristically small number of
submissions in 1993). The number of NDAs for new molecular entities
(NMEs) and priority NDAs remained relatively stable over the years.\5
Overall, 17 percent of the NDAs were for priority drugs. (See table
1.)
Table 1
Number of NDAs Submitted 1987-94\a
Type 1987 1988 1989 1990 1991 1992 1993 1994 Total
-------- ------ ------ ------ ------ ------ ------ ------ ------ ======
All NDAs 142 129 117 99 110 103 87 118 905
Priority 18 20 16 21 21 23 14 19 152
NDAs
NMEs 29 33 32 29 37 34 34 37 265
--------------------------------------------------------------------------------
\a Thirty-six percent of the NMEs are classified as priority
applications, 9 percent of non-NMEs.
A large percentage of the applications were not approved. Only 390
of the 700 NDAs submitted through 1992 had been approved by May 16,
1995. In other words, 44 percent of the applications submitted were
for drugs that FDA did not find to be safe and effective or that
sponsors chose not to pursue further. NMEs were approved at a higher
rate than non-NMEs (64 percent to 52 percent), and priority drugs
were approved more often than standard drugs (76 percent to 52
percent). This means that whether an NDA is or is not ultimately
approved is as relevant a question as how long approval takes.\6 (See
table 2.)
Table 2
Final Status of NDAs Submitted 1987-
92\a
Not
Type Approved Withdrawn Refused Approvable approvable
---------------- ---------- ---------- ---------- ---------- -- ----------
NDA
NMEs 64% 18% 2% 3% 12%
Non-NMEs 52 22 8 3 15
Priority 76 14 0 0 10
Standard 52 22 7 3 15
Sponsor
Most experienced 63 22 3 3 9
group
Least 41 17 15 2 25
inexperienced
group
All 56 21 6 3 14
--------------------------------------------------------------------------------
\a Final status as of May 16, 1995. All rows sum to 100 percent
except for rounding.
The data in table 2 show that NDAs that are submitted by experienced
sponsors and priority NDAs are more likely to be approved than
standard NDAs or NDAs submitted by sponsors with little experience
with the process. These results are supported by a regression
analysis that shows that both the NDA's priority and the sponsor's
experience are statistically significant predictors of outcome (see
appendix I for our definition of sponsor experience and appendix V
for the regression analysis). The regression analysis found that,
statistically controlling for the effects of the other explanatory
variables in the model, priority NDAs are four times more likely to
be approved than standard NDAs and that applications submitted by the
most experienced companies are three times more likely to be approved
than those submitted by less experienced sponsors.
--------------------
\5 Data on NDAs are often presented separately for NMEs (which are
drugs with active components that are new) and for priority drugs
because these NDAs may require a different type of review than other
NDAs. The types of NDAs that are classified as non-NMEs are listed
in appendix I.
\6 Some other studies of the drug review process have reported higher
rates of approval. These studies either have looked at subsets of
the population of NDAs that have higher approval rates (such as NMEs)
or have not included in their calculations applications that FDA
refused to file. In contrast, our report of a 56-percent approval
rate includes all types of NDAs and all applications listed in FDA's
records, even those that FDA refused to file.
HOW LONG DOES THE REVIEW
PROCESS TAKE?
---------------------------------------------------------- Letter :5.1
Table 3 shows for 1987-92 the average time (in months) from when NDAs
were first submitted to when final decisions were made for both NDAs
that were approved and those that were not.\7 The table also
distinguishes between all NDAs and those that were approved in three
categories: new molecular entities, priority applications, and
standard applications.
Table 3
Average Number of Months From Initial
NDA Submission to Final Decision for
NDAs submitted 1987-92
Type 1987 1988 1989 1990 1991 1992
---------------------------------- ---- ---- ---- ---- ---- ----
All NDAs 33 31 24 23 21 18
Approved NDAs 33 30 25 25 21 19
All NMEs 31 32 21 21 25 20
Approved NMEs 33 26 23 23 23 21
All priority 29 29 16 23 17 17
Approved priority 23 23 16 22 18 16
All standard 34 32 26 23 21 18
Approved standard 35 32 28 27 22 20
----------------------------------------------------------------------
As can be seen from the table, the processing time for all eight
categories of NDAs fell considerably (from 33 to 18 months, or 45
percent, for all NDAs, or from 33 to 19 months, or 42 percent for
approved NDAs). In addition, the reductions in time came for NDAs
submitted throughout the period of our study. This finding is
consistent with FDA's statements that review time has decreased in
recent years.
Alternative presentations of the data demonstrate the same result.
For example, table 4 shows that the number of months that passed
before half of all submissions were approved declined from 58 months
for NDAs submitted in 1987 to 33 months for 1992 submissions. Since
just 56 percent of the NDAs submitted between 1987 and 1992 were
approved, this measure captures the approval period for almost all
the approvals that will ultimately be granted.\8 Similarly, table 4
shows that the proportion of submitted NDAs that were approved within
2 years increased from 23 percent for NDAs submitted in 1987 to 39
percent for NDAs submitted in 1992.
Table 4
Two Alternative Measures of Review Time
for NDAs Submitted 1987-92
Months until half Percent of NDAs
of all NDAs were approved within 24
Year of submission approved months
------------------------------ ------------------ ------------------
1987 58 23%
1988 52 27
1989 41 31
1990 47 29
1991 30 36
1992 33 39
----------------------------------------------------------------------
Closer examination of the individual NDAs shows that they differed
considerably in how long it took before a final decision was made.
Some NDAs were approved within a few months (the shortest was 2
months); others took years (the slowest was 96 months). The
variation was similar among applications that were not approved.
Some were withdrawn on the day they were submitted. The longest
outstanding application was 92 months old.
This considerable variation raises the question of what
differentiates one NDA from the next: Do some factors predict the
time it will take to reach a final decision? When we tested
potential explanatory variables, we found that the priority FDA
assigned to an application and the sponsor's experience in submitting
NDAs were statistically significant predictors of how long review and
approval took. (See appendix IV.) More specifically, controlling for
the effects of the other explanatory variables in the model, our
regression analysis found that priority NDA applications are approved
10 months faster than standard applications and that applications
from the most experienced sponsors are approved 4 months faster than
applications from less experienced sponsors.
--------------------
\7 The only FDA decision that is truly "final" is the decision to
approve the NDA. All other decisions allow the sponsor to continue
to pursue an approval decision. For example, even if FDA sends a
not-approvable letter, the sponsor can address the concerns listed in
that letter and resubmit the NDA. Therefore, whenever we use the
term "final decision" in this report, it means the status of the
application as of May 16, 1995.
\8 Fifty-eight percent of the NDAs submitted in 1988 and 1991 were
approved, the years with the greatest proportion of approvals (see
appendix VI).
PROCESS MEASURES OF TIME
---------------------------------------------------------- Letter :5.2
The interval between first submission and final decision indicates
how long the public must wait for drugs after sponsors believe they
have assembled all the evidence to support an approval decision.
Alternative measures provide insight into what happens to an NDA
before FDA approves it. One such measure is the extent to which FDA
is "on time" in making decisions. We examined both the degree to
which FDA was on time and the factors that influenced whether it made
its decisions on time. The criteria for "on time" performance that
we used in this analysis were established under the Prescription Drug
User Fee Act of 1992.\9 Although on-time performance may be seen as
one indicator of FDA's efficiency, it is important to note that FDA
is not required to meet these criteria until 1997.\10
Of all the decisions FDA made on the NDAs submitted between 1987 and
1993, 67 percent were on time. Simpler decisions (for example,
refusals to file) were made on time more often than relatively
complex decisions (for example, priority applications in which the
first decision was an approval). Overall, the on-time percentage
remained relatively stable, varying between a low of 62 percent for
NDAs submitted in 1992 and a high of 72 percent for NDAs submitted in
1987.\11 In sharp contrast to the decline in overall time between
submission and final decision shown in table 3, this stability shows
that there is little relationship between the time FDA takes to reach
a final decision and whether or not it meets its deadlines for
specific actions.\12
Another process measure of review time is based on where
responsibility lies for different parts of the process--with FDA for
the intervals during which it acts on an application, or with the
sponsor, for the intervals during which FDA waits for the sponsor to
provide additional information or to resubmit the application.
Figure 2 shows how their relative times were distributed for approved
NDAs submitted between 1987 and 1992.
Figure 2: FDA and Sponsor
Times for Approved NDAs
Submitted 1987-92
(See figure in printed
edition.)
As can be seen from the figure, sponsors accounted for approximately
20 percent of the time in the NDA phase for applications that FDA
approved.\13 Importantly, the time for both sponsors and FDA
diminished for NDAs submitted between 1987 and 1992.
--------------------
\9 Upon receipt of an NDA, FDA has 60 days to determine whether the
application will be filed or refused. If the application is filed,
under the performance goals referenced in the Prescription Drug User
Fee Act, FDA is to perform a complete review of the entire
application and issue an approval letter, approvable letter, or
not-approvable letter within 6 months for priority applications and
within 12 months for standard applications. In accordance with the
act, FDA intends to fully implement these goals by the end of fiscal
year 1997.
\10 Our calculations of FDA's on-time performance were conservative,
tending to underestimate, rather than overestimate, the proportion of
FDA's actions that have been on time (see appendix II).
\11 See appendix II. We excluded the rates for the 1994 cohort from
this analysis.
\12 In commenting on a draft of this report, FDA maintained that our
on-time analysis underestimates the extent to which its performance
has improved. See appendix II for FDA's comments and our response.
\13 Our calculations of sponsor time were conservative, tending to
underestimate, rather than overestimate, the proportion of review
time accounted for by the sponsors of NDAs (see appendix I).
APPROVAL TIMES IN THE UNITED
KINGDOM
---------------------------------------------------------- Letter :5.3
Regulatory processes similar to FDA's have been mentioned as models
for reforming FDA. The one most often mentioned is the United
Kingdom's. Proponents of FDA reform have argued that the British
counterpart to the FDA, the Medicines Control Agency, performs
reviews of equivalent quality and does so significantly more quickly.
Comparisons between the Medicines Control Agency and FDA are
difficult because the workload, approval criteria, and review
procedures followed by the agency may not be exactly the same as
FDA's and because its reports cover a slightly different period than
FDA's. However, the most recent data show that overall approval
times are actually somewhat longer in the United Kingdom than they
are in this country. For the 12-month period ending September 30,
1994, the Medicines Control Agency reported that the median approval
time for applications that were apparently equivalent to NMEs was 30
months. The average time was 24 months. The fastest approval was
granted in about 4 months, the slowest in 62 months.
According to FDA, the median approval time for NMEs approved in the
United States in calendar year 1994 was 18 months, the average about
20 months. The fastest FDA approval took about 6 months and the
slowest about 40 months. (See appendix VII for a fuller comparison.)
CONCLUSION AND IMPLICATIONS
------------------------------------------------------------ Letter :6
Aside from shedding light on the central issue of time, the data we
assembled provide some interesting but rarely mentioned facts about
FDA's drug review and approval process. First, nearly half the NDAs
submitted to FDA are not approved for marketing. The 44 percent of
NDAs that were not approved in our sample either were not judged by
FDA to be safe and effective or were not pursued by their sponsors.
Second, the percentage of NDAs for drugs that are viewed by FDA as
offering an important therapeutic advance is relatively small. As we
pointed out in table 1, only 17 percent of all NDAs were given
priority status. Third, our data on drug review and approval show
that approximately one fifth of the time in that process comprises
activities for which sponsors are responsible.
With respect to time, NDAs are moving more quickly through the drug
review and approval process. Whether this improvement is because of
actions by FDA or the pharmaceutical industry or some other factors
is an issue that is beyond the scope of this report. However, the
consistency of all our results supports the conclusion that the
reduction in time is real and not an artifact of how time is
measured. Further, the magnitude of the reduction--more than 40
percent--should be considered in the ongoing discussions of the need
to change the NDA review process or the agency in order to speed the
availability of drugs to patients.
AGENCY COMMENTS
------------------------------------------------------------ Letter :7
FDA officials reviewed a draft of this report and discussed their
comments with us. They generally agreed with our analytic methods
and findings. However, they expressed concerns about some aspects of
our analysis of FDA's on-time performance. These comments, and our
responses to them, appear in appendix II. FDA also provided a number
of specific technical comments that have been incorporated into the
report where appropriate.
As we agreed with your offices, we plan no further distribution of
this report until 30 days from its date of issue, unless you publicly
announce its contents earlier. We will then send copies to the
Secretary of Health and Human Services, the Commissioner of Food and
Drugs, and to others who are interested. We will also make copies
available to others upon request.
If you have any questions regarding our report, please call me at
(202) 512-2900 or George Silberman, Assistant Director, at (202)
512-5885.
Joseph F. Delfico
Acting Assistant Comptroller General
DATA AND METHODOLOGY
=========================================================== Appendix I
THE DATA WE EXAMINED
--------------------------------------------------------- Appendix I:1
At our request, FDA provided detailed information about all new drug
applications, totaling 905, initially submitted between January 1,
1987, and December 31, 1994. This included the contents and date of
all FDA decisions and all major communications between FDA and the
NDA sponsors through May 16, 1995. The variables we used in our
analysis are described in the next section.
Our choice of this time period has important implications for the
analysis of drug review time. First, we started with 1987 because
that was the first full year following a major change in FDA's drug
review procedures. We do not believe that examining new drug
applications from before 1987 would shed any light on FDA's current
activities. Second, most reports of drug approval times, including
those published by FDA, measure time for drugs approved during a
particular period, regardless of when they were submitted. Some
approved drugs may have been submitted much earlier. By limiting our
analysis to new drug applications submitted (but not necessarily
approved) in 1987 and later, we have limited the maximum value of
review time. However, we do not believe that this has significantly
biased our findings, since relatively few drugs win approval after
exceptionally long review periods. (Appendix VI describes the
outcomes of the review process as a function of year of approval in
our sample.)
While we were unable to independently verify the accuracy of all the
data FDA provided, we did undertake a number of validation procedures
to ensure the quality of the data. First, we performed extensive
checks of the internal consistency of the databases FDA provided. In
several cases, we uncovered discrepancies in the level of detail for
different categories of drugs and between the information contained
in one data file and that contained in another file. We resolved all
these inconsistencies with FDA.
Second, we compared the information in the data files with published
sources where possible. For approved drugs, many reports (by FDA and
by others) list the names, submission dates, and approval dates. We
were able to resolve with FDA the few inconsistencies we discovered
through this method. However, it is important to note that we were
unable to do this for nonapproved drugs because there are no
published reports on them.
Third, for an earlier report, we had already obtained documentation
for all NDAs for NMEs submitted in 1989.\1 We compared those
documents with the data FDA provided us for this report, and we were
able to resolve all apparent inconsistencies.
--------------------
\1 U.S. General Accounting Office, FDA User Fees: Current Measures
Not Sufficient for Evaluating Effect on Public Health, GAO/PEMD-94-26
(Washington, D.C.: July 1994).
THE VARIABLES WE ANALYZED
--------------------------------------------------------- Appendix I:2
This section describes the variables we used in our analyses. Our
definitions of the variables do not necessarily agree with FDA's
practice. FDA provided some of the variables directly to us; we
computed others from the data FDA provided and from other sources.\2
--------------------
\2 For more information about the drug approval process, drug
characteristics, and measurement of time, see FDA User Fees, esp.
pp. 4-10 and app. I.
DRUG CHARACTERISTICS
------------------------------------------------------- Appendix I:2.1
Priority drugs. Those that FDA determines to represent a significant
therapeutic advance, either offering important therapeutic gains
(such as the first treatment for a condition) or reducing adverse
reactions. Nonpriority, or standard, drugs offer no therapeutic
advantage over other drugs already on the market.
New molecular entities. Drugs with molecular structures that have
not previously been approved for marketing in this country, either as
a separate drug or as part of a combination product. Drugs that are
not NMEs are from one of six categories defined by FDA: a new ester
or salt, a new dosage form or formulation of a previously approved
compound, a new combination of previously approved compounds, a new
manufacturer of a previously approved drug, a new indication for an
already approved drug, or drugs already marketed but without an
approved NDA (that is, drugs first marketed before FDA began
reviewing NDAs).
SUBMISSIONS TO THE REVIEW
PROCESS
------------------------------------------------------- Appendix I:2.2
Initial submission. The first submission of the application to FDA.
Resubmission. After a sponsor has withdrawn an application or FDA
has refused it for filing, sponsors can resubmit it.
Major amendments. Substantial submissions of new information by the
sponsor to FDA, either of the sponsor's own volition or in response
to an FDA query.
RESULTS OF THE REVIEW
PROCESS
------------------------------------------------------- Appendix I:2.3
Refusal to file. After FDA receives a new drug application, the
agency first determines if the application is sufficiently complete
to allow a substantive review. If not, FDA can refuse to file it.
Since the implementation of user fees in 1993, applications must be
rejected if the sponsor has failed to pay the appropriate fee to FDA.
These applications are categorized as "unacceptable for filing," not
refusal to file.
Approval. If FDA is satisfied that a drug is safe and effective, it
approves the drug for marketing for its intended use as described in
the label.
Approvable. FDA determines that a drug is approvable if there is
substantial evidence that it is safe and effective, but the sponsor
must either supply additional information or agree to some limiting
conditions before FDA grants final approval.
Not approvable. If FDA determines that the evidence submitted by the
sponsor to show that the drug is safe and effective is insufficient,
the agency notifies the sponsor that the drug is not approvable.
Withdrawal. The sponsor of an NDA may withdraw it at any time for
any reason.
Final status. We examined the data file for each NDA to see if the
drug had ever been approved. If not, we searched the file for the
last event that was a withdrawal, not approvable, approvable, or a
refusal to file, and we identified that event as the application's
final status. However, since FDA never definitively rejects
applications, some whose final status is other than approval may
ultimately be approved. (See appendix III.)
DRUG REVIEW TIME
------------------------------------------------------- Appendix I:2.4
Year of submission. The calendar year in which an application is
first submitted to FDA.
Review time. The period between the date of the initial submission
of an NDA, even if FDA refuses to file it, and the date of the
application's final status in the data file. For approved drugs,
review time is the period between the initial submission and the date
of approval.
FDA time and sponsor time. For some of the analyses, we divided the
total review time into time that is FDA's responsibility and time
that is the sponsor's responsibility. FDA time consists of periods
that begin when the agency has the information it has requested from
the sponsor for that stage of the review and that end when FDA issues
a judgment of refusal to file, approval, approvable, or not
approvable or the application is withdrawn.\3 Sponsor time consists
of periods when FDA is waiting for the sponsor to provide additional
information or to resubmit the application. FDA time and sponsor
time are complementary and together sum to total review time.
Review cycles. Each period of FDA time is one review cycle.
FDA's on-time performance. The Prescription Drug User Fee Act of
1992 established specific performance goals for each review cycle.\4
The agency must issue refusals to file within 60 days of submission
and must reach all other decisions for priority drugs within 6 months
and for standard drugs within 12 months. We applied these guidelines
retroactively to identify actions as either on time or not on time
for each review cycle for NDAs submitted between 1987 and 1994.
--------------------
\3 The beginning of a period of FDA time is clear when the NDA is
first submitted or resubmitted after a withdrawal or refusal to file.
It is less obvious when the sponsor submits amendments in response to
approvable or not-approvable letters, because sponsors frequently
submit several amendments on different dates. We chose to use the
date of the first amendment as starting a period of FDA time, even
though FDA may not have all the information it needs to continue the
review at that point. Our procedure maximizes FDA time and minimizes
sponsor time. An alternative method is to start a period of FDA time
when the last amendment before the next FDA action is received,
letting FDA review the earlier amendments during a period of sponsor
time. That procedure minimizes FDA time and maximizes sponsor time.
\4 See FDA User Fees, esp. pp. 21-22.
SPONSOR CHARACTERISTICS
------------------------------------------------------- Appendix I:2.5
Experience. We divided the sponsoring pharmaceutical companies into
four groups, based on their activities between 1987 and 1994. We
defined the most experienced companies as those that submitted 9 or
more NDAs to FDA during this period (that is, at least one per year).
Those that submitted between 5 and 8 NDAs in that period made up the
middle-experience group. The two least experienced groups submitted
4 or fewer NDAs. We further divided the least experienced companies
into one group with affiliations with other companies that sponsored
NDAs during this period and another group without such affiliations.
Affiliation meant that another sponsoring company had a significant
ownership stake in the sponsor of the NDA. We identified
affiliations by reviewing business and financial directories.
METHODOLOGY
--------------------------------------------------------- Appendix I:3
Most of our statistical analyses consist simply of listing average
review times, or the number of NDAs with a particular characteristic,
separately by year of submission or by the outcome of review.
However, we also conducted two regression analyses, one to identify
variables related to the length of the review process and another to
identify factors related to drug approval. (See appendixes IV and
V.) This allowed us to isolate the effects of one variable (for
example, drug priority) while statistically holding constant the
other predictor variables (for example, year of submission and the
experience of the sponsoring company). All our statements about
statistical significance are based on the results of the regressions,
which answer the question: If there were no differences among these
NDAs except, for example, drug priority, does drug priority influence
the chances of approval?
We performed our work in accordance with generally accepted
government auditing standards.
ALTERNATIVE MEASURES OF TIME
========================================================== Appendix II
The key statistics presented in this report are the average times to
final decisions for NDAs submitted in consecutive calendar years from
1987 onward. Previous reports on time have presented other results,
sometimes relying on slightly different measures of time, sometimes
reporting other statistics (medians rather than averages), and
usually constructing cohorts based on the years in which the NDAs
were approved rather than the years in which they were submitted. In
the sections that follow, we place our work in the context of other
studies of drug review and approval time by examining the differences
in approach.
STARTING POINTS FOR
CALCULATIONS OF TIME
-------------------------------------------------------- Appendix II:1
In our study, review time begins with the first submission of the NDA
to FDA. In FDA's statistical reports, it starts the clock with the
submission of an "accepted" NDA. The two measures would provide
similar results if the NDA were accepted on the first submission or,
if FDA refused to file it, the sponsor never resubmitted the
application. However, in any situation in which FDA refused to file
the NDA and the sponsor eventually resubmitted it, our measure of
review time would be longer by the interval between the first
submission and the date of an accepted submission. Approximately 1
in 10 NDAs (9.4 percent) fall into this category. The average time
to resubmission for these applications was a little less than 2
months (1.7 months). Therefore, our review times are slightly longer
on average than those reported by FDA.
ON-TIME PERFORMANCE
-------------------------------------------------------- Appendix II:2
Another approach to time measurement is to be less concerned with how
long the process took than with whether it was completed within a
specified period. FDA takes this approach when it reports the extent
to which the agency meets its user fee performance goals as
referenced in the Prescription Drug User Fee Act. Data on our
measure of on-time performance appear in the body of this report.
Table II.1 shows an annual breakdown of "on time" performance.\1
Table II.1
Actions Taken on Time by FDA for NDAs
Submitted 1987-94\a
Percent
Number of Number of taken
Year of submission NDAs actions "on time"
-------------------------------- ---------- ---------- ------------
1987 142 304 72%
1988 129 251 68
1989 117 206 67
1990 99 179 62
1991 110 188 63
1992 103 161 65
1993 87 116 72
1994 118 67 94
----------------------------------------------------------------------
\a Actions taken as of May 16, 1995.
As can be seen from table II.1, the percentages have changed little
over the years.\2 Interestingly, this is in contrast to the reduction
in total review time (the entire interval between submission and
approval) during this period. Seemingly, FDA has managed to reduce
the overall time even though it has not increased the proportion of
specific actions taken on time.\3
--------------------
\1 See appendix I for a discussion of our coding of an action as on
time.
\2 The data for 1994 are biased by the small number and type of
decisions that had been made for those NDAs by the time we collected
our data.
\3 The assumptions we used to calculate on-time performance served to
minimize the proportion of FDA actions that met the standard. First,
as we described in appendix I, we started each period of FDA time
with the submission of the first amendment after the last FDA action,
not the submission of the last amendment. Second, we did not extend
the deadline by 3 months if major amendments were filed close to the
original due date, as allowed in some circumstances under user fees.
We reported that 67 percent of FDA's actions are on time. Choosing
assumptions that are the most favorable to FDA (starting the clock
with the last submitted amendment and extending the deadline by 3
months in every case) would increase this figure to 78 percent.
AGENCY COMMENTS AND OUR
RESPONSE
-------------------------------------------------------- Appendix II:3
In commenting on a draft of this report, FDA officials agreed with
our general conclusions but made two points regarding our analysis of
on-time performance.
First, FDA emphasized that the 6- and 12-months guidelines used in
our analysis were not in effect during the years we studied and that
FDA is not required to meet them until 1997.
Second, while FDA believes that its review cycle on-time performance
may not have improved, the agency cautioned that the nature of its
actions has changed with the initiation of the user fee program,
particularly for not-approvable letters. Prior to the initiation of
user fees, not-approvable letters were not necessarily a complete
listing of all the deficiencies in the NDA. For example, FDA may
have sent one not-approvable letter when the review of one section of
the NDA was complete and additional not-approvable letters as other
sections of the review were completed. After user fees, FDA is
required to take complete actions, so a not-approvable letter must
contain all the deficiencies FDA identifies. In other words, FDA
must now complete more work to satisfy a post-user fee deadline than
it had to before user fees were introduced.
We agree with FDA's first point. FDA's second point argues for
caution in making comparisons of on-time performance between
different years. We agree that changes in procedure would invalidate
such comparisons. For that reason, we did not use this measure as an
indicator of whether the overall timeliness of the drug approval
process had improved. Rather, we included the trends in on-time
performance in the report in order to be comprehensive in presenting
all measures of time that others had reported.
ALTERNATIVE MEASURES OF TOTAL
REVIEW TIME
-------------------------------------------------------- Appendix II:4
AVERAGE TIMES COMPARED TO
MEDIAN TIMES
------------------------------------------------------ Appendix II:4.1
Throughout this report, we have reported the average times for NDA
review. An alternative is to report the median review time, the time
for the 50th percentile application. In this case, medians reduce
the influence of drugs with unusually long review periods and are
therefore usually somewhat lower than average review times. Table
II.2 lists the average and median approval times for the drugs we
examined by year of submission. While the median values are
generally slightly lower, they show the same pattern of consistent
decrease as the average values.
Table II.2
Average and Median Months to Approval
for NDAs Submitted 1987-92
Year of
submission Average Median
---------------------------------- ---------------- ----------------
1987 33.3 29.1
1988 30.1 26.7
1989 25.5 23.4
1990 25.1 23.1
1991 21.1 21.3
1992 19.2 18.7
----------------------------------------------------------------------
YEAR OF SUBMISSION COMPARED
TO YEAR OF APPROVAL
------------------------------------------------------ Appendix II:4.2
FDA and others frequently report time statistics for NDAs that group
the applications by the year in which they were approved rather than
the year in which they were submitted. To some extent, this reflects
FDA's general orientation away from publishing data on submissions
(given that much of that information is proprietary until they are
approved). Table II.3 compares the average approval times we
computed using year of submission with the average approval times FDA
computed using year of decision. The discussion that follows the
table indicates why grouping NDAs by year of submission is preferable
for our purpose.
Table II.3
Average NDA Approval Times in Months,
1987-94
By year of By year of
Year submission\a approval\b
------------------------------ ------------------ ------------------
1987 33.3 29.0
1988 30.1 28.9
1989 25.5 30.9
1990 25.1 30.0
1991 21.1 28.5
1992 19.2 32.6
1993 \c 33.1
1994 \c 25.5
----------------------------------------------------------------------
\a Calculated by GAO.
\b Reported by FDA.
\c We do not present values for these years because they may be
biased as a result of the censoring problem discussed in appendix VI.
Table II.3 shows an obvious difference between the decrease in
approval times when NDAs are grouped by year of submission and the
stability when they are grouped by year of approval. This difference
arises because grouping by year of approval incorporates into the
calculation whatever backlog of NDAs existed at FDA. For example,
several NDAs submitted in 1987 that had very lengthy 5-year reviews
would increase the average review time in 1987 for year-of-submission
statistics but would add to the average review time in 1992 for
year-of-approval figures.
Thus, whenever the possibility of a backlog exists, basing time on
year of approval is a less appropriate way to measure current
practice because it incorporates the older applications. In
contrast, time based on year of submission eliminates the confounding
effects of the backlog and, therefore, is the preferable measure for
assessing the current performance of the agency.
In 1987, the first year in our study, FDA had a considerable backlog
of NDAs submitted in 1986 and earlier and that backlog affected times
throughout nearly the entire period of our study. This can be seen
from table II.4.
Table II.4
Percent of NDAs Approved 1987-94 With
Approval Times Greater Than 4 Years
Percent approved
Total NDAs in more than 4
Year of approval approved years
------------------------------ ------------------ ------------------
1987 68 15%
1988 67 15
1989 87 20
1990 64 16
1991 63 16
1992 91 23
1993 70 24
1994 62 10
----------------------------------------------------------------------
Source: Adapted from FDA statistics.
As the table shows, a considerable proportion of the approvals in
every year except for 1994 were for older NDAs that had been under
review for a long time. The first years in which FDA seemed to make
progress in reducing the backlog were 1992 and 1993, when larger
percentages of older applications were approved. This progress was
reflected in the smaller percentage of older NDAs that were approved
in 1994 and in the sharp drop in times measured by year of approval
between 1993 and 1994 (see table II.3). The decrease from 33 to 26
months indicates that the backlog may have finally passed through the
system.
INTERMEDIATE OUTCOMES FROM FDA'S
REVIEW PROCESS
========================================================= Appendix III
In this appendix, we present data on what happens to the NDAs as they
move through the review process, focusing on three kinds of
activities: first actions, review cycles, and major amendments.
FIRST ACTIONS
------------------------------------------------------- Appendix III:1
Table III.1 shows the first action taken on NDAs submitted in each
successive year. It can be seen that approval is the initial
decision for relatively few NDAs.
Table III.1
First Actions on NDAs Submitted 1987-94
198 198 198 199 199 199 199 199
First action 7 8 9 0 1 2 3 4
------------------------------ --- --- --- --- --- --- --- ---
Percent of first actions on
submissions\a
Refusal to file 16% 10% 6% 13% 24% 27% 18% 11%
Withdrawal 13 18 17 15 8 13 8 4
Not approvable 46 41 37 34 32 19 20 10
Approvable 12 13 19 19 16 22 15 4
Approval 13 18 21 18 19 17 17 4
No first action\b 0 0 0 0 1 1 11 52
======================================================================
Total 100 100 100 99% 100 99% 89% 86%
% % % %
======================================================================
Total number 142 129 117 99 110 103 87 118
of submissions
----------------------------------------------------------------------
\a Percentages may not total 100 because of rounding. Percentages
for 1993 and 1994 do not total 100 because NDAs found "unacceptable
for filing" because of failure to pay user fees are not included in
the table.
\b As of May 16, 1995.
Given that approximately 55 percent of all NDAs are ultimately
approved, the data in table III.1 also show that such "negative"
decisions as refusal to file, not approvable, and withdrawal are not
necessarily fatal to an application. Of the 110 NDAs submitted from
1987 to 1992 that FDA initially refused to file, 35 (32 percent) were
ultimately approved. Similarly, 43 percent of the NDAs that had a
not-approvable first action were ultimately approved, and 27 percent
of the withdrawals were resubmitted and approved. Overall, 43
percent of the 390 drugs submitted from 1987 to 1992 that were
approved were refused, withdrawn, or found not approvable at some
point on their way to approval.
CYCLES
------------------------------------------------------- Appendix III:2
FDA reports the review cycles that an NDA goes through in its yearly
Statistical Reports. A cycle starts with the submission or
resubmission of an NDA and ends with the withdrawal of the NDA, a
refusal to file decision, or an approval, approvable, or
not-approvable letter. Each new cycle starts the review clock anew.
Table III.2 shows the number of cycles for various types of NDAs.
Table III.2
Review Cycles for NDAs Submitted 1987-
92
Tota Approved
Type of NDA 1 2 3 �4 l only
------------------------------ ---- ---- ---- ---- ---- --------
Approved\a 32% 41% 19% 9% 2.1 \b
Not approved 64 22 11 3 1.5 \b
Priority 52 38 7 4 1.6 1.7
Standard 45 31 17 7 1.9 2.2
NME 54 38 7 2 1.6 1.7
Non-NME 43 30 18 9 2.0 2.3
Year of submission
1987 32 35 25 8 2.1 2.4
1988 47 29 15 10 1.9 2.2
1989 51 30 14 5 1.8 2.0
1990 48 34 9 8 1.8 2.1
1991 49 34 14 4 1.7 2.0
1992 55 33 11 1 1.6 1.9
All 46 33 15 6 1.8 2.1
----------------------------------------------------------------------
\a Percentages may not total 100 because of rounding.
\b Not applicable.
As can be seen from table III.2, some types of NDAs are more likely
to go through multiple review cycles than others. Approved NDAs go
through more cycles on average than applications that get dropped
along the way; priority NDAs go through fewer cycles on average than
standard NDAs; and, similarly, NMEs go through fewer cycles on
average than non-NMEs. The number of cycles for both approved NDAs
and all NDAs has decreased for submissions since 1987. This decrease
is consistent with the decrease in time to final decisions.
AMENDMENTS
------------------------------------------------------- Appendix III:3
FDA has questions about almost all NDAs and requires sponsors to
submit additional data in response to those questions. The sponsors
submit these data in the form of amendments. Relatively small
amounts of data (for example, clarification of a point or correction
of a value) are classified as minor amendments, and relatively large
amounts of data (for example, a reanalysis or results of an
additional study) are classified as major amendments.
Table III.3
Major Amendments for NDAs Submitted
1987-92
6- Tota Approved
Type of NDA 0 1-5 10 �11 l only
------------------------------ ---- ---- ---- ---- ---- --------
Approved\a 12% 73% 12% 3% 3.5 \b
Not approved 45 49 5 1 1.7 \b
Priority 23 57 17 3 3.3 3.5
Standard 28 63 7 2 2.6 3.5
NME 20 59 15 5 3.8 4.6
Non-NME 29 63 6 1 2.3 3.0
Year of submission
1987 25 63 8 4 2.9 4.3
1988 20 71 6 2 2.7 3.1
1989 25 59 15 1 3.1 3.8
1990 31 54 10 5 2.9 4.4
1991 34 55 9 2 2.5 2.9
1992 27 69 4 0 2.0 2.3
All 27 62 9 2 2.7 3.5
----------------------------------------------------------------------
\a Percentages may not total 100 because of rounding.
\b Not applicable.
Table III.3 shows the number of amendments for different types of
NDAs. As expected, NDAs that are pursued through to approval have
more major amendments on the average than NDAs that drop out of the
process. NDAs for priority drugs and for NMEs required more amending
on average than applications for standard drugs and non-NMEs. As
with the data on cycles, table III.3 shows a decrease in the number
of amendments for submissions since 1987.
These data, along with those in table III.1 showing a steady decrease
in the numbers of not approvables and in table III.2 showing fewer
cycles, suggest that the drug review and approval process is getting
"cleaner." This change may result from different applications
submitted by the sponsors of new drugs, different FDA review
procedures, or both. Without additional study, it is not possible to
identify the reasons for this. However, all three sets of data (on
first action, cycles, and major amendments) are consistent with a
quicker review process.
REGRESSION ANALYSES FOR REVIEW
TIME
========================================================== Appendix IV
We conducted two regression analyses predicting review time, one for
approved new drug applications and the other for applications that
were not approved. As table IV.1 shows, we found that the length of
time until approval was significantly affected by three factors--year
of submission, drug priority, and sponsor experience. Applications
submitted in later years were approved much faster than earlier
applications (for example, 11 months quicker in 1992 than in 1987).
Drug applications given therapeutic priority by FDA were approved
nearly 10 months faster than standard drugs. Applications from
sponsors that submitted many NDAs were approved more quickly than
applications from relatively inexperienced sponsors (for example,
applications from the most experienced sponsors were approved 4
months faster than those from inexperienced sponsors that were not
affiliated with other sponsoring companies).
Table IV.1
For Approved New Drug Applications,
Regression Analysis Predicting Number of
Months From First Submission to
Approval\a
Coefficie T- Probabilit Sample
Variable\b nt\c value\d y level\e mean
----------------------------- --------- -------- ---------- ------
Year of submission
(vs. 1987)
1988 -2.44 -1.08 .28 .19
1989 -6.92 -2.95 .01 .17
1990 -6.54 -2.60 .01 .14
1991 -11.19 -4.76 .01 .16
1992 -11.39 -4.54 .01 .14
Priority drugs -9.89 -4.97 .01 .23
(vs. standard)
New molecular entity .38 .22 .82 .32
(vs. not)
Sponsor experience
(vs. inexperienced,
unaffiliated)
Inexperienced, affiliated -6.47 -1.99 .05 .07
Mid-experienced -4.14 -1.58 .12 .12
Most experienced -4.38 -2.22 .03 .65
Constant 38.25
----------------------------------------------------------------------
\a For applications first submitted from 1987 to 1992, N = 390, and
R-squared = 0.24. The mean review time is 26.36 months.
\b The list of predictor variables also included categorical
variables for the FDA reviewing divisions. We did not report those
coefficients here because they cannot readily be interpreted; the
chemical and therapeutic content and complexities of new drugs are
strongly correlated with the reviewing divisions, making it
impossible for us to distinguish the effects of drug variations from
those that are attributable to the internal operations of the
divisions. However, it is important to note that the coefficients
reported in the table are statistically independent of the effects of
the reviewing division.
\c Coefficients are from an ordinary least-squares regression
analysis with the SAS-PC software package. The coefficient indicates
the change in review time relative to that of the group left out.
For example, the coefficient of -2.44 for applications submitted in
1988 means that those applications were approved nearly 2-1/2 times
faster than applications submitted in 1987, the group left out.
\d The T-values test the statistical significance of the
coefficients.
\e Probability level refers to the chances that the coefficient
equals zero in the population. By convention, coefficients with a
probability level less than or equal to 5 percent (0.05) are regarded
as statistically significant. In this table, 0.01 indicates a
probability level less than or equal to 0.01.
In contrast, for drugs that were not approved, the only significant
factor was year of submission. Applications submitted in later years
were acted on more quickly than those submitted earlier (see table
IV.2). Neither therapeutic priority nor the experience of the
sponsor affected review time. It is important to reiterate that FDA
does not definitively reject applications it does not approve.
Therefore, FDA may take further action on some of the applications in
this analysis.
Table IV.2
For New Drug Applications Not Approved,
Regression Analysis Predicting Number of
Months From First Submission to Date of
Final Action\a
Probabil
Coefficie T- ity Sample
Variable\b nt\c value\d level\e mean
------------------------------- --------- -------- -------- ------
Year of submission (vs. 1987)
1988 .98 .27 .79 .18
1989 -10.13 -2.67 .01 .17
1990 -12.83 -3.32 .01 .15
1991 -11.90 -3.01 .01 .15
1992 -15.87 -4.14 .01 .16
Priority drugs 2.79 .71 .48 .09
(vs. standard)
New molecular entity (vs. not) 1.90 .68 .50 .22
Sponsor experience (vs.
inexperienced, unaffiliated)
Inexperienced, affiliated 7.10 1.51 .13 .07
Mid-experienced 2.01 .53 .59 .14
Most experienced 4.00 1.53 .13 .48
Constant 34.70
----------------------------------------------------------------------
\a For applications first submitted from 1987 to 1992, N = 308, and
R-squared = 0.16. Mean review time is 24.93 months.
\b The list of predictor variables also included categorical
variables for the FDA reviewing divisions. We did not report those
coefficients here because they cannot readily be interpreted; the
chemical and therapeutic content and complexities of new drugs are
strongly correlated with the reviewing divisions, making it
impossible for us to distinguish effects from drug variations from
those that are attributable to the internal operations of the
divisions. However, it is important to note that the coefficients
reported in the table are statistically independent of the effects of
the reviewing division.
\c Coefficients are from an ordinary least-squares regression
analysis with the SAS-PC software package. The coefficient indicates
the change in review time relative to that of the group left out.
For example, the coefficient of 0.98 for applications submitted in
1988 means that those applications were acted on nearly 1 month
slower than applications submitted in 1987, the group left out.
\d The T-values test the statistical significance of the
coefficients.
\e Probability level refers to the chances that the coefficient
equals zero in the population. By convention, coefficients with a
probability level less than or equal to 5 percent (0.05) are regarded
as statistically significant. In this table, 0.01 indicates a
probability level less than or equal to 0.01.
REGRESSION ANALYSIS FOR APPROVAL
=========================================================== Appendix V
Table V.1 presents the results of a logistic regression analysis
predicting NDA approval. The outcome variable is dichotomous: "1"
indicates that the drug has been approved, "0" that it has not been
approved. Fifty-six percent of the NDAs were approved. The data set
for the regression consists of the 698 drugs first submitted between
1987 and 1992 that had final status values as of May 16, 1995 (two
applications were pending).
Table V.1
Logistic Regression Analysis Predicting
NDA Approval
Probabil
Coefficie Odds Chi- ity Sample
Variable\a nt\b ratio\c square\d level\e mean
--------------------- --------- -------- -------- -------- ------
Year of
submission
(vs. 1987)
1988 .06 1.07 .06 .81 .18
1989 -.18 .84 .43 .51 .17
1990 -.30 .74 1.07 .30 .14
1991 .01 1.01 .01 .98 .16
1992 -.46 .63 2.62 .11 .15
Priority drug 1.30 3.68 23.98 .01 .17
(vs. standard)
New molecular entity .27 1.30 1.70 .19 .28
(vs. not)
Sponsor experience
(vs. inexperienced,
unaffiliated)
Inexperienced, .49 1.63 1.83 .18 .07
affiliated
Mid-experienced .60 1.81 4.03 .04 .13
Most experienced 1.10 3.01 28.21 .01 .57
Constant -.49
----------------------------------------------------------------------
\a The list of predictor variables also included categorical
variables for the FDA reviewing divisions. We did not report those
coefficients here because they are not readily interpretable; the
chemical and therapeutic content and complexities of new drugs are
strongly correlated with the reviewing divisions, making it
impossible for us to distinguish effects from drug variations from
those that are attributable to the internal operations of the
divisions. However, it is important to note that the coefficients
reported in the table are statistically independent of the effects of
the reviewing division.
\b Coefficients are from a logistic regression analysis with the
SAS-PC software package.
\c The odds ratio is the exponentiated coefficient (e\coefficient ).
The odds ratio indicates the change in the odds of approval relative
to that of the group left out. For example, the approval odds for
applications submitted in 1988 are 1.07 greater than those for
applications submitted in 1987, the group left out.
\d The chi-square values test the statistical significance of the
coefficients.
\e Probability level refers to the chances that the coefficient
equals zero in the population. By convention, coefficients with a
probability level less than or equal to 5 percent (0.05) are regarded
as statistically significant. In this table, 0.01 indicates a
probability level less than or equal to 0.01.
The regression uncovered two statistically significant factors--drug
priority and sponsor experience. Priority drugs were approved at
nearly four times the rate of nonpriority drugs. Applications from
sponsors that submitted many NDAs during this period were approved
more often than applications from relatively inexperienced sponsors
(applications from the most experienced sponsors were approved three
times more often than applications from inexperienced sponsors that
were not affiliated with other sponsoring companies; applications
from companies with mid-levels of experience were approved nearly
twice as often).
CENSORING BIAS
========================================================== Appendix VI
As mentioned in appendix II, basing our selection of NDAs for
analysis on the year of submission has one significant advantage over
the more traditional approach of examining NDAs by year of approval.
That is, our approach avoids the contamination of the averages by
whatever backlog exists. However, relying on year of submission can
introduce another form of bias in that averages for approval time
computed from all the 1993 and 1994 cohorts incorporate only a highly
selective group of NDAs from those 2 years.
As table VI.1 shows, the final status distribution for NDAs submitted
in 1993 and 1994 is radically different from that for NDAs submitted
earlier. Clearly, this is because many of the applications had not
had time to "mature" by the time we collected our data. While more
than 50 percent of NDAs submitted in every year from 1987 to 1992
were approved by May 1995, comparatively few of the NDAs submitted in
1993 and 1994 had been approved. Most importantly, the only NDAs
from 1993 and 1994 that were approved were those that had been
approved relatively quickly. As a result, the average approval time
for NDAs submitted in 1987-92 is 26.4 months, while the average time
for approved NDAs submitted in 1993 and 1994 is 12.6 months. Because
of this bias, we excluded NDAs submitted after 1992 whenever we
examined final status.
Table VI.1
Final Status for NDAs Submitted by Year
of Submission 1987-94\a
Final status 1987 1988 1989 1990 1991 1992 1993 1994
---------------------- ---- ---- ---- ---- ---- ---- ---- ----
Approved 56% 58% 56% 54% 58% 52% 33% 5%
Withdrawn 21 26 22 25 11 18 11 6
Refused to file 7 3 3 3 12 9 11 13
Approvable 1 2 2 3 5 5 7 4
Not approvable 14 12 17 15 13 16 23 11
Pending 0 0 0 0 1 1 11 51
----------------------------------------------------------------------
\a Final status as of May 16, 1995. Percentages may not total 100
because of rounding. Percentages for 1993 and 1994 do not total 100
because NDAs found "unacceptable for filing" because user fees were
not paid are not included in the table.
However, we included NDAs from 1991 and 1992 because we found no
evidence that including these years risks exposure to the censoring
bias found in 1993 and 1994. As table VI.1 shows, the approval rates
for 1991 and 1992 are equivalent to those from earlier years. That
is, almost all the NDAs from 1991 and 1992 for which approval
ultimately would be expected have already been approved by FDA.
Approval times for those years are not likely to increase much.
The question that remains is whether the trend in decreasing time
that we observed for submissions between 1987 and 1992 continued for
1993 and 1994 submissions. That question cannot be answered
definitively until the 1993 and 1994 cohorts have had time to mature.
However, preliminary evidence suggests that the trend continues.
Table VI.2 compares the percentage of all applications submitted
before 1993 that were approved quickly to the same percentage for
NDAs submitted in 1993 and 1994.
Table VI.2
Percent of NDAs Approved Quickly, 1987-
92 and 1993-94
Time between acceptance of NDA and approval 1987-92 1993-94
---------------------------------------------- ---------- ----------
Within 6 months 1% 2%
Within 9 months 4 5
Within 12 months 8 9
----------------------------------------------------------------------
As table VI.2 shows, approximately the same percentages of NDAs were
approved quickly both before and after 1992. From this evidence, we
have no reason to suspect that the trend of speedier drug approval
for 1987-92 submissions was reversed for 1993-94 submissions.
APPROVAL TIMES IN THE UNITED
KINGDOM
========================================================= Appendix VII
The United Kingdom's equivalent of FDA is the Medicines Control
Agency (MCA). MCA publishes information similar to that contained in
FDA's statistical reports, including data on workload (number and
type of submissions) and time (how long it takes to review
applications). MCA's 1994-95 annual report indicates that the
assessment of an application for a new active substance (the apparent
equivalent of what FDA terms a new molecular entity) took an average
of 56 working days. This figure stands in sharp contrast to FDA's
reports that show an average approval time of 20 months for
applications for NMEs approved in 1994. No doubt, the sharp contrast
in these two averages is one factor creating the impression that
approval times are much shorter in the United Kingdom than they are
in this country.
However, closer examination of the data in MCA's annual report shows
that they should be compared to our data on FDA with caution. Most
importantly, the drug review process in the United Kingdom is very
different from that in the United States. In the United Kingdom,
MCA's assessment is only the first step in a multistage process of
drug review and approval. All applications for new active substances
are also automatically referred to a government body called the
Committee on the Safety of Medicines (CSM). CSM's expert
subcommittees also assess the application, and these assessments,
along with those from MCA, are provided to CSM. CSM then provides
advice to the Licensing Authority, which actually grants or denies
the product license. However, the rate of rejection of applications
or requests for modifications or additional information is very high
(99 percent for applications submitted 1987-89), although many of
these issues are minor and quickly resolved. Applications with
remaining unresolved issues then go through a formal appeals process
that may involve additional work on the part of the applicant,
reassessment by MCA or CSM, and, in rare cases, the involvement of
another body called the Medicines Commission. Thus, the total time
until the license is actually granted is considerably longer than the
period of initial assessment by MCA. In contrast, the time FDA
reports includes all the steps between an accepted NDA and the final
decision on it.
When one examines total time for both processes, the United Kingdom
does not appear to be dramatically faster than the United States.
One recent study compared approval times for 11 drugs that were
approved in both countries during the period 1986-92. The median
time in the United States (about 23 months) was 15 percent longer
than the median time in the United Kingdom (20 months).\1 The most
recent data from MCA show that overall approval times are actually
somewhat longer than that.\2 These data indicate that MCA granted
licenses for applications representing 32 new active substances
during the 12-month period ending September 30, 1994. The median
time for granting a license was 30 months and the average was 24
months. The fastest license was granted in about 4 months, the
slowest in 62 months.\3
FDA's data for the calendar year ending December 31, 1994, indicate
that the agency approved a total of 22 new molecular entities. The
median approval time was 18 months, average approval time about 20
months. The fastest approval reported by FDA took about 6 months and
the slowest about 40 months.
Thus, the most recent data show that approval times for NMEs are
actually shorter in the United States. In addition, a broader
perspective shows that approval processes in many industrialized
nations may be converging.\4 Approval times over the past 10 years
for France, Germany, Japan, the United Kingdom, and the United States
all seem to be moving toward the 2-year point. The trend in the
United States (which had lengthy times throughout the mid-1980s) has
been toward more rapid times, whereas the process has been getting
slower in some of the other (originally faster) countries.
--------------------
\1 C. Harvey et al., "A Comparison of the Review of a Cohort of NCEs
by Four National Regulatory Authorities," Journal of Pharmaceutical
Medicine, 3 (1993), 65-75.
\2 From the bimonthly newsletter of the Medicines Control Agency, The
MAIL, November-December 1994.
\3 An additional complication is that MCA was starting to make the
conversion to a European-wide drug review process during this period,
meaning that the times MCA reported for 1994 may not be typical.
\4 Neal McAuslane, "A Comparison of Regulatory Review Times in
Europe, Japan, and the United States," presented at the 31st Annual
Meeting of the Drug Information Association, Orlando, Florida, June
26, 1995.
MAJOR CONTRIBUTORS TO THIS REPORT
======================================================== Appendix VIII
This report was prepared by Martin T. Gahart, Michele Orza, George
Silberman, and Richard Weston of the Program Evaluation and
Methodology Division.
RELATED GAO PRODUCTS
============================================================ Chapter 0
FDA User Fees: Current Measures Not Sufficient for Evaluating Effect
on Public Health (GAO/PEMD-94-26, July 22, 1994).
FDA Premarket Approval: Process of Approving Lodine as a Drug
(GAO/HRD-93-81, April 12, 1993).
FDA Regulations: Sustained Management Attention Needed to Improve
Timely Issuance (GAO/HRD-92-35, February 21, 1992).
FDA Drug Review: Postapproval Risks 1976-1985 (GAO/PEMD-90-15, April
26, 1990).
FDA Resources: Comprehensive Assessment of Staffing, Facilities and
Equipment Needed (GAO/HRD-89-142, September 15, 1989).