NIH Extramural Clinical Research: Internal Controls Are Key to
Safeguarding Phase III Trials Against Misconduct (Letter Report,
07/11/96, GAO/HEHS-96-117).
Pursuant to a congressional request, GAO reviewed the National
Institutes of Health's (NIH) oversight of the clinical trials it
sponsors, focusing on NIH internal controls to prevent misuse of federal
funds and safeguard the integrity of clinical trial data.
GAO found that: (1) individual NIH institutes and grantee institutions
oversee and monitor NIH-funded Phase III clinical trials; (2) internal
controls to guard against fiscal misconduct in extramural research must
comply with regulations and policies on federal funds expenditures and
be consistently applied; (3) independent auditors review grantees'
compliance with these internal control regulations during annual audits;
(4) NIH imposes sanctions on offending grantees when scientific
misconduct occurs; (5) internal controls to guard against scientific
misconduct and ensure participants' safety are generally consistent, but
they vary slightly among trials due to differences in the sponsoring
institute's management philosophy and past experience, the trial's size,
nature, and complexity, and the way the trial is funded; (6) NIH
institutes and grantees use clinical monitors and data coordinating
centers to ensure data integrity, but these controls are not
consistently applied; (7) one institute provides direct funding for
certain data verification functions which ensures prompt reporting of
data concerns and potential misconduct to NIH, but this is not an
agencywide policy for multisite trials; (8) central NIH oversight and
monitoring of clinical trials is limited; and (9) NIH has not adopted on
an agencywide basis, any of its study committee's recommendations to
improve clinical trial oversight because it believes agencywide
monitoring policies are inappropriate.
--------------------------- Indexing Terms -----------------------------
REPORTNUM: HEHS-96-117
TITLE: NIH Extramural Clinical Research: Internal Controls Are Key
to Safeguarding Phase III Trials Against
Misconduct
DATE: 07/11/96
SUBJECT: Health research programs
Medical research
Research program management
Federal funds
Ethical conduct
Internal controls
Financial management
Data integrity
Cancer research
Research grants
IDENTIFIER: NCI National Surgical Adjuvant Breast and Bowel Project
NIA Continence Program for Women
NIH Streamlined Non-Competing Award Process Data Base
******************************************************************
** This file contains an ASCII representation of the text of a **
** GAO report. Delineations within the text indicating chapter **
** titles, headings, and bullets are preserved. Major **
** divisions and subdivisions of the text, such as Chapters, **
** Sections, and Appendixes, are identified by double and **
** single lines. The numbers on the right end of these lines **
** indicate the position of each of the subsections in the **
** document outline. These numbers do NOT correspond with the **
** page numbers of the printed product. **
** **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced. Tables are included, but **
** may not resemble those in the printed version. **
** **
** Please see the PDF (Portable Document Format) file, when **
** available, for a complete electronic file of the printed **
** document's contents. **
** **
** A printed copy of this report may be obtained from the GAO **
** Document Distribution Center. For further details, please **
** send an e-mail message to: **
** **
** **
** **
** with the message 'info' in the body. **
******************************************************************
Cover
================================================================ COVER
Report to Congressional Requesters
July 1996
NIH EXTRAMURAL CLINICAL RESEARCH -
INTERNAL CONTROLS ARE KEY TO
SAFEGUARDING PHASE III TRIALS
AGAINST MISCONDUCT
GAO/HEHS-96-117
NIH Phase III Clinical Trials
(108251)
Abbreviations
=============================================================== ABBREV
HHS - Department of Health and Human Services
NCI - National Cancer Institute
NHLBI - National Heart, Lung, and Blood Institute
NIA - National Institute on Aging
NIH - National Institutes of Health
OMB - Office of Management and Budget
ORI - Office of Research Integrity
PHS - Public Health Service
SNAP - Streamlined Non-Competing Award Process
Letter
=============================================================== LETTER
B-266107
July 11, 1996
The Honorable William S. Cohen
The Honorable Nancy L. Kassebaum
United States Senate
The National Institutes of Health (NIH) is a major sponsor of
federally funded scientific research. In fiscal year 1995, NIH
sponsored about $9 billion in extramural research--research conducted
by entities outside of NIH. About $1.2 billion was spent on Phase
III clinical trials.\1
In the early 1990s, disclosure that falsified data had been used in a
large Phase III clinical trial evaluating alternative treatments for
breast cancer raised concern that the results of this multimillion
dollar trial had been compromised.\2 This case and others raised
questions about the effectiveness of internal controls in
safeguarding the integrity of clinical trial data and NIH's oversight
of federally funded research it sponsors.
Because of your concerns, you asked us to determine how NIH oversees
the clinical trials it sponsors and whether internal controls are in
place to prevent misuse of federal funds and safeguard the integrity
of clinical trial data. To respond to your request, we determined
the oversight responsibilities of NIH and identified controls used to
prevent and detect misconduct in Phase III clinical trial research.
We also reviewed NIH's approach to monitoring performance of its
institutes that sponsor clinical trials and efforts to implement
agencywide policy guidance on misconduct in research. Our review did
not include an evaluation of efforts to investigate allegations of
fiscal and scientific misconduct. The Department of Health and Human
Services' (HHS) Office of Research Integrity (ORI) investigates some
allegations of misconduct in research and oversees investigations
conducted by extramural research institutions. In our August 1995
report, we stated that ORI had made progress in its handling of
misconduct cases since its establishment in May 1992. However, it
still faced a substantial case backlog and lengthy delays in
completing its work.\3
Our review included a nonstatistical sample of four Phase III
clinical trials that had multiple sites participating in the research
and were funded through cooperative agreements--a frequently used
funding mechanism in fiscal year 1994. We conducted our work at NIH
and 2 of its 17 institutes that sponsor research--the National
Institute on Aging (NIA); National Heart, Lung, and Blood Institute
(NHLBI); and several research sites. (See app. I for a detailed
discussion of our scope and methodology.) Our work was performed
between September 1995 and May 1996 in accordance with generally
accepted government auditing standards.
--------------------
\1 NIH defines a Phase III clinical trial as any broad-based clinical
study, usually involving several hundred human participants, that
evaluates either an experimental treatment compared with a control or
a comparison of two or more existing treatments. The goal of these
trials is to develop scientific evidence as the basis for considering
changes in either the standard of care or public health policy.
Generally, Phase I trials involve experimental drug treatments,
determining toxicity and safe drug dose levels. Phase II trials
evaluate the effectiveness of a particular treatment and look for
side effects using a small number of trial participants.
\2 The National Surgical Adjuvant Breast and Bowel Project Protocol
B-06, a clinical trial sponsored by the National Cancer Institute
(NCI), included an evaluation of the value of lumpectomy and breast
irradiation for treating women in the early stages of breast cancer.
Publicity generated by the discovery that the study included
fraudulent data on patients enrolled by St. Luc Hospital in Montreal
raised concern about the overall accuracy of the data and
conclusions. An NCI audit, however, concluded that the data on which
the trial's analysis and results were based were adequate. NIH is
seeking restitution of costs related to this case of scientific
misconduct.
\3 Health Research Misconduct: HHS' Handling of Cases Is
Appropriate, but Timeliness Remains a Concern (GAO/HEHS-95-134, Aug.
3, 1995).
RESULTS IN BRIEF
------------------------------------------------------------ Letter :1
Individual NIH institutes and the institutions receiving the funds to
do the research conduct most oversight of NIH-funded Phase III
clinical trials. A number of internal controls are in place to
safeguard Phase III clinical trials against both fiscal and
scientific misconduct in extramural research. The controls that
guard against fiscal misconduct are standard procedures that must
comply with federal policies and regulations on the expenditure of
federal funds. The monitoring procedures and controls that guard
against scientific misconduct are generally consistent but vary
sometimes on the basis of the management philosophy and past
experience of the NIH institute sponsoring the trial; the trial's
size, nature, and complexity; and the way the trial is funded.
Even though controls exist to safeguard clinical trials against
misconduct, no practical level of oversight can guarantee the
complete fiscal or scientific integrity of each clinical trial NIH
sponsors. The detection of cases of potential or actual misconduct
demonstrates that abuse does occur and suggests that oversight
controls do work with some effectiveness. The application of the
controls often influences their effectiveness. For example, in the
multisite NCI-sponsored breast cancer trial in which patient data
were fabricated and falsified, an internal control procedure
discovered the data irregularities. The data problem was not
promptly reported to NIH, however, after it was detected. Instead,
it was reported to the principal investigator conducting the trial,
who waited 8 months before notifying NIH. To address this problem,
at least one institute provides direct funding for certain data
verification functions. This approach makes key data integrity
internal control functions independent of research investigators and
creates a direct link to NIH for prompt reporting of data concerns
and possible scientific misconduct. This approach, however, is not
an agencywide policy or practice for multisite trials.
In 1994, NIH launched a study to determine how the institutes oversee
and manage the Phase III clinical trials they sponsor. An internal
working committee on clinical trial monitoring determined that the
type of trial, its funding mechanism, and the sponsoring institute
greatly influence the implementation of certain internal control
measures. The committee recommended that NIH establish some
agencywide guiding principles for managing trials. These principles
would cover such areas as quality assurance and site monitoring,
level of NIH staff monitoring, and patient confidentiality. NIH has
not adopted any of the committee's recommendations agencywide,
although in the past it has implemented some agencywide policies
designed to discourage misconduct in scientific research. NIH
believes adopting agencywide policies such as those the committee
recommended is inappropriate because all clinical trials should not
be monitored in the same way. Some institutes have selectively
adopted some of the committee's recommendations.
BACKGROUND
------------------------------------------------------------ Letter :2
NIH is a Public Health Service (PHS) agency within HHS. It consists
of a director's office and 14 staff offices that oversee the
operations of 24 separate units. These units include 17 institutes,
each focused on specific health or medical issues, such as cancer or
aging; six research centers; and the National Library of Medicine.
Each unit separately awards funds for the research it sponsors.
NIH's Office of Extramural Research is responsible for agencywide
activities concerning oversight of Phase III clinical trials, such as
developing policy on the review, funding, and management of
extramural grants.
NIH's extramural research units (generally referred to in this report
as "institutes") used various methods to fund the 470 Phase III
clinical trials they sponsored in fiscal year 1994. As figure 1
shows, the largest number of trials (180) were funded through
cooperative agreements. Regardless of the method used to fund the
trials, the institutions that are awarded the funds are referred to
as "grantee institutions" or "grantees."
Figure 1: Number of Extramural
Phase III Clinical Trials
Sponsored by NIH, Fiscal Year
1994
(See figure in printed
edition.)
Source: NIH's Office of Extramural Research.
Most Phase III clinical trials involving multiple sites are funded
through contracts and cooperative agreements. Trials funded through
contracts are typically planned, initiated, and controlled by the
sponsoring NIH institute. The institute details the trial's
objectives, protocols, and controls.
Under cooperative agreements, however, the grantees and principal
investigators have more flexibility in planning, managing, and
conducting the trial. Although the sponsoring institute is expected
to make substantial contributions to the trial, such as providing
technical assistance, coordinating the trial's activities, and
helping to manage the trial, operational control of the trial rests
with the grantee.
NIH INSTITUTES AND GRANTEES
PROVIDE OVERSIGHT OF CLINICAL
TRIALS
------------------------------------------------------------ Letter :3
The institutes and research centers at NIH along with the grantee
institutions directly oversee and monitor Phase III clinical trials.
These entities are to ensure that controls are in place to prevent or
detect the misuse of federal funds and the falsification of data in
conducting extramural clinical research. According to NIH, these
institutes and grantee institutions know the nature and objectives of
the trials and are therefore in the best position to develop
monitoring procedures to ensure safety and data integrity. At the
sites we visited, controls that safeguard against fiscal misconduct
are consistently applied among institutes and trials. Some controls
that safeguard against scientific misconduct, however, are not always
consistently used for various reasons, including the type of trial
and the sponsoring institute's management philosophy.
FISCAL OVERSIGHT POLICIES
AND CONTROLS ARE CONSISTENT
AMONG INSTITUTES AND
GRANTEES
---------------------------------------------------------- Letter :3.1
Although each institute independently oversees the clinical trials it
sponsors, the controls established to prevent and detect fiscal
misconduct were consistent among the institutes in our review. The
control procedures must conform with federal requirements and
policies on the expenditure of federal funds. Grantee institutions
are responsible for ensuring that their research scientists and other
employees comply with all applicable federal rules, regulations, and
policies on the use of federal funds. Independent auditors review
grantee compliance annually in a required financial audit.
Most grantee institutions receive federal funds from several federal
agencies. The grantees must adhere to a uniform series of
regulations laid out by the Office of Management and Budget (OMB).
Chief among these policies are cost principles that grantees must
adhere to as specified in OMB Circulars A-21, A-87, or A-122.\4
These principles provide guidance on what expenses a grantee may
incur and charge against an NIH grant award. Grantees must also
follow a uniform set of administrative requirements in OMB Circulars
A-102 or A-110, detailing how grant funds should be managed.\5
Foremost among these requirements are standards for such areas as
fiscal reporting, accounting records, internal controls, and cash
management. Other administrative requirements cover procurement and
property standards.
Independent auditors annually audit grantees' fiscal management of
federal funds as required by OMB Circulars A-128 and A-133.\6 It was
such an audit that detected the embezzlement of more than $700,000 of
NIH grant funds in the early 1990s. This case of fiscal misconduct
by a manager of grants accounting occurred at the New York Medical
College--the grantee. Because the grantee institution is responsible
for ensuring that federal grant funds are properly used, the college
was required to fully refund these funds to NIH.
At the five grantee institutions we visited, we reviewed the annual
financial audits. The audits included a review of internal controls
established by the grantees to safeguard federal funds. In each
case, the audits disclosed that grantees had complied with federal
guidelines and no material weaknesses were detected in the internal
controls.
When grantee institutions fail to establish and maintain adequate
internal controls and proper accounting procedures to safeguard
federal funds, NIH can impose requirements that the grantee must
comply with to continue receiving and managing grant awards. In
1995, NIH designated the University of Minnesota, a grantee, an
"exceptional organization" because of its failed internal controls
and poor accounting procedures. This designation enabled NIH more
oversight of its funds than would be feasible under the
administrative procedures normally associated with its grant
programs. NIH increased the conditions and restrictions attached to
the University's grant award. It also required the University to
develop and successfully implement a corrective action program to
address the deficiencies before NIH would consider removing the
exceptional organization designation.
--------------------
\4 The cost principles that a grantee must follow depend on the
nature of the grantee institution. The applicable circulars are OMB
Circular A-21, Cost Principles for Educational Institutions; OMB
Circular A-87, Cost Principles for State and Local Governments; and
OMB Circular A-122, Cost Principles for Non-Profit Organizations,
which include medical centers and hospitals.
\5 The applicable circulars for administrative requirements are OMB
Circular A-102, Grants and Cooperative Agreements With State and
Local Governments, and OMB Circular A-110, Uniform Administrative
Requirements for Grants and Agreements with Institutions of Higher
Education, Hospitals and Other Non-Profit Organizations.
\6 For most grantees, this annual financial audit must comply with
OMB Circular A-128, Audits of State and Local Governments, or OMB
Circular A-133, Audits of Institutions of Higher Education and Other
Nonprofit Institutions.
GRANTS MANAGEMENT OFFICERS
MONITOR FISCAL INTEGRITY
---------------------------------------------------------- Letter :3.2
Each institute assigns grants management officers to clinical trials
to oversee the use of federal funds awarded to grantee institutions.
One method used by the grants management officers to reduce the
agency's risk is to limit the amount of funds readily available to
the grantees. For instance, because a cooperative agreement usually
covers more than 1 year, the initial award specifies how much funding
will be provided each year for the life of the agreement. However,
funding is provided on a year-to-year basis only. The grantee must
apply each year for a continuation award for additional funding even
though the total grant amount is committed. Institutes release funds
on the basis of satisfactory performance as detailed in the annual
progress reports that principal investigators must submit. If a
grantee's progress is not satisfactory, a grants management officer
may reserve all or some of the funding until grantee progress
improves.
Institutes awarding funds for clinical trial research issue award
notices that include a section indicating whether any of the funding
is restricted and what must be done to lift the restriction. If a
grantee's funds are restricted, the grants management officer might
release the funds but restrict their use until the grantee has
completed certain tasks. For example, in one trial we reviewed, the
officer restricted administrative funds until the sites developed a
contractual agreement indicating how they would work together.
In addition to annual progress reports, the grantee must include a
summary of annual expenditures in its Financial Status Report to the
grants management officer. This allows the officer to compare the
reported overall expenditure totals with the original budget and
progress reports. If the officer finds any significant differences,
the grantee is expected to explain them. For Phase III clinical
trials funded through contracts or cooperative agreements, according
to NIH, when grantees do not spend funds as budgeted, grants
management officers must approve all requests to rebudget funds as
well as requests to carry over funds from one year to the next.
PROGRAM OFFICERS AND
OVERSIGHT BOARDS MONITOR
SCIENTIFIC CONDUCT
---------------------------------------------------------- Letter :3.3
Institutes' oversight monitoring of clinical trials has some
consistent safeguards against scientific misconduct and protections
for the safety of trial participants. Each institute usually
requires specific monitoring methods. For example, an NIH program
officer is assigned by the sponsoring institute to monitor each Phase
III trial. Program officers are research scientists with expertise
in the area being studied. Each institute trains and develops its
own program officers in monitoring and managing clinical trials.
Therefore, program officers' training can vary by institute.\7 Also,
their responsibilities often vary by the institute's management
philosophy, the type of trial, and the funding method--contracts or
cooperative agreements. Typically, program officers, at a minimum,
rely on basic oversight controls in monitoring clinical trials,
including annual progress and budget reports and trial participants'
recruitment and retention statistics.
Oversight boards also monitor trials. For example, most clinical
trials that pose a potential hazard to human trial participants must
be monitored by a Data and Safety Monitoring Board or an equivalent.
This board, composed of scientists not connected with the trial,
monitors a trial's clinical data and progress. The board also
focuses on reported adverse events--adverse changes in the health
status of a human research subject in a clinical trial. In addition
to a Data and Safety Monitoring Board, each grantee institution must
establish an Institutional Review Board to approve and monitor all
research involving human subjects. An important function of this
board is to review and approve informed consent forms, making sure
they have been signed. All prospective research subjects must sign
consent forms that explain the objectives, risks, and benefits of the
proposed research before they can participate in a trial.\8
In our review of clinical records at the five sites we visited, we
did not find any cases in which a consent form had not been signed by
a trial participant. However, according to a report on the
NCI-sponsored breast cancer trial,\9 only about 71 percent of trial
participants gave written informed consent before surgery; consent
forms were missing or not available or data were insufficient for 7
percent of the participants.
--------------------
\7 In January 1995, NIH's Office of Extramural Research instituted an
agencywide training program for all new program officers hired by the
institutes. This program was designed to provide all new program
officers basic training on their role in managing any type of
NIH-sponsored research. The NIH official in charge of this training
stated that NIH is considering expanding this program to include
training specifically related to monitoring and managing clinical
trials.
\8 In our report, Scientific Research: Continued Vigilance Critical
to Protecting Human Subjects (GAO/HEHS-96-72, Mar. 8, 1996), we
reviewed federal oversight procedures for protecting human subjects
in scientific research.
\9 Michaele C. Christian and others, "The National Cancer Institute
Audit of the National Surgical Adjuvant Breast and Bowel Project
Protocol B-06," The New England Journal of Medicine, Vol. 333, No.
22 (1995), pp. 1469-1474.
CONTROLS THAT PROTECT TRIAL
DATA INTEGRITY ARE NOT
ALWAYS CONSISTENTLY APPLIED
---------------------------------------------------------- Letter :3.4
Clinical trials have controls that safeguard against scientific
misconduct, including direct data verification to ensure data
integrity. Because institutes and grantees, however, have
flexibility in deciding how these controls are used, the application
of the controls often varies by institute and type of trial.
One control designed to safeguard trials against scientific
misconduct is the use of clinical monitors to review trial data.
These monitors visit clinical trial sites to verify that the
established protocols are being followed and that the data being
reported match the data in the clinical records. In one trial
sponsored by the National Eye Institute, clinical monitors found that
clinical test results were being entered on data collection sheets
and not in the patients' medical records. Clinical research policy
states that medical records are the acceptable source documents for
clinical test results so monitors required that the site also enter
reported data in the patients' medical records.
Because clinical monitors add both expense and time to a trial,
institutes tend to use them only in the large and more complex
trials. For example, clinical monitors are being used in NIA's
largest and most expensive ongoing Phase III clinical
trial--alternative therapies for Alzheimer's disease. This trial is
being conducted at 35 research sites and costs $16.9 million in NIH
funds. The NIA program officer for the Alzheimer's trial estimated
that using clinical monitors in this trial delayed data entry by 6
months. This delay is acceptable, however, because of the increased
quality assurance that clinical monitors bring to the trial,
according to the program officer.
Another internal control procedure to protect data integrity is the
use of data coordinating centers. Most Phase III clinical trials
that have multiple research sites use data coordinating centers to
process patient clinical data generated during a trial. These
centers inspect the data for inconsistencies among the sites,
irregularities, and fraud.
In one NIA-sponsored trial, Continence Program for Women, the data
coordinating center detected data inconsistencies between two
clinical sites and alerted both the institute and the Data and Safety
Monitoring Board. The inconsistency was caused by a different
classification of patients by the two sites. However, the center's
detection of the data problem allowed the problem to be corrected.
In an NHLBI-sponsored trial, the data coordinating center questioned
test results from one laboratory. Further investigation by NIH's
Office of Research Integrity revealed that a lab technician had not
conducted the tests as required and had reported false test results.
NHLBI took corrective action to minimize the damage to the trial.
The institute also recovered funds paid to the laboratory, and the
technician was sanctioned.
BENEFITS OF DATA
COORDINATING CENTERS'
INDEPENDENCE
-------------------------------------------------------- Letter :3.4.1
NHLBI officials believe that the independence of data coordinating
centers is an essential part of internal controls. It is a way for
the institute to create a direct link to a key data verification
point and to help ensure prompt notification of potential scientific
misconduct or other data irregularities. For this reason, NHLBI
directly funds data coordinating centers and requires that the heads
of the centers report directly to the institute's program officer and
the Data and Safety Monitoring Board. This approach places data
coordinating centers beyond the direct control of a trial's principal
investigator.
Other institutes that have not provided for data coordinating
centers' independence in trials have experienced problems with
researchers' influence over the centers. For example, for the three
NIA trials included in our review, data coordinating centers were
funded through a subcontract with research centers. This arrangement
allowed a lead researcher, in a dispute with the center, to withhold
the center's operating funds. The institute's program officer had to
intervene to resolve the situation. During our review, NIA's policy
was to independently fund data coordinating centers for most of its
multisite clinical trials.
In the breast cancer trial, NCI permitted the trial's principal
investigator to oversee the operations of the data coordinating
center. When the center detected suspect data, the principal
investigator was notified. The investigator took about 3 months to
establish that fraud had occurred and another 5 months before
notifying NCI. The investigator's failure to promptly notify NCI as
required delayed corrective action and jeopardized the integrity of
the trial. NCI had to spend time and resources to revalidate the
trial's initial results.
CENTRALIZED NIH OVERSIGHT OF
AND GUIDANCE ON MANAGING
CLINICAL TRIALS IS LIMITED
------------------------------------------------------------ Letter :4
NIH conducts limited centralized monitoring of Phase III clinical
trials. No agencywide registry or database exists to track progress
and performance of all clinical trials and provide NIH's management
with comprehensive reports for oversight and decisionmaking
purposes.\10 Although periodic meetings occur to discuss progress of
ongoing trials, no data are systematically collected nor used to
provide centralized oversight. Furthermore, NIH has not adopted its
internal committee's recommendations to develop agencywide guidance
on quality assurance measures and data monitoring procedures for
institutes to use in managing clinical trials. According to NIH,
some of its institutes have selectively adopted some of the
committee's recommendations, but the agency believes adopting these
policies agencywide is inappropriate because this erroneously assumes
that all clinical trials should be monitored in the same manner.
Nonetheless, NIH and HHS have implemented some agencywide measures in
the past designed to discourage misconduct in federally funded
research.
--------------------
\10 The NIH Revitalization Act of 1993 mandates that women and
minorities be adequately represented in all appropriate NIH-sponsored
research, including Phase III clinical trials. To ensure compliance
with this requirement, NIH's Office of Research on Women's Health has
developed a database that monitors the inclusion of women and
minorities in NIH-funded research. The act also required NIH to
develop a registry of clinical trials involving women's health
issues.
NIH CONDUCTS LIMITED CENTRAL
OVERSIGHT AND MONITORING OF
PHASE III CLINICAL TRIALS
---------------------------------------------------------- Letter :4.1
Even though NIH's Office of Extramural Research is responsible for
centralized activities concerning oversight of extramural research,
such as developing policy on the review, funding, and management of
clinical trials, it has limited knowledge of and data on the Phase
III clinical trials NIH funds and the performance of individual
institutes and grantees. The office does review institutes' initial
requests for Phase III clinical trial research. Once a request is
reviewed and ultimately approved, however, the awarding of the grants
and most of a trial's oversight and management are left to individual
institutes.
The Office of Extramural Research might learn of a trial's progress
from meetings of the Extramural Program Management Committee, whose
members are staff from each institute. The committee meets regularly
to discuss, among other issues, those related to Phase III clinical
trials and to exchange ideas. However, unless an institute's
representative mentions a problem with a trial or raises concern
about fiscal or scientific misconduct, the committee or the Office of
Extramural Research would not likely know about it.
NIH has not yet implemented a centralized database or a central trial
registry to improve its oversight of the clinical trials it funds.
An automated database of all clinical trials could track progress and
performance and generate reports that would increase management's
knowledge about the trials and improve its ability to oversee them.
Because no active central trial registry exists, NIH would have to
survey each institute just to determine the total number of Phase III
clinical trials it funds. The NIH Revitalization Act of 1993
required NIH to develop a registry of clinical trials involving
women's health issues. NIH, however, decided not to limit this
registry to trials involving women's health but to include other
trials. NIH's Office of Extramural Research is developing the
Streamlined Non-Competing Award Process (SNAP) database as a pilot
experiment. According to NIH, this database will allow it to
interact with the grantee institutions and monitor research progress.
NIH expects that when SNAP is expanded to include all clinical
trials, NIH staff will be able to monitor trial progress in areas
such as recruitment.
NIH HAS NOT ADOPTED A
RECOMMENDATION TO DEVELOP
AGENCYWIDE GUIDANCE ON
MANAGING CLINICAL TRIALS
---------------------------------------------------------- Letter :4.2
Because the NIH institutes and grantees have more flexibility in
deciding how to manage clinical trials funded through cooperative
agreements, the scientific controls used in such trials vary. Aware
of this variability, NIH's Office of Extramural Research established
the NIH Working Committee on Clinical Trial Monitoring in June 1994
to determine how its institutes manage clinical trials. The
committee members were representatives from the institutes and
research centers and were selected for their expertise in various
aspects of clinical research.
The committee's task was to specifically review how the institutes
manage the Phase III clinical trials they sponsor. On the basis of
its review, the committee decided in 1995 that attempting to develop
standards to dictate how these trials are managed is inadvisable
given the unique characteristics of each Phase III clinical trial and
the diverse nature of the institutes. The committee did recommend,
however, that NIH consider formulating guiding principles for all the
institutes to follow in managing the trials.
The principles the committee recommended covered such areas as
quality assurance and site monitoring, patient confidentiality, level
of NIH staff involvement, and data access. Specifically, for
example, in the area of quality assurance and site monitoring, the
committee recommended that the institutes, at a minimum, conduct
regular on-site monitoring of all clinical centers and monitor key
outcome data. It also recommended that trials involving multiple
clinical centers, large study populations, or potentially harmful
interventions have the substantial involvement of and close oversight
by the sponsoring institute. NIH has decided not to adopt any of the
committee's recommendations agencywide. According to NIH, adopting
agencywide policies such as those the committee recommended is
inappropriate because it assumes that all clinical trials should be
monitored in the same way.
At one of the data coordinating centers we visited, officials
expressed frustration because standards and procedures for data
collection differ by institute as well as among program officers at
the same institute. They believe that minimal data collection
standards and procedures should be established for all trials.
HHS AND NIH HAVE IMPLEMENTED
SOME AGENCYWIDE POLICIES ON
SCIENTIFIC MISCONDUCT
---------------------------------------------------------- Letter :4.3
Prompted by legislation and on their own initiative, HHS and NIH have
taken steps to discourage misconduct in federally sponsored research,
including clinical trials. These efforts have focused mainly on
establishing proper scientific conduct and conflict-of-interest
reporting requirements for grantee institutions.
In response to the Health Research Extension Act of 1985, HHS
required each grantee institution to develop a formal process
delineating the steps to be taken to resolve allegations of
scientific misconduct. In addition, institutions are required to
diligently try to protect the positions and reputations of
whistleblowers. ORI monitors compliance with this requirement.
As required by the NIH Revitalization Act of 1993, HHS recently took
action to ensure that the design, conduct, or reporting of PHS-funded
research is not affected by researchers' outside financial interests.
This applies also to all NIH-sponsored research. Specifically, HHS
issued a regulation effective October 1, 1995, requiring that each
grantee institution develop a conflict-of-interest policy applicable
to all staff benefitting from PHS funding. To comply with this
regulation, researchers must file annual financial disclosure forms
that allow the institution to determine if a conflict of interest
exists. All applications for PHS funding must contain a
certification by the institution that no conflict of interest exists.
Each of the five grantee institutions we visited had developed and
implemented conflict-of-interest policies. Because of the newness of
the policies, however, officials said it would take time to see how
these policies operated in practice and how effective the policies
would be.
CONCLUSIONS
------------------------------------------------------------ Letter :5
A large percentage of NIH-sponsored Phase III clinical trials are
funded through cooperative agreements so both the institutes and
grantees are involved in managing the trials and developing
procedures for conducting them, according to NIH. The trials have
controls designed to safeguard against fiscal and scientific
misconduct that the institutes, grantee institutions, and research
sites can apply in overseeing the trials. However, no practical
level of oversight and controls can completely eliminate the
potential for misconduct.
Most oversight of these trials is decentralized and performed
independently by each of the different institutes that sponsor
clinical research and by the grantee institutions. Because of the
large number of diverse Phase III clinical trials NIH funds and the
independent nature of its institutes, NIH charged a working committee
with determining how such trials are managed. The committee
recommended that NIH develop some agencywide guidance for all
institutes to follow in managing these trials. The guidance was
recommended for areas such as quality assurance, site monitoring, and
the level of NIH staff involvement. Although some institutes have
implemented some of the principles, NIH believes adopting them
agencywide is inappropriate.
In the past, NIH has done little centralized oversight and monitoring
of the trials it funds and the institutes that sponsor them, except
for tracking women's and minorities' participation in clinical
trials. NIH is, however, developing a database that it expects will
allow for monitoring elements of clinical trials' progress and
performance.
AGENCY COMMENTS
------------------------------------------------------------ Letter :6
In commenting on a draft of this report, NIH agreed in general with
our conclusions and noted that the report provides a balanced
discussion of the relevant issues. (See app. II.) NIH also provided
technical comments, which we incorporated as appropriate.
---------------------------------------------------------- Letter :6.1
We are sending copies of this report to appropriate congressional
committees, the Secretary of HHS, the Director of NIH, and other
interested parties. We also will make copies available to others on
request.
If you or your staff have any questions about this report, please
call me at (202) 512-7119. Other major contributors to this report
are listed in appendix III.
Sarah F. Jaggar
Director, Health Financing
and Public Health Issues
SCOPE AND METHODOLOGY
=========================================================== Appendix I
To determine how NIH provides oversight to protect Phase III clinical
trials from fiscal and scientific misconduct, we conducted audit work
at NIH; National Institute on Aging (NIA); and National Heart, Lung,
and Blood Institute (NHLBI). We selected NIA because it is among the
institutes that provide the smallest amount of funding for Phase III
clinical trials and NHLBI because it is among the institutes that
provide the largest amount of funding. In fiscal year 1995, NIA
sponsored 7 clinical trials costing about $9 million, and NHLBI
sponsored 42 trials costing about $73 million. Selecting these
institutes for review provided some perspective on whether oversight
might be influenced by the size of an institute's clinical trial
portfolio. Also, these two institutes offered a variety of trials
from which to select for review.
We limited the scope of our review to Phase III clinical trials
funded through cooperative agreements. Under cooperative agreements,
grantee institutions have more flexibility in planning, conducting,
and managing the trials than under contracts, the other major funding
method for Phase III clinical trials. NIH institutes that sponsor
the trials are expected to provide assistance to and oversight of the
trials. Our review included a nonstatistical sample of four
multisite clinical trials that varied in nature, size, complexity,
and number of sites (see table I.1). We visited five of the clinical
research sites that participated in the trials and two data
coordinating centers that processed and monitored the clinical data.
The clinical sites we visited were either state or private
institutions located in Virginia, Connecticut, and Massachusetts.
The data coordinating centers we visited differed in how they were
funded.
Table I.1
Four Phase III Clinical Trials Reviewed
Sponsori
ng FY 95
Clinical institut Number funding
trial e Objective of sites (000s)
------------ -------- -------------- -------- --------
BARI\a NHLBI To compare 17 $3,104
coronary
artery bypass
surgery with
angioplasty
STOP/IT\b NIA To compare the 4 1,919
effects of
exercise,
calcium,
vitamin D, and
hormones on
bone density
Continence NIA To compare the 3 0\c
Program for effects of
Women exercise,
estrogen, and
surgery in
treating
incontinence
Alzheimer's NIA To examine the 35 4,116
Disease effects of
Cooperative various drug
Study therapies on
Alzheimer's
disease
----------------------------------------------------------
\a Bypass Angioplasty Revascularization Investigation.
\b Sites Testing Osteoporosis Prevention/Intervention Treatment.
\c No new funds were obligated in fiscal year 1995. Activities were
funded through the use of funds carried over from previous years.
To determine the oversight roles played by NIH, the institutes, and
the institutions receiving research funds, we conducted interviews,
reviewed NIH rules and regulations, examined NIH studies and reports,
and reviewed grant documents on the chosen Phase III clinical trials.
We interviewed agency officials from NIH, NIA, and NHLBI. Within
NIH, we interviewed officials from the Office of Extramural Research
and the Office of Research on Women's Health. At NIA and NHLBI, we
interviewed senior officials, grants management personnel, and
program management officers. We also met with staff from HHS' Office
of Research Integrity to discuss their role in investigating
allegations of misconduct and the Office of the Inspector General,
which was investigating allegations of scientific misconduct.
We also met with the principal research investigators, key research
personnel, grants and fiscal management officials, and internal audit
staff at the research sites to get their views on oversight
responsibilities and controls that protect trials against misconduct.
We reviewed grantee institutions' policies and procedures for
preventing, detecting, and resolving scientific misconduct, conflicts
of interest, and fiscal mismanagement. Also, we examined research
documentation, clinical records, correspondence, and external audits
of the institutions.
To determine what controls exist at the central data processing point
to help ensure clinical data integrity, we visited two data
coordinating centers. One of the centers was funded independently of
the clinical sites; the other's funding was included in the research
center's grant award. At the coordinating centers, we observed their
operation, reviewed their policies and procedures, and interviewed
key personnel about the centers' data collection and analysis role
and responsibilities. We examined reports generated by the centers
and observed the procedures they use to ensure consistency of each
clinical site's data collection and recording methodology. We also
established how research data are analyzed to detect data problems
and reviewed the follow-up procedures the centers use when potential
problem data are discovered.
Our work was performed between September 1995 and May 1996 in
accordance with generally accepted government auditing standards.
(See figure in printed edition.)Appendix II
COMMENTS FROM THE DEPARTMENT OF
HEALTH AND HUMAN SERVICES
=========================================================== Appendix I
MAJOR CONTRIBUTORS TO THIS REPORT
========================================================= Appendix III
James O. McClyde, Assistant Director, (202) 512-7152
Frank F. Putallaz, Evaluator-in-Charge, (617) 565-7527
Thomas S. Taydus, Senior Evaluator
*** End of document. ***