National Practitioner Data Bank: Major Improvements Are Needed to Enhance
Data Bank's Reliability (Letter Report, 11/17/2000, GAO/GAO-01-130).
While the National Practitioner Data Bank (NPDB) is presently the
nation's only central source of medical malpractice payment information,
it is not clear that all such data are being properly reported. While
GAO sampled 1 month's submissions, its review suggests that NPDB
information may not be as accurate, complete, or as timely as it should
be. Inaccuracies in the way reported information was coded could confuse
or mislead querying organizations about the severity of actions taken
against practitioners. Additionally, duplicate reports overstate the
amount of information the NPDB has on a particular practitioner. The
Health Resources and Services Administration (HRSA) has not established
criteria for the descriptive information that must be reported by states
and other entities when notifying the data bank of the disciplinary
actions taken. Moreover, HRSA does not have procedures for ensuring that
reporters adhere to the criteria it has established for medical
malpractice reports, including inappropriate references to patients'
names. Furthermore, the practitioner notification and dispute resolution
processes have not ensured that inaccurate and erroneously reported
information is removed from the data bank and not released to entities
seeking information on specific practitioners. Finally, without an
examination of its financial operations, HRSA has little assurance that
its NPDB user fees are appropriate. An analysis of its cash balances and
cash flows--user fee collections and disbursements--would be the best
way for HRSA to determine the appropriateness of fees.
--------------------------- Indexing Terms -----------------------------
REPORTNUM: GAO-01-130
TITLE: National Practitioner Data Bank: Major Improvements Are
Needed to Enhance Data Bank's Reliability
DATE: 11/17/2000
SUBJECT: Medical information systems
Data integrity
Internal controls
Malpractice (medical)
Statistical data
Health care personnel
Reporting requirements
IDENTIFIER: HHS National Practitioner Data Bank
HHS Healthcare Integrity and Protection Data Bank
******************************************************************
** This file contains an ASCII representation of the text of a **
** GAO Testimony. **
** **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced. Tables are included, but **
** may not resemble those in the printed version. **
** **
** Please see the PDF (Portable Document Format) file, when **
** available, for a complete electronic file of the printed **
** document's contents. **
** **
******************************************************************
GAO-01-130
A Report to the Chairman, Subcommittee on National Economic Growth, Natural
Resources and Regulatory Affairs, Committee on Government Reform, House of
Representatives
November 2000 NATIONAL PRACTITIONER DATA BANK
Major Improvements Are Needed to Enhance Data Bank's Reliability
Letter 3 Appendixes Appendix I: Scope and Methodology 36
Appendix II: Comments From the Department of Health and Human Services 40
Appendix III: GAO Contacts and Staff Acknowledgments 46 Tables Table 1:
Malpractice Reports Submitted to NPDB, by Items of
Information Missing 21 Table 2: Disciplinary Action Reports Submitted to
NPDB 23 Table 3: 5- Year Trend in Accumulated NPDB Fees (in Millions of
Dollars) 27 Figures Figure 1: Percentage of Report Types Submitted to NPDB,
September 1999 18 Figure 2: Lateness of State Licensure and Medical
Malpractice
Reports, by Percentage of Reports 19
Abbreviations
DEA Drug Enforcement Administration DFO Division of Financial Operations EFT
electronic funds transfer HIPDB Healthcare Integrity Protection Data Bank
HHS Department of Health and Human Services HHS/ OIG HHS/ Office of
Inspector General HRSA Health Resources and Services Administration NAIC
National Association of Insurance Commissioners NPDB National Practitioner
Data Bank
Lett er
November 17, 2000 The Honorable David M. McIntosh Chairman, Subcommittee on
National Economic Growth,
Natural Resources and Regulatory Affairs Committee on Government Reform
House of Representatives
Dear Mr. Chairman: To address concerns that states were hampered in their
ability to protect the public from incompetent health care practitioners who
cross state lines to continue the practice of medicine, the Health Care
Quality Improvement Act of 1986 authorized the Secretary of Health and Human
Services (HHS) to create the National Practitioner Data Bank (NPDB). 1
Administered by HHS' Health Resources and Services Administration (HRSA),
NPDB is the nation's only central source of information on physicians,
dentists, and other health care practitioners who either have been
disciplined by a state licensing board, professional society, or health care
provider or have been named in a medical malpractice settlement or judgment.
Hospitals and other health care providers periodically access NPDB, for a
fee, to obtain information on practitioners who are currently on staff,
under contract, or who have applied for clinical privileges. Because NPDB
information can affect a practitioner's reputation and livelihood, the
integrity of the data bank's information has been of great concern.
Since its beginning in 1990, questions have arisen about NPDB's operational
efficiency and effectiveness. We studied NPDB's early development and
recommended operational and security- related improvements. 2 HRSA officials
responsible for ensuring that the data bank has comprehensive information
have questioned whether medical malpractice insurers and health care
providers report all practitioners, as required. Officials from HHS' Office
of Inspector General (HHS/ OIG), who have studied and reported on the data
bank, determined that a relatively
1 P. L. 99- 660, title IV. 2 Information System: National Health
Practitioner Data Bank Has Not Been Well Managed (GAO/ IMTEC- 90- 68, Aug.
21, 1990), Practitioner Data Bank: Information on Small Medical Malpractice
Payments( GAO/ IMTEC- 92- 56, July 7, 1992), and Health Information Systems:
National Practitioner Data Bank Continues to Experience Problems( GAO/
IMTEC- 93- 1, Jan.
29, 1993).
small number of disciplinary actions were reported by hospitals and other
health care providers and recommended that HRSA do more to address potential
underreporting. In addition, various organizations representing the health
care industry have periodically questioned the accuracy of information
submitted to NPDB. The industry has also questioned the appropriateness of
fees charged to access data and HRSA's use of these fees. Accordingly, you
asked that we (1) assess HRSA's efforts to address potential underreporting
to the data bank, (2) evaluate the accuracy, completeness, and timeliness of
NPDB data, and (3) assess the adequacy of internal controls over user fees
and expenditures to determine whether these fees are set at the appropriate
level.
To address issues related to underreporting, we reviewed HRSA's operational
and research plans for NPDB, related studies and documentation, and
interviewed officials from HRSA, HHS/ OIG, and selected health care industry
representatives. To assess the accuracy, completeness, and timeliness of
reported data, we worked with HRSA officials and chose September 1999 as a
typical reporting period. We analyzed the reports submitted to NPDB during
that month. Additionally, we obtained and analyzed information from NPDB on
34 practitioners who were reported to NPDB during September 1999. Finally,
to assess the adequacy of internal controls over user fees and expenditures,
we interviewed HRSA officials to understand how NPDB's user fees are
determined, collected, and disbursed. We also reviewed applicable laws,
regulations, and other guidance concerning user fees, and tested a sample of
the data bank's disbursements made between October 1994 and May 2000. We
conducted our audit work between January 2000 and September 2000 in
accordance with generally accepted government auditing standards. (See app.
I for more detailed information on our scope and methodology.)
Results in Brief Although HRSA has long been concerned that underreporting
weakens NPDB's reliability, steps for addressing such issues are not part of
the
agency's strategic plan. As a result, HRSA's efforts to quantify or minimize
underreporting have been unsuccessful. For example, the agency has focused
on the underreporting of malpractice payments even though HHS/ OIG and HRSA-
sponsored studies conclude that underreporting of clinical privilege
restrictions by hospitals and other health care providers is a more pressing
issue. Industry experts also agree, pointing out that disciplinary actions
taken by health care providers and states are better indicators of
professional competence than medical malpractice. However,
HRSA has made little progress in addressing suspected underreporting by
health care providers. HRSA officials said that additional resources and
skills are needed to monitor and sanction nonreporters effectively. Also,
HRSA has not implemented a 13- year- old law that expanded NPDB to include
information on nurses and other health care practitioners. As a result,
disciplinary actions taken against nurses and other practitioners are not
reported to NPDB, despite these individuals' increasing importance in the
delivery of health care.
Problems that we identified in the data submitted to NPDB during September
1999 raise concerns about the effectiveness of HRSA's management of the data
bank and of the two mechanisms�practitioner notification and dispute
resolution- that are intended to ensure the quality of reported information.
We identified problems particular to each of the three types of reports we
reviewed. The data in medical malpractice payment reports- representing
about 80 percent of the information in NPDB- generally did not meet HRSA's
criteria for completeness. For example, over 95 percent of the medical
malpractice reports we reviewed did not note whether the standard of patient
care had been considered when the claim was settled or adjudicated. Further,
our analysis of 252 reports of state licensure actions revealed that about
30 percent were submitted late and 11 percent contained inaccurate or
misleading information on the severity or number of times practitioners had
been disciplined. We also found inaccurate information in about one- third
of the 79 clinical privilege restriction reports we reviewed.
Finally, our review disclosed that HRSA has not adequately examined whether
the level of user fees used to finance NPDB operations is appropriate. HRSA
does not have a plan that projects cash flows such as revenue,
disbursements, and capital investments. Such a plan is needed to determine
if the level of fees is appropriate and if HRSA's long- standing policy of
maintaining a cash balance of 4 to 6 months of operating expenses is still
reasonable. HRSA has not reassessed the amount needed to cover operating
expenses since 1994. As of the end of fiscal year 1999, it had a $6. 8
million cash balance. We also found that controls over NPDB transactions did
not ensure that all collections were received and that disbursements were
for authorized purposes. For example, HRSA and HHS' Division of Financial
Operations (DFO), which performs the accounting functions, do not have
adequate controls to ensure that all assessed user fees are collected and
properly recorded in HRSA's general ledger. HRSA and DFO also could not
ensure that user fees collected electronically- presently about 30 percent
of NPDB's receipts- were properly allocated
between NPDB and another data bank the agency also manages. Additionally, we
found that controls over disbursements were not effective, as supporting
documentation was sometimes missing or inadequate.
We are making several recommendations to the Secretary of HHS, the
Administrator of HRSA, and the Director of HHS/ DFO to improve both the
operation and the financial management of the data bank. In its written
comments on a draft of this report, HHS concurred with our recommendations
to improve compliance monitoring and enforcement, allocate user fees
appropriately, and develop criteria for the narrative section of
disciplinary action reports. HHS also described actions it is taking or
plans to take. HHS did not concur with our specific recommendations to
improve the reliability of reported information and to strengthen its
internal controls over NPDB user fee collections and disbursements. However,
we believe that actions on these recommendations are necessary to enhance
the accuracy, completeness, and timeliness of NPDB's information and to
improve internal controls and financial operations.
Background In 1986, the Congress found that there was a need nationally to
restrict the ability of incompetent practitioners to move between states
without
disclosure or discovery of their professional histories. Moreover, it was
determined that states and individual organizations, acting independently,
might not be able to do so. While there were several private and nonprofit
organizations that collected data on state disciplinary actions, these
groups did not have access to information either on the disciplinary actions
taken by health care providers or on medical malpractice cases. As a result,
the Congress created NPDB as the nation's central source of such information
on health care practitioners.
HRSA has federal oversight responsibility for NPDB. As such, it has
developed rules and regulations for reporting information and accessing
NPDB. The instructions for reporting practitioner information to NPDB and
for accessing the data bank, which is known as querying, are spelled out in
the NPDB Guidebook, updated January 1999. HRSA is also responsible for
ensuring health care industry compliance with reporting and querying
requirements. A private contractor operates the data bank for HRSA. 3
In 1988, HRSA commissioned a group of health care industry representatives
and advocates to provide continual advice to its contractor on NPDB
operational issues. This group, the NPDB Executive Committee, includes
various health care industry representatives from organizations such as
accrediting bodies and licensing boards, hospitals and other providers,
malpractice insurers, professional societies, and others. With the advice of
the NPDB Executive Committee, HRSA and its contractors developed and
customized the software applications used to collect reports on
practitioners and respond to user queries.
The Health Care Quality Improvement Act of 1986 also established criteria
for reporting practitioners to NPDB. The requirements for reporters-
malpractice insurers, health care providers, state licensing boards, and
federal agencies- essentially parallel their areas of responsibility.
Entities such as insurance companies must report practitioners on whose
behalf medical malpractice payments are made. State licensing boards must
report practitioners whom they have disciplined. 4 Health care providers
such as hospitals and health plans must report disciplinary actions
restricting practitioners' clinical privileges for more than 30 days. In
addition, professional societies such as the American Medical Association
and the American Dental Association must report actions that adversely
affect a practitioner's membership in the society. Finally, the law directed
HRSA to negotiate Memorandums of Understanding with selected federal
3 Several different private contractors have operated and maintained NPDB
since it began operations Sept. 1, 1990. The current contractor has been
operating NPDB since June 1995. 4 According to the NPDB Guidebook, state
licensing boards are required to report disciplinary actions such as
revocations, suspensions, reprimands, and fines associated with license
restrictions.
agencies, outlining the terms for reporting practitioners that they employ,
insure, or regulate. 5
Time frames for reporting the required information are set in the law,
regulation, or NPDB Guidebook. Medical malpractice payments must be reported
to NPDB within 30 days of the date of the initial payment. Health care
providers that report electronically have up to 15 days to report
simultaneously to NPDB and the applicable state licensing board. Providers
submitting paper reports have up to 15 days to send reports to the
applicable state licensing board. State boards have 15 days to forward paper
reports to NPDB. State licensing actions against practitioners must be
reported within 30 days. Professional societies must report actions taken
against practitioners' memberships within 15 days. Some federal agencies, in
their Memorandums of Understanding with HRSA, also agreed to report
malpractice payments and disciplinary actions within 30 days of the payment
or action.
Since 1986, NPDB has been expanded to include additional information and
other categories of health care practitioners who must be reported. The
Medicare and Medicaid Patient and Program Protection Act of 1987, as
amended, requires that states have a system for reporting licensure actions
taken against nurses and other state- licensed health care practitioners
such as chiropractors, emergency medical technicians, and physical
therapists to NPDB. 6 Since 1997, under an agreement among the HHS/ OIG,
HRSA, and the Health Care Financing Administration, practitioners who are
excluded from participation in the Medicare and Medicaid federal health care
5 The law specifically directed HRSA to negotiate Memorandums of
Understanding with the Department of Defense, the Department of Veterans
Affairs, and the Drug Enforcement Administration (DEA). HRSA also has
agreements with the Department of Transportation (U. S. Coast Guard), the
Bureau of Prisons, and with the U. S. Public Health Service for reporting
its physicians and dentists, including those working in community health
centers or the Indian Health Service.
6 P. L. 100- 93.
programs due to fraudulent or abusive activities or who default on federal
loan agreements are also reported to NPDB. 7
The law also has provisions regarding access to and use of information
contained in the data bank. Hospitals are required to query NPDB whenever a
practitioner applies for clinical privileges and every 2 years for
practitioners already on staff. State licensing boards, professional
societies, and certain other types of health care providers are permitted to
query but are not required to do so. Individual practitioners can query NPDB
but only to obtain information on themselves. 8 Under current law,
malpractice insurers, advocacy groups, and the public cannot query NPDB;
however, selected information that does not identify individual
practitioners is available for purchase in a public use data file. 9
NPDB's operations are to be completely funded by the fees charged to users.
Fees are imposed for each practitioner's name queried and must be sufficient
to cover the cost of collecting reports and releasing query information. 10
HRSA is responsible for setting these fees. 11 In fiscal year 1999, HRSA
collected about $14 million in user fees, disbursed about $12 million for
NPDB expenses, and had a cash balance of $6. 8 million.
7 The HHS/ OIG's exclusion list provides information on individuals and
organizations that are excluded from participation in Medicare, Medicaid,
and other federal health care programs because of criminal convictions
related to Medicare or state health programs, patient abuse or neglect,
felony convictions related to controlled substances, health care fraud, and
other criteria such as defaulting on federal loans and license revocations.
As of January 1999, there were more than 15, 000 individuals and entities
excluded from program participation.
8 Practitioners who query the data bank for information about themselves are
charged $10. They complete an Internet- based form that can be accessed from
NPDB's home page. The completed form must be notarized and mailed to the
NPDB contractor for processing.
9 Plaintiffs' attorneys or plaintiffs acting on their own behalf may query
NPDB only if they can independently prove that a hospital did not perform
the query, as required by law, and a medical malpractice suit against that
hospital, naming the specific practitioner among the defendants, has been
filed in court. However, they may not query for information when suing
practitioners.
10 Users who submit queries via the Internet are charged $4 per practitioner
name, while those submitting queries on diskette are charged $7 per
practitioner. 11 The Secretary of Health and Human Services approves user
fees for NPDB queries and publishes these fees periodically in the Federal
Register.
Civil penalties can be assessed for nonreporting and for unauthorized use of
NPDB information. Entities failing to report medical malpractice payments
can be assessed up to $11,000 for each unreported payment. HRSA can also
impose penalties of up to $11, 000 for each instance of unauthorized access
or improper distribution of NPDB information. There are no financial
penalties for states, health care providers, or federal agencies that do not
report practitioners to NPDB. HRSA officials said that several organizations
have been fined for unauthorized access but none for not reporting to the
data bank. HRSA cannot penalize organizations that do not report the
required information on time.
Efforts to Address Although HRSA has long suspected that some organizations
do not report
Underreporting Have practitioners as required, the agency has not included
steps for addressing
underreporting in its strategic plan, nor has it taken a systematic approach
Been Unsuccessful
to the problem. Most of HRSA's efforts to address underreporting have
focused on medical malpractice insurers, while HHS/ OIG and HRSAsponsored
studies have concluded that underreporting of clinical privilege
restrictions by hospitals and other health care providers is a larger and
more pressing issue. Moreover, experts widely agree that disciplinary
actions taken by state licensing boards and health care providers are better
indicators of professional competence than malpractice settlements. Yet,
very little has been done to address suspected underreporting among health
care providers. Further, disciplinary actions taken against nurses and other
health care practitioners are not being reported to NPDB because HRSA has
not yet implemented the law. According to HRSA's management, additional
staff and resources would be needed for the agency to identify and take
effective action against organizations suspected of underreporting to the
data bank.
Medical Malpractice Although HRSA has been concerned that malpractice
payments are
Underreporting is a LongStanding underreported, it has not been able to
determine the magnitude of the
Problem problem despite many years of effort. Medical malpractice payments
can
be underreported in two ways, neither of which has been successfully
quantified. First, agency officials believe that some insurers may be using
a technicality in NPDB's reporting requirements to avoid reporting some
practitioners. Second, agency officials believe that some insurers and
selfinsured organizations such as HMOs and other health plans should report
to NPDB but do not. However, HRSA has not yet identified or fined any
organizations for failing to report the required information. Agency
officials told us that they are reluctant to impose fines because they
believe that the
cost of levying and collecting civil penalties often exceeds the $11,000
maximum amount that can be assessed.
Soon after NPDB began operating in 1990, HRSA officials became aware that
under the data bank's regulations, some practitioners, who may have
committed malpractice, were not being reported because of what has become
known as the “corporate shield.” NPDB regulations require that
only the practitioners named in final malpractice settlements be reported to
the data bank. The corporate shield occurs when individuals filing
malpractice claims remove the practitioner's name from the claim, leaving
only the hospital or another corporate entity identified as the responsible
party. When this happens, no report is submitted to NPDB. HRSA officials
believe that practitioners who have committed malpractice use the corporate
shield to avoid being reported. However, they have not been able to quantify
the extent to which the corporate shield is used for such purposes. In
addition, the agency has not found a means of successfully addressing this
issue in a way that would also have the support of industry representatives
on NPDB's Executive Committee, who could facilitate compliance by persuading
member organizations to adopt this policy change.
In December 1998, HRSA proposed changing NPDB's malpractice payment
reporting regulations. The proposal would have required that insurers report
all practitioners for whose benefit a payment is made, including those
practitioners who might not have been named in the final settlement or even
in the initial malpractice claim. The health care industry- including those
organizations on NPDB's Executive Committee- overwhelmingly opposed the
proposal, arguing that it would interfere with settlement negotiations
between the insurer and the claimant. The industry also argued that
reporting all initially named practitioners would deny due process to those
not found liable by the court. HRSA subsequently withdrew the proposal and
initiated other strategies to solve this problem while working to gain NPDB
Executive Committee support for a change in medical malpractice reporting
requirements.
HRSA officials have begun to work more closely with the NPDB Executive
Committee to obtain its input and gain consensus before finalizing a new
proposal. Two proposals have recently emerged from this collaboration and
will be circulated within HRSA and the full Executive Committee for comment.
The first proposal would require insurers to report to NPDB the names of
corporations and individual practitioners named in malpractice settlements
or judgments. HRSA officials told us that by collecting
information on corporations, they will have more complete data on the total
number of claims settled or adjudicated, which will help them identify
specific instances when the corporate shield has been used. However, they
acknowledge that the proposal to report corporations does not fully solve
the problem.
The second proposal would permit peer review organizations to determine
which practitioners involved in malpractice settlements should be reported
to NPDB. The Department of Defense and the Department of Veterans Affairs-
two large federal health care providers- both have peer review processes for
reporting practitioners to NPDB. As outlined in their Memorandums of
Understanding with HRSA, only those identified by their agencies' peer
review processes as responsible for injuring a patient or violating
standards of patient care are reported. However, HRSA officials told us that
they are presently concerned about the limited quantity and timeliness of
reports that are submitted following the federal agencies' peer review
processes. Further, this proposed alternative might require congressional
action because NPDB's authorizing legislation does not provide for peer
review of malpractice settlements or specify that HRSA can use the fees it
collects for queries to fund this activity.
In addition to these efforts to alleviate the use of the corporate shield,
HRSA officials told us that, since early 2000, they have been trying to
identify insurers that have paid medical malpractice claims but have not
reported the involved practitioners to NPDB. Using malpractice claims data
that insurance companies voluntarily report to the National Association of
Insurance Commissioners (NAIC), the agency identified 41 insurers that
reported payments to NAIC but not to NPDB. HRSA contacted these companies
seeking explanations regarding the differences in the reported payments. As
of September 2000, 17 of the 41 companies have adequately explained the
discrepancies to HRSA. For instance, NAIC data, for some companies, reflect
total payments made by their corporations- combining payments made on behalf
of individual practitioners with payments made on behalf of organizations.
NPDB data only represent payments made on behalf of individual
practitioners. Of the remaining 24 companies, 18 recognized their omissions
and agreed to file the delinquent reports. The other six companies have not
responded to HRSA's inquiries and have been warned by the agency that they
will be reported to HHS/ OIG for possible enforcement action.
Although HRSA has had some success in identifying nonreporters using NAIC
data, agency officials acknowledged that these data have some
significant limitations. NAIC's medical malpractice data are not
comprehensive because companies report this information voluntarily.
Moreover, they do not include payments made by self- insured organizations,
such as health maintenance organizations and other health plans that do not
report to NAIC. Also, as previously noted, NAIC data combine the payments
made on behalf of practitioners with those made on behalf of institutions.
Because HRSA could not independently reconcile NAIC and NPDB data, agency
officials had to rely on insurers' explanations as to whether reports should
have been submitted or not.
Underreporting of Clinical HRSA and the HHS/ OIG have been concerned about
the relatively low
Privilege Restrictions Is number of reported clinical privilege restrictions
since NPDB's early years
Another Long- Standing of operation. While early estimates projected that as
many as 10,000 clinical
Concern privilege restrictions would be reported annually, fewer than 9, 000
reports
were submitted from 1990 through 1999. Concerned with the contrast between
the early estimates and the number of clinical privilege restrictions being
reported, HRSA management asked HHS/ OIG and others to study the issue. HHS/
OIG concluded that providers are more likely to report if there are
penalties for nonreporting and recommended that HRSA seek legislative
authority to fine nonreporting providers, comparable to its authority to
fine malpractice insurers. Although HRSA generally concurred with HHS/ OIG's
July 1999 recommendation, the agency did not act on it until late July 2000.
HRSA officials acknowledge that the agency has not been successful in
encouraging provider compliance with clinical privilege reporting
requirements. HRSA officials believe that to improve compliance
significantly, the agency needs more than the ability to fine providers.
They noted that the states report licensure actions, as required, but
providers' reporting of clinical privilege restrictions have always fallen
far short of the agency's projections. Before NPDB began operations, the
Public Health Service projected that about 5, 000 clinical privilege
restrictions would be reported annually. The American Medical Association
estimated there would be as many as 10, 000 reports per year. As of the end
of calendar year 1999- after 9 years of operation- NPDB had received fewer
than 8, 600 clinical privilege restriction reports.
HRSA officials told us that the original estimates may have been too high
and that, over time, changes in industry practices may have resulted in
different approaches to disciplining practitioners. Industry representatives
told us that hospitals now provide more monitoring and training to address
performance problems than at the time the Public Health Service and the
American Medical Association estimates were made. This new approach to
disciplining practitioners may reduce the number of restrictions that
hospitals impose for more than 30 days and thus reduce the number of
individuals who would be reported to NPDB. NPDB's authorizing legislation
does not require that the data bank collect information on practitioners
targeted for special monitoring or training.
In July 1999, an HHS/ OIG study recommended that HRSA seek authority to fine
nonreporting providers. HRSA officials told us that in late July 2000, they
asked HHS to pursue legislation allowing the agency to fine health care
providers up to $25, 000 when specific instances of noncompliance are
identified. However, HRSA does not currently have the authority to access
the confidential peer review records that hospitals and other health care
providers maintain on practitioner performance. HRSA officials told us that
the agency would need this additional authority and staff skilled in
investigating specific instances of noncompliance to monitor and sanction
nonreporters effectively. Recognizing that additional funding and skilled
staff might not be forthcoming, agency officials have begun to develop a
compliance monitoring plan that less specialized personnel could perform.
Agency officials said they are hopeful that the plan would be implemented in
fiscal year 2001.
13- Year- Old Law Awaits HRSA has not implemented a law passed in 1987 that
would have
Implementation significantly increased the information reported to NPDB. The
Medicare
and Medicaid Patient and Program Protection Act of 1987 directed the states
to have systems of reporting licensure actions taken against nurses and
other licensed health care practitioners. Today, nurses and other licensed
practitioners play an even more important role in the provision of health
care. The law was amended in 1990 to include state reporting of adverse
actions taken by peer review and accrediting organizations against nurses
and other practitioners. HRSA officials told us that they did not implement
this law when NPDB began operating in 1990 because the agency lacked the
funding to include information on these additional practitioners in the data
bank. According to HRSA officials, the HHS General Counsel initially advised
the agency that it could not impose user fees to cover the cost of
collecting and disseminating this additional information, but it has since
reversed that opinion.
By July 1998, HRSA had drafted reporting regulations and had modified the
data bank's software to accommodate additional categories of
practitioners. Nonetheless, implementation was postponed pending startup of
a new fraud and abuse data bank, the Healthcare Integrity Protection Data
Bank (HIPDB), which HRSA manages for HHS/ OIG. 12 HRSA officials told us
that they made this decision because, in their opinion, expanding NPDB at
the same time the agency initiated HIPDB might have confused the data banks'
users. For instance, some state actions, such as denied licensure renewals,
are reported to both data banks. Other actions, such as denied initial
licenses, are only reported to HIPDB.
Recognizing the potential burden and confusion that users might face, the
Congress directed that duplicative reporting requirements be avoided. As a
result, HRSA developed a single system for users to access both data banks.
This Internet- based system became operational in November 1999 and by
October 2000 was the only way authorized organizations could report to or
query the data banks. 13 With this system, users only need to report
information once. Information is automatically distributed to one or both
data banks, as appropriate. For example, state licensure actions taken
against physicians and dentists are routed to both data banks, while actions
taken against nurses and other licensed practitioners are presently routed
only to HIPDB.
Currently, hospitals are not authorized access to HIPDB. As a result, they
cannot obtain information on licensure actions that states take against
nurses and other licensed practitioners. For instance, while the state of
Illinois reports at least 15 such actions each month, hospitals cannot
obtain that information from HIPDB. HRSA officials told us that they have
suggested a technical modification to HIPDB's authorizing legislation that
would allow hospitals to access the data bank. While this would provide
hospitals access to information on licensing actions, we believe that this
is only a partial solution because the actions taken by peer review and
accrediting organizations are not reported to HIPDB.
12 As authorized by the Health Insurance Portability and Accountability Act
of 1996, HIPDB is a central source of information on final actions that
states and courts have taken against individuals and companies found guilty
of health care fraud or abuse. It contains data on health- care- related
criminal convictions and civil judgments, as well as the names of
individuals and companies excluded from participation in federal health
programs. NPDB, on the other hand, collects information on individuals whose
professional competence may be at issue.
13 Individual practitioners, who can only access information about
themselves, must submit their queries on paper.
HRSA officials told us that they support HHS/ OIG's suggestion that NPDB and
HIPDB be combined into a single data bank and are working with members of
HHS/ OIG General Counsel's office to develop a legislative proposal.
However, work on the legislative proposal has just begun and, if enacted,
may take several years before a combined data bank would be available to
users. In the interim, HHS/ OIG officials have informed HRSA that they are
concerned that the agency might again delay implementing the 1987 law. HRSA
management officials told us that aside from seeking statutory changes-
allowing hospital access to HIPDB and combining the two data banks- they do
not have any other immediate plans for including actions taken against
nurses and other practitioners in NPDB.
Weaknesses in NPDB The quality of some of the reports we reviewed suggests
weaknesses in the
Data Limit Their data bank's reliability. We found problems with each of
three types of
reports we analyzed- malpractice payments, state licensure actions, and
Usefulness
clinical privilege restrictions. Medical malpractice payment reports, which
comprise 80 percent of the data bank, generally did not meet HRSA's criteria
for completeness. We also found that reports from state licensing boards and
health care providers were, at times, untimely, inaccurate, or submitted in
duplicate, which made it appear that twice the number of disciplinary
actions against a practitioner had been taken. Moreover, when mistakes were
made, practitioners had difficulty getting the reported information
corrected.
HRSA officials acknowledged that there are problems with the accuracy and
completeness of the data and that they have been working with consultants to
revise the way information reported to the data bank is coded. Agency
officials said they are considering revisions to the coding scheme to
improve the accuracy and completeness of reports. They have also begun
working with the NPDB contractor to remove duplicate reports from the data
bank. They also acknowledged that some reports are submitted late, but they
have not sought the additional authority to fine late reporters. Agency
officials also realize that practitioners can face difficulties in
correcting reported information. However, they said that NPDB's practitioner
notification and dispute resolution processes adequately address individual
concerns while maintaining the data bank's integrity.
Test Results Revealed Lags To evaluate the reliability of NPDB information,
we obtained electronic
and Gaps in NPDB copies of the 1, 645 reports submitted to the data bank
during September
Submissions 1999. In general, we analyzed the timeliness of reports by
comparing the
dates they were submitted with NPDB's reporting time frames. We assessed the
completeness of reports by comparing information in them with NPDB's
criteria for the items of information that should be reported. Because NPDB
is the only central source for much of the information it contains, we
assessed accuracy by determining the internal consistency of the narrative
and coded information in individual reports. (See app. I for a more detailed
description of the types of reports submitted in September 1999.)
As figure 1 indicates, nearly 80 percent of the reports submitted to NPDB
during September 1999 were related to medical malpractice payments. This
percentage is somewhat comparable to the data bank's cumulative totals.
Since 1990, almost 173, 000 out of approximately 228,000 NPDB reports- or 76
percent- involved malpractice.
Figure 1: Percentage of Report Types Submitted to NPDB, September 1999
State Licensure Reports (N= 252)
Clinical Privilege Restrictions (N= 79)
Other Reports (N= 14)
1% 5%
15% 79%
Medical Malpractice (N= 1,300)
Source: GAO analysis of September 1999 NPDB reports.
Clinical privilege restrictions comprise 5 percent of the September 1999
reports and less than 4 percent of NPDB's cumulative totals. On average,
fewer than 1, 000 such reports are submitted annually. HRSA officials
estimate that about 60 percent of the nation's hospitals had never reported
a practitioner to NPDB. Officials arrived at this figure by comparing the
list of authorized reporters with those entities that have submitted at
least one report to NPDB since it began operating in 1990. While the
estimate may include entities that may no longer exist or that may have more
than one authorization number, it appears that many of the nation's
hospitals have never reported to NPDB.
Our analysis revealed weaknesses in the timeliness and currency of medical
malpractice payment reports. About 25 percent (331) of the 1,300 malpractice
reports received in September 1999 were not submitted to NPDB within 30 days
of the initial payment, as required. On average, these reports were about 85
days late. About 30 percent (76) of the state
licensure reports submitted during September 1999 were late by an average of
61 days. As noted in figure 2, our analysis of these late submissions showed
that one- third of state licensure reports and almost one- half of medical
malpractice reports were 31 or more days late. We did not measure the
timeliness of reports submitted by hospitals and other health care
providers. 14
Figure 2: Lateness of State Licensure and Medical Malpractice Reports, by
Percentage of Reports
31- 60 Over
1- 30 Days
61- 90 90 Days Late
Late Days Late
Days Late
State Licensure 67
9 15 9 Actions
Medical 51
17 11
21 Malpractice
0 10 20 30 40 50 60 70 80 90 100
Percentage
Source: GAO analysis of 331 medical malpractice and 76 state licensure
reports submitted to NPDB in September 1999.
HRSA does not track the timeliness of reports submitted and does not have
the authority to sanction late reporters. Agency officials told us that
penalizing late reporters may have a chilling effect on submissions.
The timely submission of information does not necessarily ensure that
information about practitioners is quickly available. The malpractice
payments reported in September 1999 involved incidents that occurred, on
average, 4- 1/ 2 years earlier. The median time was 4 years, which is not an
14 Health care providers have two options for submitting reports to NPDB,
with different reporting deadlines for each. Electronic submissions have a
15- day deadline, while paper submissions pass through the state licensing
board and are allowed up to 30 days to reach NPDB. From the information we
obtained from NPDB, we could not determine which reporting option was used.
As a result, we could not measure the timeliness of clinical privilege
restrictions.
unusual length of time to resolve malpractice claims. During the time it
takes to resolve claims and report malpractice payment information,
practitioners could move between states or change health care providers.
In addition to the lateness and dated nature of reported information, our
analysis also revealed some delays in getting reports into NPDB. For 512
reports, or more than 30 percent of the September reports, we noted delays
between the date the report was submitted to NPDB and the date that the
information was incorporated into the data bank. These delays ranged from 5
days to more than 1 year. The median processing delay was about 13 days.
HRSA officials were unaware of the lengthier delays. They explained the
shorter delays by noting that, at times, organizations do not submit reports
on the dates indicated. However, we could not determine how frequently
reports had the wrong submission date and could not adjust our analysis to
take this into consideration. Nonetheless, late and delayed reports can
weaken NPDB's reliability as a mechanism for alerting others of potential
problems with a practitioner's past performance.
HRSA officials told us that NPDB's new Internet- based reporting and
querying system would alleviate processing delays by instantaneously
incorporating submitted information into the data bank. As of October 1,
2000, the new Internet- based system became the primary means of reporting
information to NPDB. However, instantaneous processing, without other
improvements in the data bank's software controls, may exacerbate the
problems of incomplete and inaccurate reporting that we found.
Malpractice Payment We found that the usefulness of NPDB's medical
malpractice data was
Reports Were Incomplete further compromised by the data bank's acceptance of
incomplete report
and Included Inappropriate submissions. We selected 250 of the 1, 300
malpractice reports submitted in
Information September 1999 for a more detailed review and found that only 1
met
NPDB requirements for disclosing the circumstances associated with payments.
The NPDB Guidebookrecommends that narrative descriptions include at least
seven items of information describing the events leading up to the medical
malpractice claim. Such information can help users identify potential
weaknesses or problems in a practitioner's past performance. Some items are
descriptive, such as patient age, gender, and inpatient or outpatient
status. However, others, such as the initial event or diagnosis and standard
of patient care, relate more to the quality of practitioner performance. As
table 1 shows, more than 95 percent of the malpractice reports in our sample
did not mention whether the standard of patient care
had been considered when the claim was settled or adjudicated. Moreover, of
those reports whose narrative mentioned that the standard of patient care
had been considered, only one noted the actual determination. 15
Table 1: Malpractice Reports Submitted to NPDB, by Items of Information
Missing Number of reports
Percentage of GAO Items of information missing information sample
Patient age 134 53. 6 Patient gender 108 43. 2 Patient type 199 79. 6
Initial event (procedure/ diagnosis) 68 27. 2 Subsequent event 37 14.8
Damages (medical or legal) 61 24. 4 Standard of patient care determination
239 95. 6 Source: GAO analysis of 250 reports.
HRSA officials acknowledged that medical malpractice reports are often
incomplete and explained that reports are submitted electronically and are
not manually screened before acceptance into the data bank. They explained
that NPDB's software only checks for the presence of text in the narrative
section of malpractice reports. It does not verify that all seven items of
information are present. They also told us that NPDB contract staff do not
routinely review the narratives and thus would not request additional
narrative information, even if the narrative was incomplete or
uninformative. NPDB contract staff only examine reports when there is a need
to verify that a query has resulted in identifying the correct practitioner.
16 If staff note obvious errors or questionable information, the reporting
institution is contacted and, if necessary, asked to submit a corrected
report. Contract staff are not authorized to change any of the information
reported to NPDB.
15 Our analysis did not reveal any substantive difference in the
completeness of reports involving settlements compared with those involving
court judgments. 16 NPDB uses a matching algorithm that compares queries
with information in the data bank. Before NPDB determines that it has
matched a query with the correct practitioner, a certain level of
information must be identical. If NPDB identifies a potential but not
definite match, an NPDB contract staff member compares information to verify
the match.
In addition to the problems of untimely and incomplete submissions, we also
found that 71 of the 250 medical malpractice reports included patient and
practitioner names in the narrative sections of the reports, in violation of
NPDB reporting instructions. HRSA officials said that they were aware of the
problem but had not found a cost- effective method for removing names. At
one point, the NPDB contractor tested a “name- filtering”
program that could be added to NPDB's software to detect and remove names
inappropriately included in the narrative sections. However, the test was
not successful because the program could not distinguish between
individuals' names that should be removed and other names that could be
included, such as those of institutions or street names. HRSA officials said
they do not ask entities reporting information to NPDB to revise their
submissions when names are included in the narrative.
Licensure Reports Were Our analysis of 266 licensure action reports, which
includes 14 actions
Inaccurate, Inconsistent, reported by the U. S. Drug Enforcement
Administration (DEA) and
and Submitted in Duplicate professional societies, indicated additional
weaknesses in NPDB's
reliability. 17 As table 2 shows, 24 of the 252 reports submitted by state
licensing boards contained errors that could confuse or mislead querying
organizations about the severity of sanctions imposed. These errors were
related to the way sanctions were coded in the reports submitted by state
licensing boards. For example, several reports indicated that practitioners'
licenses had been restored or reinstated when, in fact, they had been placed
on probation. Other reports indicated that practitioners had been
reprimanded when, instead, restrictions had been placed on their licenses.
Other reports did not contain sufficient detail in the narrative section for
us to determine whether they had been coded accurately.
17 NPDB classifies reports submitted by DEA and professional societies as
licensure actions.
Table 2: Disciplinary Action Reports Submitted to NPDB Detail insufficient
to Source of reports submitted to NPDB Miscoded verify coding Number in
sample
State licensing boards 24 57 252 Health care providers (clinical privilege
restrictions) 23 3 79 Professional societies 0 4 7 DEA 0 7 7
Total 47 71 345
Source: GAO analysis.
HRSA has not established criteria for the information that should be
included in the narrative sections of state licensure reports. Our analysis
of the reports submitted in September 1999 revealed considerable variation
in the amount and quality of narrative information. Some reports included
sufficient detail to indicate why practitioners were disciplined. Others,
however, contained insufficient explanations of disciplinary actions. For
example, 26 of the state licensure reports we reviewed were based on actions
taken by another state. The narrative sections of more than onehalf of these
reports did not note which state initially took action or why. In theory,
organizations querying NPDB should receive information from all the states
that have sanctioned practitioners. However, if the initial action was
reported late- as 30 percent of the state reports we reviewed were- or not
at all, organizations querying NPDB might not be able to identify the
appropriate state to contact to obtain additional information on the initial
licensure action.
We also found reports that may have been inadvertently submitted twice to
NPDB, making it appear as though practitioners had been disciplined more
than once for the same offense in a relatively short time. We queried NPDB
for information on four practitioners who were reported at least twice
during September 1999 and found that the narrative sections of state
licensure reports, in particular, lacked sufficient detail to determine
whether they were duplicates or reports of separate actions taken against
practitioners. For example, a state reported that a practitioner's license
was surrendered twice within 1 week. The response we obtained to our query
indicated that the second report had been erroneously submitted.
In another instance, a state reported issuing a public letter of reprimand
because of the poor condition of a practitioner's medical records.
Approximately 1 week later, the state submitted an identical report to
NPDB. The information we received in response to our query did not provide
sufficient detail to determine if the practitioner had been reprimanded once
or twice for poor recordkeeping. Although NPDB software routinely generates
notices to practitioners who have been reported to NPDB, practitioners may
not realize that a second report notification may indicate that a duplicate
report had been submitted. HRSA officials informed us that they have
directed the NPDB contractor to begin identifying and removing duplicate
reports from the data bank during the next contract year.
Clinical Privilege As with state licensure action reports, the reports that
hospitals and other
Restriction Reports Were health care providers submitted on clinical
privilege restrictions also
Miscoded and Included contained errors affecting the accuracy of NPDB
information. We found
Inappropriate Information coding errors in about one- third of the 79
clinical privilege restriction
reports we reviewed. Several health care providers used codes that indicated
licensure actions had been taken when, in fact, the practitioners' clinical
privileges had been restricted. In another instance, a provider coded a
report as though the practitioner's privileges had been restricted, while
the narrative section stated that the application for privileges had been
denied. While the narrative sections of clinical privilege reports generally
contained sufficient information to discern which actions were taken, those
purchasing copies of NPDB's public use file do not receive the narratives
and thus might be misled about the severity of disciplinary actions taken
against practitioners.
HRSA has not set criteria for the narrative sections of clinical privilege
restriction reports but has been working with consultants to identify ways
to improve the level of detail and consistency of reported information. A
recently completed study recommended that HRSA revise NPDB's new Internet-
based reporting format so that guidance specific to each type of
disciplinary action is displayed as the reporter keys in the narrative
information. For example, health care providers submitting reports on
clinical privilege restrictions imposed due to alcohol or substance abuse
would be instructed to include information in the narrative about the
specific circumstances under which the practitioner displayed a substance
abuse problem. Similarly, providers reporting practitioners whose privileges
were restricted because of incompetence would be instructed to state
specifically what the practitioner did or did not do.
HRSA officials told us that some of the study's recommended changes might be
too detailed to implement. They said that, in the past, reporters
have tended to select the top few choices for coding actions and might not
review an even longer list before selecting the most appropriate code.
Furthermore, HRSA officials' analysis of the extensive use of the “not
otherwise classified” category has led them to believe that some
reporters prefer to be less specific when reporting practitioners to the
data bank. As of December 31, 1998, 49 percent of the reports concerning
disciplinary actions were coded as not otherwise classified, while 34
percent of the malpractice reports were so coded. HRSA officials said that
they are reviewing the study's recommendations to determine which ones are
feasible for implementation.
Controls Do Not Ensure HRSA officials cited practitioner notifications and
the dispute resolution
Reporting Accuracy process as two control mechanisms that ensure the
accuracy of
information reported to NPDB. However, our analysis of reports submitted to
the data bank and the results of our queries for information on particular
practitioners suggest that these controls have not prevented erroneous
information from remaining in the data bank once it is reported. As
previously noted, there are instances- such as duplicate reports- when
practitioners are notified but may not realize that the same information has
been erroneously reported twice.
One NPDB Executive Committee member we spoke with told us that it is very
difficult to get information in the data bank corrected- and costly, if
practitioners get legal assistance. We found several examples of this. For
instance, on September 1, 1999, a hospital reported restricting a
practitioner's privileges because of poor recordkeeping. The practitioner
disputed the report, noting that the hospital planned to monitor his medical
records and not restrict clinical privileges. About 1 week later, the
hospital attempted to correct the information, requesting that NPDB cancel
the initial report. However, in doing so, the hospital incorrectly coded the
action as a state license revocation. As of July 2000, when we queried NPDB,
the incorrect information on the initial restriction and the erroneously
reported licensure revocation were still in the data bank.
Our July 2000 query also yielded information on a practitioner that, based
on our analysis, should no longer be available to organizations querying the
data bank. In this instance, a state reported revoking a license because the
practitioner did not meet its continuing medical education requirements. The
practitioner disputed the report and supplied evidence to the state of its
error. Although the state reported the mistake to NPDB in February 2000, we
received both reports in response to our query, indicating that the
information had not been expunged. These reports would likely be of
particular concern to the practitioner because this was the only information
that NPDB had on this individual. HRSA officials said that while there may
be instances when practitioners have difficulty getting reported information
corrected, the practitioner notification and dispute resolution processes
are generally adequate to address most problems.
User Fee Structure Not As stated earlier, NPDB operations are funded by the
fees that users pay to
query the data bank for information on practitioners. 18 HRSA does not
Validated and Controls
receive a separate appropriation for these purposes. In fiscal year 1999,
Over Collections and
HRSA collected $14 million in user fees, disbursed about $12 million, and
Disbursements are
had a $6.8 million cash balance at the end of fiscal year 1999. In recent
Inadequate
years, HRSA has not adequately examined whether the level of the user fees
to finance NPDB operations is appropriate. In reviewing the collection and
disbursement activities, we also found that controls over NPDB transactions
did not ensure that all collections were received and that disbursements
were for authorized purposes.
At the end of fiscal year 1994, NPDB had a cash balance of $3.3 million. As
table 3 shows, this balance has fluctuated over the last 5 years. Officials
told us these fluctuations occurred because some of these funds were used
for software and hardware enhancements to NPDB.
18 Section 427( b) of the Health Care Quality Improvement Act of 1986, as
amended in 1987, states that user fees may not exceed the costs of
“processing the requests for disclosure and providing such
information.” However, beginning with the HHS appropriation act for
fiscal year 1993 and each year through fiscal year 2000, an additional
provision has been included regarding user fees. The provision states that,
in addition to user fees authorized by section 427( b) of the 1986 act, fees
shall be collected for the full disclosure of information and be
“sufficient to recover the full costs of operating” the data
bank.
Table 3: 5- Year Trend in Accumulated NPDB Fees (in Millions of Dollars) FY
95 FY 96 FY 97 FY 98 FY 99
Beginning user fee balance $3. 3 $4.6 $2.6 $2.4 $3.1 Collections 10. 8 7. 6
9. 3 12. 0 14. 0 Recoveries a 0 0. 2 0.3 0 1.6 Total available 14. 1 12. 4
12. 2 14. 4 18. 7 Disbursements (9. 5) (9. 8) (9. 8) (11. 3) (11.9) Ending
user fee balance $4. 6 $2. 6 $2. 4 $3. 1 $6. 8 a At the beginning of the
year an estimated amount of NPDB funds are set aside for NPDB's portion of
HRSA's overhead. At the end of the year, funds that are not used are
reported as recoveries.
HRSA officials told us that the agency does not have a plan for its
financial operations that would project cash flows such as revenue,
disbursements, and capital investments. Neither has it reassessed the amount
it needs to cover NPDB operating expenses. While the accumulated fee balance
in fiscal year 1999 is consistent with HRSA's long- standing policy of
retaining about 4 to 6 months of operating expenses, HRSA has not confirmed
that this is an appropriate time frame. Performing an analysis could also
help HRSA determine whether the balance could be used to adjust the rates
charged for NPDB queries.
HRSA's management is responsible for establishing internal controls to
account for and manage user fees properly. The Comptroller General's
Standards for Internal Control in the Federal Governmentcontain the criteria
that federal agencies should follow in establishing and maintaining internal
controls. 19 As such, HRSA management is responsible for developing the
detailed policies, procedures, and practices that fit its agency's
operations. Specifically, this includes implementing procedures to (1)
assess user fees properly, (2) collect and record user fees, and (3)
reconcile user fees assessed with those collected and recorded.
HRSA and the Division of Financial Operations (DFO) did not have controls to
ensure that all assessed user fees were collected and properly recorded in
the general ledger. For example, unique identifying numbers that NPDB
assigns to each batch of queries for credit card transactions do not remain
with the transactions once they are entered into the commercial
19 The Comptroller General's Standards, as updated in November 1999, were
issued pursuant to the Federal Managers' Financial Integrity Act of 1982.
bank for processing. When the batch is electronically submitted to the
commercial bank for collection, the bank assigns a new identifying number,
deposits the funds in HRSA's Department of Treasury account, and sends a
daily deposit ticket to DFO, which records the funds in HRSA's general
ledger.
However, because the commercial bank assigned the batch of queries a
different identifier than the one originally assigned by NPDB, HRSA cannot
track the amounts of assessed user fees for credit card transactions to the
related collection amounts in the general ledger. Officials at the
commercial bank told us that they did not know that HRSA needed a unique
identifier for credit card transactions and that HRSA officials had not
contacted them about this issue. Without a common identifier, HRSA cannot be
assured that all assessed fees have been collected and may be foregoing
income that it is due. DFO and bank officials told us that, as a result of
our review, they have begun discussions about ways to correct this problem.
DFO officials realized that there were discrepancies between the amount of
user fees assessed and the amount collected and had conveyed this
information to the division within HRSA that oversees NPDB operations.
However, the discrepancies between amounts assessed and actual collections
were not reconciled because HRSA and DFO officials have not agreed on which
organization is responsible for performing these reconciliations. HRSA
officials acknowledged that reconciliations should be performed but stated
that DFO maintains the necessary documents and that HRSA does not have
access to them.
DFO reported that about $8.3 million in user fees were collected during the
first 8 months of fiscal year 2000, while HRSA's contractor reported $8.7
million in fees assessed in the same period. At the time of our review, an
analysis had not been performed to determine the reasons for this
difference. However, DFO officials speculated that the difference could be
due to denied credit card transactions, electronic funds transfer (EFT)
charges, or differences in when the commercial bank and HRSA post
transactions.
HRSA officials told us that DFO compares the collections recorded in the
general ledger to Treasury's records; however, this procedure is not
sufficient because the collections that are recorded in the general ledger
may not be accurate. As noted above, HRSA does not reconcile assessments
with actual collections. Reconciliation procedures are a
control necessary to ensure accurate reporting of user fee receipts. Until a
reconciliation is performed between the user fees assessed in NPDB and the
user fees collected and recorded in HRSA's general ledger, HRSA cannot be
assured that the general ledger is accurate. The Comptroller General's
Standards for Internal Control in the Federal Governmentstates that internal
control activities help ensure that management directives are carried out.
These activities include reconciliations and maintenance of related records
that provide evidence that these activities were executed and appropriately
documented.
HRSA and DFO also cannot ensure that the user fees collected electronically-
about 30 percent of NPDB's receipts- are properly allocated between NPDB and
HIPDB. HRSA's contractor operates NPDB and HIPDB and assigns unique
identifying numbers to each query processed by the data banks. However, the
bank commingles EFT transactions for the two data banks and sends deposit
information to DFO without differentiating between NPDB and HIPDB
transactions. Because DFO cannot independently determine how much should be
allocated to each data bank, it subtracts total HIPDB assessments- as shown
in the contractor's report- from the total deposits to arrive at the amount
credited to NPDB. This allocation process assumes that all assessed user
fees are collected.
Given the current allocation process, neither HRSA nor DFO can ensure that
the amounts that either data bank allocates in EFT- related collections are
accurate and that the collections posted to the general ledger for each data
bank are accurate. Without this knowledge, HRSA cannot be assured that it is
receiving all fees it is due nor can it ascertain whether these collections
stem from NPDB or HIPDB queries. Although EFT transactions accounted for
only about 30 percent of HRSA's total user fee receipts for fiscal years
1998 and 1999, these transactions are expected to increase. According to
HRSA officials, the agency plans to request that all users of NPDB pay for
queries electronically to reduce processing costs. When this procedure is
fully implemented, EFT transactions will become an even larger part of
NPDB's transactions. HRSA and DFO officials also told us that as a result of
our review, they are revising the allocation process so that it more
accurately reflects collections for each data bank.
Based on our review of NPDB disbursements, we determined that controls were
not effective. After reviewing and testing 118 statistically selected
disbursements from a population of 102,393, we estimate that HRSA and DFO
could not provide adequate documentation for 7, 810 transactions. 20
We also estimate that HRSA and DFO could not provide any documentation for
6, 942 disbursement transactions. 21 The Comptroller General's Standards for
Internal Control in the Federal Governmentstates that all transactions and
significant events need to be clearly documented and that documentation
should be readily available for examination.
Conclusions Quantifying and reducing underreporting to NPDB are admittedly
difficult, but without a coherent strategy for systematically addressing the
areas of
greatest significance, agency efforts may continue to be ineffective. While
NPDB is presently the nation's only central source of medical malpractice
payment information, it is not clear that all such data are being properly
reported. Underreporting of clinical privilege restrictions is another area
of particular concern because these reports are seen as better indicators of
professional competence and involve events far more recent than medical
malpractice settlements and judgments. However, HRSA only recently requested
that HHS seek the additional legislative authority that the HHS/ OIG
recommended as necessary for addressing noncompliance by hospitals and other
health care providers. Even more troubling is HRSA's failure to implement
the law regarding nurses and other practitioners, despite their increasing
importance in the delivery of health care services.
While we only sampled 1 month's submissions, our review suggests that NPDB
information may not be as accurate, complete, or timely as it should be.
Nearly one- third of the reports involving disciplinary actions were either
miscoded or did not have sufficient detail to determine what action was
taken and why. Inaccuracies in the way reported information was coded could
confuse or mislead querying organizations about the severity of actions
taken against practitioners. Additionally, duplicate reports overstate the
amount of information that NPDB has on a particular practitioner. Some
reporters may have purposely submitted vaguely coded and uninformative
reports; however, HRSA bears part of that responsibility. The agency has not
established criteria for the descriptive information that must be reported
by states and other entities when notifying the data bank of the
disciplinary actions taken. Moreover, the agency does not have
20 We are 95 percent confident that the actual total lies between 3,973 and
13, 563 disbursement transactions. 21 We are 95 percent confident that the
actual total lies between 3,325 and 12, 533 disbursement transactions.
procedures for ensuring that reporters adhere to the criteria it has
established for medical malpractice reports, including inappropriate
references to patients' names. Furthermore, the practitioner notification
and dispute resolution processes have not ensured that inaccurate and
erroneously reported information is removed from the data bank and not
released to entities seeking information on specific practitioners.
Finally, without an examination of its financial operations, HRSA has little
assurance that its NPDB user fees are appropriate. An analysis of its cash
balances and cash flows�user fee collections and disbursements�would be the
best way for HRSA to determine the appropriateness of fees. Moreover, HRSA
needs to improve controls over its collection and disbursement activities.
For example, HRSA and DFO did not have adequate controls to ensure that all
assessed user fees were collected and properly recorded in its general
ledger. As a result, HRSA could be foregoing income that it is due. Until
monthly reconciliations of user fee information are performed, HRSA cannot
be assured that its assessments and collections are accurate and complete.
In addition, neither HRSA nor DFO have procedures to ensure proper
allocation of EFT user fee receipts between NPDB and HIPDB. Without these
procedures, HRSA cannot ascertain whether its collections stem from NPDB or
HIPDB. Also, controls over NPDB disbursements were not effective because
supporting documentation that would provide confidence that disbursements
were for authorized purposes was too often missing or inadequate.
Recommendations for To address underreporting, we recommend that the
Secretary of Health and
Executive Action Human Services determine what resources and authorities are
required to
monitor and enforce compliance with NPDB's reporting requirements
efficiently and effectively, and then seek the necessary legislative
remedies to carry out these responsibilities. Additionally, the Secretary
should require the Administrator of HRSA and the Director of DFO to work
together to accomplish the following:
Develop an annual financial plan for projecting cash flows- including
revenue, operating expenses, and capital investments- as a basis for
assessing operating cash needs. This includes assessing the adequacy of the
human capital and technical resources needed for NPDB operations. Further,
taking into consideration existing cash balances and projected cash flows,
they should evaluate whether current user fees are appropriate.
Develop procedures to ensure that all assessed user fees are collected,
including (1) establishing an audit trail of user fees from the NPDB system
to the general ledger and (2) periodically reconciling user fees. Develop
procedures to ensure that user fees are properly allocated
between NPDB and HIPDB. Ensure that NPDB disbursements are adequately
documented. This
could be done by establishing internal controls that require original
support and a clear audit trail for all disbursements.
We also recommend that the Administrator of HRSA Take immediate action to
incorporate information on the disciplinary
actions taken against nurses and other health care practitioners into NPDB.
Incorporate NPDB into the agency's strategic plan, including the
measures needed to improve the reliability of reported information. Develop
criteria for the information that should be included in the
narrative sections of reports concerning disciplinary actions taken against
practitioners. Develop procedures for routinely checking the accuracy and
completeness of information reported to NPDB and for obtaining corrections
from reporters, when necessary. Revise NPDB user and practitioner
notifications to include disclosures
on the limitations of the data and warnings regarding duplicate submissions
as an interim measure until procedures to monitor data quality are
implemented.
Agency Comments and In written comments (reprinted in app. II) on a draft of
this report, HHS
Our Evaluation said that it generally agreed with the report's findings. HHS
concurred with
three of our recommendations and described actions it is taking. It
disagreed with the rest of our recommendations.
HHS concurred with our recommendation concerning compliance monitoring and
enforcement. The Department agreed that it needs to assess the additional
resources and authorities needed to address noncompliance proactively.
However, HHS noted that, to improve compliance with reporting requirements,
HRSA needs to coordinate its efforts with OIG and the health care community.
HHS also concurred with our recommendation to properly allocate user fees
between NPDB and HIPDB. The Department noted that HRSA has directed its
commercial bank to implement procedures separating collections between NPDB
and
HIPDB. Finally, HHS concurred with our recommendation to develop criteria
for the narrative section of disciplinary action reports and indicated that
HRSA has begun taking steps to do so.
HHS did not concur with our recommendation to develop an annual financial
plan. HHS indicated that this is unnecessary because HRSA projects its
revenue, disbursements, and capital investments annually, and monitors
income and expenditures on a monthly basis. We acknowledge that, although
HRSA may make projections to adjust user fees, it could not provide us with
a written plan during our study indicating how, and how often, these
projections are made.
Similarly, HHS did not agree with our recommendation that it develop
procedures over the collection process, including establishing an audit
trail of user fees from NPDB to the general ledger and periodically
reconciling these fees. It indicated that NPBD user fees are collected
promptly and properly. Despite its disagreement with our recommendation, HHS
stated that HRSA's contractor and bank will implement procedures to create
an audit trail and DFO will routinely reconcile amounts processed with
amounts deposited and recorded in HRSA's general ledger.
HHS also did not concur with our recommendation regarding disbursements. It
acknowledged that it could not provide documentation for some transactions,
but explained that these disbursements occurred before HHS adopted its new
accounting system. HHS said that its new system ensures effective internal
controls over disbursements and a clear audit trail. Further, HHS noted that
the organization managing the NPDB accounting system for HRSA�HHS' Program
Support Center- had received clean opinions from its independent auditor on
its internal controls for fiscal years 1998 and 1999. However, our test
results showed that HRSA's controls over disbursements are not effective.
Several of the disbursements for which HRSA could not provide documentation
occurred after the new system was implemented. In addition, we believe that
HHS' statement that the Program Support Center has received clean opinions
on its internal controls is misleading. Our review of these internal control
reports showed that the audits involved computer system controls and not the
detailed testing of disbursements that was covered by our audit.
HHS disagreed with our recommendation to immediately incorporate into NPDB
disciplinary action information against nurses and other health care
practitioners. Instead, it indicated that it needs to review the Medicare
and Medicaid Patient and Program Protection Act of 1987- in light of more
recent legislation that established HIPDB- before it can take any action. We
believe that HHS has had ample time to study this issue because the original
Act became effective more than 13 years ago and HIPDB was established in
legislation that was passed more than 4 years ago.
HHS did not concur with our recommendations to improve the reliability of
information contained in NPDB. In regard to our recommendation to include
NPDB operations into HRSA's strategic plan, HHS stated that it does not
include individual programs in a plan that covers broad programmatic areas.
Instead, it indicated that HRSA's 2001 Annual Performance Plan contains
information about NPDB operations. While HRSA believes that its Performance
Plan may be an appropriate place to address NPDB operations, there is no
mention in this plan of NPDB or measures associated with improving the
reliability of its information. We continue to believe that this information
should be incorporated into the agency's strategic plan.
Finally, HHS also disagreed that it should develop procedures to ensure the
accuracy and completeness of NPDB information and that it should revise its
notification to users regarding limitations in the data. HHS responded that
HRSA already has adequate procedures in place to ensure the integrity of
NPDB information. It also said that users are properly informed about the
contents and limitations of NPDB data. However, we believe that the results
of our detailed tests raise serious concerns about the integrity of NPDB
information. For example, over 95 percent of the medical malpractice reports
we reviewed were missing information on standard of patient care
determinations. Accordingly, we continue to believe that our warnings about
the data's limitations are warranted.
HHS also suggested several technical comments, which we incorporated where
appropriate.
As agreed with your offices, unless you announce its contents earlier, we
plan no further distribution of this report until 30 days after its issuance
date. At that time, we will send copies to the Honorable Donna E. Shalala,
Secretary of Health and Human Services; the Honorable Claude E. Fox,
Administrator of HRSA; and interested congressional committees. Copies of
this report will also be made available to others upon request.
If you have any questions about HRSA's operation of NPDB as described in
this report, please contact Leslie G. Aronovitz at (312) 220- 7600. If you
have
questions about HRSA's financial operations relative to NPDB, please call
Gloria Jarmon at (202) 512- 4476. Other GAO contacts and staff
acknowledgments are listed in appendix III.
Leslie G. Aronovitz Director, Health Care Program Administration and
Integrity Issues
Gloria L. Jarmon Managing Director External Liaison
Appendi Appendi xes xI
Scope and Methodology To address issues related to underreporting, we
reviewed NPDB's authorizing legislation and regulations and the NPDB
Guidebookto identify the reporting requirements and instructions given to
those accessing the data bank. We interviewed HRSA officials and reviewed
the agency's fiscal year 2000 and 2001 performance plans and the fiscal year
1999 performance report to determine how NPDB fits into HRSA's overall
strategic plan. 1 Additionally, we reviewed NPDB's annual reports for
calendar years 1993 2 through 1999 and internal research proposals prepared
by HRSA's Division of Quality Assurance, the unit overseeing NPDB
operations. We interviewed HHS/ OIG officials and reviewed their reports on
the data bank to obtain information on NPDB's weaknesses and open
recommendations. We also reviewed HRSA- sponsored studies on issues related
to underreporting, including Hospital Peer Review and the National
Practitioner Data Bank( July 1999), The Roundtable on Hospital Reporting to
the NPDB( 1996 ), HRSA's Report to the Congress on Small Malpractice Payment
Issues( 1996), and the data bank's user satisfaction surveys.
We reviewed HRSA's December 24, 1998, notice of proposed rulemaking,
comments the agency received on the proposal, and the Federal Register
notice that subsequently withdrew the proposal. We reviewed the minutes of
meetings held since late 1998 and interviewed 17 of the 24 health care
industry representatives and advocacy groups on NPDB's Executive Committee.
This included interviewing officials from medical and dental professional
societies such as the American Medical Association, the American Dental
Association, American Association of Dental Examiners, the Federation of
State Medical Boards, and the National Council of State Boards of Nursing.
In addition, we interviewed officials of the Physicians Insurers Association
of America and Harvard Risk Management Foundation, which represent the
medical malpractice industry. We also interviewed representatives of the
American Hospital Association, the National Committee for Quality Assurance,
the Joint Commission on Accreditation of Healthcare Organizations, and
representatives of advocacy groups such as Public Citizen and the American
Association of Retired Persons. Finally, we reviewed the federal Memorandums
of Understanding that HRSA negotiated with the Departments of Defense,
1 The Government Performance and Results Act of 1993 (P. L. 103-62)
specifically requires that federal agencies develop multiyear strategic
plans, annual performance plans, and annual performance reports.
2 The 1993 annual report covered the period Sept. 1, 1992, to Aug. 31, 1993.
Transportation, and Veterans Affairs; the Drug Enforcement Administration;
the Indian Health Service; and the Public Health Service.
To evaluate the accuracy, completeness, and timeliness of NPDB data, we
obtained electronic copies of the 1, 645 reports submitted to NPDB during
September 1999 and electronic copies of the 447 reports that were submitted
as corrections, changes, or in dispute of the September reports, as of June
2000. 3 We categorized these reports by type of information reported-
medical malpractice payment, state licensure action, and clinical privilege
restrictions. 4 Because these three types of reports have different
requirements for coding and descriptive information, we analyzed each type
separately.
We used NPDB's reporting time frames to gauge the timeliness of reports. We
compared the dates malpractice payments were made or disciplinary actions
were taken (date of action) with the dates that the reports were submitted
(the certification date) to NPDB. In total, we analyzed 1,552 reports for
timeliness, including 1,300 medical malpractice payment reports and 252
state licensure actions. We did not analyze the timeliness of clinical
privilege restrictions because their submission deadlines vary by the method
used to transmit the information to NPDB, and we could not determine which
method had been used. Reports submitted electronically have a 15- day
deadline, while those submitted on paper pass through state licensing boards
and are allowed up to 30 days to reach NPDB.
We also analyzed the currency of information included in the 1,300 medical
malpractice reports submitted to NPDB during September 1999. We compared the
dates of the events initiating the claims (date of act or omission) with the
dates that the payments were reported to NPDB. We could not analyze the
currency of state and health care provider reports because they do not
contain comparable information.
We assessed only medical malpractice payment reports for completeness
because this was the one type of report that had NPDB- prescribed criteria
3 We omitted reports concerning 298 practitioners that HHS/ OIG submitted to
NPDB as being excluded from participation in the Medicare and Medicaid
health care programs. This present study was focused on the accuracy,
completeness, and timeliness of reports involving malpractice payments and
disciplinary actions taken against practitioners.
4 We grouped the Drug Enforcement Administration and professional society
reports together with state licensure actions because NPDB classified all
three as licensure actions.
on the data that should be included in narrative descriptions. We selected
250 of the 1,300 medical malpractice payment reports to determine the
frequency with which seven of the items of information were present. 5 We
randomly selected 125 reports, then added to that number 101 reports
involving practitioners who had been reported more than once during
September 1999 and 24 reports that were disputed.
We assessed the accuracy of state licensure actions and clinical privilege
restriction reports by determining the internal consistency of the narrative
and coded information contained in individual reports. As part of this
analysis, we also identified the frequency with which reporters identified
why a particular action was taken against a practitioner. In total, we
analyzed 345 reports for accuracy, including those involving 252 state
licensure actions, 79 clinical privilege restrictions, 7 actions limiting
professional society memberships, and 7 DEA actions curtailing
practitioners' authorization to prescribe controlled substances.
We also queried NPDB for information on 34 practitioners reported during
September 1999. We selected these 34 practitioners due to the nature of the
reported information, such as apparently erroneous or duplicate report
submissions. We did this to determine what information NPDB would provide on
these practitioners and to gauge the impact of potentially erroneous
reports.
Two limitations affect our analysis of information reported to NPDB. First,
we had to rely on NPDB's own criteria and the internal consistency of
reports to gauge timeliness, accuracy, and completeness. There was no
independent, single source for much of the information contained in NPDB.
Second, we only had a snapshot of the information in the data bank. Working
with HRSA officials, we selected 1 month's submissions to NPDB for our
analysis. We did not find any evidence that would lead us to believe that
September 1999 was an atypical month for NPDB. Besides the 34 practitioners
for which we obtained query results, we do not know what other information
has been reported on the practitioners included in our September 1999
sample.
5 As specified in the NPDB Guidebook, medical malpractice reports should
include information on the patients' age, gender, inpatient or outpatient
status, the events (initial and subsequent) precipitating the claim, and the
medical or legal damages incurred. The reports are also to include
information on whether a standard of patient care determination had been
made in connection with the settlement or judgment.
To review the adequacy of HRSA's internal controls to ensure proper
accountability and management of user fees, we interviewed officials from
DFO and HRSA to understand how user fees are determined, assessed,
collected, recorded, and disbursed. We also interviewed and reviewed the
workpapers of independent public accountants who in fiscal year 1999
performed work technically known as “agreed- upon procedures”
for userfee- related issues. 6 The accountants told us they could not
develop an audit trail for user fee transactions. To independently verify
the accountants' work, we selected one credit card and one electronic funds
transfer (EFT) to trace pertinent data from the point at which a user fee
was assessed to its posting to HRSA's general ledger.
In addition, we selected and tested a statistical sample of the disbursement
transactions from HRSA's general ledger that occurred between October 1,
1994, and May 31, 2000. 7 We traced the sampled disbursements from the
general ledger to supporting documentation. We also reviewed the supporting
documentation to determine whether the disbursements had been properly
approved and reviewed pertinent laws, regulations, and guidance related to
NPDB user fees to determine whether the disbursements were used for
authorized purposes. Finally, we discussed with HRSA officials their reasons
for maintaining excess user fees and reviewed documentation supporting
management's decision to maintain these additional funds.
We performed our work between January 2000 and September 2000 in accordance
with generally accepted government auditing standards.
6 The term “agreed- upon procedures” means that the client and
accountant have agreed that specific work will be performed in areas
involving certain items of the financial statement. The final report is
limited to the results (findings) of the work performed.
7 We statistically selected a probability sample of 118 disbursements from
HRSA's population of 102,392. With this statistically valid probability
sample, each disbursement had a nonzero chance of being included in the
sample. Each sample element was subsequently weighted in the analysis to
account statistically for all disbursements in the population, including
those that were not selected.
Comments From the Department of Health
Appendi xII and Human Services
Appendi xI II
GAO Contacts and Staff Acknowledgments GAO Contacts Geraldine Redican-
Bigott, (312) 220- 7678 Rosa Ricks Harris, (202) 512- 9492 Staff
Acknowledgments Enchelle Bolden, Marian Cebula, Tiffani Clark, Lynn Filla-
Clark, Tarunkant
Mithani, and Barbara Mulliken also made key contributions to this report.
(201024/ 916331) Lett er
Ordering Information The first copy of each GAO report is free. Additional
copies of reports are $2 each. A check or money order should be made out to
the Superintendent of Documents. VISA and MasterCard credit cards are
accepted, also.
Orders for 100 or more copies to be mailed to a single address are
discounted 25 percent.
Orders by mail: U. S. General Accounting Office P. O. Box 37050 Washington,
DC 20013
Orders by visiting: Room 1100 700 4th St. NW (corner of 4th and G Sts. NW)
U. S. General Accounting Office Washington, DC
Orders by phone: (202) 512- 6000 fax: (202) 512- 6061 TDD (202) 512- 2537
Each day, GAO issues a list of newly available reports and testimony. To
receive facsimile copies of the daily list or any list from the past 30
days, please call (202) 512- 6000 using a touchtone phone. A recorded menu
will provide information on how to obtain these lists.
Orders by Internet: For information on how to access GAO reports on the
Internet, send an e- mail message with “info” in the body to:
info@ www. gao. gov or visit GAO's World Wide Web home page at: http:// www.
gao. gov
To Report Fraud,
Contact one:
Waste, or Abuse in
Web site: http:// www. gao. gov/ fraudnet/ fraudnet. htm
Federal Programs
e- mail: fraudnet@ gao. gov 1- 800- 424- 5454 (automated answering system)
GAO United States General Accounting Office
Page 1 GAO- 01- 130 National Practitioner Data Bank
Contents
Page 2 GAO- 01- 130 National Practitioner Data Bank
Page 3 GAO- 01- 130 National Practitioner Data Bank United States General
Accounting Office
Washington, D. C. 20548 Page 3 GAO- 01- 130 National Practitioner Data Bank
Page 4 GAO- 01- 130 National Practitioner Data Bank
Page 5 GAO- 01- 130 National Practitioner Data Bank
Page 6 GAO- 01- 130 National Practitioner Data Bank
Page 7 GAO- 01- 130 National Practitioner Data Bank
Page 8 GAO- 01- 130 National Practitioner Data Bank
Page 9 GAO- 01- 130 National Practitioner Data Bank
Page 10 GAO- 01- 130 National Practitioner Data Bank
Page 11 GAO- 01- 130 National Practitioner Data Bank
Page 12 GAO- 01- 130 National Practitioner Data Bank
Page 13 GAO- 01- 130 National Practitioner Data Bank
Page 14 GAO- 01- 130 National Practitioner Data Bank
Page 15 GAO- 01- 130 National Practitioner Data Bank
Page 16 GAO- 01- 130 National Practitioner Data Bank
Page 17 GAO- 01- 130 National Practitioner Data Bank
Page 18 GAO- 01- 130 National Practitioner Data Bank
Page 19 GAO- 01- 130 National Practitioner Data Bank
Page 20 GAO- 01- 130 National Practitioner Data Bank
Page 21 GAO- 01- 130 National Practitioner Data Bank
Page 22 GAO- 01- 130 National Practitioner Data Bank
Page 23 GAO- 01- 130 National Practitioner Data Bank
Page 24 GAO- 01- 130 National Practitioner Data Bank
Page 25 GAO- 01- 130 National Practitioner Data Bank
Page 26 GAO- 01- 130 National Practitioner Data Bank
Page 27 GAO- 01- 130 National Practitioner Data Bank
Page 28 GAO- 01- 130 National Practitioner Data Bank
Page 29 GAO- 01- 130 National Practitioner Data Bank
Page 30 GAO- 01- 130 National Practitioner Data Bank
Page 31 GAO- 01- 130 National Practitioner Data Bank
Page 32 GAO- 01- 130 National Practitioner Data Bank
Page 33 GAO- 01- 130 National Practitioner Data Bank
Page 34 GAO- 01- 130 National Practitioner Data Bank
Page 35 GAO- 01- 130 National Practitioner Data Bank
Page 36 GAO- 01- 130 National Practitioner Data Bank
Appendix I
Appendix I Scope and Methodology
Page 37 GAO- 01- 130 National Practitioner Data Bank
Appendix I Scope and Methodology
Page 38 GAO- 01- 130 National Practitioner Data Bank
Appendix I Scope and Methodology
Page 39 GAO- 01- 130 National Practitioner Data Bank
Page 40 GAO- 01- 130 National Practitioner Data Bank
Appendix II
Appendix II Comments From the Department of Health and Human Services
Page 41 GAO- 01- 130 National Practitioner Data Bank
Appendix II Comments From the Department of Health and Human Services
Page 42 GAO- 01- 130 National Practitioner Data Bank
Appendix II Comments From the Department of Health and Human Services
Page 43 GAO- 01- 130 National Practitioner Data Bank
Appendix II Comments From the Department of Health and Human Services
Page 44 GAO- 01- 130 National Practitioner Data Bank
Appendix II Comments From the Department of Health and Human Services
Page 45 GAO- 01- 130 National Practitioner Data Bank
Page 46 GAO- 01- 130 National Practitioner Data Bank
Appendix III
United States General Accounting Office Washington, D. C. 20548- 0001
Official Business Penalty for Private Use $300
Address Correction Requested Bulk Rate
Postage & Fees Paid GAO Permit No. GI00
*** End of document. ***