Health Professions Education: Clarifying the Role of Title VII and VIII
Programs Could Improve Accountability (Stmnt. for the Rec., 04/25/97,
GAO/T-HEHS-97-117).
GAO discussed reauthorizing health professions education programs
established under Title VII and Title VIII of the Public Health Service
Act.
GAO noted that: (1) the effectiveness of Title VII and VIII programs
will remain difficult to measure as long as they are authorized to
support a broad range of health care objectives without common goals,
outcome measures, and reporting requirements; (2) the implementation of
the Government Performance and Results Act and the "clustering" concept
offer an opportunity to address these problems by providing the
framework and flexibility to clarify the role of Title VII and VIII
programs and direct federal efforts to achieve desired outcomes; and (3)
however, unless steps are taken with a clear eye to addressing the
problems, these developments could meet with little success.
--------------------------- Indexing Terms -----------------------------
REPORTNUM: T-HEHS-97-117
TITLE: Health Professions Education: Clarifying the Role of Title
VII and VIII Programs Could Improve Accountability
DATE: 04/25/97
SUBJECT: Aid for education
Minority education
Health care personnel
Physicians
Dentists
Medical education
Cost effectiveness analysis
IDENTIFIER: Medicare Program
******************************************************************
** This file contains an ASCII representation of the text of a **
** GAO report. Delineations within the text indicating chapter **
** titles, headings, and bullets are preserved. Major **
** divisions and subdivisions of the text, such as Chapters, **
** Sections, and Appendixes, are identified by double and **
** single lines. The numbers on the right end of these lines **
** indicate the position of each of the subsections in the **
** document outline. These numbers do NOT correspond with the **
** page numbers of the printed product. **
** **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced. Tables are included, but **
** may not resemble those in the printed version. **
** **
** Please see the PDF (Portable Document Format) file, when **
** available, for a complete electronic file of the printed **
** document's contents. **
** **
** A printed copy of this report may be obtained from the GAO **
** Document Distribution Center. For further details, please **
** send an e-mail message to: **
** **
** **
** **
** with the message 'info' in the body. **
******************************************************************
Cover
================================================================ COVER
Before the Subcommittee on Public Health and Safety, Committee on
Labor and Human Resources, U.S. Senate
For Release on Delivery
Expected at 9:30 a.m.
Friday, April 25, 1997
HEALTH PROFESSIONS EDUCATION -
CLARIFYING THE ROLE OF TITLE VII
AND VIII PROGRAMS COULD IMPROVE
ACCOUNTABILITY
Statement for the Record by Bernice Steinhardt, Director
Health Services Quality and Public Health Issues
Health, Education, and Human Services Division
GAO/T-HEHS-97-117
GAO/HEHS-97-117T
(108326)
Abbreviations
=============================================================== ABBREV
GPRA - Government Performance and Results Act of 1993
HHS - Department of Health and Human Services
HRSA - Health Resources and Services Administration
PHS - Public Health Service
HEALTH PROFESSIONS EDUCATION:
CLARIFYING THE ROLE OF TITLE VII
AND VIII PROGRAMS COULD IMPROVE
ACCOUNTABILITY
============================================================ Chapter 0
Mr. Chairman and Members of the Subcommittee:
We are pleased to contribute this statement for the Subcommittee's
deliberations on reauthorizing health professions education programs
established under titles VII and VIII of the Public Health Service
Act.\1 In fiscal year 1996, the Congress provided nearly $300 million
for about 40 programs under these titles. These programs,
administered through the Department of Health and Human Services
(HHS),\2 provide direct student assistance, such as loans, as well as
grants to institutions for expansion or maintenance of health
professions education and training.
When it last reauthorized titles VII and VIII in 1992, the Congress
required us to report on whether these programs were effective in
advancing three key purposes--increasing the numbers of health
professionals, improving their distribution in locations that have a
shortage of health professionals, and adding minorities to their
ranks. Since our 1994 report,\3 two important developments have
occurred with regard to these programs. First, the Government
Performance and Results Act of 1993 (GPRA), which was newly enacted
when we issued our report, requires federal agencies to be more
accountable for the results of their efforts. Second, both the
Congress and the administration have proposed placing many existing
Title VII and VIII programs into five or six "clusters" as part of
efforts to streamline government. In preparing this statement, we
relied on our earlier report and conducted a limited amount of
follow-up work to review the current status of the Title VII and VIII
programs and to place our earlier findings in the context of these
new developments.
Our comments will focus on (1) problems we identified in linking
these programs to changes in the supply, distribution, and minority
representation of health professionals and the impact of these
changes on access to care; (2) the potential for implementation of
GPRA to address these problems; and (3) the opportunities associated
with consolidating the separate programs into program clusters.
In brief, we found that the effectiveness of Title VII and VIII
programs will remain difficult to measure as long as they are
authorized to support a broad range of health care objectives without
common goals, outcome measures, and reporting requirements. The
implementation of GPRA and the "clustering" concept offer an
opportunity to address these problems by providing the framework and
flexibility to clarify the role of Title VII and VIII programs and
direct federal efforts to achieve desired outcomes. However, unless
steps are taken with a clear eye to addressing the problems, these
developments could meet with little success.
--------------------
\1 42 U.S.C. 292-298b-7 (1994).
\2 HHS' Health Resources and Services Administration (HRSA)
administers these programs.
\3 Health Professions Education: Role of Title VII/VIII Programs in
Improving Access to Care Is Unclear (GAO/HEHS-94-164, July 8, 1994).
ROLE OF PROGRAMS IN IMPROVING
ACCESS IS UNCLEAR
---------------------------------------------------------- Chapter 0:1
In 1994, we reported that the supply of nearly all types of health
professionals had increased faster than the population. Moreover,
the number of underrepresented minorities in health professions
education for which data were available\4 had increased faster than
the number for all races combined. For most professions, however,
data were not available to demonstrate whether this increased supply
and minority representation translated into more access to care in
rural and underserved areas. For the two professions with the most
data available--primary care physicians and general dentists--supply
increased in many rural areas but not in those urban and rural areas
where the greatest shortages existed.
We also reported that evaluations of Title VII and VIII programs have
not linked these programs to the changes in the supply, distribution,
and minority representation of health professionals. HHS is not
required to evaluate the effectiveness of each program, and 6 of the
23 programs established before 1990 have never been evaluated.\5
Evaluations conducted on the remaining programs generally addressed
the impact at individual institutions and found that the programs
have assisted schools in improving or enhancing curricula, funding
innovative projects, and providing seed money for starting new
programs. However, the results of virtually all of these evaluations
could not be generalized to determine the national impact of the
programs in the three key areas. Such a relationship is difficult to
establish for several reasons, including (1) the wide variety of
often unrelated objectives that the programs addressed and (2)
problems with the data and criteria used to measure the outcomes of
what the programs were accomplishing.
--------------------
\4 HHS has identified African-Americans, Native Americans, and
Hispanics as underrepresented in the health professions. Only data
for physicians, dentists, and registered nurses were available for
minority applicants, first-year enrollments, and graduates of health
professions schools.
\5 The Secretary of HHS is authorized by statute to set aside up to 1
percent of Public Health Service (PHS) appropriations for
evaluations. We previously reported that implementation of this
set-aside has been less than fully effective in providing information
to the Congress on PHS programs. See Public Health Service:
Evaluation Set-Aside Has Not Realized Its Potential to Inform the
Congress (GAO/PEMD-93-13, Apr. 8, 1993).
MULTIPLE PROGRAM OBJECTIVES
-------------------------------------------------------- Chapter 0:1.1
While over the past 2 decades congressional interest in Title VII and
VIII programs has focused on their utility in adding to the number of
health care professionals, placing these professionals in underserved
areas, and training more minority health professionals, the programs
themselves have a variety of objectives. Titles VII and VIII,
established in 1963 and 1964 and amended over time, authorize funding
for a number of programs with diverse objectives. While most of the
programs address at least one of the three key areas of improving the
supply, distribution, and minority representation of health
professionals, they also address other objectives as well. These
other objectives, such as improving the quality of education and
training, may only indirectly result in improvements to the three key
areas. Furthermore, HHS officials identified some programs,
including grants for chiropractic demonstration projects, that do not
have objectives related to any of the three key areas.
The large number and piecemeal approach of Title VII and VIII program
objectives make evaluating program impact difficult. For example,
one institution received a $300,590 family medicine grant to further
the achievement of 12 separate objectives. One of the 12 objectives
was to directly improve distribution and minority representation;
none was for increasing supply. The other 11 were for various
curricula improvements, such as expanding the behavioral science
curriculum and maintaining the physician practice-management
curriculum. While these 11 other objectives may be valuable in their
own right, they represent federally funded activities that could not
be directly linked to, and thus evaluated as affecting, these three
key areas.
PROBLEMS WITH OUTCOME
MEASURES
-------------------------------------------------------- Chapter 0:1.2
Another problem hindering evaluation was that none of the Title VII
and VIII programs at the time of our review had established specific
program outcome measures--that is, the desired results--against which
to gauge their effectiveness. Establishing results-oriented measures
is difficult because to set such measures, HHS must move beyond what
it controls--that is, the activities--to focus on what it merely
influences--the results. We found that some grantees reported on the
process they established to achieve results, rather than on the
results themselves. For example, a grantee reported that it
instituted a recruitment activity but did not report how many
students were actually recruited through federal funding of this
activity.
We also identified problems in the cases in which HHS had begun
collecting data to measure program outcomes. For example, data
provided to HHS to qualify for a funding preference for placing
graduates in medically underserved communities were not necessarily
complete or comparable among schools, and HHS had not established a
way to validate the data provided. Even if these data problems are
resolved, other work we have conducted shows that the underlying
criteria used to identify some medically underserved communities are
outdated and flawed.\6 For example, more than half of the locations
designated as underserved may be invalid because the data are
outdated or do not consider a significant number of primary care
providers, such as nurse practitioners or physician assistants.
Without valid criteria and data against which to measure grantee
performance, it is difficult to determine whether grantee efforts
under Title VII and VIII programs are needed and will be successful,
or if other federal programs would be more appropriate.
--------------------
\6 More specifically, the designation systems we evaluated were the
Health Professional Shortage Areas and Medically Underserved Areas.
See Health Care Shortage Areas: Designations Not a Useful Tool for
Directing Resources to the Underserved (GAO/HEHS-95-200, Sept. 8,
1995).
IMPLEMENTATION OF THE
GOVERNMENT PERFORMANCE AND
RESULTS ACT PROVIDES AN
OPPORTUNITY TO ADDRESS
IDENTIFIED PROBLEMS
---------------------------------------------------------- Chapter 0:2
GPRA was intended, in part, to deal with the types of problems we
identified and provides HHS and the Congress with an opportunity to
address them. Concerned that federal agencies such as HHS have not
always effectively managed their activities to ensure accountability,
the Congress has created a legislative framework to address
long-standing management challenges throughout the federal
government. The centerpiece of this framework is GPRA. Under GPRA,
every major federal agency must now ask some basic questions: What
is our mission? What are our goals and how can we achieve them? How
can we measure our performance? How will we use that information to
improve? GPRA requires a strategic plan to be prepared in
consultation with the Congress--this plan is due in September 1997.
Since HHS is still finalizing its required plans, it is unclear
whether implementation of GPRA for the Title VII and VIII programs
will resolve the problems we identified regarding the number and
variation of program goals. One unresolved issue is the degree to
which Title VII and VIII program goals will be considered in
relationship to the other HHS programs for health professions
education and training. Because HHS' influence on education and
training involves multiple efforts spanning several of its agencies,
Title VII and VIII programs should not be considered in a vacuum.
HHS officials responsible for administering Title VII and VIII
programs cite the influences of other, larger HHS programs on health
professions education and training programs. For example, the
officials said they believe the incentives for primary care education
and training provided by Title VII programs are counteracted by the
billions of dollars of federal funding from the Medicare program to
support the training of specialists and from the National Institutes
of Health to support biomedical research at medical schools. HHS
officials added that goals of improving supply, distribution, or
minority representation nationally are unrealistic for the few
hundred million dollars that fund Title VII and VIII programs given
the multibillion dollar training environment. Clarifying the
intended nature and extent of the impact of Title VII and VIII
programs remains an important step in overcoming the problems we
identified.
The consultation process under GPRA gives the Congress and HHS an
opportunity to reach an understanding of what role Title VII and VIII
programs should play in this broader context of programs--and what
goals and desired outcomes should be set specifically for Title VII
and VIII programs. In discussing the challenges of GPRA
implementation, HHS officials noted that funding for Title VII and
VIII programs is often intended to have impact at the margin--that
is, to affect an individual institution or recipient--and therefore
long-term outcomes solely attributable to funding these programs are
difficult to measure. HHS officials are currently developing
cross-cutting goals and indicators for the Title VII and VIII
programs. Although benchmarks and specific goals against which to
measure the success of these programs have not yet been established,
the officials said they plan on measuring the impact on projects that
receive Title VII and VIII funding, such as counting the number of
enrollees at program-supported institutions, rather than measuring
changes in national indicators.
For successful GPRA implementation, performance information must be
used to direct resources where federal intervention would have a
greater impact. One area that would benefit from this process is
HHS' goal of increasing minority representation in the health
professions. Although some minority groups are underrepresented in
the health professions when compared with their overall percentage in
the U.S. population, some groups are not underrepresented when the
comparison is based more narrowly on the segment of the population
with the necessary educational background to enter into health
professions education and training. This may mean that federal
efforts could be better spent on bringing more minority students to
the point of being able to enter health professions education,
instead of on helping the relatively few who already have those
qualifications. GPRA, with its emphasis on targeting federal efforts
on more cost-effective ways to achieve agreed-upon goals, could help
to surface such considerations.
PROPOSALS TO CLUSTER TITLE VII
AND VIII PROGRAMS COULD PROVIDE
FLEXIBILITY TO TARGET RESOURCES
---------------------------------------------------------- Chapter 0:3
Like GPRA, recent legislative proposals provide an opportunity to
focus Title VII and VIII moneys in the most effective ways. In
response to national efforts to streamline government, recent
reauthorization proposals by the Senate and the administration
combine about 40 Title VII and VIII categorical programs into 5 or 6
program clusters.\7
Under such an approach, the Congress could authorize and appropriate
funds for each cluster of programs instead of authorizing and
appropriating funds for each program. HHS could have the authority
to fund programs within the cluster but would no longer be required
to fund each individual program. This would give HHS more
flexibility, in conjunction with GPRA, to determine how to spend the
money to achieve stated goals.
This cluster concept could provide greater flexibility to target
resources to the most effective programs and to discontinue federal
support when agreed-upon goals have been achieved. However, as with
GPRA, achieving greater success with this approach is not automatic.
Increased agency flexibility and related discretion would make it
even more critical that the desired outcomes of the programs be
clarified so that resources could be allocated on the basis of the
need for and effectiveness of specific programs. Unless these issues
are addressed, the risk continues that money will be spent without a
clear idea of what is being accomplished--and whether spending it
differently would produce greater results.
--------------------
\7 The Senate proposal would also combine a Title III program, the
National Health Service Corps, with the Title VII and VIII programs
in one cluster.
CONCLUDING OBSERVATIONS
---------------------------------------------------------- Chapter 0:4
An appropriate number and mix of health professionals are vital to
ensuring that all Americans have adequate access to health care. Our
work points to the need to clarify the role of Title VII and VIII
programs in improving the supply, distribution, and minority
representation of health professionals and whether these programs are
intended to affect the health professions at the national level. If
these programs are to specifically improve supply, distribution, and
minority representation of health professionals, federal efforts need
to be directed to activities that clearly support those goals and
whose results can be measured and reported in terms of those goals.
Similarly, if the programs are to meet other goals, such as improving
curricula to address emerging national health issues, federal efforts
need to be directed to the most effective means of achieving them.
Regardless of which direction is chosen, once goals are defined,
performance measures and targets are critical to determine when
federal intervention is no longer required, or when federal
strategies are not successful and should be redirected.
The implementation of GPRA and reauthorization of the programs
provide an ideal opportunity to identify where Title VII and VIII
programs fit within the federal government's overall strategy for
addressing national health workforce issues. In doing so, HHS and
the Congress can establish vital national goals and common outcome
measures for HHS programs and allocate limited federal funds to those
programs, including programs outside of Title VII and VIII, based on
demonstrated effect and relative need in meeting national goals.
CONTRIBUTORS
---------------------------------------------------------- Chapter 0:5
This statement was prepared under the direction of Bernice
Steinhardt, Director, Health Services Quality and Public Health
Issues, who may be reached at (202) 512-7119 if there are any
questions. Other key contributors include Frank Pasquier, Assistant
Director, and Lacinda Baumgartner, Kim Yamane, and Stan Stenersen,
Evaluators.
*** End of document. ***