Federal Education Funding: Multiple Programs and Lack of Data Raise
Efficiency and Effectiveness Concerns (Testimony, 11/06/97,
GAO/T-HEHS-98-46).
GAO discussed: (1) the amount and complexity of federal support for
education; (2) additional planning, implementation, and evaluative
information needed by agencies and the Congress on federal education
programs; and (3) some of the challenges of obtaining more and better
information.
GAO noted that: (1) billions in federal education dollars are
distributed through hundreds of programs and more than 30 agencies; (2)
agencies and the Congress need information to plan, implement, and
evaluate these programs; (3) to gauge and ensure the success of these
programs, the Congress and agencies need several kinds of information;
(4) they need to know which specific program approaches or models are
most effective, the circumstances in which they are effective, and if
the individual programs are working nationwide; (5) they also need to be
able to look across all programs that are designed to help a given
target group to see if individual programs are working efficiently
together and whether the federal effort is working effectively overall;
(6) GAO believes a close examination of these multiple education
programs is needed; (7) the current situation has created the potential
for inefficient service and reduced overall effectiveness; (8) basic
information about programs and program results is lacking and there are
many challenges in obtaining this important information; and (9) the
Government Performance and Results Act of 1993 (GPRA) holds promise as a
tool to help agencies manage for results, coordinate their efforts with
other agencies, and obtain the information they need to plan and
implement programs and evaluate program results.
--------------------------- Indexing Terms -----------------------------
REPORTNUM: T-HEHS-98-46
TITLE: Federal Education Funding: Multiple Programs and Lack of
Data Raise Efficiency and Effectiveness Concerns
DATE: 11/06/97
SUBJECT: Federal funds
Aid for education
Education program evaluation
Interagency relations
Data collection
Information systems
Statistical data
******************************************************************
** This file contains an ASCII representation of the text of a **
** GAO report. Delineations within the text indicating chapter **
** titles, headings, and bullets are preserved. Major **
** divisions and subdivisions of the text, such as Chapters, **
** Sections, and Appendixes, are identified by double and **
** single lines. The numbers on the right end of these lines **
** indicate the position of each of the subsections in the **
** document outline. These numbers do NOT correspond with the **
** page numbers of the printed product. **
** **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced. Tables are included, but **
** may not resemble those in the printed version. **
** **
** Please see the PDF (Portable Document Format) file, when **
** available, for a complete electronic file of the printed **
** document's contents. **
** **
** A printed copy of this report may be obtained from the GAO **
** Document Distribution Center. For further details, please **
** send an e-mail message to: **
** **
** **
** **
** with the message 'info' in the body. **
******************************************************************
Cover
================================================================ COVER
Before the Education Task Force, Committee on the Budget, United
States Senate
For Release on Delivery
Expected at 9:30 a.m.
Thursday, November 6, 1997
FEDERAL EDUCATION FUNDING -
MULTIPLE PROGRAMS AND LACK OF DATA
RAISE EFFICIENCY AND EFFECTIVENESS
CONCERNS
Statement of Carlotta C. Joyner, Director
Education and Employment Issues
Health, Education, and Human Services Division
GAO/T-HEHS-98-46
GAO/HEHS-98-46T
(104909)
Abbreviations
=============================================================== ABBREV
HHS - Department of Health and Human Services
GPRA - ABC
K-12 - kindergarten to twelfth grade
NCES - National Center for Education Statistics
OMB - Office of Management and the Budget
PPG - Performance Partnership Grants
FEDERAL EDUCATION FUNDING:
MULTIPLE PROGRAMS AND LACK OF DATA
RAISE EFFICIENCY AND EFFECTIVENESS
CONCERNS
============================================================ Chapter 0
Mr. Chairman and Members of the Education Task Force:
I am pleased to be here today to discuss federal support for
education, concerns raised by the multiple programs dispersed through
over 30 agencies, and the lack of data about the programs and their
impact.
Hundreds of federal education programs--the specific number differs
depending on how education is defined and who is counting the
programs--are administered by more than 30 federal agencies.
Although billions of federal dollars support education, state and
local governments and the private sector spend even more. For
example, federal spending for public elementary and secondary
education is only about 7 percent of all funding for kindergarten
through high school (K-12) education. State and local sources
provide 47 and 46 percent of the funding, respectively.
Figure 1: Federal Share of
Public K-12 Education Costs Is
Small
(See figure in printed
edition.)
My testimony today is based on work we have done over several years
and on a recent analysis by the Department of Education's National
Center for Education Statistics (NCES).\1 (A list of related GAO
products appears at the end of this testimony.) I will focus on (1)
the amount and complexity of federal support for education; (2)
additional planning, implementation, and evaluative information
needed by agencies and the Congress on federal education programs;
and (3) some of the challenges of obtaining more and better
information.
In summary, billions of federal education dollars are distributed
through hundreds of programs and more than 30 agencies. Agencies and
the Congress need information to plan, implement, and evaluate these
programs. Moreover, to gauge and ensure the success of these
programs, the Congress and agencies need several kinds of
information. First, they need to know which specific program
approaches or models are most effective and the circumstances in
which they are effective. They also need to know if individual
programs are working nationwide. In addition, they need to be able
to look across all programs that are designed to help a given target
group to see if individual programs are working efficiently together
and whether the federal effort is working effectively overall. We
believe closely examining these multiple education programs is
definitely needed. The current situation has created the potential
for inefficient service and reduced overall effectiveness. Basic
information about programs and program results is lacking, and we
face many challenges in obtaining this important information. The
Government Performance and Results Act of 1993 (GPRA) holds promise
as a tool to help agencies manage for results, coordinate their
efforts with other agencies, and obtain the information they need to
plan and implement programs and evaluate program results.
--------------------
\1 Federal Support for Education: Fiscal Years 1980 to 1997,
Department of Education, NCES (Washington, D.C.: Sept. 1997).
BILLIONS OF FEDERAL EDUCATION
DOLLARS DISTRIBUTED THROUGH
HUNDREDS OF PROGRAMS AND
MULTIPLE AGENCIES
---------------------------------------------------------- Chapter 0:1
Overall federal support for education includes not only federal
funds, but also nonfederal funds associated with federal legislation.
More than 30 departments or agencies administer federal education
dollars, although the Department of Education administers the most,
accounting for about 43 percent. Of the about $73 billion
appropriated to support education, half supports elementary and
secondary education. Overall, six program areas account for almost
two-thirds of all budgeted education funding. Many departments and
programs may target funds to the same target groups, such as poor
children. Although some coordination takes place and some programs
have been consolidated, much more needs to be done to coordinate the
multiple education programs scattered throughout the federal
government.
DISTRIBUTION OF FEDERAL
SUPPORT FOR EDUCATION
-------------------------------------------------------- Chapter 0:1.1
NCES estimates federal support for education, excluding tax
expenditures, as approximately $100.5 billion in fiscal year 1997.
This figure is an estimate of the value of the assistance to the
recipients--not the cost to the government. NCES describes this
support as falling into two main categories: funds appropriated by
the Congress (on-budget) and a combination of what NCES calls
"off-budget"\2 funds and nonfederal funds generated by federal
legislation. Appropriated funds include items such as grants,
federal matching funds, and the administration and subsidy costs for
direct and guaranteed student loans. Off-budget funds are the
portion of direct federal loans anticipated to be repaid. Nonfederal
funds generated by federal legislation include nonfederal (generally
state or local) funds provided to obtain federal matching funds and
capital provided by private lenders for education loans. According
to NCES, in fiscal year 1997, appropriated funds constituted
approximately three-
quarters of the total: $73.1 billion.
--------------------
\2 The term "off-budget" as used by NCES differs from the technical
definition of the term. Off-budget actually refers to transactions
that belong on budget because they are a government cost but that are
required by law to be excluded from the budget. The transactions
associated with direct loans that NCES describes as off budget would
more appropriately be described as nonbudgetary.
MULTIPLE DEPARTMENTS AND
PROGRAMS
-------------------------------------------------------- Chapter 0:1.2
To ensure that all Americans have equal access to educational
opportunities, the federal government often targets its education
funds to groups, such as poor children, that for various reasons have
not had equal access to educational opportunities. The government
may also target funds to ensure that all children have access to
vital resources--such as well-trained teachers and technology. These
concerns have helped disperse federal education programs to over 30
departments or agencies.
The Department of Education spends the most, accounting for about 43
percent of appropriations or an estimated $31 billion in fiscal year
1997. (See fig. 2.) The Department of Health and Human Services
(HHS) spends the next largest amount, with about 18 percent or an
estimated $13 billion. Over half of this amount ($7.1 billion)
funded research; another $4 billion funded the Head Start program.
Other Departments with federal education dollars include the
Departments of Agriculture, Labor, and Defense, with 13, 6, and 5
percent, respectively. The remaining 15 percent is spent by more
than 30 additional departments or agencies.
Figure 2: Many Federal
Departments Fund Education
Programs
(See figure in printed
edition.)
Six program areas account for almost two-thirds of all on-budget
education funding. As figure 3 shows, the child nutrition programs
at the Department of Agriculture account for the largest percentage
of funding--16 percent. The two programs accounting for the next
largest percentages of funding are Department of Education programs:
title I for disadvantaged preschool, elementary, and secondary
children, with about 14 percent, and Pell Grants for postsecondary
education for low-income students, with about 12 percent. The
Department of Labor runs other major programs (Job Corps and other
job training programs) as do HHS and the Department of Defense.
Figure 3: Six Program Areas
Account for Almost Two-Thirds
of All Budgeted Education
Dollars
(See figure in printed
edition.)
Figure 4 shows the fiscal year 1997 funding for the six largest
program activities. Funding ranges from $8.3 billion for child
nutrition programs to $3.4 billion for special education activities.
Figure 4: Six Largest
Education Program Areas, Fiscal
Year 1997
(See figure in printed
edition.)
Elementary and secondary education programs account for half of all
budgeted federal education dollars. (See fig. 5.) In addition, the
federal government provides funds for postsecondary education
(generally as grants and loan guarantees), research (through such
Departments as HHS, Energy, and Defense, along with the National
Science Foundation), and other activities such as rehabilitative
services.
Figure 5: Elementary and
Secondary Education: Half of
On-Budget Education Dollars
(See figure in printed
edition.)
MANY AGENCIES AND PROGRAMS
TARGET SPECIFIC GROUPS
-------------------------------------------------------- Chapter 0:1.3
Federal funds are generally targeted to specific groups. However,
many education programs administered by separate agencies may target
any single group. Although we have no comprehensive figures on the
number of programs targeted to different groups, figure 6 shows the
number of programs in various agencies targeted to three specific
groups--young children, at-risk and delinquent youth, and teachers.
Figure 6: Three Target Groups
Served by Multiple Programs and
Agencies
(See figure in printed
edition.)
Note: Circled numbers indicate number of programs.
In September 1997, we reported that the federal government funded a
wide array of programs dedicated to at-risk and delinquent youth.\3
More specifically, 15 federal departments and agencies administered
127 at-risk and delinquent youth programs in fiscal year 1996. HHS
and the Departments of Justice, Labor, and Education administered 98
programs--
about 77 percent of all programs. HHS and Justice administered the
most--59 and 22, respectively--but Department of Labor programs
received the most money--$2.2 billion. About 43 percent of the
funded programs received at least $15 million each. Many programs,
often located in different federal departments and agencies, appear
to fund similar services. For example, in 1996, 47 federal programs
provided substance abuse prevention, 20 provided substance abuse
treatment, and 57 provided violence prevention. Thirteen federal
departments and agencies administered these programs and received
about $2.3 billion.\4 In addition, the same department or agency
administered many programs providing similar services. Justice, for
example, had nine programs providing substance abuse prevention
services to youth in 1996. Furthermore, many individual programs
funded multiple services: about 63 percent of the programs funded
four or more services each in 1996, according to our review.
We also examined programs that provide teacher training. For this
target group, multiple federal programs exist in a number of federal
agencies. For example, the federal government funded at least 86
teacher training programs in fiscal year 1993 in nine federal
agencies and offices. For the 42 programs for which data were
available, Department officials reported that over $280 million was
obligated in fiscal year 1993.\5
Similarly, in fiscal years 1992 and 1993, the government funded over
90 early childhood programs in 11 federal agencies and 20 offices,
according to our review. Our analysis showed that one disadvantaged
child could have possibly been eligible for as many as 13 programs.
Many programs, however, reported serving only a portion of their
target population and maintained long waiting lists.\6
Secretary of Education Riley testified recently before this Task
Force that the Department of Education has made progress in both
eliminating low-priority programs and consolidating similar programs.
He noted, for example, that the reauthorization of the Individuals
With Disabilities Education Act reduced the number of programs from
14 to 6. In addition, the Department has proposed eliminating or
consolidating over 40 programs as part of the reauthorization of the
Higher Education Act.
The multiple education programs scattered throughout the federal
government have created the potential for inefficient service as well
as difficulty for those trying to access the most appropriate
services and funding sources. Federal programs that contribute to
similar results should be closely coordinated to ensure that goals
are consistent and, as appropriate, program efforts are mutually
reinforcing.\7 Uncoordinated program efforts can waste scarce funds,
confuse and frustrate program customers, and limit the overall
effectiveness of the federal effort.\8
--------------------
\3 See At-Risk and Delinquent Youth: Fiscal Year 1996 Programs
(GAO/HEHS-97-211R, Sept. 2, 1997).
\4 This does not include programs in the armed services in the
Department of Defense. The services--
Air Force, Army, Marine Corps, National Guard, and Navy--administered
seven programs totaling $48.8 million. Of those, four programs
provided substance abuse prevention, and all of them provided
violence prevention services. None of the programs provided
substance abuse treatment.
\5 Multiple Teacher Training Programs: Information on Budgets,
Services, and Target Groups (GAO/HEHS-95-71FS, Feb. 22, 1995).
\6 Early Childhood Programs: Multiple Programs and Overlapping
Target Groups (GAO/HEHS-95-4FS, Oct. 31, 1994).
\7 Managing for Results: Using the Results Act to Address Mission
Fragmentation and Program Overlap (GAO/AIMD-97-146, Aug. 29, 1997).
\8 The Government Performance and Results Act: 1997 Governmentwide
Implementation Will Be Uneven (GAO/GGD-97-109, June 2, 1997).
MORE INFORMATION IS NEEDED
ABOUT THE FEDERAL
PRE-KINDERGARTEN-12 EDUCATION
EFFORT AND ITS IMPACT
---------------------------------------------------------- Chapter 0:2
The large numbers of programs and agencies supporting education
activities and target groups make management and evaluation
information critical to the Congress and agency officials.
Information about the federal education effort is needed by many
different decisionmakers, for different reasons, at different times,
and at different levels of detail. Much of that information,
however, is not currently available.
KINDS OF INFORMATION NEEDED
-------------------------------------------------------- Chapter 0:2.1
To efficiently and effectively operate, manage, and oversee programs
and activities, agencies need reliable, timely program performance
and cost information and the analytic capacity to use that
information. For example, agencies need to have reliable data during
their planning efforts to set realistic goals and later, as programs
are being implemented, to gauge their progress toward reaching those
goals. In addition, in combination with an agency's performance
measurement system, a strong program evaluation capacity is needed to
provide feedback on how well an agency's activities and programs
contributed to reaching agency goals. Systematically evaluating a
program's implementation can also provide important information about
the program's success or lack thereof and suggest ways to improve
it.\9
Moreover, to gauge and maximize success of federal efforts,
congressional and agency officials and decisionmakers need many kinds
of information. First, agencies and the Congress need to know
specific models that work and the circumstances in which they are
effective, such as specific classroom activities for preventing
substance abuse. Second, the Congress and agencies also need to know
if the program, such as the activities funded by the Safe and
Drug-Free Schools and Community Act, is working nationwide. Finally,
the Congress needs the ability to look across all programs designed
to help a given target group to assess how the programs are working
together and whether the overall federal effort is accomplishing a
mission such as preventing substance abuse among youths.
In addition, for specific oversight purposes, congressional
decisionmakers sometimes want specific kinds of information. For
example, this Task Force has indicated that two types of information
would be particularly useful to its mission: knowing which federal
education programs target which groups and knowing what characterizes
successful programs.
--------------------
\9 Managing for Results: Building on Agencies' Strategic Plans to
Improve Federal Management (GAO/T-GGD/AIMD-98-29, Oct. 30, 1997).
AREAS IN WHICH SOME
INFORMATION EXISTS
-------------------------------------------------------- Chapter 0:2.2
Some information is available about preK-12 programs that do not
appear to be achieving the desired results and others that appear to
be successful. Secretary Riley, for example, has testified that the
Department will be doing more to disseminate the latest information
on what works in education.
Our clearest evidence about a lack of positive effect from federal
expenditures comes from one of the largest programs: title I. Title
I of the Elementary and Secondary Education Act is the largest
federal elementary and secondary education grant program. It has
received much attention recently because of an Education Department
report showing that, overall, title I programs do not ultimately
reduce the effect of poverty on a student's achievement.\10 For
example, children in high-poverty schools began school academically
behind their peers in low-poverty schools and could not close this
gap as they progressed through school. In addition, when assessed
according to high academic standards, most title I students failed to
exhibit the reading and mathematics skills expected for their
respective grade levels. The study concluded that students in
high-poverty schools were the least able to demonstrate the expected
levels of academic proficiency.
Many of our studies sought to determine what is working and identify
information available or needed to make better determinations about
program results. For example, we have looked at promising practices
or strategies in several areas: school violence,\11
substance abuse prevention,\12 and school-to-work transition.\13 In
1995, we also prepared an overview of successful and unsuccessful
practices in schools and workplaces.\14 Our reviews identified
several important program characteristics: strong program
leadership, linkages between the program and the community, and a
clear and comprehensive approach.
The Department of Education also has contracts for evaluating what
works. For example, the Prospects study--in addition to providing
the data on the overall limited effect of title I--analyzed the five
high-
performing, high-poverty schools in its sample of 400 schools.
Although the number of schools is too small for conclusive
generalizations, the study described the characteristics of these
schools as "food for thought" for future research on successful
programs. These schools had an experienced principal; low teacher
and pupil turnover; an emphasis on schoolwide efforts that seek to
raise the achievement of every student; a greater use of tracking by
student ability; a balanced emphasis on remedial and higher order
thinking in classroom involvement; and higher parent support and
expectations than low-performing, high-poverty schools.
Similarly, the Department's Special Strategies study of 10 promising
alternatives for title I practices, found two--the Success for All
and Comer School Development programs--to be especially successful
approaches. Overall, the study concluded that "some programs, well
implemented, appear to help students make dramatic academic progress;
that pursuing schoolwide change may be well worth the effort; that
intensive early intervention may yet be the best bet; and that after
a third of a century of research on school change, we still have not
provided adequate human and fiscal resources, appropriately targeted,
to make large-scale program improvements a reliably consistent
reality in schools serving students placed at risk."\15 \16
--------------------
\10 Prospects: Final Report on Student Outcomes, Abt Associates,
Inc. (Cambridge, Mass.: Apr. 1997). The study notes, however,
that although it was not able to discern a compensatory effect of
chapter 1, this is not really an indication of program failure.
Without chapter 1 services, students may have fallen further behind.
(The program now called title I was called chapter 1 from the early
1980s through 1994.)
\11 School Safety: Promising Initiatives for Addressing School
Violence (GAO/HEHS-95-106, Apr. 25, 1995).
\12 Drug Control: Observations on Elements of the Federal Drug
Control Strategy (GAO/GGD-97-42, Mar. 14, 1997).
\13 Transition From School to Work: States Are Developing New
Strategies to Prepare Students for Jobs (GAO/HRD-93-139, Sept. 7,
1993).
\14 GAO/PEMD-95-28, Aug. 31, 1995.
\15 Urban and Suburban/Rural Special Strategies for Educating
Disadvantaged Children: Findings and Policy Implications of a
Longitudinal Study, Department of Education (Washington, D.C.: Apr.
1997).
\16 The latter point has recently been reinforced by an analysis of
education productivity. See David W. Grissmer, Education
Productivity, Washington, D.C., Council for Educational Development
and Research (Washington, D.C.: 1997).
SIGNIFICANT INFORMATION GAPS
-------------------------------------------------------- Chapter 0:2.3
Significant information gaps exist, however, about both programs and
their outcomes. Currently, no central source of information exists
about all the programs providing services to the same target groups
among different agencies or about those providing a similar service
to several target groups. Instead, we have had to conduct the
specific analyses previously described for at-risk and delinquent
youth, young children, and teachers--as well as others--to obtain
this information.
Moreover, in our evaluations of specific programs--some of which get
billions of federal dollars each year--the most basic information is
lacking. For example, our study of the Safe and Drug-Free Schools
Program revealed that the program has no centralized information
about what specific services the funds pay for--much less whether the
money is being spent effectively. In our ongoing work on Head Start,
we found that no list of Head Start classrooms and their locations
existed.
A second important theme from our report on successful and
unsuccessful practices was that few evaluations of successful
strategies exist, and many of the existing evaluations lack the
methodological rigor needed to determine effectiveness.\17 We further
documented this theme in our review of Head Start evaluations.\18
Since Head Start's inception in 1965, federal funding for the program
has increased significantly. Since 1990, Head Start funding has more
than doubled--increasing from $1.5 billion in fiscal year 1990 to
almost $4 billion in fiscal year 1997. During this period, Head
Start also received additional federal funds to, among other things,
increase participation and improve program quality. Yet, little
research has focused on program impact, and the body of Head Start
research available is inadequate for use in drawing conclusions about
the impact of the current Head Start program. We do not know what is
working and what is not in today's programs--the early research on
Head Start, conducted over 20 years ago, may no longer apply to
today's program because of program changes and changes in the
population served. We have recommended that HHS include in its
research plan an assessment of the impact of regular Head Start
programs. Although the Department believes that clear evidence
exists of the positive impacts of Head Start services, it does have
plans to evaluate the feasibility of conducting such studies.
More promising, but still incomplete, is the information available
for Safe and Drug-Free Schools programs.\19 Information on
effectiveness and impact has not been collected, although overall
evaluations of the Safe and Drug-Free Schools program have not been
completed. However, Education's evaluative activities focus on
broader aspects of program implementation and not the effectiveness
of all Safe and Drug-Free Schools programs nationwide. Moreover, the
lack of uniform information requirements on program activities and
effectiveness may create a problem for federal oversight.
--------------------
\17 GAO/PEMD-95-28, Aug. 31, 1995.
\18 Head Start: Research Provides Little Information on Impact of
Current Program (GAO/HEHS-97-59, Apr. 15, 1997).
\19 Safe and Drug-Free Schools: Balancing Accountability With State
and Local Flexibility (GAO/HEHS-98-3, Oct. 10, 1997).
CHALLENGES IN OBTAINING
IMPORTANT INFORMATION
---------------------------------------------------------- Chapter 0:3
If (1) process information is critical for program, agency, and
interagency management of federal elementary and secondary programs,
and (2) outcome and impact information is needed to assess results
and focus efforts on what works, why is information not readily
available? The challenges to collecting that information include
-- competing priorities--such as reducing paperwork and regulatory
burden and promoting flexibility in program implementation--that
restrict data collection and evaluation activities;
-- the cost of data collection;
-- the secondary role of education in many programs;
-- the difficulty of obtaining impact evaluation information (under
any circumstances);
-- the special challenge to assessing overall effects on federal
efforts involving multiple federal programs in multiple
agencies; and
-- until recently, a lack of focus on results and accountability.
COMPETING PRIORITIES LIMIT
DATA COLLECTION ACTIVITIES
-------------------------------------------------------- Chapter 0:3.1
The goal of having enough information for accountability and program
management is always competing with the goals of providing local
agencies with the flexibility they need to meet local needs and
lessening regulatory burden. For example, the Paperwork Reduction
Act has given the Office of Management and Budget (OMB) the
responsibility of approving collections of information done by the
federal government, whether through questions, surveys, or studies.
This can limit the burden on state and local governments and others;
however, it can also limit the amount of information collected by the
Department of Education.
Similarly, the challenge of balancing flexibility and accountability
is apparent in efforts to provide certain federal education funds as
block grants. Agencies face the challenge of balancing the
flexibility block grants afford states to set priorities on the basis
of local need with their own need to hold states accountable for
achieving federal goals.\20 For example, the Safe and Drug-Free
Schools program allows a wide range of activities and permits states
to define the information they collect on program activities and
effectiveness. With no requirement that states use consistent
measures, the Department faces a difficult challenge in assembling
the triennial state reports to develop a nationwide picture of the
program's effectiveness.
One promising strategy as an alternative to traditional block grants
is the use of Performance Partnership Grants (PPG). Under PPGs, the
states and the federal government negotiate an arrangement that
identifies specific objectives and performance measures regarding
outcomes and processes. This approach gives the states more control
over their funding decisions, while encouraging them to accept
greater accountability for results.\21
--------------------
\20 For more information on ensuring accountability in block grants,
see Block Grants: Issues in Designing Accountability Provisions
(GAO/AIMD-95-226, Sept. 1, 1995).
\21 See, for example, a description of use of this approach in grants
for substance abuse prevention and treatment services in Substance
Abuse and Mental Health: Reauthorization Issues Facing the Substance
Abuse and Mental Health Services Administration (GAO/T-HEHS-97-135,
May 22, 1997).
INFORMATION IS COSTLY
-------------------------------------------------------- Chapter 0:3.2
Obtaining and analyzing information to manage and evaluate programs
requires significant resources. For example, the Department of
Education's strategic plan cites the need to improve the quality of
performance data on programs and operations and to promote the
integration of federal programs with one another as well as with
state and local programs. Towards this end, in fiscal year 1997, the
Department of Education was appropriated about $400 million for
educational research and improvement. Education estimates an
additional $367 million was obligated by the Department for
information technology for Department operations.
In addition, evaluation research is costly. For example, in fiscal
year 1993, the Department awarded 38 contracts totaling more than $20
million for evaluating elementary and secondary education programs.
Contract amounts ranged from $38,000 to fund a program improvement
conference to $6.8 million for implementing the chapter 1
longitudinal study (Prospects). But this accounted for only 1 year
of this multiyear study: this longitudinal study to assess the
impact of significant participation in title I programs on student
and young adult outcomes cost about $25 million over a 4-year
period.\22 The median cost for an evaluation contract was about
$180,000 in fiscal year 1993.
In our testimony last spring on challenges facing the Department of
Education, we noted that the Department needed more information to
determine how its programs are working and that additional
departmental resources may be needed to manage funds and provide
information and technical assistance.\23 For example, title I is
intended to promote access to and equity in education for low-income
students. The Congress modified the program in 1994, strengthening
its accountability provisions and encouraging the concentration of
funds to serve more disadvantaged children. At this time, however,
the Department does not have the information it needs to determine
whether the funding is being targeted as intended. Although the
Department has asked for $10 million in its fiscal year 1998 budget
request to evaluate the impact of title I, it has only just begun a
small study of selected school districts to examine targeting to
identify any necessary mid-course modifications. The ultimate impact
of the 1994 program modifications could be diminished if the funding
changes are not implemented as intended.
--------------------
\22 Biennial Evaluation Report, Fiscal Years 1993-1994, Department of
Education (Washington D.C.: 1995).
\23 Department of Education: Challenges in Promoting Access and
Excellence in Education (GAO/T-HEHS-97-99, Mar. 20, 1997).
MANY PROGRAMS INVOLVING
EDUCATION HAVE OTHER PRIMARY
PURPOSES
-------------------------------------------------------- Chapter 0:3.3
Many federal programs involving education have other primary
purposes. For example, the Department of Agriculture's child
nutrition program provides school breakfast and school lunch
programs. The Head Start program also emphasizes health and
nutrition as well as parenting skills; cognitive development is only
one of six program goals. In addition, Safe and Drug-Free Schools
Act money can be used to provide comprehensive health education,
whose major goals and objectives are broader than just drug and
violence prevention.
IMPACT EVALUATION
INFORMATION IS DIFFICULT TO
OBTAIN
-------------------------------------------------------- Chapter 0:3.4
Good evaluative information about program effects is difficult to
obtain. Each of the tasks involved--measuring outcomes, ensuring the
consistency and quality of data collected at various sites,
establishing the causal connection between outcomes and program
activities, and distinguishing the influence of extraneous
factors--raises formidable technical or logistical problems. Thus,
evaluating program impact generally requires a planned study and,
often, considerable time and expense. Program features affect the
relative difficulty of getting reliable impact information. The more
varied the program activities and the less direct the connection
between the provider and the federal agency, the greater the
difficulty of getting comparable, reliable data on clients and
services. For example, a federal agency whose own employees deliver
a specified service can probably obtain impact data more easily than
one that administers grants that states then pass on to several local
entities to be used different ways. Also, due to the absence of
contrasting comparison groups, it is extremely difficult to estimate
the impact of a long-standing program that covers all eligible
participants.\24
--------------------
\24 Program Evaluation: Improving the Flow of Information to the
Congress (GAO/PEMD-95-1, Jan. 30, 1995).
MULTIPLE FEDERAL PROGRAMS
MANAGED BY DIFFERENT
AGENCIES
-------------------------------------------------------- Chapter 0:3.5
The sheer number of departments and agencies that spend federal
education dollars makes it hard to aggregate existing information
among federal programs for certain issues or target groups. Each
program may have its own measures on the federal, state, and local
levels. Even for a single program, each state may use different
measures (as mentioned earlier regarding the Safe and Drug-Free
Schools and Communities Act programs), creating difficult challenges
to developing a nationwide picture of the program's effectiveness.
Yet this is just 1 of the 127 programs administered by 15 agencies
that target at-risk and delinquent youth. If the Congress wanted to
know the overall effectiveness of the federal effort in helping
at-risk and delinquent youth, the task would be even more daunting
than that the Department of Education faces in developing a
nationwide picture of one flexibly administered program.
PAST LACK OF EMPHASIS ON
RESULTS AND ACCOUNTABILITY
-------------------------------------------------------- Chapter 0:3.6
Federally funded programs have historically placed a low priority on
results and accountability. Therefore, until recently, the statutory
framework has not been in place to bring a more disciplined approach
to federal management and to provide the Congress and agency
decisionmakers with vital information for assessing the performance
and costs of federal programs.
In recent years, however, governments around the world, including
ours, have faced a citizenry that is demanding that governments
become more effective and less costly.\25 These two demands are
driving the move to a performance-based approach to managing
public-sector organizations.
GPRA is the centerpiece of a statutory framework provided by recent
legislation to bring needed discipline to federal agencies'
management activities. Other elements are the expanded Chief
Financial Officers Act, the Paperwork Reduction Act of 1995, and the
Clinger-Cohen Act of 1996. These laws each responded to a need for
accurate, reliable information for executive branch and congressional
decision-making. In combination, they provide a framework for
developing (1) fully integrated information about an agency's mission
and strategic priorities, (2) performance data for evaluating the
achievement of these goals, (3) the relationship of information
technology investments to meeting performance goals, and (4) accurate
and audited financial information about the costs of meeting the
goals.
GPRA requires that agencies clearly define their missions, establish
long-term strategic goals as well as annual goals linked to them,
measure their performance according to the goals they have set, and
report on their progress. In addition to ongoing performance
monitoring, agencies are also expected to perform discrete
evaluations of their programs and to use information obtained from
these evaluations to improve their programs. Agencies are also
expected to closely coordinate with other federal agencies whose
programs contribute to similar results to ensure that goals are
consistent and, as appropriate, that program efforts are mutually
reinforcing. Each agency was required to submit to OMB and the
Congress a strategic plan explaining its mission, long-term goals,
and strategies for meeting these goals by September 30, 1997, and the
Department of Education did so.
Beginning in fiscal year 1999, agencies are to use their strategic
plans to prepare annual performance plans. These performance plans
are to include annual goals linked to the activities noted in their
budget presentations as well as the indicators the agency will use to
measure performance according to results-oriented goals. Agencies
are subsequently to report each year on the extent to which goals are
met, explain why these goals are not met, if needed, and discuss the
actions needed to meet any unmet goals. In addition, by early 1998,
OMB must submit to the Congress governmentwide performance plans
based on agencies' plans as part of the president's fiscal 1999
budget.
For federal education programs, this shift to a focus on results can
help inform decisionmakers about effective program models and the
actual activities and characteristics of individual federal programs.
GPRA provides an incentive for agency and program personnel to
systematically assess their programs and identify and adapt
successful practices of similar programs. The act also provides an
early warning system for identifying goals and objectives that are
not being met so that agency and program staff can replace
ineffective practices with effective ones.
The act's emphasis on coordination among similar programs and linking
results to funding also provides a way to better understand the
overall effect of federal activities and to identify programs that
might be abolished, expanded, or consolidated with others. If
agencies and OMB use the annual planning process to highlight
crosscutting program issues, the individual agency performance plans
and the governmentwide performance plan should provide the Congress
with the information needed to identify agencies and programs
addressing similar missions. Once these programs are identified, the
Congress can consider the associated policy, management, and
performance implications of crosscutting program issues. This
information should also help identify the performance and cost
consequences of program fragmentation and the implications of
alternative policy and service delivery options. These options, in
turn, can lead to decisions about department and agency missions and
allocating resources among those missions.\26
Achieving the full potential of GPRA is a particularly difficult
challenge because of the multiple programs and many departments
involved in the federal effort to improve public K-12 education.
Meanwhile, this challenge--combined with the current limited data
available about the programs and their effectiveness--is precisely
why GPRA is needed. It is also why we believe it holds promise to
help improve the information available to decisionmakers and, thus,
the federal effort in this important area.
--------------------
\25 See, for example, Managing for Results: Experiences Abroad
Suggest Insights for Federal Management Reform (GAO/GGD-95-120, May
2, 1995); Managing for Results: State Experiences Provide Insights
for Federal Management Reforms (GAO/GGD-95-22, Dec. 21, 1994); and
Government Reform: Goal-Setting and Performance
(GAO/AIMD/GGD-95-130R, Mar. 27, 1995).
\26 GAO/T-GGD/AIMD-98-29, Oct. 30, 1997.
-------------------------------------------------------- Chapter 0:3.7
Mr. Chairman, this concludes my prepared statement. I would be
pleased to respond to any questions you or members of the Task Force
may have.
RELATED GAO PRODUCTS
=========================================================== Appendix 1
Managing for Results: Building on Agencies' Strategic Plans to
Improve Federal Management (GAO/T-GGD/AIMD-98-29, Oct. 30, 1997).
Safe and Drug-Free Schools: Balancing Accountability With State and
Local Flexibility (GAO/HEHS-98-3, Oct. 10, 1997).
Education Programs: Information on Major Preschool, Elementary, and
Secondary Education Programs (GAO/HEHS-97-210R, Sept. 15, 1997).
Education Programs: Information on Major Postsecondary Education,
School-to-Work, and Youth Employment Programs (GAO/HEHS-97-212R,
Sept. 15, 1997).
At-Risk and Delinquent Youth: Fiscal Year 1996 Programs
(GAO/HEHS-97-211R, Sept. 2, 1997).
Managing for Results: Using the Results Act to Address Mission
Fragmentation and Program Overlap (GAO/AIMD-97-146, Aug. 29, 1997).
Substance Abuse and Violence Prevention: Multiple Youth Programs
Raise Questions of Efficiency and Effectiveness (GAO/T-HEHS-97-166,
June 24, 1997).
The Government Performance and Results Act: 1997 Governmentwide
Implementation Will Be Uneven (GAO/GGD-97-109, June 2, 1997).
Head Start: Research Provides Little Information on Impact of
Current Program (GAO/HEHS-97-59, Apr. 15, 1997).
Department of Education: Challenges in Promoting Access and
Excellence in Education (GAO/T-HEHS-97-99, Mar. 20, 1997).
Schools and Workplaces: An Overview of Successful and Unsuccessful
Practices (GAO/PEMD-95-28, Aug. 31, 1995).
Block Grants: Issues in Designing Accountability Provisions
(GAO/AIMD-95-226, Sept. 1, 1995).
Multiple Teacher Training Programs: Information on Budgets,
Services, and Target Groups (GAO/HEHS-95-71FS, Feb. 22, 1995).
*** End of document. ***