Welfare Reform: Assessing the Effectiveness of Various Welfare-to-Work
Approaches (Letter Report, 09/07/1999, GAO/HEHS-99-179).
Pursuant to a congressional request, GAO reviewed research on the
effectiveness of various welfare-to-work approaches, focusing on: (1)
the effectiveness of a rapid employment approach, an education-based
approach, and a combination of these two approaches in improving
employment-related outcomes for welfare recipients and other low-income
women with children; and (2) the effect of welfare recipients'
educational attainment, including postsecondary education, on the
educational attainment of their children.
GAO noted that: (1) research conducted to date on the effectiveness of
different welfare-to-work approaches suggests that programs with a
combined approach--including both job search assistance and some
education and training--tend to be more effective than either approach
alone in increasing employment and earnings while reducing welfare
payments; (2) five evaluations begun in the 1980s with 5-year results
indicated that programs focusing on rapid employment and job search
activities combined with eduction and training activities more often
increased employment and earnings and reduced welfare payments, compared
with programs that focused solely on job search activities or those that
placed the greatest emphasis on education; (3) in addition, early
results from a more recent ongoing evaluation--the only evaluation
designed explicitly to compare the effectiveness of a rapid-employment
approach with an education-based welfare-to-work approach--found that
while each approach has increased participants' employment and earnings,
neither approach has proven clearly better than the other; (4) the rapid
employment approach did cost about half as much per person as the
education-based approach; (5) while these studies provide useful
information, more needs to be known about how well different approaches
are performing in the environment created by the enactment of welfare
reform in 1996, which none of these evaluations cover; (6) while
research indicates that parents' educational attainment has a positive
effect on children's educational attainment, little information is
available on this relationship specifically within the welfare
population; (7) recent studies have identified factors affecting
cognitive development of children in welfare families; (8) this
research, while limited in scope, indicates that a mother's higher level
of educational attainment is one factor that may positively affect
children's development; and (9) in addition, a body of research that
focuses on the effects of poverty on children's educational attainment
suggests a significant positive relationship between the educational
attainment of parents and their children among both the welfare and the
nonwelfare populations.
--------------------------- Indexing Terms -----------------------------
REPORTNUM: HEHS-99-179
TITLE: Welfare Reform: Assessing the Effectiveness of Various
Welfare-to-Work Approaches
DATE: 09/07/1999
SUBJECT: Workfare
Employment or training programs
Disadvantaged persons
State-administered programs
Comparative analysis
Children
Single parents
Welfare benefits
Public assistance programs
IDENTIFIER: AFDC Work Incentive Program
Job Opportunities and Basic Skills Training Program
HHS Temporary Assistance for Needy Families Program
Aid to Families with Dependent Children Program
AFDC
National Evaluation of Welfare-to-Work Strategies
******************************************************************
** This file contains an ASCII representation of the text of a **
** GAO report. Delineations within the text indicating chapter **
** titles, headings, and bullets are preserved. Major **
** divisions and subdivisions of the text, such as Chapters, **
** Sections, and Appendixes, are identified by double and **
** single lines. The numbers on the right end of these lines **
** indicate the position of each of the subsections in the **
** document outline. These numbers do NOT correspond with the **
** page numbers of the printed product. **
** **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced. Tables are included, but **
** may not resemble those in the printed version. **
** **
** Please see the PDF (Portable Document Format) file, when **
** available, for a complete electronic file of the printed **
** document's contents. **
** **
** A printed copy of this report may be obtained from the GAO **
** Document Distribution Center. For further details, please **
** send an e-mail message to: **
** **
** **
** **
** with the message 'info' in the body. **
******************************************************************
Cover
================================================================ COVER
Report to Congressional Committees
September 1999
WELFARE REFORM - ASSESSING THE
EFFECTIVENESS OF VARIOUS
WELFARE-TO-WORK APPROACHES
GAO/HEHS-99-179
Welfare-to-Work Approaches
(205393)
Abbreviations
=============================================================== ABBREV
AFDC - Aid to Families With Dependent Children
GAIN - Greater Avenues for Independence
HHS - Department of Health and Human Services
JOBS - Job Opportunities and Basic Skills Training
MDRC - Manpower Demonstration Research Corporation
SWIM - Saturation Work Initiative Model
TANF - Temporary Assistance for Needy Families
WIN - Work Incentive
Letter
=============================================================== LETTER
B-282174
September 9, 1999
Congressional Committees
The Personal Responsibility and Work Opportunity Reconciliation Act
of 1996 (P.L. 104-193), enacted in August 1996, significantly
changed the nation's cash assistance program for needy families with
children. Title I of the law replaced the Aid to Families With
Dependent Children (AFDC) cash assistance program with fixed block
grants to states to provide Temporary Assistance for Needy Families
(TANF) and ended families' entitlement to assistance. Several goals
of the TANF program are specified in the law, including that of
ending welfare dependence by promoting work over welfare and
self-reliance over dependency. Over the years, states'
welfare-to-work programs have emphasized different goals and
philosophies for moving individuals into work and have provided
different types of services and activities to program participants to
help them reach those goals. Programs with the goal of rapid
employment emphasize quick exposure to and entry into the labor
force, reflecting the belief that participants can best acquire
employment-related skills when they are working, regardless of the
quality of the job. These programs' service strategies tend to rely
heavily on job search activities but can make use of education and
training to some extent. Other programs have the goal of skill
building, often called an education-based approach, which usually
involves a greater initial investment in participants' education and
occupational skills, so that when they do enter the labor market,
they can obtain �good� jobs�those with higher pay, health benefits,
and opportunity for advancement. The 1996 welfare reform law
emphasizes the importance of moving welfare recipients into
employment and gives states greater flexibility to tailor their
programs to meet their own goals and needs.
To help it assess how best to assist welfare recipients, the Congress
directed us in the Higher Education Amendments of 1998 (P.L.
105-244) to review research on the effectiveness of various
welfare-to-work approaches. Specifically, we were asked to examine
the research findings on (1) the effectiveness of a rapid employment
approach, an education-based approach (including adult vocational and
postsecondary education), and a combination of these two approaches
in improving employment-related outcomes for welfare recipients and
other low-income women with children and (2) the effect of welfare
recipients' educational attainment, including postsecondary
education, on the educational attainment of their children. In
performing this work, we reviewed studies that had been published
from 1988 through mid-1999 that assessed the effectiveness of
welfare-to-work approaches by comparing outcomes related to
employment and earnings as well as declines in welfare payments for
those in the programs with those not in the programs. Because some
welfare-to-work programs produce results in the longer term rather
than in the short term, we focused on evaluations having relatively
long-term results (5 years). In addition, because many of the
welfare recipients who participated in the welfare-to-work programs
evaluated had less than a high school education, the education-based
approaches evaluated in the studies we assessed primarily provided
basic education services rather than postsecondary education. As a
result, none of the studies evaluated the effectiveness of a college
education on improving employment-related outcomes for welfare
recipients. We also identified and reviewed studies about the effect
of welfare recipients' educational attainment on their children.
(See app. I for a full discussion of our scope and methodology.)
RESULTS IN BRIEF
------------------------------------------------------------ Letter :1
Research conducted to date on the effectiveness of different
welfare-to-work approaches suggests that programs with a combined
approach--including both job search assistance and some education and
training--tend to be more effective over a 5-year period than either
approach alone in increasing employment and earnings while reducing
welfare payments. Five evaluations begun in the 1980s with 5-year
results indicated that programs focusing on rapid employment and job
search activities combined with education and training activities
more often increased employment and earnings and reduced welfare
payments, compared with programs that focused solely on job search
activities or those that placed the greatest emphasis on education.
In addition, preliminary results (2-year findings) from a more recent
ongoing evaluation (started in 1992)--the only evaluation designed
explicitly to compare the effectiveness of a rapid-employment
approach with an education-based welfare-to-work approach--found that
while each approach has increased participants' employment and
earnings, so far, neither approach has proven clearly better than the
other. The rapid employment approach did, however, cost about half
as much per person as the education-based approach. While these
studies provide useful information, more needs to be known about how
well different approaches are performing in the current environment
created by the enactment of welfare reform in 1996, which none of
these evaluations cover.
While research indicates that parents' educational attainment has a
positive effect on children's educational attainment, little
information is currently available on this relationship specifically
within the welfare population. Recent studies have identified
factors affecting cognitive development of children in welfare
families. This research, while limited in scope, indicates that a
mother's higher level of educational attainment is one factor that
may positively affect children's development. In addition, a body of
research that focuses on the effects of poverty on children's
educational attainment suggests a significant positive relationship
between the educational attainment of parents and their children
among both the welfare and the nonwelfare populations.
BACKGROUND
------------------------------------------------------------ Letter :2
AFDC was created by the Social Security Act of 1935 to provide cash
assistance to families with needy children who had been deprived of
the support of one of their parents--at that time, mostly children
living with widowed mothers. The program was not designed to promote
employment, because, at the time, mothers were generally not expected
to work outside the home. Over the past several decades, however,
the public has come to believe that most welfare families should be
at least partly self-supporting. Efforts to provide education,
training, and job search assistance to help welfare recipients
prepare for and find jobs can be traced back at least to the 1960s,
when the Congress mandated that every state operate a Work Incentive
(WIN) program to encourage AFDC recipients to become self-sufficient.
WIN began primarily as a voluntary program focusing on job search
assistance and immediate employment. Starting in 1981, WIN
demonstration projects were established that gave states greater
flexibility to design their own programs, and states could now
require welfare recipients with children aged 6 and older to
participate. Our reviews of the WIN program showed that it often
served those most likely to find employment on their own rather than
those less job-ready, who needed the most help to become employed.
The Family Support Act of 1988 eliminated the WIN program and created
the Job Opportunities and Basic Skills Training (JOBS) program.
Research conducted before passage of that act showed that welfare
recipients were a diverse group, making use of the AFDC program in
different ways. While most who used AFDC did so for short periods of
time, the majority of AFDC's resources were devoted to providing
benefits to long-term recipients. This research also identified
several factors that were associated with long-term welfare
dependence, including recipients' low level of education,
single-parent status, higher number of children, disability, and
limited work experience.\1 To better ensure that AFDC recipients
received assistance that would help them avoid long-term welfare
dependence, JOBS required state programs to include a broad range of
services, including education and training assistance,\2 and to
provide financial assistance with support services such as child care
and transportation. The population that could be required to
participate was changed from those with children aged 6 and above to
those with children aged 3 and above. In addition, for the first
time, states were required to place a specified minimum percentage of
nonexempt welfare recipients in education, training, and work-related
activities and to target resources to long-term recipients and those
considered at risk of long-term welfare dependence.
The Family Support Act also called for an evaluation with a random
assignment design, which would control for other factors that could
affect outcomes, to assess the effectiveness of various
welfare-to-work programs. The Department of Health and Human
Services (HHS), with support from the Department of Education,
contracted with the Manpower Demonstration Research Corporation
(MDRC), a research organization that analyzes education- and
employment-related programs, to conduct this evaluation, which
focused on mandatory welfare-to-work programs at seven sites. This
evaluation is referred to as the National Evaluation of
Welfare-to-Work Strategies, formerly known as the JOBS Evaluation.
In the mid-1980s, California also contracted with MDRC to conduct an
evaluation of its state welfare-to-work program, called Greater
Avenues for Independence (GAIN), which became the state's JOBS
program after 1988. At the beginning of the GAIN evaluation,
California had about one-sixth of the nation's AFDC caseload, and
GAIN expended about 13 percent of federal JOBS funds.
In response to JOBS' increased emphasis on education and training and
a general belief that these activities could help improve welfare
recipients' financial well-being, as they do for the general
population, many state and local JOBS programs emphasized the
provision of education and training. In our 1991 report on the
implementation of JOBS, we found that almost half of the states
reported a shift from an emphasis on immediate job placement under
their previous welfare-to-work programs toward a new emphasis on
long-term education or training.\3 In 1995, we reported that JOBS
participants nationwide were enrolled in a variety of education and
training activities�postsecondary education (17 percent), high school
or preparation for the general equivalency diploma (16 percent), job
skills training (13 percent), adult basic or remedial education (7
percent), and English as a Second Language training (2 percent).\4
Several years after the implementation of JOBS, our review of state
JOBS programs nationwide showed that only about 11 percent of welfare
recipients were involved in JOBS activities each month.\5 In
addition, AFDC caseloads rose to their highest levels ever, peaking
at 5 million families in 1994. Also during the 1990s, under waivers
of the federal rules, several states experimented with changes in
their AFDC and JOBS programs. These changes included encouraging
welfare recipients to work by allowing them to keep more of their
earnings without losing welfare benefits, strengthening and more
strongly enforcing work requirements, and imposing limits on the
length of time a family could receive aid.
To encourage and facilitate innovation by the states and to address
continuing concerns among policymakers about growing welfare
dependence, the Personal Responsibility and Work Opportunity
Reconciliation Act of 1996, commonly referred to as welfare reform
legislation, overhauled the nation's welfare system by abolishing the
AFDC program and establishing TANF block grants. Under TANF, which
is administered by HHS, states are provided up to $16.8 billion each
year through 2002 to provide aid to needy families with children.
While the states have great flexibility to design programs that meet
their own goals and needs, they must also meet several federal
requirements designed to emphasize the importance of work and the
temporary nature of TANF aid. TANF established stronger work
requirements for those receiving aid than the requirements of its
predecessor program, and the population that can be required to work
now includes all parents, regardless of the ages of their children.\6
In addition, states must enforce a 5-year limit (less at state
option) on the length of time a family may receive federal TANF
assistance.\7
Our June 1998 report and other studies of TANF implementation show
that many states and localities have taken steps to transform their
welfare offices into job placement centers and are encouraging or
requiring those seeking aid to engage in job search activities as
soon as they apply.\8 Along with this increased emphasis on work,
welfare offices and workers are also focusing more on helping clients
address and solve problems that interfere with employment. States'
implementation of more work-focused programs, undertaken under
conditions of strong economic growth, has been accompanied by a
45-percent decline in the number of families receiving welfare�from a
high of about 5 million families in 1994 to 2.7 million families as
of December 1998.\9 A nationally representative survey of families
who left welfare from 1995 to 1997 found that 61 percent of former
welfare recipients were working at the time of the survey, although
often at low-paying jobs.\10
--------------------
\1 David Ellwood, Targeting "Would-Be" Long-Term Recipients of AFDC
(Princeton, N.J.: Mathematica Policy Research, Inc., 1986).
\2 Under JOBS, states were to assess the needs and skills of welfare
recipients, prepare them for employment through education and
training as needed, and place them in jobs. Federal rules specified
certain activities that each state's JOBS program was required to
offer. These included education activities, job skills training,
job-readiness activities, and job development and placement services.
States also had to offer at least two of the four WIN activities (job
search, on-the-job training, work supplementation programs, and
community work experience programs). Postsecondary education was
optional under the federal JOBS rules, and states could assign
participants to this activity on an individual basis.
\3 Welfare to Work: States Begin JOBS, but Fiscal and Other Problems
May Impede Their Progress (GAO/HRD-91-106, Sept. 27, 1991).
\4 Welfare to Work: Participants' Characteristics and Services
Provided in JOBS (GAO/HEHS-95-93, May 2, 1995).
\5 Welfare to Work: Most AFDC Training Programs Not Emphasizing Job
Placement (GAO/HEHS-95-113, May 19, 1995).
\6 The required minimum participation rate began at 25 percent in
fiscal year 1997 and rises to 50 percent in fiscal year 2002. States
receive credit for the degree to which their caseloads have declined
since fiscal year 1996. While states have considerable flexibility
in designing their welfare-to-work programs, the legislation
prescribes the activities that states may count toward their work
participation rate. For example, vocational education is limited as
a countable work activity both in the percentage of recipients who
can be engaged in vocational training and count toward the
participation rate (30 percent) and the length of time a recipient
can be in vocational training (up to 12 months). Moreover, unless
states include it in their definition of vocational education
training, the legislation does not allow postsecondary education to
be counted as a work activity toward the states' participation rate.
\7 States may exempt from time limits up to 20 percent of those
receiving TANF aid and may use their own funds to provide aid beyond
the federal time limit.
\8 Welfare Reform: States Are Restructuring Programs to Reduce
Welfare Dependence (GAO/HEHS-98-109, June 17, 1998).
\9 Welfare Reform: States' Implementation Progress and Information
on Former Recipients (GAO/T-HEHS-99-116, May 27, 1999).
\10 Pamela Loprest, "Families Who Left Welfare: Who Are They and How
Are They Doing?" Urban Institute Discussion Paper 99-02 (July 1999).
See also Welfare Reform: Information on Former Recipients' Status
(GAO-HEHS-99-48, Apr. 28, 1999).
RESEARCH SHOWS THAT A VARIETY
OF WELFARE-TO-WORK APPROACHES
HAVE POSITIVE
EMPLOYMENT-RELATED OUTCOMES
------------------------------------------------------------ Letter :3
The current research on the relative merits of the rapid employment
and education-based approaches does not conclusively show that one
approach is more effective than the other in increasing welfare
recipients' employment and earnings and reducing their welfare
payments. Of the six evaluations of different approaches we
identified and reviewed, five evaluations, with 5-year results,
covered a range of programs, with some combining elements of a rapid
employment approach with an education-based approach. The results of
these evaluations were mixed and, while not conclusive, indicated
that programs that combined the approaches had more positive
effects--and that these effects covered a broader cross section of
the welfare population--than programs that focused more exclusively
on providing only job search activities or only education. Only one
evaluation, part of the National Evaluation of Welfare-to-Work
Strategies, directly compared the results of a rapid employment
approach with those of an education-based approach. The results for
the first 2 years of this ongoing study showed that the outcomes were
roughly comparable for the two approaches, with both modestly
increasing participants' employment and earnings and reducing welfare
payments. The rapid employment approach was only about half as
costly per participant as the education approach. None of the
studies evaluated the effectiveness of a college education in
improving employment-related outcomes for welfare recipients. All of
the evaluations we reviewed provided information on welfare-to-work
programs operated in the 1980s and 1990s; none included results on
programs operated since welfare reform was enacted in 1996.
Currently, HHS is funding 23 studies in 20 states on welfare reforms
that began under waivers of the AFDC program but that are continuing
in the new welfare environment. These studies will provide more
information on effective approaches for moving welfare recipients
into work.\11
--------------------
\11 For more information, see Web sites
http://www.acf.dhhs.gov/programs/opre/rd&e.htm and
http://aspe.os.dhhs.gov/hsp/hspwelfare.htm
LITTLE RESEARCH IS AVAILABLE
THAT COMPARES THE
EFFECTIVENESS OF VARIOUS
APPROACHES
---------------------------------------------------------- Letter :3.1
We reviewed five evaluations conducted by MDRC that focused on
mandatory welfare-to-work programs and for which 5 years of follow-up
data were available. Because the results of the rapid employment and
education-based approaches are expected to unfold in different ways,
with more immediate results from the rapid employment approach and
more delayed results from the education-based approach, we focused
only on those evaluations with results from 5 years of follow-up.
(Other studies we identified that did not include 5-year results are
listed in the bibliography.) Four evaluations were begun in the early
1980s and covered a variety of programs across the country; the fifth
evaluation, begun in the late 1980s, included six sites in
California. As shown in table 1, these programs generally used some
combination of the rapid employment and education-based approaches,
although each tended to emphasize one approach more than the other.
By randomly assigning welfare recipients to different groups--program
participants and nonparticipants--evaluators can determine which
changes in people's employment, earnings, and welfare payments were
due to their participation in the program. This random assignment
method cannot tell analysts which particular aspects of the program
caused the changes or definitively show that the program effects were
caused by the particular approach used, rather than other program
features. But such an evaluation can determine whether the way a
program was operated at a particular site was effective.
While evaluations of welfare-to-work programs have been conducted
over the years, they generally have not been designed to determine
the effectiveness of one particular approach compared with another.
As part of the National Evaluation of Welfare-to-Work Strategies,
formerly the JOBS Evaluation, required by the Family Support Act of
1988, MDRC started a largely unprecedented effort to compare the
effect of two distinct types of welfare-to-work approaches--the rapid
employment approach versus the education-based approach.\12 Under the
rapid employment approach, it is expected that individuals will move
quickly into employment with immediate payoffs in increased earnings
and welfare savings and the potential to earn more over time. Under
the education-based approach, an initial investment in education and
training is expected to pay off in the future, with increased
earnings and welfare savings once the training is completed. This
study, the sixth MDRC study that we reviewed, will provide up to 5
years of follow-up data and will analyze the effects of each approach
on a wider array of outcomes, including those for children, in the
future. Only the results from the first 2 years of this ongoing
comparison study are currently available.
Table 1
Evaluation Studies That Assessed the
Effectiveness of Welfare-to-Work
Programs
Number of
programs with
Start specific Basic program
Evaluation date\a findings approach
-------------- ---------- -------------- --------------
WIN-era programs
----------------------------------------------------------
Baltimore 1982 Combined Mixed service
Options results for 10 strategy
Program of 18 city
welfare
offices
Arkansas WORK 1983 Combined Rapid
Program results for 2 employment,
counties with a primary
focus on job
search
activities
Virginia 1983 Combined Rapid
Employment results for 11 employment,
Services country with a primary
Program welfare focus on job
agencies search
activities
San Diego 1985 Combined Mixed service
Saturation results for 2 strategy
Work of 7 city
Initiative welfare
Model (SWIM) offices
JOBS-era program
----------------------------------------------------------
California's 1988 Separate Mixed service
Greater results for strategies
Avenues for six counties: that emphasize
Independence Alameda; rapid
(GAIN) Program Butte; employment or
Los Angeles; an education-
Riverside; San based approach
Diego; and to different
Tulare degrees
National 1992 Separate Comparison of
Evaluation of results for rapid
Welfare-to- three sites: employment and
Work Atlanta, Ga.; education-
Strategies: Grand Rapids, based
Evaluation of Mich.; approaches
Two Welfare- Riverside,
to-Work Calif.
Program
Approaches
----------------------------------------------------------
\a All programs were evaluated for 5 years except for the JOBS-era
evaluation of two welfare-to-work approaches�2-year results are
available for this ongoing comparison study.
The MDRC comparison study of two welfare-to-work approaches was
conducted at three sites�Atlanta, Georgia; Grand Rapids, Michigan;
and Riverside, California. At each site, welfare recipients were
randomly assigned to the rapid employment program, the
education-based program, or a control group. The study compared
outcomes for those assigned to the program groups with outcomes for
those assigned to the control group, and compared outcomes for those
assigned to one program with outcomes for those assigned to the other
program. Operating the two programs at the three sites
simultaneously controlled for the economic and programmatic
environments that could affect participants' outcomes. Randomly
assigning recipients to one of the two programs or a control group
helped to eliminate any bias from, or effect of, differences in
recipients' characteristics that could affect outcomes. Differences
in participant outcomes could then be attributed to the program in
which they participated. In this comparison study and in other
evaluations we reviewed, not all individuals assigned to a
welfare-to-work program group necessarily participated in program
activities; also, some individuals not assigned to a program group
sought out and received services at their own initiative. In this
report, hereafter, we use the term "participants" to refer to
individuals assigned to a welfare-to-work program group and the term
"nonparticipants" to refer to those assigned to a control group.
--------------------
\12 This comparison evaluation is part of the National Evaluation of
Welfare-to-Work Strategies, which also includes other components,
such as a study, focusing on children who were between the ages of 3
and 5 when their mothers entered the study, to measure outcomes on
children's cognitive development and academic achievement, safety and
health, problem behavior and emotional well-being, and social
development. In the national evaluation, over 55,000 individuals at
seven sites were randomly assigned to groups that remained eligible
for specific welfare-to-work programs or to groups that did not
participate in these programs. In addition to the 2-year findings
reported in a study of three sites, 2-year impacts are available for
one site in Portland, Oreg. Also, MDRC has completed evaluations at
the remaining three sites in Columbus, Ohio; Detroit, Mich.; and
Oklahoma City, Okla.; however, these studies had not been published
at the time of our review.
EVALUATIONS SHOWED THAT
PROGRAMS THAT COMBINE
APPROACHES MORE OFTEN HAD
POSITIVE EFFECTS OVER 5
YEARS THAN THOSE THAT
EMPHASIZED ONE APPROACH
---------------------------------------------------------- Letter :3.2
Although they were not designed to assess the effect of one approach
compared with another, the four WIN-era and one JOBS-era evaluations
provide insights into the effectiveness of various approaches. The
five evaluations covered 10 programs--seven in California--with
varying approaches for moving welfare recipients into work, and had
mixed results. Some programs had effects in all three
areas--employment, earnings, and welfare savings. In some cases, a
program increased earnings for those considered less job-ready but
did not increase earnings for those considered job-ready. Although
not definitive, the studies show that programs combining elements of
both the rapid employment and education-based approaches--having a
rapid employment focus but relying on education and training for some
participants--tended to have a greater effect on participants'
employment and earnings, and on welfare savings, than approaches that
emphasized just job search activities or just longer-term education
and training.
FIVE-YEAR EVALUATIONS OF
FOUR WIN-ERA PROGRAMS
-------------------------------------------------------- Letter :3.2.1
A study of the 5-year results of four programs operated during the
WIN era offers a look at two programs that relied primarily on job
search activities to move people into jobs in comparison with two
programs that provided education and training in addition to job
search activities (see table 2). The Arkansas and Virginia programs
are generally considered examples of programs focusing on job search
only that were relatively low cost and provided little in the way of
education and training, although some Virginia program participants
did participate in education and training opportunities but
participated in such activities only slightly more than the control
group. The Baltimore Options program featured an initial assessment
of each participant's needs and then provided participants a choice
of activities. SWIM, in San Diego, was designed to involve as many
welfare recipients as possible in ongoing activities, following a set
sequence of activities: job search, work experience, and then
education or training.
Table 2
Emphasis and Other Features of Four WIN-
Era Programs
Percentage
of
participants
Net who were
Program emphasis program ever in
Evaluated and service cost per education
program strategy person\a and training
------------ ------------------ ---------- ------------
Arkansas Rapid employment $220 0
WORK with job search
first and 3-
month unpaid work
assignments
Virginia Rapid employment 598 12
Employment with job search
Services and some 3-month
unpaid work
assignments
Baltimore After initial 1,325 17
Options assessment,
participants are
given some choice
among job search,
unpaid work
assignments, or
education and
training
SWIM Involve as many 1,212 24
participants as
possible in
activities for as
long as possible,
starting with job
search, then 3-
month unpaid work
assignment, then
some education
and training
----------------------------------------------------------
\a Represents the average cost per participant less the average cost
per nonparticipant, adjusted to 1993 dollars. Note that net program
costs per person are lower for these WIN-era programs compared with
the JOBS-era programs because welfare recipients with younger
children were not required to participate in WIN-era programs.
Consequently, child care costs were lower.
The SWIM and Baltimore Options programs significantly increased total
earnings for participants over the 5-year study period, while the
earnings increases for the Arkansas and Virginia programs were not
statistically significant. Arkansas WORK did, however, have
significant savings in welfare payments.\13 As shown in figure 1,
only the SWIM program both produced welfare savings and increased
participant earnings. Over the 5 years, the SWIM program increased
total earnings of participants more than $2,000 above the total
earnings increase for nonparticipants and reduced welfare payments
almost $2,000 per participant. On the basis of a detailed analysis
of these studies' findings, researchers concluded that the Baltimore
Options program was the only program that helped some participants
find higher-paying jobs than the jobs nonparticipants found. The
programs that relied primarily on job search generally resulted in
those participants who would have worked eventually beginning to work
sooner, and motivated some participants who would not have worked to
get a job. For the near term, this resulted in participants relying
more on their own earnings than on welfare, although it did not
increase their overall financial well-being or increase their
earnings capacity.\14
Figure 1: Effect on Total
Earnings and Welfare Savings
per Program Participant Over a
5-Year Study Period for Four
WIN-Era Programs
(See figure in printed
edition.)
Note: Earnings differences between program participants and
nonparticipants were statistically significant only for Baltimore
Options and SWIM; welfare savings were statistically significant only
for Arkansas WORK and SWIM.
--------------------
\13 For our interpretations, we used a common significance level of 5
percent (.05) or less, which was stricter than that used in some
evaluations. Total earnings impact represents the difference between
the total average earnings between the program participants and
nonparticipants. Welfare savings represent the differences between
the total average welfare payments for participants and
nonparticipants and do not take into account the costs of the
program. In averaging the earnings or savings, those individuals
with a zero value were included. The average earnings of only those
with earnings would be higher.
\14 Daniel Friedlander and Gary Burtless, Five Years After: The
Long-Term Effects of Welfare-to-Work Programs (New York, N.Y.:
Russell Sage Foundation, 1995).
FIVE-YEAR EVALUATION OF
SIX JOBS-ERA PROGRAMS IN
CALIFORNIA
-------------------------------------------------------- Letter :3.2.2
In 1988, MDRC began a 5-year evaluation of California's GAIN program
in six counties. This six-site evaluation affords an opportunity to
examine the performance of programs that varied in key ways,
including program emphasis and service strategy. While all of the
county programs used a mixed service strategy, including job search
activities and education and training, the extent to which they
emphasized rapid employment or longer-term education and training
varied, as shown in table 3. The study researchers noted that, among
the six counties, the Riverside program promoted the strongest
message about the importance of moving quickly into employment. The
other counties' programs placed more emphasis on skill building, with
the Alameda, Los Angeles, and Tulare programs placing participants in
education and training longer than the other counties.
The county programs and environments differed in other key ways that
can affect the operation and success of a welfare-to-work program.
For example, as shown in table 3, some counties had much higher
percentages of welfare recipients generally considered hard to employ
(or less job-ready) than other counties, as measured by the
percentage of program participants assessed by each program to be in
need of basic education. A key feature of the GAIN program at the
time of the evaluation in these six counties was its emphasis on
adult basic education. Depending on participants' need for basic
education, GAIN placed participants into groups that were considered
more job-ready and less job-ready. GAIN assigned the job-ready
participants to job search initially and, in general, assigned the
less job-ready to basic education, although sometimes these
participants were given the option to look for work or to look for
work and participate in basic education concurrently. As shown in
table 3, the percentage of participants who were enrolled in
education and training ranged from 28 to 53 percent.
Table 3
Program Emphasis and Other Features of
the Six GAIN County Programs
Participants Participants
assessed to enrolled in
need adult education and
Program emphasis Net program basic training
County and service cost per education activities Special
program strategy person\a (percentage) (percentage) features
------------ ---------------- ------------ ------------ -------------- ---------- ------------
Alameda Emphasized $5,597 65 53 Targeted to
longer-term long-term
education and welfare
training; used recipients
mixed service only
strategy
Butte Placed more 2,904 49 28 Used more
emphasis on intensive
education and case
training than on management
rapid than other
employment; used programs
mixed service
strategy
Los Angeles Emphasized 5,789 81 44 Targeted to
longer-term long-term
education and welfare
training; used recipients
mixed service only
strategy
Riverside Stressed rapid 1,597 60 36 None
employment but
included short-
term education;
used mixed
service strategy
San Diego Placed more 1,912 56 37 None
emphasis on
education and
training than on
rapid
employment; used
mixed service
strategy
Tulare Emphasized 2,734 65 49 None
longer-term
education and
training; used
mixed service
strategy
----------------------------------------------------------------------------------------------------
Note: The information on these programs generally represents the
programs and the status of their clients at the end of 3 years. The
programs changed to some extent over the 5-year period. For example,
some of the original nonparticipants were allowed to participate in
some JOBS activities.
\a Represents the average cost per participant less the average cost
per nonparticipant, adjusted to 1993 dollars.
The six programs had mixed results for participants in increasing
employment, earnings, and welfare savings, compared with
nonparticipants. Of the sites with statistically significant
employment increases, three--Los Angeles, Riverside, and San
Diego--increased the percentage of participants who had been employed
sometime during the 5 years by at least 5 percentage points above the
percentage for nonparticipants. (See fig. 2.) The Riverside program
had the greatest effect: the percentage of those who had held a job
during the 5-year study period was 10 percentage points higher for
participants than for nonparticipants (72 percent versus 62 percent).
Figure 2: Employment Rates for
Program Participants and
Nonparticipants Over the 5-Year
Study Period in Six GAIN
Counties
(See figure in printed
edition.)
Note: Employment rate differences between program participants and
nonparticipants were statistically significant only in Los Angeles,
Riverside, and San Diego.
Two of the programs that had statistically significant effects on the
employment of participants compared with nonparticipants�Riverside
and San Diego�also had significant effects on earnings for their
participants (see fig. 3). The Butte program did not result in
increased employment but did result in increased earnings. For the
programs with statistically significant differences, the total effect
on earnings per participant compared with nonparticipants over the 5
years ranged from almost $3,000 in San Diego to just over $5,000 in
Riverside. Four programs--Alameda, Los Angeles, Riverside, and San
Diego--significantly reduced welfare payments; these savings ranged
from about $1,400 to about $2,700 per participant (see fig. 4).
Figure 3: Total Earnings of
Program Participants and
Nonparticipants Over the 5-Year
Study Period in Six GAIN
Counties
(See figure in printed
edition.)
Note: Differences in earnings were statistically significant between
program participants and nonparticipants only in Butte, Riverside,
and San Diego.
Figure 4: Average Total
Welfare Payments for Program
Participants and
Nonparticipants Over the 5-Year
Study Period in Six GAIN
Counties
(See figure in printed
edition.)
Note: Differences in welfare payments between program participants
and nonparticipants were statistically significant only in Alameda,
Los Angeles, Riverside, and San Diego.
Because of the importance of understanding how a welfare-to-work
program works for different segments of the welfare population, the
GAIN evaluation also looked at results for two key subgroups:
participants and nonparticipants who were job-ready (those determined
not to be in need of basic education) and participants and
nonparticipants who were less job-ready (those determined to be in
need of education). Job-ready participants in three programs had
statistically significant higher earnings or lower welfare payments
when compared with job-ready nonparticipants, with the Riverside and
San Diego programs having both higher earnings and lower welfare
payments. These two programs had relatively large effects on
earnings�almost $6,000 and more than $5,000, respectively�for the
job-ready group.
For those considered less job-ready, three programs--Butte,
Riverside, and Tulare--had statistically significant increases in
earnings, as shown in figure 5, with the effect on earnings ranging
from about $2,700 in Tulare to more than $5,000 in Butte. Five of
the programs had significant welfare savings, as shown in figure 6.
Two programs--Butte and Riverside--had both higher participant
earnings and welfare savings. The Butte program performed relatively
well for the less job-ready with earnings of participants $5,000
higher than those of nonparticipants and per-participant welfare
savings of about $3,900. While the Butte, Riverside, and Tulare
programs all had a statistically significant effect on participant
earnings for those needing basic education, the effects occurred
later in the study period in the Butte and Tulare programs than in
the Riverside program. This demonstrates that some programs that do
not produce results in a short time frame may do so in the longer
term.
Among the six programs, participants determined in need of basic
education were generally assigned to adult basic education
activities.\15
The programs generally increased participation in these activities
for participants compared with nonparticipants, but a majority of
those who were in need of basic education did not obtain a general
equivalency diploma. For those programs that had higher earnings and
welfare savings, it is unclear the extent to which provision of basic
education contributed to these effects.\16
Figure 5: Total Earnings for
Program Participants and
Nonparticipants Determined to
Be in Need of Basic Education
Over the 5-Year Study Period in
Six GAIN Counties
(See figure in printed
edition.)
Note: Differences between program participants and nonparticipants
were statistically significant only in Butte, Riverside, and Tulare.
Figure 6: Welfare Payments to
Program Participants and
Nonparticipants Determined to
Be in Need of Basic Education
Over the 5-Year Study Period in
Six GAIN Counties
(See figure in printed
edition.)
Note: Differences between program participants and nonparticipants
were statistically significant in Alameda, Butte, Los Angeles,
Riverside, and San Diego.
Among the six programs, only the Riverside and San Diego programs had
statistically significant effects in all three areas--total
employment, earnings, and welfare savings--over the 5 years. In
addition, the Riverside program, the least costly, achieved results
in earnings and welfare savings for both the job-ready and less
job-ready participants. The study researchers attributed the success
of the Riverside program to a combination of factors: conveying a
strong, consistent message about the importance of quick employment
for participants, even for those who began the program with education
and training; relying on a mixed strategy including job search,
education and training, and other activities and services; enforcing
participation requirements; devoting some staff to job development
activities to help identify employment opportunities for
participants; and focusing staff on results by, for example, using
performance standards to measure their performance.\17
The Riverside program is considered one of the most successful
large-scale mandatory welfare-to-work programs on the basis of its
significant effect on a range of outcomes and the fact that it
reaches a broad cross section of the welfare caseload. Even so, the
program still did not end participants' dependence on welfare or lead
to employment at wages above poverty levels for many families. The
3-year results showed that, after entering the program, 41 percent of
participants were still receiving welfare and 81 percent had income
at or below the poverty line. The percentage of those still
receiving welfare is no doubt affected to some extent by the
relatively high maximum earnings levels for AFDC in California at the
time of the study. For example, in California in fiscal year 1991, a
three-person family would have needed monthly earnings of $959 to
become ineligible for AFDC.\18 In a state with a relatively high
earnings limit such as California, many more welfare recipients are
likely to combine work and welfare, while in states with lower
earnings limits, any job, even at the minimum wage, can result in a
family's moving off welfare.
--------------------
\15 In the Riverside program, all participants--whether they needed
education or not--were strongly encouraged to start with job search
as their first activity.
\16 For more information on adult education for welfare recipients,
see Edward Pauly, The JOBS Evaluation: Adult Education for People on
AFDC: A Synthesis of Research (Washington, D.C.: Department of
Education and HHS, Dec. 1995). Also see Janet Quint, The JOBS
Evaluation: Educating Welfare Recipients for Employment and
Empowerment: Case Studies of Promising Programs (Washington, D.C.:
Department of Education and HHS, 1997).
\17 The GAIN Riverside program discussed here differs somewhat from
the Riverside program currently being operated as a �pure� rapid
employment approach as part of the MDRC comparison evaluation under
way. The researchers noted that the GAIN Riverside program
emphasized a broader range of activities for participants, placing
more of them in education and training than the Riverside rapid
employment approach in place now.
\18 We estimated that, in fiscal year 1991, the monthly amount of
earnings needed for a family to no longer be eligible for AFDC varied
widely across the states, from a low of $385 to a high of $1,111,
with a median of $632. See Self Sufficiency: Opportunities and
Disincentives on the Road to Economic Independence (GAO/HRD-93-23,
Aug. 6, 1993).
MDRC COMPARISON STUDY SHOWS
THAT RESULTS FROM RAPID
EMPLOYMENT AND
EDUCATION-BASED APPROACHES
DID NOT DIFFER IN THE SHORT
TERM
---------------------------------------------------------- Letter :3.3
In the ongoing comparison study conducted at three sites (Atlanta,
Grand Rapids, and Riverside) by MDRC as part of the National
Evaluation of Welfare-to-Work Strategies, the welfare-to-work
programs implemented at the study sites were designed to provide
�pure� examples of the rapid employment and education-based
approaches. In the rapid employment approach, though, the program
could assign participants to education and training activities if
they were not able to find a job during the initial job search
period.
The rapid employment approach emphasizes quick job placement,
assuming that work habits and skills are better learned at a job than
in a classroom and that any job can be a stepping-stone to a better
one. Program staff only briefly assess participants assigned to this
program before they attend a 3- to 5-week job club. The job club
includes classroom instruction on how to look for a job and provides
supervised job search. Participants who have not found a job at the
end of this period are more fully assessed by staff to see what
activities and services would help them get a job. Depending on that
assessment, participants are assigned to more time in job club,
individual job search, or short-term (up to 9 months) basic
education, vocational training, or work experience. A participant
who completes such an assignment without finding a job is assessed
again and assigned to the same or another of these activities.
The education-based approach assumes that participants need to invest
some time in education or training before seeking employment so that
they can acquire skills that will help them get good jobs and thus
leave welfare permanently. Participants assigned to this approach
first undergo a detailed assessment by program staff to determine
their job-related skills and interests and to identify potential
barriers that they face to getting employment. Depending on the
assessment, these participants are assigned to up to 2 years of basic
education, vocational training, college, or work experience.\19 A
participant who remains unemployed after completing the assignment is
reassessed and assigned to the same or another of these activities, a
job club, or an individual job search.
The 2-year earnings and welfare payment outcomes of both the rapid
employment and education-based program participants were
significantly better than outcomes for nonparticipants, but the
outcomes of the participants in the two approaches were not different
enough from each other to conclude that, overall, one approach is
more effective than the other. In Atlanta and Grand Rapids, there
was no statistically significant difference in the total earnings
effects between the two program approaches.\20 Among participants
with a high school diploma or equivalent, there were no differences
between the two approaches in terms of total earnings. The rapid
employment approach did, however, show greater effects on earnings
than the education-based approach among participants in Grand Rapids
who did not have a high school diploma or equivalent (42 to 44
percent of the total sample at the three sites). In Grand Rapids,
the rapid employment approach produced significantly greater savings
in total welfare payments than the education-based approach, but that
difference was not statistically significant in Atlanta.
While both approaches produced positive outcomes, they did so at very
different program costs. Across the three sites, the average
per-participant cost for the rapid employment approach (above what
was spent on nonparticipants) was $1,550. The average
per-participant cost for the education-based approach was $3,077,
nearly twice the cost of the rapid employment approach.
--------------------
\19 There were not many differences between the intended and actual
sequence of program activities and emphases. According to the
researchers, assessments were generally not in-depth. For example,
only one program had an up-front assessment that was longer than a
few hours--the program in Grand Rapids, using an education-based
approach, had an up-front assessment that lasted a week in a
classroom setting. In addition, at all three education-based sites,
basic education was by far the most commonly assigned first program
activity, followed by vocational training. Assignments to work
experience or college were very rare.
\20 A comparison of the outcomes of the full sample of the two groups
can be made only in Atlanta and Grand Rapids, because in Riverside
the education-based approach was available only to participants who
did not have a high school diploma or equivalent or who achieved
relatively low scores on basic skills tests administered at
orientation.
THE EFFECT OF WELFARE MOTHERS'
EDUCATIONAL ATTAINMENT ON THEIR
CHILDREN'S EDUCATIONAL
ATTAINMENT IS CURRENTLY UNKNOWN
------------------------------------------------------------ Letter :4
While research indicates that parents' educational attainment is
positively related to children's educational attainment, little
information is currently available on this relationship specifically
within the welfare population. Recent studies have identified
factors affecting the cognitive development of children in welfare
families. This research, while limited in scope, indicates that one
factor that may positively affect children's development is the level
of their mothers' educational attainment. This issue has also been
analyzed within a body of research focusing on the effects of poverty
on children's educational attainment. In these studies, which did
not sample welfare recipients exclusively, analysis has included
measuring and controlling for welfare receipt in order to determine
whether the welfare population is different from the general
population. In general, these research results show a significant
positive relationship between the educational attainment of parents
and their children among both the welfare and nonwelfare populations.
Findings from research that focuses on the development of the
children within the welfare population are inconsistent. One
longitudinal study of 614 children whose families received AFDC found
that mothers' prior education corresponded to higher reading scores
but not to higher math or vocabulary scores for children.\21 This
study found no significant effect over the study period from a
mother's increasing her education over the first 5 years of her
child's life. As part of the National Evaluation of Welfare-to-Work
Strategies currently being conducted, researchers will identify how
the welfare-to-work experiences of mothers affect their children.\22
A preliminary descriptive report on preschool-aged children in this
evaluation found that a child's development was associated with the
mother's educational attainment.\23 Future reports from this
evaluation will include school data for children approximately 5
years later. In addition to this evaluation, several states have
included a component in evaluations of their welfare
programs--partially funded by HHS--to look at the effects of various
welfare reforms on children, including children's school
achievement.\24 Although these studies will not measure the
completion of schooling for these children, they will provide
additional information on how mothers' educational attainment affects
the progress of their children's education.
Other applicable research, which has not analyzed welfare recipients
exclusively, has attempted to determine whether and how the welfare
population differs from the general population in terms of children's
educational attainment. These studies show that a parent's various
circumstances, such as economic status and educational level, have a
significant effect on children's educational attainment across the
sample, even when analysis controls for welfare receipt. In all of
the studies that provided detailed results, mothers' educational
attainment consistently had a significant effect on the educational
attainment of their children. These findings suggest that a
significant relationship between a mother's and her children's
educational attainment may also hold true for the welfare population.
--------------------
\21 Hirokazu Yoshikawa, �Welfare Dynamics, Support Services, Mothers'
Earnings, and Child Cognitive Development: Implications for
Contemporary Welfare Reform,� Child Development, Vol. 70, No. 3
(May/June 1999). The study examined mothers' education level 1 year
before the birth of the child in the study.
\22 The three sites for this study are Atlanta, Ga.; Grand Rapids,
Mich.; and Riverside, Calif. This study is contained within MDRC
comparison study referred to earlier that compares the rapid
employment and education-based welfare-to-work approaches.
\23 Kristin A. Moore and others, The JOBS Evaluation: How Well Are
They Faring, AFDC Families With Preschool-Aged Children in Atlanta at
the Outset of the JOBS Evaluation (Washington, D.C.: HHS, Office of
the Assistant Secretary for Planning and Evaluation, 1995).
\24 States include Connecticut, Iowa, Minnesota, Indiana, and
Florida.
CONCLUDING OBSERVATIONS
------------------------------------------------------------ Letter :5
Our review of research conducted over the past two decades shows that
a welfare-to-work approach with a strong employment focus can have
positive effects on participant earnings and employment and on
welfare costs. However, we do not yet definitively know, especially
in the long term, whether a rapid employment or education-based
approach works best for increasing the employment and earnings of
welfare recipients and reducing their dependence on welfare. Future
results from an ongoing study designed specifically to compare the
effectiveness of a rapid employment approach with that of an
education-based approach may shed more light on this issue. In the
meantime, our review of 10 evaluated programs that had 5-year results
indicates that those welfare-to-work programs that combine elements
of both approaches--emphasizing rapid employment but tailoring
services to some extent to meet the differing needs of welfare
recipients--may best meet the goals of increasing employment and
earnings for welfare recipients while at the same time reducing
welfare payments. These types of programs can play an important role
in moving welfare recipients into the labor force and increasing the
extent to which they rely on their own earnings rather than
government aid. Nevertheless, even the most successful program in
Riverside did not usually lead families to higher-paying jobs or move
them out of poverty during the time period studied. Future research
will need to focus on longer-term program outcomes, the effect of the
1996 welfare reform legislation, and what works best for particularly
hard-to-employ populations.
AGENCY COMMENTS AND OUR
EVALUATION
------------------------------------------------------------ Letter :6
We obtained comments on a draft of this report from HHS, which stated
that, overall, it agreed with the report. More specifically, HHS
agreed that, of the welfare-to-work approaches that have been tested,
programs with a combined approach--emphasizing rapid employment but
providing education and training when appropriate--appear to be most
effective. HHS commended us in our selection of rigorous studies
using random assignment; however, HHS noted that this criterion
eliminated any studies regarding postsecondary education, since no
rigorous evaluations have been done of the effectiveness of such
programs for welfare recipients.
HHS made several suggestions to further clarify information contained
in the report. First, HHS pointed out that the costs for the WIN-era
programs would naturally be lower than the costs for the JOBS-era
programs because welfare recipients with younger children were exempt
from participating in the earlier programs and, consequently, costs
for child care would be lower. We agree with this suggestion and
have included it in the report. Second, HHS noted that while we
focused on effects on cumulative earnings over the 5-year follow-up
period, which is a good measure of the overall impact of a program,
the effects on earnings at the end of the follow-up period might be a
better indicator of future earnings. We agree with HHS that
assessing the effects on earnings at the end of the follow-up period
can provide important information. However, our analysis of the
results of the six-county GAIN evaluation showed that the only
counties that showed statistically significant increases in earnings
for the last year of the follow-up period (Butte, Riverside, and San
Diego) also showed significant increases over the 5-year follow-up
period; as a result, we did not report separately on the effects at
the end of the study time period. Furthermore, we noted that for
participants considered to be less job-ready, effects occurred later
in the study period for several counties, indicating that some
programs that do not produce results in a short time frame may do so
in the longer term. Finally, regarding the section about the effect
of welfare mothers' educational attainment on their children's
educational attainment, HHS noted that the report should more clearly
explain how studies documented a mother's educational background. We
have made revisions to this section to clarify whether the education
level was documented at the onset of a study or during the study
period. However, we could not indicate the percentage of mothers in
the sample with increases in education during the study period. In
addition, HHS provided technical comments, which we incorporated in
the report where appropriate.
We also provided copies of the draft report to the Departments of
Education and Labor for technical review; they had no technical
comments.
---------------------------------------------------------- Letter :6.1
We are sending copies of this report to the Honorable Donna E.
Shalala, Secretary of Health and Human Services; the Honorable Alexis
M. Herman, Secretary of Labor; the Honorable Richard W. Riley,
Secretary of Education; and state TANF directors. We will also make
copies available to others upon request.
If you or your staffs have any questions about this report, please
call me on (202) 512-7215. Other GAO contacts and staff
acknowledgments are listed in appendix III.
Cynthia M. Fagnoni
Director, Education, Workforce, and
Income Security Issues
List of Addressees
The Honorable William V. Roth, Jr.
Chairman
The Honorable Daniel Patrick Moynihan
Ranking Minority Member
Committee on Finance
United States Senate
The Honorable James M. Jeffords
Chairman
The Honorable Edward M. Kennedy
Ranking Minority Member
Committee on Health, Education, Labor, and Pensions
United States Senate
The Honorable William F. Goodling
Chairman
The Honorable William L. Clay
Ranking Minority Member
Committee on Education and the Workforce
House of Representatives
The Honorable Bill Archer
Chairman
The Honorable Charles B. Rangel
Ranking Minority Member
Committee on Ways and Means
House of Representatives
SCOPE AND METHODOLOGY
=========================================================== Appendix I
To address the mandated objectives, we identified relevant evaluation
studies of welfare-to-work programs that help welfare recipients and
other low-income women with children become employed. To be included
in our review, evaluations had to meet the following criteria:
-- A program could have started before 1988, but its evaluation had
to have been published since 1988, after the passage of the
Family Support Act.
-- A study had to measure the effect of welfare-to-work approaches
on employment-related outcomes such as employment, earnings, and
welfare payments.
-- A study had to rigorously evaluate the program by controlling
for factors that could affect employment-related outcomes.
To identify the relevant evaluations as well as identify information
on the impact of welfare recipients' educational attainment on the
educational attainment of their children, we searched several on-line
bibliographic databases. These databases included Sociological
Abstracts, Social SciSearch, ERIC, the Welfare Information Network,
and ECONLIT. We also reviewed bibliographies of research studies on
these issues and consulted with experts on welfare-to-work issues to
identify other studies we should consider. We met with officials at
the Departments of Health and Human Services, Labor, and Education to
obtain further information on pertinent evaluations.
We identified only one evaluation that compared the effectiveness of
a rapid employment approach with that of an education-based approach
and included this evaluation in our review. In selecting other
welfare-to-work evaluations for review, we included only evaluations
with impacts for follow-up periods of at least 5 years.
Consequently, we identified for review six evaluations, which
evaluated a total of 13 programs. Because many of the welfare
recipients who participated in the welfare-to-work programs evaluated
had less than a high school education, the education-based approaches
evaluated in the studies we assessed primarily provided basic
education services rather than postsecondary education. As a result,
none of the studies evaluated the effectiveness of a college
education on improving employment-related outcomes for welfare
recipients. All the evaluations used research designs that
controlled for other factors that could affect outcomes. For
example, participants were randomly assigned to either a program
group, which was subject to the program being evaluated, or to a
control group, which continued under a previous program or no
program. The experience of the control group members�who, at their
initiative, could use services elsewhere in the community�indicates
what would have happened to the program groups in the absence of
special intervention, providing a benchmark for measuring program
effects. The principal outcomes measured in the evaluations were
employment, earnings, and welfare savings. For each outcome in each
study, the researchers had compared results for the participants
receiving program services with those for participants in the control
group and identified statistically significant differences that were
deemed to be program impacts or effects. The evaluation reports
estimated the likelihood that these differences occurred by chance by
using standard tests of statistical significance. We did not
independently verify the information in the evaluation reports.
We conducted our work in accordance with generally accepted
government auditing standards between February 1999 and July 1999.
(See figure in printed edition.)Appendix II
COMMENTS FROM THE DEPARTMENT OF
HEALTH AND HUMAN SERVICES
=========================================================== Appendix I
(See figure in printed edition.)
(See figure in printed edition.)
GAO CONTACTS AND STAFF
ACKNOWLEDGMENTS
========================================================= Appendix III
GAO CONTACTS
Gale C. Harris, (202) 512-7235
Sigurd R. Nilsen, (202) 512-7003
STAFF ACKNOWLEDGMENTS
In addition to those named above, the following individuals made
important contributions to this report: Lara L. Carreon, Betty S.
Clark, Margaret A. Holmes, Denise D. Hunter, Susan A. Riedinger,
and Megan V. Smith.
BIBLIOGRAPHY
=========================================================== Appendix 0
WELFARE-TO-WORK EVALUATIONS
REVIEWED IN THIS REPORT
Freedman, Stephen, Daniel Friedlander, Winston Lin, and others. "The
GAIN Evaluation: Five-Year Impacts on Employment, Earnings and AFDC
Receipt." Working Paper 96.1, Manpower Demonstration Research
Corporation, 1996.
Friedlander, Daniel. Subgroup Impacts and Performance Indicators for
Selected Welfare Employment Programs. New York, N.Y.: Manpower
Demonstration Research Corporation, 1988.
Friedlander, Daniel, and Gary Burtless. Five Years After: The
Long-Term Effects of Welfare-to-Work Programs. New York, N.Y.:
Russell Sage Foundation, 1995.
Friedlander, Daniel, and Gayle Hamilton. The Saturation Work
Initiative Model in San Diego: A Five-Year Follow-Up Study. New
York, N.Y.: Manpower Demonstration Research Corporation, 1993.
Hamilton, Gayle, Thomas Brock, Mary Farrell, and others. National
Evaluation of Welfare-to-Work Strategies: Evaluating Two
Welfare-to-Work Approaches, Two-Year Findings on the Labor Force
Attachment and Human Capital Development Programs in Three Sites.
Washington, D.C.: Departments of Education and Health and Human
Services, 1997.
Riccio, James, Daniel Friedlander, Stephen Freedman, and others.
GAIN: Benefits, Costs, and Three-Year Impacts of a Welfare-to-Work
Program. New York, N.Y.: Manpower Demonstration Research
Corporation, 1994.
OTHER WELFARE-TO-WORK EVALUATIONS
AND RELATED STUDIES
Auspos, Patricia, George Cave, and David Long. Maine: The
Demonstration of State Work/Welfare Initiatives: Final Report on the
Training Opportunities in the Private Sector Program. New York,
N.Y.: Manpower Demonstration Research Corporation, 1988.
Bloom, Dan. After AFDC: Welfare-to-Work Choices and Challenges for
States. New York, N.Y.: Manpower Demonstration Research
Corporation, 1997.
Bloom, Dan, Mary Farrell, James Kemple, and others. The Family
Transition Program: Implementation and Interim Impacts of Florida's
Initial Time-Limited Welfare Program. New York, N.Y.: Manpower
Demonstration Research Corporation, 1998.
Bos, Hans, Aletha Huston, Robert Granger, and others. New Hope for
People With Low Incomes: Two-Year Results of a Program to Reduce
Poverty and Reform Welfare. New York, N.Y.: Manpower Demonstration
Research Corporation, 1999.
Fein, David, Erik Beecroft, and John D. Blomquist. The Ohio
Transitions to Independence Demonstration: Final Impacts for JOBS
and Work Choice. Bethesda, Md.: Abt Associates, 1994.
Fein, David, Erik Beecroft, William Hamilton, and others. The
Indiana Welfare Reform Evaluation: Program Implementation and
Economic Impacts After Two Years. Cambridge, Mass.: Abt Associates,
1998.
Fein, David J., and Jennifer A. Karweit. The ABC Evaluation: The
Early Economic Impacts of Delaware's A Better Chance Welfare Reform
Program. Cambridge, Mass.: Abt Associates, 1997.
Fraker, Thomas M., Lucia A. Nixon, Jonathan E. Jacobson, and
others. Iowa's Family Investment Program: Two-Year Impacts.
Washington, D.C.: Mathematica Policy Research, Inc., 1998.
Freedman, Stephen, Jan Bryant, and George Cave. New Jersey: The
Demonstration of State Work/Welfare Initiatives, Final Report on the
Grant Diversion Project. New York, N.Y.: Manpower Demonstration
Research Corporation, 1988.
Freedman, Stephen, Marisa Mitchell, and David Navarro. "The Los
Angeles Jobs-First GAIN Evaluation: Preliminary Findings on
Participation Patterns and First-Year Impacts." Working Paper,
Manpower Demonstration Research Corporation, 1998.
Gueron, Judith M., and Edward Pauly. From Welfare to Work. New
York, N.Y.: Russell Sage Foundation, 1991.
Kemple, James, Daniel Friedlander, and Veronica Fellerath. Florida's
Project Independence: Benefits, Costs, and Two-Year Impacts of
Florida's JOBS Program. New York, N.Y.: Manpower Demonstration
Research Corporation, 1995.
Long, Sharon K., Demetra Smith Nightingale, and Douglas A. Wissoker.
The Evaluation of the Washington State Family Independence Program.
Washington, D.C.: Urban Institute Press, 1994.
Miller, Cynthia, Virginia Knox, Patricia Auspos, and others. Making
Welfare Work and Work Pay: Implementation and 18-Month Impacts of
the Minnesota Family Investment Program. New York, N.Y.: Manpower
Demonstration Research Corporation, 1997.
Nightingale, Demetra Smith, Douglas A. Wissoker, Lynn C. Burbridge,
and others. Evaluation of the Massachusetts Employment and Training
(ET) Program. Washington, D.C.: Urban Institute Press, 1991.
Olson, Jerome A., Deanna T. Schexnayder, Daniel P. O'Shea, and
others. Participation Patterns and Program Impacts of Hawaii's JOBS
WORKS! Demonstration Project. Austin, Tex.: Lyndon B. Johnson
School of Public Affairs, University of Texas at Austin, 1997.
O'Neill, June. Work and Welfare in Massachusetts: An Evaluation of
the ET Program. Boston, Mass.: Pioneer Institute for Public Policy
Research, 1990.
Orr, Larry L., Howard S. Bloom, Stephen H. Bell, and others. Does
Training for the Disadvantaged Work? Evidence From the National JTPA
Study: An Abt Associates Study. Washington, D.C.: Urban Institute
Press, 1996.
Schexnayder, Deanna T., and Jerome A. Olson. Texas JOBS Program
Evaluation: Second Year Impacts. Austin, Tex.: Center for the
Study of Human Resources, University of Texas at Austin, 1995.
Scrivener, Susan, Gayle Hamilton, Mary Farrell, and others. National
Evaluation of Welfare-to-Work Strategies: Implementation,
Participation Patterns, Costs and Two-Year Impacts of the Portland
(Oregon) Welfare-to-Work Program. Washington, D.C.: Departments of
Education and Health and Human Services, 1998.
Werner, Alan, and Robert Kornfeld. Final Impact Report�The
Evaluation of To Strengthen Michigan Families. Cambridge, Mass.:
Abt Associates, 1997.
Werner, Alan, David Rodda, Elsie Pan, and others. Final Report:
Evaluation of the Alabama Avenues to Self-Sufficiency Through
Employment and Training Services (ASSETS) Demonstration. Cambridge,
Mass.: Abt Associates, 1997.
RESEARCH EXPLORING THE
RELATIONSHIP BETWEEN WELFARE
MOTHERS' EDUCATIONAL ATTAINMENT
AND THAT OF THEIR CHILDREN, AND
RELATED STUDIES
Brooks-Gunn, Jeanne, Greg J. Duncan, and Nancy Maritato. �Poor
Families, Poor Outcomes: The Well-Being of Children and Youth,�
Consequences of Growing Up Poor, eds. Greg J. Duncan and Jeanne
Brooks-Gunn. New York, N.Y.: Russell Sage Foundation, 1997.
Duncan, Greg J., Rachel Dunifon, Morgan Ward Doran, and W. Jean
Yeung. "How Different ARE Welfare and Working Families? And Do
Those Differences Matter for Children's Achievement?" Working Paper
No. 2, The Northwestern University/University of Chicago, Joint
Center for Poverty Research, July 1998.
Duncan, Greg J., and Wei-Jun J. Yeung. �Extent and Consequences of
Welfare Dependence Among America's Children,� Children and Youth
Services Review, Vol. 17, Nos. 1 and 2 (1995), pp. 157-82.
Haveman, Robert, and Barbara Wolfe. �The Determinants of Children's
Attainments: A Review of Methods and Findings,� Journal of Economic
Literature, Vol. 33 (Dec. 1995), pp. 1829-78.
_____. Succeeding Generations: On the Effects of Investments in
Children. New York, N.Y.: Russell Sage Foundation, 1994.
Haveman, Robert, Barbara Wolfe, and James Spaulding. �Childhood
Events and Circumstances Influencing High School Completion,�
Demography, Vol. 28, No. 1 (Feb. 1991), pp. 133-57.
Menaghan, Elizabeth, Susan Jekielek, Frank Mott, and Elizabeth
Cooksey. "Work and Family Circumstances and Child Trajectories:
When (and For What) Does AFDC Receipt Matter?" Working Paper No. 3,
The Northwestern University/University of Chicago, Joint Center for
Poverty Research, July 1998.
Moore, Kristin A., Martha J. Zaslow, Mary Jo Coiro, and others. The
JOBS Evaluation: How Well Are They Faring? AFDC Families With
Preschool-Aged Children in Atlanta at the Outset of the JOBS
Evaluation. Washington, D.C.: Department of Health and Human
Services, Office of the Assistant Secretary for Planning and
Evaluation, 1995.
Teachman, Jay D., Kathleen M. Paasch, Randal D. Day, and Karen P.
Carver. �Poverty During Adolescence and Subsequent Educational
Attainment,� Consequences of Growing Up Poor, eds. Greg J. Duncan
and Jeanne Brooks-Gunn. New York, N.Y.: Russell Sage Foundation,
1997.
Yoshikawa, Hirokazu. �Welfare Dynamics, Support Services, Mothers'
Earnings, and Child Cognitive Development: Implications for
Contemporary Welfare Reform,� Child Development, Vol. 70, No. 3
(May/June 1999), pp. 779-801.
Zill, Nicholas, Kristen A. Moore, Ellen Wolpow Smith, and others.
�The Life Circumstances and Development of Children in Welfare
Families: A Profile Based on National Survey Data,� Escape From
Poverty: What Makes a Difference for Children? eds. P. Lindsay
Chase-Lansdale and Jeanne Brooks-Gunn. New York, N.Y.: Cambridge
University Press, 1995.
RELATED GAO PRODUCTS
Welfare Reform: States' Implementation Progress and Information on
Former Recipients (GAO/T-HEHS-99-116, May 27, 1999).
Welfare Reform: Information on Former Recipients' Status
(GAO/HEHS-99-48, Apr. 28, 1999).
Welfare Reform: States' Experiences in Providing Employment
Assistance to TANF Clients (GAO/HEHS-99-22, Feb. 26, 1999).
Welfare Reform: States Are Restructuring Programs to Reduce Welfare
Dependence (GAO/HEHS-98-109, June 17, 1998).
Welfare to Work: State Programs Have Tested Some of the Proposed
Reforms (GAO/PEMD-95-26, July 14, 1995).
Welfare to Work: Most AFDC Training Programs Not Emphasizing Job
Placement (GAO/HEHS-95-113, May 19, 1995).
Welfare to Work: Participants' Characteristics and Services Provided
in JOBS (GAO/HEHS-95-93, May 2, 1995).
Welfare to Work: States Begin JOBS, but Fiscal and Other Problems
May Impede Their Progress (GAO/HEHS-91-106, Sept. 27, 1991).
*** End of document. ***