Unemployment Insurance: More Guidance and Evaluation of
Worker-Profiling Initiative Could Help Improve State Efforts
(14-JUN-07, GAO-07-680).
Changes to the U.S. economy have led to longer-term unemployment.
Many unemployed workers receive Unemployment Insurance (UI),
which provided about $30 billion in benefits in 2006. In 1993,
Congress established requirements--now known as the Worker
Profiling and Reemployment Services (WPRS) initiative--for state
UI agencies to identify claimants who are most likely to exhaust
their benefits, and then refer such claimants to reemployment
services. To assess the implementation and effect of the
initiative, GAO examined (1) how states identify claimants who
are most likely to exhaust benefits, (2) to what extent states
provide reemployment services as recommended by the Department of
Labor (Labor), and (3) what is known about the effectiveness of
the initiative in accelerating reemployment. To answer these
questions, we used a combination of national data; review of
seven states, including visits to local service providers in four
states; and existing studies and interviews with Labor and
subject matter experts.
-------------------------Indexing Terms-------------------------
REPORTNUM: GAO-07-680
ACCNO: A70869
TITLE: Unemployment Insurance: More Guidance and Evaluation of
Worker-Profiling Initiative Could Help Improve State Efforts
DATE: 06/14/2007
SUBJECT: Claims
Data collection
Employment
Employment assistance programs
Evaluation methods
Federal/state relations
Labor statistics
Program evaluation
Reemployment
State-administered programs
Statistical data
Statistical methods
Unemployment compensation programs
Unemployment insurance
Unemployment rates
Program implementation
Unemployment Insurance Program
******************************************************************
** This file contains an ASCII representation of the text of a **
** GAO Product. **
** **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced. Tables are included, but **
** may not resemble those in the printed version. **
** **
** Please see the PDF (Portable Document Format) file, when **
** available, for a complete electronic file of the printed **
** document's contents. **
** **
******************************************************************
GAO-07-680
* [1]Results in Brief
* [2]Background
* [3]Most States Use Statistical Models to Identify Claimants Lik
* [4]Most States Use Statistical Models instead of More Limited C
* [5]Statistical Models Vary in Key Elements
* [6]Although Models Require Periodic Maintenance to Ensure Predi
* [7]Most Study States Did Not Take the In-Depth Approach Recomme
* [8]Data Collected by Labor Provided a Limited Picture of States
* [9]Six of Seven States We Studied Referred Claimants and Enforc
* [10]Six of Seven States We Studied Provided Limited Reemployment
* [11]Little Is Known about Program's Effectiveness because There
* [12]Early Research Showed Some Positive Outcomes for Those Refer
* [13]Outcomes Data Collected by Labor Are Limited and Not Consist
* [14]Conclusions
* [15]Recommendations for Executive Action
* [16]Agency Comments
* [17]GAO Contact
* [18]Staff Acknowledgments
* [19]GAO's Mission
* [20]Obtaining Copies of GAO Reports and Testimony
* [21]Order by Mail or Phone
* [22]To Report Fraud, Waste, and Abuse in Federal Programs
* [23]Congressional Relations
* [24]Public Affairs
Report to Congressional Requesters
United States Government Accountability Office
GAO
June 2007
UNEMPLOYMENT INSURANCE
More Guidance and Evaluation of Worker-Profiling Initiative Could Help
Improve State Efforts
GAO-07-680
Contents
Letter 1
Results in Brief 2
Background 4
Most States Use Statistical Models to Identify Claimants Likely to Exhaust
Benefits, but Many Have Not Updated Them to Account for Changing Economic
Conditions 9
Most Study States Did Not Take the In-Depth Approach Recommended by Labor
to Ensure That Profiled Claimants Obtain Reemployment Services 19
Little Is Known about Program's Effectiveness because There Are No Current
Studies and Labor's Data Are of Limited Usefulness 24
Conclusions 29
Recommendations for Executive Action 30
Agency Comments 31
Appendix I Objectives, Scope, and Methodology 32
Appendix II Average Percentage of Claimants Profiled, Referred to, and
Completing Services for 2002-2006 and Average Claimant Outcomes for
2002-2005, by State 36
Appendix III Bibliography of Research Studies on the Worker-Profiling
Initiative--Exhaustive List Identified from the Literature Review 38
Appendix IV Summary of the Impact of Referral to Services on Claimant
Outcomes from the Literature Review 39
Appendix V Comments from the Department of Labor 40
Appendix VI GAO Contacts and Acknowledgments 42
Related GAO Products 43
Tables
Table 1: Labor-Recommended Factors 10
Table 2: Selected Types of Variables Used in State Models beyond Federally
Recommended Variables 15
Table 3: Research Studies Included in Our Literature Review 25
Table 4: Summary of Research Study Findings on the Effect of Referral to
Services on Claimant Outcomes 26
Table 5: National Averages and Ranges of State Averages on Outcomes for
Claimants Profiled and Referred to Services, 2002 to 2005 28
Figures
Figure 1: Process for Profiling, Referring, and Providing Reemployment
Services to Claimants 7
Figure 2: Profiling Techniques Used in the States 12
Figure 3: States' Adjustments of Model Coefficients 17
Figure 4: Passage of Time since States Adjusted Models by Changing
Variables 18
Figure 5: Percentage of Claimants Referred to Services, of Those Profiled
20
Figure 6: Percentage of Claimants Completing Services, of Those Profiled
21
Abbreviations
ETA Employment and Training Administration
UI Unemployment Insurance
WIA Workforce Investment Act
This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed in
its entirety without further permission from GAO. However, because this
work may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this material
separately.
United States Government Accountability Office
Washington, DC 20548
June 14, 2007
The Honorable Jerry Weller
Ranking Member
Subcommittee on Income Security and Family Support
Committee on Ways and Means
House of Representatives
The Honorable Wally Herger
House of Representatives
Changes to the U.S. economy--including the contraction of entire
industries as a result of changes in technology and overseas
competition--have led to increases in the length of unemployment.
Unemployed workers are now less likely to be rehired by their previous
employers and are at a greater risk of long-term unemployment than in the
past. Over the past five decades, the average duration of unemployment has
been gradually increasing, so that during 2006, periods of unemployment
grew to an average of 15 weeks, compared with 11 weeks during the 1950s.
Many unemployed workers receive temporary, partial wage replacement
through the Unemployment Insurance (UI) program. Under most state
programs, claimants can obtain regular benefits for up to 26 weeks. In
order to help facilitate workers' return to work, Congress established
requirements now known as the Worker Profiling and Reemployment Services
initiative in 1993. Under this initiative, state UI agencies are required
to identify those who are most likely to exhaust their benefits, a process
known as profiling, and refer them to reemployment services. Changes to
the U.S. economy--including the contraction of entire industries as a
result of changes in technology and overseas competition--have led to
increases in the length of unemployment. Unemployed workers are now less
likely to be rehired by their previous employers and are at a greater risk
of long-term unemployment than in the past. Over the past five decades,
the average duration of unemployment has been gradually increasing, so
that during 2006, periods of unemployment grew to an average of 15 weeks,
compared with 11 weeks during the 1950s. Many unemployed workers receive
temporary, partial wage replacement through the Unemployment Insurance
(UI) program. Under most state programs, claimants can obtain regular
benefits for up to 26 weeks. In order to help facilitate workers' return
to work, Congress established requirements now known as the Worker
Profiling and Reemployment Services initiative in 1993. Under this
initiative, state UI agencies are required to identify those who are most
likely to exhaust their benefits, a process known as profiling, and refer
them to reemployment services.
Statistics on the UI program underscore the importance of addressing
benefit exhaustion. In 2006 about 7 million claimants received UI
payments, totaling about $30 billion. Of those claimants, about 35 percent
used all the benefits available to them. If they had used 1 week less of
benefits, it would have saved the state UI trust funds roughly $600
million. Statistics on the UI program underscore the importance of
addressing benefit exhaustion. In 2006 about 7 million claimants received
UI payments, totaling about $30 billion. Of those claimants, about 35
percent used all the benefits available to them. If they had used 1 week
less of benefits, it would have saved the state UI trust funds roughly
$600 million.
You asked us to assess how the worker-profiling initiative has been
implemented, and its effect on shortening the length of unemployment.
Specifically, you wanted to know: You asked us to assess how the
worker-profiling initiative has been implemented, and its effect on
shortening the length of unemployment. Specifically, you wanted to know:
1. How do states identify unemployment claimants who are most
likely to exhaust benefits? 1. How do states identify unemployment
claimants who are most likely to exhaust benefits?
2. To what extent do states provide reemployment services as
recommended by Labor?
3. What is known about the effectiveness of the worker-profiling
initiative in accelerating the reemployment of unemployment
insurance claimants?
To answer these questions, we used a combination of national data,
in-depth site visits, existing studies, and interviews with subject matter
experts. We analyzed national data collected by the Department of Labor
(Labor) from states on the worker-profiling initiative, including data on
the 50 states, as well as the District of Columbia, Puerto Rico, and the
Virgin Islands from a 2006 Labor-sponsored survey on models states use to
profile. We also analyzed data on profiling, reemployment services, and
outcomes that states report to Labor using the reemployment service
activity and outcomes reports maintained by Labor's Employment and
Training Administration. In addition, we interviewed state officials in
seven states: California, Delaware, Illinois, Kentucky, Texas, Washington,
and Wisconsin. We selected the seven states to include the range of
approaches states take to identify and serve those likely to exhaust
benefits. We visited local service providers in four of these states. Our
site visit states were selected to provide a range of state experiences
with the worker-profiling initiative and to ensure variation in geography
and population size. We identified six studies examining the impact of the
worker-profiling programs and, after evaluating the methodological
soundness and the validity of the results and conclusions, determined that
five of the six studies were sufficiently rigorous to use in this report.
Further, we interviewed Labor officials and other experts on worker
profiling and UI and reviewed other reports, including academic and Labor
research on profiling systems, best practices, and the outcomes of
profiled UI claimants. We performed our work in accordance with generally
accepted government auditing standards between August 2006 and April 2007.
Results in Brief
The large majority of states use statistical models to identify claimants
who are most likely to exhaust their unemployment benefits. However, many
states have not recently adjusted these models to ensure predictive
accuracy. Forty-five of the 53 states and territories use statistical
models to identify clients likely to exhaust benefits, while 7 states use
more limited screening tools that do not enable states to rank claimants
by probability of exhausting benefits. One state--Florida---allows the
local areas in the state to select the profiling technique. The size and
complexity of the statistical models used by the 45 states vary
considerably. For example, 11 of the 45 states reported using models that
include the 5 variables recommended by the Department of Labor, while 34
states reported using additional variables. In our site visits, we learned
that these additional variables can be limited to just 1 or 2, as in
California, to over 50 variables used by Kentucky. We further found that
many states do not regularly update their models. A 2006 Labor-sponsored
survey of the states revealed that many states continue to use models that
have not been adjusted in a decade or more. For example, 30 states have
not revised their models since implementing them in the mid-1990s. This
raises concerns because Labor officials, state officials, and other
modeling experts have stated that a model may lose predictive accuracy if
it is not revised every few years to adjust for changes to the labor
market and other economic conditions. For example, a 2003 California study
found that the state's model underestimated benefit exhaustion and
recommended an update to the model, a process California has begun.
Officials in states we contacted explained that they face a number of
obstacles to regularly updating the models. For example, updating and
maintaining the statistical models can be impeded by technical and data
difficulties, and other priorities for limited UI administrative funds and
staff resources. Although Labor provides technical assistance to states
requesting it, the agency does not regularly monitor state efforts to
adjust their models.
It is difficult to determine the extent to which all states are providing
reemployment services to claimants because some of Labor's data are
unreliable. However, we determined that 6 of the 7 states we studied did
not provide the in-depth services that Labor originally recommended.
Nationally, according to Labor's data, 15 percent of the profiled
claimants were referred to services between 2002 and 2006. These referrals
ranged from as few as 1 percent in Wyoming to as many as 52 percent in
Washington. We could not determine the services received by those referred
for all states because Labor's data on services were not sufficiently
reliable. In the states we studied, the services provided were only some
of those recommended by Labor. Six of the 7 states referred claimants to
services, enforced consequences for failure to attend these services, and
provided one or more sessions that included orientation to services and
instruction on various job search skills. However, only 1 state performed
in-depth individual assessments and created individualized reemployment
plans, both of which are recommended in Labor's guidance. State officials
cited various challenges to providing these reemployment services to UI
claimants. These included the discontinuation of federal grants that some
states had used to fund services and the difficulty of serving a disparate
clientele that ranged from people in upper management to laborers.
Although Labor recognizes these constraints, officials said the program's
purpose is to target the funding that does exist to those who need it
most.
Little is known about the current effectiveness of the worker-profiling
initiative. The few early studies, which all used data from 1994 to 1996,
generally indicated that claimants who were referred to services received
less in unemployment benefits and collected them for shorter lengths of
time than comparable groups of claimants that did not receive services.
These studies found that those referred to services received $55 to $320
less in benefits and remained on unemployment insurance for 0.2 to 4 weeks
less. Since 1999, Labor has not published any studies on the effect of the
initiative and, according to Labor officials, has no plans to study the
effects of profiling. Although Labor collects data on the outcomes of
those profiled and referred to services, only some of the data were
reliable enough to report. For example, on average about 40 percent of
claimants who were referred to services exhausted their benefits between
2002 and 2005. Furthermore, although Labor and some states make limited
use of the data, the data are not consistently being used to evaluate the
initiative. According to Labor officials, the data were intended for
states to evaluate the effectiveness of the initiative. However, officials
from 4 of the 7 states we studied said they did not use Labor's data
primarily because they do not meet their management needs. For example,
data are aggregated at the state level, and some state officials said they
would prefer local-level data.
In this report, we recommend that the Secretary of Labor ensure that the
Employment and Training Administration reevaluate data collection for the
worker-profiling initiative to determine whether it is sufficient for its
intended purpose, take a more active role in ensuring the accuracy of
state profiling models, encourage states to adhere to Labor's vision for
reemployment services, and evaluate the impact of the worker-profiling
program. In responding to a draft of this report, Labor generally agreed
with our findings and recommendations.
Labor also provided technical comments on the draft report, which we have
incorporated where appropriate.
Background
Beginning in the mid-1970s, major structural changes took place in the
American economy, as advances in technology, international competition,
plant closings, and corporate streamlining resulted in the dislocation of
thousands of workers from their jobs. Some of these individuals possessed
skills that were no longer in demand; others suffered from a lack of job
search skills. In the 1980s and early 1990s, demonstration projects were
conducted in New Jersey, Nevada, Minnesota, and Washington. The New Jersey
and Minnesota projects showed the efficacy of using statistical methods
and administrative data to identify those who are likely to experience
long periods of joblessness. For example, the New Jersey demonstration
project screened claimants with various eligibility requirements and found
that the screening allowed the state to direct services to those who
generally faced reemployment difficulties.^1 Further, results from all
four states showed that providing more intensive job search assistance to
this population reduced the duration of insured unemployment and UI
expenditures.^2
In response to these events, the Clinton administration proposed
legislation to implement worker profiling in 1993. In the same year,
Congress enacted the Unemployment Compensation Amendments, amending the
Unemployment Insurance program legislation. ^3 The law requires that
states establish and utilize a system of profiling all new claimants for
UI regular compensation. The system must identify those claimants that
will be likely to exhaust regular compensation and refer them to
reemployment services, such as job search assistance services. Typically,
such claimants receive services at one of the local "one-stop" employment
services centers that exist throughout the nation. States are required to
collect information on the type of services claimants receive, their
participation, and their subsequent employment outcomes. The last could
include such information as whether referred claimants obtained new jobs
and the related wage levels.
In 1994, Labor issued guidance to help states establish profiling tools
and provide necessary reemployment services. In profiling claimants, Labor
required that states consider factors that include whether the claimant
has a date for being recalled to work, union status, first unemployment
benefit payment, and previous industry or occupation of employment. Labor
recommended also considering some additional factors such as claimants'
education, tenure at previous job, and the state unemployment rate.^4
^1Walter Corson, Paul T. Decker, Shari Mill Dunstan, and Anne R. Gordon,
"The New Jersey Unemployment Insurance Reemployment Demonstration Project"
(April 1989).
^2Randall Eberts, "The Use of Profiling in the United States for Early
Identification and Referral of Less Employable Unemployment Insurance
Recipients," Employability: Concepts and Policies (May 1999).
^3Unemployment Compensation Amendments of 1993 (Pub. L. No. 103-152).
Labor outlined recommended processes for providing reemployment services
to referred claimants, including (1) an orientation session for claimants
that would, among other things, explain the availability and benefit of
reemployment services; (2) an assessment of the specific needs of each
claimant, if appropriate; and (3) based on the assessment, development of
an individual plan for services that would guide a claimant's further
services. (See fig. 1.) Under the law, states must also require that
claimants who have been referred to reemployment services participate in
those services as a condition of eligibility for receiving compensation.
^4Labor also prohibited the use of certain data elements, such as age,
race or ethnic group, sex, disability, and religion, as Labor determined
that use of such characteristics would be in violation of federal law.
Figure 1: Process for Profiling, Referring, and Providing Reemployment
Services to Claimants
Labor may withhold UI administrative grants from a state if Labor finds,
after notice and an opportunity for a hearing, that a state has failed to
comply with worker-profiling requirements. These include identifying
claimants most likely to exhaust benefits, referring claimants to
reemployment services, and collecting follow-up information on services
received and subsequent employment outcomes.
The law required that Labor report to Congress on the operation and
effectiveness of the profiling system within three years of its enactment.
Labor issued a report to Congress in March 1997, and published a final
report in 1999 on the program's implementation and operation nationwide
and the effectiveness in six early implementation states. Labor has
published no studies on the effectiveness of the initiative since then.
The agency's strategic plan for fiscal years 2006 through 2011, in
providing an overview of program evaluation, includes no ongoing or future
research topics addressing the impact of the worker-profiling initiative.
Labor has conducted impact evaluations as part of its program evaluations
in the past.^5 In fiscal year 2007, Labor was appropriated $17.7 million
for pilots, demonstrations, and research.
Funding for the worker-profiling program is provided from a variety of
sources. Federal funding for the creation and maintenance of profiling
models can come from UI administrative funds, which are financed by a
federal UI tax on employers.^6 Reemployment services can be funded through
a variety of sources. For example, states can use Wagner-Peyser Employment
Services grants as well as other state sources of funding to provide
reemployment services to profiled UI claimants.^7 From 2001 to 2005, Labor
also provided Reemployment Services grants to all states in order to
enhance and target services to claimants through the nation's network of
one-stop employment service centers.^8
^5Many researchers consider impact evaluations to be the best method for
determining the extent to which the program itself, rather than other
factors, is causing participant outcomes. Impact evaluations can be
designed in several ways, but fall into two basic design categories:
experimental and quasi-experimental. Experimental designs randomly assign
eligible individuals either to a group that will receive services from the
program being studied or to a group that will not receive services from
the program. If random assignment is successful, the only difference
between the two groups is their access to program services. The relevant
outcomes of these two groups are measured and compared, and any
differences found between the two can be attributed to the programs. When
randomly assigning individuals to a control group is not a feasible
option, quasi-experimental impact evaluations can be used to compare the
outcomes of program participants to those of individuals not in the
program. In a quasi-experimental design, methods other than random
assignment are used to create a comparison group. A comparison group can
be developed in a variety of ways. One way is to use a set of individuals
who have similar characteristics as the group receiving the program
services under study. Although quasi-experimental studies do not use
random assignment, it is still possible to determine the impact of a
program through statistical methods or other research design techniques.
^6In 2002 the federal government distributed $8 billion of the
unemployment tax revenue it had held in reserve. This was known as a Reed
Act distribution. As long as a state has a specific appropriation for its
legislature, it could use the funds for administrative costs of state UI.
^7The Wagner Peyser-funded activities are an integral part of the nation's
one-stop delivery system that provides employment-related services so that
workers, job seekers, and businesses can access the services they need in
a central location.
Most States Use Statistical Models to Identify Claimants Likely to Exhaust
Benefits, but Many Have Not Updated Them to Account for Changing Economic
Conditions
The large majority of states use statistical models to identify
unemployment recipients who are most likely to exhaust benefits. However,
many states have not recently adjusted their models, risking the
possibility that these models may lose predictive accuracy over time.
Forty-five states use statistical models to identify and rank clients by
their likelihood to exhaust benefits, while 7 states use characteristic
screens that do not rank claimants. One state--Florida--allows the local
areas to decide whether to use statistical models or screening tools.
Among the states using statistical models, the detailed specifications of
these models vary considerably from state to state. Further, many states
do not regularly update their models, a fact that can lead to a loss of
predictive accuracy over time. A survey of the states reveals that many
have not revised or updated their models in many years. Officials in
states we contacted explained that they face a number of impediments to
doing so.
Most States Use Statistical Models instead of More Limited Characteristic
Screens
Under worker profiling, state UI agencies are to identify claimants who
are most likely to exhaust benefits in two steps. States screen claimants
in order to eliminate claimants who are unemployed but job-attached or
would otherwise not qualify for referral to services from the profiling
process.^9 After the initial screening, states profile remaining
claimants--that is, they consider a range of personal and economic
variables related to a claimant and determine whether or not he or she is
likely to exhaust benefits. Although states have considerable flexibility
in determining what variables to use, Labor has recommended the use of
five variables, as outlined in table 1.^10
^8Reemployment Services grants could be used to fund services and are
different from the Reemployment Eligibility and Assessment (REA) grants
awarded by Labor to some states. REA grants are to be used by one-stop
centers to conduct in-person interviews of certain UI recipients to assess
their continuing eligibility for benefits and need for reemployment
services. They cannot be used to fund services, according to Labor
officials.
^9Labor requires that states screen out claimants who will be recalled to
work or who have a union hiring hall agreement. It also requires that
states exclude claimants who do not receive a first payment for total
unemployment and those who receive first payment for only partial claims.
Some states also exclude other claimants from profiling, such as
interstate claimants, and seasonal workers.
^10Labor also developed a prototype statistical model that some states
substantially adopted.
Table 1: Labor-Recommended Factors
Factor^a Impact on unemployment
Education Claimants with less education are more likely to exhaust
benefits.
Job tenure Claimants with long attachment to a specific employer
have more difficulty in finding equivalent employment
elsewhere.
Industry^a Claimants who worked in industries that are declining,
relative to others in a state, experience greater
difficulty in finding new employment than claimants who
worked in expanding industries.
Occupation^a Workers in low demand occupations experience greater
reemployment difficulty than workers in occupations with
high demand.
Unemployment rate Reemployment difficulty is closely related to economic
conditions, and in areas of high unemployment, workers
will have greater difficulty becoming reemployed than
workers in areas of low unemployment.
Source: U.S. Department of Labor, Employment and Training Administration,
Unemployment Insurance Program Letter No. 41-94, August 1994.
aLabor requires that state profiling models consider either a claimant's
industry or occupation. The other factors are recommended but optional.
We found that states used one of two methods to identify claimants who are
most likely to exhaust benefits--the statistical model or characteristics
screening. Both of these look at a range of personal and economic factors
that help predict exhaustion.
Forty-five of the 53 states and territories use statistical models to
identify clients likely to exhaust benefits. (See fig. 2.) Using various
statistical techniques, these models consider the combined quantitative
influence of various personal and labor market characteristics and produce
a measurement of a claimant's likelihood to exhaust. In statistical
models, each characteristic--commonly referred to as a variable--is
associated with a specific mathematical weight that quantifies the
variable's contribution to the claimant's probability of exhaustion. If,
for example, a claimant's last job was in a steeply declining industry,
the industry variable would have a positive effect on the score,
indicating a claimant's likelihood to exhaust. Conversely, if a claimant's
last job was in an expanding industry, it would have the opposite effect.
Essentially a statistical model produces a weighted average of the effect
of all the variables combined. As a result, states that use statistical
models can rank claimants from greatest to least likelihood of exhaustion,
and target reemployment services to claimants with the greatest likelihood
of exhausting. According to an official of the Upjohn Institute for
Employment Research,^11 such models, if properly developed and maintained,
are a powerful and effective means of identifying particular populations
for a range of social service programs.^12
^11The Upjohn Institute for Employment Research is a not-for-profit,
nonpartisan research organization founded to conduct research into the
causes and effects of unemployment and measures for the alleviation of
unemployment.
^12Programs using a statistical method for early identification of those
most likely to have long spells of unemployment have been used in other
countries, such as Australia and Canada, as well.
Figure 2: Profiling Techniques Used in the States
Seven of the 8 remaining states use characteristic screens that do not
allow them to rank claimants. One state, Florida, delegates the selection
of profiling tools to the local areas because state officials believe
profiling can be done more accurately at that level. Like statistical
models, characteristic screens may consider various factors associated
with the likelihood to exhaust benefits, but treat them as yes-no decision
points. Either the claimant has the attribute or does not. The relative
importance of any one variable in relation to others is not considered.
Claimants selected through this process must have each of the screening
criteria. For example, the characteristic screen used by Delaware
considers whether or not a claimant meets specific criteria relating to
industry, occupation, and job tenure. In Delaware, a claimant passes the
job tenure screen if he or she has 2 or more years of tenure with his or
her last employer. However, since claimants cannot be ranked, states using
screens must develop a method, such as random assignment, to refer
identified claimants to services if they are unable to serve all claimants
that pass the screens. For example, Delaware used to refer claimants who
passed the screen on a random basis, but now refers all claimants who pass
the screen. Labor encourages the use of statistical models over
characteristic screens because they are more efficient and precise in
identifying claimants likely to exhaust.
Statistical Models Vary in Key Elements
Although all statistical models are supposed to identify claimants who are
likely to exhaust benefits, the states can vary in how they specifically
define this exhaustion. The model originally proposed by Labor is designed
to predict the probability of exhaustion as a yes-or-no
outcome--exhaustion or no exhaustion--and the claimant's profiling score
would reflect the probability of the yes outcome. Most states have adopted
this definition. However, as Labor explained in 1998 guidance, this
approach does not distinguish between claimants who almost exhaust
benefits and those who do not come close to exhausting benefits.^13 This
is significant, because the claimant with nearly exhausted benefits may be
in greater need of reemployment services than the clamant who uses a
comparatively small portion of his or her benefits. Consequently, some
states predict exhaustion as the amount of benefits a claimant will
potentially use. For example, the profiling score produced by Kentucky's
model produces a number between 1 and 20. A claimant with a score of 20 is
likely to use 95 to 100 percent of benefits; a claimant with a score of 19
is likely to use 90 to 95 percent of benefits, and so on.
^13Marisa L. Kelso, "Worker Profiling and Reemployment Services Profiling
Methods: Lessons Learned," U.S. Department of Labor, Unemployment
Insurance Occasional Paper 99-5, June 1998.
State models can differ considerably in how they define similar variables,
including those corresponding to the factors recommended by Labor. For
example, California uses six categories to measure the job tenure
variable, ranging from 1 year or less on the low end to more than 15 years
on the high end. In contrast, Texas uses only two categories--job tenure
of less than 1 year on the low side and tenure of more than 10 years on
the high end. The Kentucky model, on the other hand, measures job tenure
on a continuous scale--specifically, the length of time that a claimant
held his or her last job. The definitions of variables associated with
education, industry, and other variables can also differ among state
models. For example, Kentucky includes "completed vocational education" as
part of its education variable, while Wisconsin does not.
The number and nature of the additional variables can also differ
significantly by state. The large majority of states using statistical
models (34 of 45) use models that consider factors in addition to the five
factors recommended by Labor, while about one-quarter do not. (See fig.
2.) Among the 6 states that we contacted that use statistical models, the
number of additional variables used ranged from 1 in California to over 50
in Kentucky. For example, 2 of the 7 states we contacted--Texas and
Illinois--consider the time lapse between the loss of a job and the
application for UI benefits. According to Texas officials and Labor,
delays in filing a claim are indicative of a difficult job search, thus
increasing the likelihood of benefit exhaustion.
Table 2: Selected Types of Variables Used in State Models beyond Federally
Recommended Variables
Increased
State likelihood
of
Variable category Calif. Ill. Ky. Tex. Wash. exhaustion
Delay in filing of Claimants who
claim delay filing
claims,
indicating an
unsuccessful job
search
Potential duration Claimants with
short duration
of eligibility
for benefits
Exhaustion rate by Claimants who
sub-state region live in areas of
the state that
have a high rate
of exhaustion
Past wages (base Claimants with
period wages) higher past
wages
Prior UI recipient Claimants who
have prior UI
claims
Reason for Claimants who
unemployment have been
discharged from
work for reasons
other than being
laid off
Number of recent Claimants with
employers more than one
employer
Employer's history Claimants whose
of layoffs^a employer has a
high propensity
to lay off
workers
Source: GAO document review and analyses of interviews with state
officials.
Note: This table does not include two of the seven states we contacted.
The Wisconsin profiling model uses only the five variables recommended by
Labor, and Delaware uses a characteristic screen that uses three of the
five recommended variables.
aAn employer's propensity to lay off workers serves as the basis of an
employer's unemployment insurance tax rate, and is known as an experience
rating.
Although Models Require Periodic Maintenance to Ensure Predictive Accuracy, Many
States Have Not Updated Their Models since 2000 or Before
While Labor has recommended that states update models periodically to
reflect changes in economic conditions, many states have not done so in
many years. If not periodically updated, statistical models can lose
predictive accuracy over time because of changes in the labor market, the
general economy, or other factors. Labor has emphasized the importance of
updating models, and noted in 1998 guidance that models represent the
historical period in which they were developed, and that old models become
increasingly unrealistic and less useful over time. Labor has further
recommended that models be assessed, and if necessary adjusted,
approximately every 3 years. Officials in some of the 7 states we
contacted also stressed the importance of updating models from time to
time. For example, Washington officials noted that although a 2002
analysis of their model update showed that it accurately identified the
majority of claimants who exhausted, this adjustment of their model was
based on data collected in 1999 and 2000, and subsequent changes in their
labor market and the general economy have made the model outdated. Also, a
2003 California study found that the state's model underestimated benefit
exhaustion and recommended an update to the model. Similarly, an official
of the Upjohn Institute for Employment Research told us that the
institute's analysis of 1 state's model found that before the model was
updated, its results were little better than random selection of
claimants. Officials in Washington and California told GAO that the models
would be updated in the next year.
Models can be adjusted by modifying the mathematical weights associated
with specific variables, and by adding, deleting, or redefining variables
to enhance a model's predictive power. This is necessary over time
because, although a particular variable--such as a claimant's
industry--can remain an important predictor of exhaustion, its relative
importance in the model can change significantly. For example, if a
variable's weight was estimated based on data from a historical period of
large changes of employment levels in a particular industry or industries,
the model might produce misleading results if used in a period of greater
industrial and employment stability. Similarly, a variable that once
served as an important predictor in a model may lose predictive value as
the labor market and economic circumstances change, and conversely, other
variables that may not have been relevant in one time period may become
important at another time. For example, Texas deleted education as a
variable from the model used in that state. According to a Texas official,
statistical work performed for the model update revealed that the
education variable did not measurably add to its predictive power.
Factors other than the labor market and general economy can affect the
reliability of models as well. For example, in the past 10 years
standardized coding used to identify both industries and occupations has
changed, ^14 and some of the states we contacted had not updated their
model to reflect this change. Illinois' analysis of its model showed that
while the model had generally retained predictive accuracy, areas of
concern existed. For example, as a result of outdated occupational codes,
certain occupations associated with greater likelihood of exhaustion were
no longer being targeted, while others not associated with exhaustion
were.
Although Labor has taken a number of actions to encourage and assist
states in updating their profiling models, some states have not done so
for many years. Labor has noted the importance of updating models in
written guidance, sponsored occasional seminars where best modeling
practices are shared with state staff, and provides on-demand technical
assistance to states. However, Labor has not established requirements for
updating models, and has not undertaken ongoing monitoring of state
models.^15 A recent Labor-commissioned survey revealed that many states
have not updated their profiling models in recent years.^16 (See figs. 3
and 4.) For example, although 21 states reported taking actions such as
adjusting variable weights since 2003, many others have not. Specifically,
18 states have not done so since 1999 or before, and 12 of these reported
never having done so.
^14Specifically, the Standard Industrial Code system has been replaced by
the North American Industrial Classification System, and the Dictionary of
Occupational Titles has been replaced by Standard Occupational
Classification System.
Figure 3: States' Adjustments of Model Coefficients
Note: Number of states and territories does not total 53 because 7 states
did not respond to this query.
^15The Unemployment Compensation Amendments of 1993 (Pub. L. No. 103-152)
do not require this type of monitoring of state performance.
^16In 2004, Labor commissioned a study of state profiling models, the
goals of which included determining the effectiveness of current models,
and developing guidance on best practices in operating and maintaining
worker profiling models. Labor conducted a survey of states in 2006 and
expects to publish this report in 2007.
According to Labor's survey results, states have been even less inclined
to adjust their models by taking actions such as changing or redefining
variables in the models. As figure 4 shows, 30 states reported that they
had not made such changes since implementation, and 23 states reported
having done so. Only 11 of these 23 states reported having done so since
2003.
Figure 4: Passage of Time since States Adjusted Models by Changing
Variables
Note: No states adjusted models between 1997 and 1999.
Labor's survey did not inquire about factors influencing the frequency
with which states update their models, but our contacts with 7 states
reveal a variety of reasons that some states have not updated their
models. Officials in California said that they had more pressing
priorities for UI administrative funds, and thus would have difficulty
funding model updates. Wisconsin officials said that revising the models
required expertise that they did not have, either in-house or from other
sources, such as a state university. Although Labor provides technical
guidance and advice, and has offered seminars on updating models, state
officials indicated they still need more continuous access to expertise in
order to keep models updated. A Texas official said that sometimes
historical data needed to determine a variable's impact on exhaustion of
benefits are not available, and so the variable cannot be included in the
model. Relatedly, if the necessary data on claimants are not collected, or
cannot be transmitted and used by the model, the related variable cannot
be used. For example, a Texas official told us that certain variables,
such as the number of a claimant's dependents or spousal income, might be
good predictive variables. However, the standard Texas application form
for UI benefits does not ask about the number of dependents or spousal
income, so these variables cannot be used.
Most Study States Did Not Take the In-Depth Approach Recommended by Labor to
Ensure That Profiled Claimants Obtain Reemployment Services
Labor data provide a limited picture of states' implementation of worker
profiling, and some aspects of these data were not reliable. Further, 6 of
our 7 study states did not offer the in-depth approach to services
prescribed by Labor. These states generally referred claimants to
services, held them accountable for attending the services, and provided
them with an orientation and some instruction on job search skills.
However, 6 of the 7 states did not adhere to Labor's guidance recommending
an in-depth individual needs assessment and a tailored reemployment
service plan for referred UI claimants.
Data Collected by Labor Provided a Limited Picture of States' Implementation of
Reemployment Services
Between 2002 and 2006, about 94 percent of the UI claimants who received a
first payment were profiled.^17 To the extent that reemployment services
are available, Labor requires that states refer profiled claimants to
these services. Of those profiled, an average of 15 percent were referred
to services, with states ranging from 5-year averages of 1 percent
(Wyoming) to 52 percent (Washington) (See fig. 5.) While 3 states referred
between 29 and 52 percent of profiled claimants to services, 28 states
referred 14 percent or fewer. Further, of those claimants profiled, an
average of 11 percent completed services, with states ranging from 1
percent (Arkansas, Colorado, Idaho, Michigan, and Wyoming) to 39 percent
(Texas). (See fig. 6.)^18 In 2 states, more than 27 percent of profiled
claimants completed services. However, in 33 states, 13 percent or fewer
of claimants did so. See appendix II for the average percentages of
profiled claimants referred to and completing services by state from 2002
to 2006.
^17The total number of claimants profiled can exceed the total number of
claimants who receive a first UI benefit payment because some states
profile claimants at the initial claim, and these claimants may never
receive a payment.
^18Labor collects these data from the states on Form ETA 9048, Worker
Profiling and Reemployment Services Activity. Appendix I contains a
description of how we derived these summary statistics using the raw data
from Labor.
Figure 5: Percentage of Claimants Referred to Services, of Those Profiled
Figure 6: Percentage of Claimants Completing Services, of Those Profiled
Labor's data are not sufficiently reliable to provide any information on
the specific services provided to claimants--such as orientation,
counseling, job search workshops, or job clubs. Specifically, Labor and
state officials told us that definitions of these services can vary across
states and within states over time as they change the content of their
programs. For example, California officials told us that the state's
definitions of services provided were established over 10 years ago and
that the nature of the services may have changed since then.
Six of Seven States We Studied Referred Claimants and Enforced Compliance with
Referrals
We found that 6 of the 7 study states had, as required by Labor, referred
profiled claimants to services and made claimants ineligible for benefits
if they failed to attend reemployment services. In contrast, officials in
1 state told us that referrals had been delegated to local workforce
areas, and that they did not know whether claimants were being referred to
services statewide. We subsequently contacted some local workforce
development offices in this state and learned that several had not been
referring UI profilees to reemployment services for years. In addition,
officials in this state told us that there are no consequences for those
who fail to attend reemployment services. They further said they do not
track information at the state level on whether claimants attend services.
While Labor requires that states hold claimants ineligible for benefits
for any week in which they fail to attend services, Delaware goes further
and holds the UI benefits of claimants who do not attend services until
they reschedule.
Some of the study states took additional steps to ensure compliance with
service referrals, while others did not. Of the states that referred
claimants to services, Delaware and Washington required that claimants
reschedule if they failed to attend required services, while Texas and
Wisconsin attempted to reschedule claimants in some cases and the
remaining states did not do so. ^19 Officials in Delaware reported that
they go so far as to have staff call claimants early during the week of
their scheduled orientation to remind them to attend; officials in
Washington said that some local workforce centers do this. Officials cited
the large flow of claimants into the program, the complexity of the
rescheduling process, and the scarcity of staff resources as reasons they
did not reschedule referred claimants.
Six of Seven States We Studied Provided Limited Reemployment Services and Did
Not Develop Individual Assessments and Service Plans
The reemployment services offered in the states we contacted generally did
not conform to the robust service process originally outlined by Labor.
Labor's 1994 guidance states that after initial orientation, the service
provider should determine the specific needs of each worker through an
assessment process, such as vocational testing.^20 Only one of our study
states, Delaware, required that case managers conduct an initial
assessment to determine what services claimants might need, such as
Workforce Investment Act (WIA) training, depending on their job readiness
level. Washington and Wisconsin required that claimants complete a
self-assessment. For example, claimants at one one-stop center were
expected to complete a one-page self-assessment that asks questions,
including what educational level they attained, whether they had a current
resume, and whether they had difficulty filling out a job application. The
4 other states we studied required no assessment of any kind.
According to state officials, our study states also generally did not
require, as recommended by Labor, that local offices develop or document a
reemployment services plan that could serve as the basis for determining
satisfactory participation. Only Delaware required case managers to
develop service plans and meet with claimants on a monthly basis after
each claimant's assessment. In California and Wisconsin, claimants
developed their own plans, which involved selecting an additional service
session on a topic the claimants felt would be most helpful. For example,
California required that UI claimants attend an orientation and choose an
additional service, such as a WIA service or job club, that would
constitute their individual reemployment plan.
^19At the time of our contact, a Washington official said that the state
sometimes rescheduled claimants for services, but that effective April 2,
2007, the state would require that claimants be rescheduled for services.
^20U.S. Department of Labor, Employment and Training Administration, Field
Memorandum No. 35-94, March 1994.
All 7 study states cited lack of or declining funding as an issue that
affected the provision of reemployment services. Specifically, some states
mentioned the loss of Labor's Reemployment Services grants, which had been
awarded to all states between 2001 and 2005 to enhance and target
integrated core services to claimants through the one-stop centers and
were used by some states to fund program-related services. A Wisconsin
official said that when the grant funds end in summer 2007, the state
would only be providing worker profiling services in 6 to 12 of its 75
local workforce development offices. State officials also mentioned
continuing declines in Wagner-Peyser, or Employment Services, funding. A
local workforce manager in Washington said that there is a vast gap
between the need for services and the resources and that the state only
has resources for about 5 percent of the 50,000 to 60,000 UI claimant
population. In order to help address this issue, officials in Washington
told us that a special surtax is applied to UI taxes, and a small portion
of this is diverted to worker-profiling service activities. While state
officials were concerned with the availability of funding, Labor officials
said that the purpose of the worker-profiling initiative is to target the
funding that does exist to those claimants who need it most and that the
program does not mandate that states serve any claimants they did not
serve prior to its implementation.
Officials also cited various day-to-day challenges in providing effective
reemployment services. A single services session can include claimants
ranging from former upper management employees to construction and factory
production workers, according to a Kentucky official. The same official
said that pitching the class so that it is effective for both types of
claimants can be difficult. Claimants' language skills also can be a
challenge. However, California addresses this by excusing non-English
speakers from the session, and directing them to job service centers or
community-based partners that provide reemployment services in their own
language, unless the orientation is available in their native language.
Little Is Known about Program's Effectiveness because There Are No Current
Studies and Labor's Data Are of Limited Usefulness
Little is known about the current effectiveness of the worker-profiling
initiative. Research studies, while generally finding that profiling and a
referral to services had a positive impact on claimants, used data from
the early implementation of the initiative--1994 to 1996. Although Labor
collects data on the outcomes of those profiled and referred to services,
we found portions of it to be unreliable.^21 In addition, state officials
said they do not use Labor's data for evaluation purposes.
Early Research Showed Some Positive Outcomes for Those Referred to Services, but
There Are No Current Studies
Five methodologically sound studies looking at the impacts of the
worker-profiling initiative after it was first implemented found that the
program had some desired effects. Examining data from 1994 to 1996, the
studies generally indicated that a referral to services under worker
profiling led to a reduction in claimants' duration on UI, a reduction in
the amount of UI benefits that were paid out, and an increase in
subsequent employment earnings. Though the methodologies varied, all the
studies evaluated the impacts of the referral to services using
statistical analyses to compare the outcomes of claimants who were
referred to services against those of claimants who were not.^22 As table
3 indicates, these studies cover a total of only 7 states, and no national
study exists. Further, no study using current data exists. Labor sponsored
the two multistate studies published in 1997 and 1999, but has not
published any subsequent studies.^23 According to Labor officials, the
agency has no current plans to study the effects of profiling. Because
data in all the studies were from the period when worker profiling was
first implemented, the profiling process and reemployment services
provided then may not reflect what states are currently offering.^24
^21The claimant outcomes data are descriptive data only and do not indicate
the effect of the worker-profiling program. Experimental and
quasi-experimental research studies that evaluate the impact of the
worker-profiling program may indicate how claimant outcomes differ due to
program participation.
^22These studies controlled for a variety of factors such as location;
profiling score; time period; personal characteristics, such as age, race,
sex, and education; and employment characteristics, such as base period
earnings, job tenure, industry, and previous occupation.
^23Texas, Washington, and Wisconsin officials said that state-sponsored
impact studies conducted on various aspects of the worker-profiling
initiative in their states were not complete or not yet published.
California state officials conducted an impact evaluation study of worker
profiling in the state, which was published in 2003, but the methodology
was not sufficiently rigorous to include in our report.
Table 3: Research Studies Included in Our Literature Review
Research study Data: states and time frame^a
Kentucky studies
Black and others 2003 Kentucky, 1994-1996
Black and others 2007 Kentucky, 1994-1996
Noel 1998^b Kentucky, 1994-1996
Multistate studies
Dickinson and others 1997 Delaware^c, Kentucky, New Jersey, 1994-1995
Dickinson and others 1999 Connecticut, Illinois, Kentucky, Maine, New
Jersey, South Carolina, 1995-1996
Source: GAO analysis of relevant studies. See bibliography for full
citations.
aDates indicate when claimants filed their UI claim or received their
first UI benefit payment.
bUnpublished dissertation.
cDelaware was included in this study, but its sample size was too small to
detect any significant impacts.
While these early studies showed positive impacts for referred claimants
with regard to reducing duration, reducing amount of UI benefits, and
increasing employment earnings, there were mixed results for whether the
program reduced the percentage of claimants who exhausted their benefits
or improved subsequent employment rates (See table 4.)^25 According to the
studies, claimants who were referred to services had a decreased UI
duration and received lower total amounts of UI benefits.^26 Most of the
studies found that claimants who were referred to services increased
earnings in the year following the UI claim. However, the largest
multistate study was unable to draw any conclusions about the impact on
earnings because of contradictory data. Evidence that a referral to
services reduced the percentage of claimants who exhausted their UI
benefits was mixed. For example, one study showed a decrease in the
percentage of claimants who exhausted their UI benefits in 3 states, but
an increase in 2 states. The effect of a referral to services on
employment rates was also inconclusive. According to the two multistate
studies, the effect was minimally positive for one state, but the other 6
states showed insignificant or contradictory results. Most of the studies,
however, did not examine subsequent employment rates.
^24The reemployment services received by claimants in these studies
typically included an orientation and then on average between one and two
additional services after orientation.
^25See appendix IV for more detailed information on the claimant outcome
effects broken out by research study.
^26Typically, a claimant can receive a maximum of 26 weeks of regular UI
benefits in a benefit year, though this duration can lengthen due to
partial benefits receipt or federally funded extensions in periods of high
unemployment rates. The amount of UI benefits received varies depending on
a claimant's previous employment earnings and state UI laws.
Table 4: Summary of Research Study Findings on the Effect of Referral to
Services on Claimant Outcomes
Range of effect found in research
Claimant outcome studies
Duration of UI receipt Reduced by 0.2 to 4 weeks
Amount of UI benefits received Reduced by $55 to $320
Lessened likelihood of UI benefit Inconclusive
exhaustion
Earnings following UI claim Increased by $218 to $1,054^a
Employment rates following UI claim Inconclusive
Source: GAO analysis of relevant studies.
aClaimant earnings subsequent to the UI claim may be underreported because
not all employers are covered by the UI system, and claimant earnings are
not tracked if the claimant moves to another state.
Research studies of other work search programs corroborate the generally
favorable results found in the impact evaluation studies of the
worker-profiling initiative. ^27 Though the methodologies varied, these
studies demonstrated that work search assistance reduced the duration
claimants received UI benefits, among other beneficial impacts. In two
demonstration projects, UI claimants who received job search assistance
received fewer weeks of UI benefits. The reemployment services offered in
these demonstration projects, however, were more robust; for example, in
one study, claimants were required to attend an orientation, testing, a
job search workshop, and a one-on-one assessment interview. As such, they
may not reflect what is offered through the states' worker-profiling
programs currently.
^27The research studies include the following: D. H. Klepinger, T. R.
Johnson, and J. M. Joesch. "Effects of Unemployment Insurance Work-Search
Requirements: The Maryland Experiment." Industrial and Labor Relations
Review, Vol. 56, No. 1. (October 2002), and P. T. Decker, R. B. Olsen, L.
Freeman, and D. H. Klepinger. "Assisting Unemployment Insurance Claimants:
The Long-Term Impacts of the Job Search Assistance Demonstration." U.S.
Department of Labor, Employment and Training Administration (February
2000). Both research studies used data from the mid-1990s.
Even though they were unable to provide supporting data, officials from
our study states said that worker profiling was a useful program for UI
claimants. They said it had enabled states to advertise their job search
and training services and target claimants who are most likely to exhaust
their UI benefits. In the process of referring claimants to services,
states are also educating the community on the many services and resources
available at the one-stop service centers. They also said the initiative
was a way to focus resources on those who would benefit from job search
assistance the most.
Outcomes Data Collected by Labor Are Limited and Not Consistently Used for
Evaluation Purposes
Due to reliability issues, Labor's claimant outcomes data are of limited
value. Labor's claimant outcomes data^28 were sufficiently reliable for us
to report only certain outcomes, including benefits exhaustion, weeks of
benefit receipt, and reemployment. ^29 Those data showed that less than
half of profiled claimants exhausted benefits, that on average they
received benefits for about two-thirds of the typical maximum time
allowed, and that about half found employment within 1 year of the
referral to services (see table 5).
^28Labor collects claimant outcomes data from the states on Form ETA 9049,
Worker Profiling and Reemployment Services Outcomes.
^29Three states and two territories were dropped from our analyses due to
large amounts of missing data. Also, as previously mentioned, we limited
our analysis to data collected since 2002, as Labor instituted data edit
checks that year. Despite the edit checks, we still found inaccuracies in
the outcomes data collected since 2002. For example, seven states have
been improperly reporting the claimants' wage data based on Labor's
definition and relative to the rest of the states. To the extent possible,
we estimated missing or incorrect data. See appendix I for a detailed
description of our methodology.
Table 5: National Averages and Ranges of State Averages on Outcomes for
Claimants Profiled and Referred to Services, 2002 to 2005
Range of state
Claimant outcome National average averages^a
Claimants who exhausted their UI 40 percent 13 percent to 60
benefits percent
Number of weeks that claimants received 17 weeks 7 to 27 weeks
UI benefits^b
Claimants who found employment at some 53 percent 22 percent to 87
point during the year subsequent to the percent
referral to services
Source: GAO analysis of U.S. Department of Labor data.
Note: Data on claimant outcomes are for the four quarters after the
referral to services or for the benefit year. See appendix I for a
description of the methodology used to calculate national and state
averages.
aIndividual states averages were approximately evenly distributed around
the national average.
bTypically, a claimant can receive a maximum of 26 weeks of regular UI
benefits in a benefit year, though this duration can lengthen due to
partial benefits receipt or federally funded extensions in periods of high
unemployment rates.
In addition to reliability issues, other characteristics, such as the lack
of a comparison group and long time lags, limit the usefulness of both the
reemployment services data and claimant outcomes data for states. First,
the outcomes data reflected only the experience of those who were referred
to services, and did not include an adequate point of comparison. It was
therefore impossible to know if these outcomes were different than they
would have been had the claimants not been referred to or completed
reemployment services.^30 Second, according to Labor officials, the data
were originally intended for states to evaluate the effectiveness of the
worker-profiling initiative. However, we found that neither Labor nor the
states used the data for this purpose.^31 Several state officials said the
time lag and aggregated nature of the data were insufficient for program
management purposes. The claimant outcomes data were not reported for more
than a year after claimants were referred to services, and some state
officials said they needed more timely data. Both the reemployment
services and claimant outcomes data were aggregated to the state level,
and some state officials said that local-level data would better meet
their management needs. Four of our seven study states indicated that they
did not utilize the reemployment services data or claimant outcomes data,
and some only reported them because it was required by Labor; the
remaining states said they used the reemployment services data for
nonevaluative purposes, such as determining how many services were
provided to claimants or the volume of claimants served under the
worker-profiling program.
^30Data for all UI claimants, which would include those profiled and
referred to services, show that between 2002 and 2005, on average
claimants received 16 weeks of benefits and 41 percent of claimants
exhausted benefits. According to a Labor official, in 2007, the Department
of Labor began collecting data on the percentage of all UI claimants who
find employment, and not all states have submitted the data.
^31Labor officials said they made limited use of the data. For example,
Labor used the data to verify that states comply with the statutory
requirements to profile and refer claimants, and they have used the data
for special project needs that have not included evaluating the
effectiveness of profiling and reemployment services.
In light of these data limitations, several state officials said they
developed their own program performance measures and reports instead of
using the reemployment services data and claimant outcomes data. For
example, Washington developed its own data warehouse system that links
data on UI benefits, reemployment services, and claimant wages. According
to officials, on a monthly basis they review performance indicators, such
as the number of UI claimants that find employment and the amount of time
it takes before finding employment.^32
Conclusions
Our findings suggest that although states continue to profile and refer
claimants to reemployment services, the worker-profiling initiative is not
a high priority at the federal level or in many states. In the past Labor
has set out broad guidelines for states on the design and maintenance of
profiling models. However, our analyses indicate that these have been
inadequate. Labor's 2006 survey of state profiling techniques revealed
that many states had not updated their profiling models for many years. As
a result, it is possible that many models have lost predictive accuracy,
and are referring claimants to services who are not in need of them, or
failing to refer claimants that are in need of them. However, the
worker-profiling program is required by law, and if there is to be a
continued federal mandate, it may be that a more assertive federal role is
necessary to ensure the integrity of those models.
A long time has passed since Labor articulated its vision of reemployment
services, and our review of seven states indicates that what is being
practiced is a diminished version of that vision. While the states we
studied indicated they provided orientation sessions that seemed to convey
important information, including job search skills, Labor's guidance
implies a more tailored and in-depth approach to services. It may be that
the original vision is no longer realistic or perhaps, in the states'
experience, necessary. Absent clarification at the federal level, it will
remain unclear what Labor expects from the states.
^32These performance indicators are for all UI claimants, not just the
claimants that are profiled and referred to services under the
worker-profiling program.
The national data on the worker-profiling initiative is of very limited
usefulness as a measure of program activity, outcomes, and effectiveness.
Many of the data are not usable because of inconsistent or incorrect
reporting, and neither Labor nor the states we contacted use the data for
evaluating the worker-profiling initiative. Further, even if all the
outcomes data were reported consistently and accurately, these data
cannot, by themselves, be used to measure the impact of the program. In
the end, by requiring the submittal of data that are of such limited
reliability and value, Labor is potentially wasting both its own and the
states' resources. Finally, absent information about the program's current
impact, Labor may find it more difficult to make decisions regarding the
best means for returning the unemployed to work more quickly.
Recommendations for Executive Action
To better ensure that claimants who need and could benefit from
reemployment services are referred, and to ensure that resources are not
unnecessarily expended on claimants not needing them, we recommend that
the Secretary of Labor:
1. Reevaluate the agency's worker-profiling data collection to
determine whether it is sufficient for its intended purpose. The
agency might assess gaps in data, evaluate data consistency,
confer with states on what data would be beneficial to them,
determine the purpose of the data collection and for whose benefit
the data are collected, and modify what Labor requires states to
collect.
2. Ensure that the Employment and Training Administration takes a
more active role to help ensure the accuracy of the state
profiling models. The agency might track states' management of
their models and actively encourage review and updating of models
in specific states where there have been no efforts to adjust the
model for a number of years. The agency could also assess whether
an expanded technical assistance effort is needed, and, if so,
take the lead in developing one.
3. Encourage states to adhere to Labor's vision for in-depth
reemployment services, such as conducting individualized needs
assessments and developing individual service plans, or issue
updated guidance if this original vision would be too burdensome
for the states.
4. Evaluate the impact of the worker-profiling program on the
reemployment of UI recipients to ensure the benefits are
commensurate with the resources invested.
Agency Comments
We provided a draft of this report to Labor for review and comment. In
general, Labor agreed with our findings and recommendations. Labor's
formal comments are reproduced in appendix V.
Labor also provided technical comments on the draft report, which we have
incorporated where appropriate.
We are sending copies of the report to interested congressional committees
and members, and the Secretary of Labor. We will also make copies
available to others upon request. In addition, our report will be
available at no charge on GAO's Web site at http://www.gao.gov .
A list of related GAO products is included at the end of the report. If
you or your staff has any questions about this report, please contact me
at (202) 512-7215. You may also reach me by e-mail at [email protected]
. Key contributors to this report are listed in appendix VI.
Sigurd R. Nilsen
Director, Education, Workforce, and Income Security
Issues
Appendix I: Objectives, Scope, and Methodology
Our objectives were to answer the following questions:
1. How do states identify unemployment claimants who are most
likely to exhaust benefits?
2. To what extent do states provide reemployment services as
recommended by Labor?
3. What is known about the effectiveness of the worker-profiling
initiative in accelerating the reemployment of unemployment
insurance claimants?
To answer the first question, we reviewed Labor's guidance about the
worker-profiling initiative, and reviewed literature and interviewed
experts with the Department of Labor and the Upjohn Institute for
Employment Research regarding profiling techniques. We also obtained and
analyzed the results of a 2006 Department of Labor-sponsored survey of the
53 states and territories.^1 This survey made numerous inquiries about the
structural and operational aspects of the profiling tools--such as
statistical models or characteristic screens--in use in the states.
Finally, we contacted officials in 7 states--California, Delaware,
Illinois, Kentucky, Texas, Washington, and Wisconsin. We selected some
states to ensure that we included certain aspects of worker profiling; for
example, we selected Kentucky because it had a very complex statistical
model with numerous variables, and we selected Delaware because it was one
of the few states that profiled claimants using a characteristic screen
instead of a statistical model. We also selected these states because they
ensured geographic dispersion and a range of populations sizes. In each of
these states, we reviewed documents describing the profiling model that
the state uses, and interviewed knowledgeable officials about the
variables used in the model, the degree to which the model has been
assessed and updated, and other matters.
To answer the second question, we reviewed Labor guidance regarding
reemployment services provided to Unemployment Insurance (UI) claimants
referred through the worker-profiling initiative, and obtained and
analyzed national data collected by the Department of Labor from states on
the Employment and Training Administration (ETA) 9048 Worker Profiling and
Reemployment Services Activity report. In this report, states submit to
Labor, by quarter, information such as the number of UI claimants
profiled, referred to services, and completing services.^2 During our
contacts with the 7 states mentioned above, we also obtained and reviewed
state documents describing policies about referral and reemployment
services for claimants profiled under the worker-profiling initiative. We
also interviewed knowledgeable state officials about these policies,
including referral and notification of claimants, enforcement of
participation requirements, and the type of reemployment services that are
offered to claimants. In 6 of these states, we also contacted officials at
local one-stop offices or regional offices to discuss how reemployment
services are managed and delivered. In 4 of these states, we also attended
the initial reemployment services session for claimants referred through
the worker-profiling initiative and recorded our observations on a
standard template.
^1The survey, which resulted in a 100 percent response rate, encompassed
the 50 states, as well as the District of Columbia, Puerto Rico, and the
Virgin Islands.
To answer the third question, we identified and reviewed six research
studies that evaluated the impact of profiling and the referral to
services on claimant outcomes. All the studies used regression techniques
to estimate the impact of a referral to services on a claimant's UI claims
experience or the subsequent earnings and employment activities.^3 A GAO
economist reviewed these studies and determined whether each study's
findings were generally reliable by evaluating the methodological
soundness of the studies and validity of the results and conclusions that
were drawn. On the basis of this assessment, we determined that five of
the six studies were methodologically rigorous enough to use in this
report. We confirmed with Labor and national experts on unemployment
insurance that these remaining five studies constituted the definitive
work done to date on the impact of the worker-profiling initiative.
Additionally, we reviewed these studies to assess the reemployment
services offered under the worker-profiling initiative. Finally, we
reviewed several studies on other work search programs that also evaluated
impacts on claimant outcomes. We also obtained and analyzed national data
collected by Labor from states via the ETA 9049 Worker Profiling and
Reemployment Services Outcomes report.^4 In this report, states report to
Labor on a quarterly basis information on the outcomes of referred
claimants, such as the average duration claimants received UI benefits and
the number of claimants that found employment in the year following
referral. Finally, in our contacts with the 7 states mentioned above, we
interviewed knowledgeable officials regarding the data collected by Labor
and their general views about the worker-profiling initiative, and in
particular whether they believed the initiative was having the intended
outcomes.
^2Data are reported for the quarter in which the activity occurred.
^3One study used a Wald estimator, a simple nonparametric regression.
^4All outcomes data were analyzed with respect to the cohort of claimants
referred to services in a report quarter rather than at the individual
claimant level. The date of the outcomes data is the quarter when the
claimants were referred to services.
We conducted a data reliability assessment on the ETA 9048 and ETA 9049
reports data, which included electronically checking the data and
interviewing Labor and state officials on the reliability of the data. On
the basis of our reliability assessment and interviews, we found that some
of the ETA 9048 and ETA 9049 reports had missing or inaccurate data. As a
result, we took the following actions to ensure the accuracy of the data.
First, because Labor instituted data edit checks starting in 2002, we
limited the time frame of our analysis to 2002 to the most recent
available, September 2006 and March 2005 for the ETA 9048 and ETA 9049,
respectively. Second, we disregarded data from states that had excessive
amounts of missing data reports. Specifically, from the ETA 9048, we
excluded Louisiana, New Mexico, Puerto Rico, and the Virgin Islands, and
for the ETA 9049, we also excluded Idaho and New Jersey, in addition to
those states dropped for the ETA 9048.^5 Third, we estimated data values,
if possible, for states that had sporadically missing reports or data that
were anomalous or illogical, for example, when the number of claimants who
found employment exceeded the number referred to services. Of the data we
reported from the ETA 9048 and ETA 9049, we estimated approximately 1
percent of these data; because of this small proportion, we believe that
any errors arising from our estimation process did not significantly
affect the state and national averages we reported. Some possible issues
resulting from our estimation process were the following:
o We utilized logical relationships between data to estimate
values, and at times, these values were based on other estimated
data. Any errors resulting from the previous estimation would be
carried over to the following estimated value.
o Some states had volatile data, and as our estimation process was
based on the existing state data, it is uncertain how accurate our
estimates were.
o At times, our estimated values were the highest or the lowest in
the data series, and it is possible that the estimation procedure
resulted in an inaccurate value.
^5Two of the three outcomes data we report for the ETA 9049 are calculated
using data from the ETA 9048, and hence states dropped for the ETA 9048
also were dropped for the ETA 9049.
Fourth, we excluded data from states that we confirmed were reported
incorrectly. Specifically, for the ETA 9049, California and Georgia were
excluded from calculations using the number of claimants who become
employed, and Illinois was dropped from all analyses of both the ETA 9048
and ETA 9049 data. Last, we did not use any of the detailed reemployment
services data, such as the number of claimants that completed an
orientation, assessment, and so forth, because both Labor and state
officials said these data were not comparable within and between states.
Appendix II: Average Percentage of Claimants Profiled, Referred to, and
Completing Services for 2002-2006 and Average Claimant Outcomes for
2002-2005, by State
Claimants Length of
receiving time
first UI Profiled referred
benefit Profiled claimants Referred Referred claimants
payment who claimants who claimants claimants receive UI
were referred to completed who exhaust who become benefits
profiled services services UI benefits employed (weeks)
State (2002-2006) (2002-2006) (2002-2006) (2002-2005) (2002-2005) (2002-2005)
Ala. 102% 3% 3% 47% 56% 18.3
Alaska 89% 8% 5% 42% 60% 14.2
Ariz. 86% 16% 9% 37% 51% 14.6
Ark. 95% 2% 1% 46% 38% 21.0
Calif. 59% 14% 7% 32% Not 21.3
available
Colo. 100% 2% 1% 45% 58% 16.3
Conn. 82% 13% 8% 47% 58% 17.7
Del. 162% 4% 3% 58% 87% 20.6
D.C. 93% 6% 4% 57% 52% 20.7
Fla. 90% 36% 21% 29% 29% 27.0
Ga. 74% 25% 24% 49% Not 15.6
available
Hawaii 94% 11% 7% 40% 60% 18.2
Idaho 114% 2% 1% Not Not Not
available available available
Ind. 97% 8% 5% 57% 61% 13.3
Iowa 37% 21% 13% 39% 58% 18.9
Kan. 71% 6% 6% 60% 52% 12.8
Ky. 136% 13% 10% 43% 50% 18.1
Maine 100% 18% 11% 40% 64% 15.1
Md. 100% 24% 12% 51% 43% 20.9
Mass. 100% 26% 20% 53% 48% 20.2
Mich. 95% 2% 1% 36% 55% 17.0
Minn. 114% 21% 19% 39% 59% 17.5
Miss. 115% 22% 13% 44% 59% 16.0
Mo. 73% 8% 6% 51% 56% 16.6
Mont. 171% 4% 3% 53% 64% 7.4
Neb. 22% 28% 24% 31% 85% 16.1
Nev. 96% 4% 3% 31% 46% 7.3
N.H. 112% 26% 26% 13% 74% 15.7
N.J. 86% 14% 14% Not Not Not
available available available
N.Y. 93% 12% 11% 60% 52% 20.9
N.C. 37% 12% 7% 18% 22% 20.1
N.D. 123% 20% 15% 18% 70% 7.8
Ohio 94% 13% 7% 21% 62% 21.0
Okla. 115% 27% 23% 48% 51% 18.5
Ore. 74% 13% 9% 42% 45% 24.0
Pa. 96% 15% 10% 30% 28% 22.7
R.I. 74% 20% 18% 48% 43% 18.0
S.C. 73% 21% 14% 37% 52% 10.2
S.D. 86% 7% 6% 26% 55% 13.8
Tenn. 80% 10% 8% 35% 40% 21.2
Tex. 89% 49% 39% 39% 39% 17.8
Utah 94% 26% 24% 45% 38% 15.0
Vt. 99% 7% 5% 22% 48% 17.2
Va. 55% 12% 7% 31% 31% 24.2
Wash. 116% 52% 38% 26% 65% 14.8
W.Va. 122% 21% 18% 45% 54% 19.7
Wis. 104% 8% 7% 44% 55% 16.7
Wyo. 115% 1% 1% 43% 47% 15.4
National 94% 15% 11% 40% 53% 17.3
Average
Source: GAO analyses of Labor data, 2002-2006.
aGAO analysis based on 47 states and the District of Columbia.
Notes: For the percentages profiled, referred to services, and completed
services, Illinois, Louisiana, New Mexico, Puerto Rico, and the Virgin
Islands were excluded because of reliability concerns or missing data. For
the percentage exhausted, percentage employed, and length of UI benefits
received, Idaho and New Jersey were also excluded due to missing data.
California and Georgia were excluded only from the percentage employed
averages due to reliability concerns. See appendix I for further detail.
Appendix III: Bibliography of Research Studies on the Worker-Profiling
Initiative--Exhaustive List Identified from the Literature Review
Black, Dan A., Jeffrey A. Smith, Mark C. Berger, and Brett J. Noel. "Is
the Threat of Reemployment Services More Effective than the Services
Themselves? Evidence from Random Assignment in the UI System." The
American Economic Review, Vol. 93, No. 4 (November 2003).
Black, Dan A., Jose Galdo, and Jeffrey A. Smith. "Evaluating the Worker
Profiling and Reemployment Services System Using a Regression
Discontinuity Approach." Paper presented at the American Economic
Association conference in January 2007. Submitted to The American Economic
Review for the May 2007 Papers and Proceedings Issue.
Dickinson, Katherine P., Suzanne D. Kreutzer, and Paul T. Decker.
"Evaluation of Worker Profiling and Reemployment Services Systems: Report
to Congress." U.S. Department of Labor, Employment and Training
Administration (March 1997).
Dickinson, Katherine P., Suzanne D. Kreutzer, Richard W. West, and Paul T.
Decker. "Evaluation of Worker Profiling and Reemployment Services: Final
Report." U.S. Department of Labor, Employment and Training Administration
Research and Evaluation Report Series 99-D (1999).
Noel, Brett J. "Two Essays on Unemployment Insurance: Claimant Responses
to Policy Changes." Dissertation submitted for the degree of Doctor of
Philosophy at the Graduate School of the University of Kentucky, UMI
Number: 9922624 (1998).
Appendix IV: Summary of the Impact of Referral to Services on Claimant
Outcomes from the Literature Review
Employment
Duration Benefit Earnings rate
Research of UI Amount of UI exhaustion following UI following UI
study Data^a receipt benefits rate claim^b claim
Kentucky studies
Black and KY, Reduced Reduced by Not Increased by Not
others 1994-1996 by 2.2 $143^c significant $1,054 in available
2003 weeks the year
following UI
claim
Black and KY, Reduced Inconsistent Not Increased by Not
others 1994-1996 by 0.4 results^d available $648 to available
2007 to 2.3 $1,054 in
weeks the year
following UI
claim
Noel KY, Reduced Reduced by Not Increased by Not
1998^e 1994-1996 by 2.2 $65 to $320 available $218 to available
to 4 $1,054 in
weeks the year
following UI
claim
Multi-state studies
Dickinson DE^f, KY, KY, NJ: KY, NJ: NJ: Reduced NJ: NJ:
and NJ, Reduced Reduced by by 4 Increased by Increased by
others by 0.6 $96 to $109 percentage $190 and 1 percentage
1997 1994-1995 to 0.7 points $226 in the point^c in
weeks first and first
second quarter
quarters,
respectively
Dickinson CT, IL, CT, IL, CT, IL, ME, CT, ME, NJ: Inconsistent Inconsistent
and KY, ME, KY, ME, NJ: Reduced Reduced by results results
others NJ, SC, NJ: by $55^c to 1.4 to 4.3
1999 1995-1996 Reduced $139 percentage
by 0.2^c points SC,
to 1 KY:
week Increased
by 1.1^c to
4.1
percentage
points
Source: GAO analysis from literature review. See bibliography for full
citations.
Note: Only results significant at the 95 percent confidence level are
included unless otherwise noted.
aDates indicate when claimants filed their UI claim or received their
first UI benefit payment.
bThe earnings may be underreported because not all employers are covered
by the UI system, and claimant earnings are not tracked if the claimant
moves to another state.
cSignificant at the 90 percent confidence level.
dThe results with the least likelihood of error show a reduction in the
amount of UI benefits received of $175.
eUnpublished dissertation.
fDelaware was included in this study, but its sample size was too small to
detect any significant impacts.
Appendix V: Comments from the Department of Labor
Appendix VI: GAO Contacts and Acknowledgments
GAO Contact
Sigurd Nilsen, Director, (202) 512-7215 or [email protected]
Staff Acknowledgments
Patrick di Battista, Assistant Director, and Michael Hartnett, managed
this engagement.
Shannon Groff, Rosemary Torres Lerma, and Winchee Lin also made
significant contributions throughout the engagement. Susan Bernstein
helped develop the report's message. Jay Smale, Stuart Kaufman, Rhiannon
Patterson, Robert Dinkelmeyer, and Greg Dybalski contributed to the
analysis of Labor data and reviews of external studies. Jessica Botsford
provided legal support.
Related GAO Products
Workforce Investment Act: Employers Found One-Stop Centers Useful in
Hiring Low-Skilled Workers; Performance Information Could Help Gauge
Employer Involvement. [27]GAO-07-167 . Washington, D.C.: December 22,
2006.
Unemployment Insurance: States' Tax Financing Systems Allow Costs to Be
Shared among Industries. [28]GAO-06-769 . Washington, D.C.: July 26, 2006.
Unemployment Insurance: Enhancing Program Performance by Focusing on
Improper Payments and Reemployment Services. [29]GAO-06-696T . Washington,
D.C.: May 4, 2006.
Unemployment Insurance: Factors Associated with Benefit Receipt and
Linkages with Reemployment Services for Claimants. [30]GAO-06-484T .
Washington, D.C.: March, 15, 2006.
Unemployment Insurance: Factors Associated with Benefit Receipt.
[31]GAO-06-341 . Washington, D.C.: March 7, 2006.
Workforce Investment Act: Labor and States Have Taken Actions to Improve
Data Quality, but Additional Steps Are Needed. [32]GAO-06-82 . Washington,
D.C.: November 14, 2005.
Unemployment Insurance: Better Data Needed to Assess Reemployment Services
to Claimants. [33]GAO-05-413 . Washington, D.C.: June 24, 2005.
Workforce Investment Act: Labor Should Consider Alternative Approaches to
Implement New Performance and Reporting Requirements. [34]GAO-05-539 .
Washington, D.C.: May 27, 2005.
Unemployment Insurance: Information on Benefit Receipt. [35]GAO-05-291 .
Washington, D.C. March 17, 2005.
Workforce Investment Act: Employers Are Aware of, Using, and Satisfied
with One-Stop Services, but More Data Could Help Labor Better Address
Employers' Needs. [36]GAO-05-259 . Washington, D.C.: February 18, 2005.
Workforce Investment Act: States and Local Areas Have Developed Strategies
to Assess Performance, but Labor Could Do More to Help. [37]GAO-04-657 .
Washington, D.C.: June 1, 2004.
Workforce Investment Act: One-Stop Centers Implemented Strategies to
Strengthen Services and Partnerships, but More Research and Information
Sharing Is Needed. [38]GAO-03-725 . Washington, D.C.: June 18, 2003.
(130587)
GAO's Mission
The Government Accountability Office, the audit, evaluation and
investigative arm of Congress, exists to support Congress in meeting its
constitutional responsibilities and to help improve the performance and
accountability of the federal government for the American people. GAO
examines the use of public funds; evaluates federal programs and policies;
and provides analyses, recommendations, and other assistance to help
Congress make informed oversight, policy, and funding decisions. GAO's
commitment to good government is reflected in its core values of
accountability, integrity, and reliability.
Obtaining Copies of GAO Reports and Testimony
The fastest and easiest way to obtain copies of GAO documents at no cost
is through GAO's Web site ( www.gao.gov ). Each weekday, GAO posts
newly released reports, testimony, and correspondence on its Web site. To
have GAO e-mail you a list of newly posted products every afternoon, go to
www.gao.gov and select "Subscribe to Updates."
Order by Mail or Phone
The first copy of each printed report is free. Additional copies are $2
each. A check or money order should be made out to the Superintendent of
Documents. GAO also accepts VISA and Mastercard. Orders for 100 or more
copies mailed to a single address are discounted 25 percent. Orders should
be sent to:
U.S. Government Accountability Office 441 G Street NW, Room LM Washington,
D.C. 20548
To order by Phone: Voice: (202) 512-6000
TDD: (202) 512-2537
Fax: (202) 512-6061
To Report Fraud, Waste, and Abuse in Federal Programs
Contact:
Web site: www.gao.gov/fraudnet/fraudnet.htm
E-mail: [email protected]
Automated answering system: (800) 424-5454 or (202) 512-7470
Congressional Relations
Gloria Jarmon, Managing Director, [email protected] (202) 512-4400 U.S.
Government Accountability Office, 441 G Street NW, Room 7125 Washington,
D.C. 20548
Public Affairs
Paul Anderson, Managing Director, [email protected] (202) 512-4800
U.S. Government Accountability Office, 441 G Street NW, Room 7149
Washington, D.C. 20548
www.gao.gov/cgi-bin/getrpt?GAO-07-680 .
To view the full product, including the scope
and methodology, click on the link above.
For more information, contact Sigurd Nilsen at (202) 512-7215 or
[email protected].
Highlights of [46]GAO-07-680 , a report to congressional requesters
June 2007
UNEMPLOYMENT INSURANCE
More Guidance and Evaluation of Worker-Profiling Initiative Could Help
Improve State Efforts
Changes to the U.S. economy have led to longer-term unemployment. Many
unemployed workers receive Unemployment Insurance (UI), which provided
about $30 billion in benefits in 2006. In 1993, Congress established
requirements--now known as the Worker Profiling and Reemployment Services
(WPRS) initiative--for state UI agencies to identify claimants who are
most likely to exhaust their benefits, and then refer such claimants to
reemployment services.
To assess the implementation and effect of the initiative, GAO examined
(1) how states identify claimants who are most likely to exhaust benefits,
(2) to what extent states provide reemployment services as recommended by
the Department of Labor (Labor), and (3) what is known about the
effectiveness of the initiative in accelerating reemployment. To answer
these questions, we used a combination of national data; review of seven
states, including visits to local service providers in four states; and
existing studies and interviews with Labor and subject matter experts.
[47]What GAO Recommends
GAO recommends that Labor reevaluate worker-profiling data collection,
take a more active role in ensuring the accuracy of state models,
encourage states to adhere to Labor's vision for reemployment services,
and consider evaluating the impact of the program. The Secretary of Labor
generally agreed with our findings and recommendations.
Forty-five of the 53 states and territories use statistical models that
facilitate the ranking of claimants by their likelihood to exhaust
benefits, while 7 states use more limited screening tools that do not
facilitate a ranking. Florida delegates the selection of profiling tools
to local areas in the state. Factors used to determine the probability of
exhaustion include a claimant's education, occupation, and job tenure.
Many states have not regularly maintained their models, and as a result,
the models in some states may not be accurately identifying claimants who
are likely to exhaust benefits.
Although Labor data provide a limited picture of states' implementation of
the worker-profiling initiative, 6 of the 7 states we studied did not
provide the in-depth approach to services as recommended by Labor.
Overall, an average of 15 percent of profiled UI claimants were referred
to reemployment services, and 11 percent completed these services between
2002 and 2006. Six of the 7 states we contacted referred claimants to
services, held them accountable for attending the services, and provided
an orientation. However, only 1 of the 7 states provided individualized
needs assessments, and developed service plans, as recommended.
Little is known about the effectiveness of the worker-profiling initiative
as it is currently operating. Although studies using data from the 1990s
generally indicated that claimants who were referred to services had
reduced reliance on UI, there are no more up-to-date studies. Further,
some of the program data collected by Labor are not reliable, and the data
are not being used by Labor or states to evaluate the initiative.
Profiling Techniques Used in the United States
References
Visible links
27. http://www.gao.gov/cgi-bin/getrpt?GAO-07-167
28. http://www.gao.gov/cgi-bin/getrpt?GAO-06-769
29. http://www.gao.gov/cgi-bin/getrpt?GAO-06-696T
30. http://www.gao.gov/cgi-bin/getrpt?GAO-06-484T
31. http://www.gao.gov/cgi-bin/getrpt?GAO-06-341
32. http://www.gao.gov/cgi-bin/getrpt?GAO-06-82
33. http://www.gao.gov/cgi-bin/getrpt?GAO-05-413
34. http://www.gao.gov/cgi-bin/getrpt?GAO-05-539
35. http://www.gao.gov/cgi-bin/getrpt?GAO-05-291
36. http://www.gao.gov/cgi-bin/getrpt?GAO-05-259
37. http://www.gao.gov/cgi-bin/getrpt?GAO-04-657
38. http://www.gao.gov/cgi-bin/getrpt?GAO-03-725
46. http://www.gao.gov/cgi-bin/getrpt?GAO-07-680
*** End of document. ***