Veterans' Employment and Training Service: Further Changes Needed
to Strengthen Its Performance Measurement System (07-JUN-01,
GAO-01-757T).
The Veterans' Employment and Training Service (VETS) administers
programs and activities designed to help veterans obtain
employment and training assistance. Recently, policymakers have
focused increasing attention on VETS and its programs, advocating
changes to the structure and administration of the program and in
the way it assesses program performance. This testimony discusses
VETs' efforts to improve its performance measurement system. VETS
has proposed changes to its performance measurement system that
will move it closer to implementing an effective accountability
system. However, additional changes are needed so that VETS can
effectively determine whether its programs and services are
fulfilling its mission. VETS continues to send a mixed message to
states about what services to provide and to whom. In addition,
two of the proposed measures may provide nearly identical
results, and neither helps VETS to monitor whether more intensive
services are being provided to veterans or whether these services
are successful. Furthermore, through its planning documents and
proposed performance measures, VETS continues to inconsistently
identify the groups of veterans that it wants states to help.
This testimony summarizes the May report, GAO-01-580.
-------------------------Indexing Terms-------------------------
REPORTNUM: GAO-01-757T
ACCNO: A01089
TITLE: Veterans' Employment and Training Service: Further
Changes Needed to Strengthen Its Performance Measurement System
DATE: 06/07/2001
SUBJECT: Strategic planning
Data collection
Reporting requirements
Performance measures
State-administered programs
Veterans employment programs
DOL Disabled Veterans Outreach Program
DOL Local Veterans Employment Reps
Program
Unemployment Insurance Program
Wage Record Interchange System
******************************************************************
** This file contains an ASCII representation of the text of a **
** GAO Testimony. **
** **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced. Tables are included, but **
** may not resemble those in the printed version. **
** **
** Please see the PDF (Portable Document Format) file, when **
** available, for a complete electronic file of the printed **
** document's contents. **
** **
******************************************************************
GAO-01-757T
Before the Subcommittee on Benefits, Committee on Veterans' Affairs, House
of Representatives
United States General Accounting Office
GAO For Release on Delivery Expected at 9: 30 a. m. EDT Thursday, June 7,
2001 VETERANS'
EMPLOYMENT AND TRAINING SERVICE
Further Changes Needed to Strengthen Its Performance Measurement System
Statement of Sigurd R. Nilsen, Director Education, Workforce, and Income
Security Issues
GAO- 01- 757T
Page 1 GAO- 01- 757T
Mr. Chairman and Members of the Subcommittee: Thank you for inviting me here
today to discuss our recent report on the Veterans? Employment and Training
Service?s (VETS) performance measurement system. VETS, as part of the
Department of Labor, administers programs and activities designed to help
veterans obtain employment and training assistance. Recently, policymakers
have focused increasing attention on VETS and its programs, advocating
changes to the structure and administration of the program and in the way it
assesses program performance. This Subcommittee introduced legislation
during the 106th Congress that would restructure the program and require a
new, comprehensive performance accountability system. In addition, in 1999,
the Congressional Commission on Servicemembers and Veterans? Transition
Assistance recommended that the Congress establish effective outcome
measures for VETS. This focus on reform comes at a time when other federally
funded employment and training programs are changing the way they provide
services and measure performance. For example, the Workforce Investment Act
(WIA), which provides employment and training assistance for youth, adults,
and dislocated workers through onestop centers, recently established outcome
measures that are similar to those that VETS now proposes.
In our recently released report, requested by this Subcommittee, we reviewed
VETS? efforts to improve its performance measurement system. 1 In that
report, we examined VETS? proposed performance measures, the data source
VETS proposes to use in the new system and other measurement issues that may
affect the comparability of states? performance data. Our work was based on
interviews and discussions we had with over 45 officials in 15 states,
interviews with VETS officials, and a review of government documents and
other relevant reports.
In summary, VETS? proposed performance measures are an improvement over the
current performance accountability system because they focus more on what
its programs achieve and less on the number of services provided by staff
serving veterans. They also no longer require states to compare the level
and associated service outcomes provided to veterans with those provided to
nonveterans and are more closely aligned with WIA program measures, making
it easier for service providers to report on
1 Veterans? Employment and Training Service: Proposed Performance
Measurement System Improved, But Further Changes Needed (GAO- 01- 580, May
15, 2001).
Page 2 GAO- 01- 757T
outcomes. However, our work revealed a few areas of concern with the
proposed measures. A comparison of the performance measures with the
strategic plan indicates that VETS is sending a mixed message to states
about what services to provide and to whom. VETS? strategic plan suggests
that states focus their efforts on providing staff- assisted services to
veterans, including case management, yet none of the proposed measures
specifically gauge the success of these services. In addition, VETS?
proposal includes one measure- the number of federal contractor jobs listed
with local employment offices- that is not only process- oriented but also
focuses on outcomes that are beyond the control of staff serving veterans.
VETS also proposes that all states use a single data source- Unemployment
Insurance (UI) wage records- to identify veterans who get jobs. Using wage
record data will greatly improve the comparability and reliability of the
new measures, however, these data also present some challenges that states
will need to overcome. States generally do not have access to wage records
from other states and, therefore, may not be able to track individuals who
receive services in one state but get a job in another. In addition, certain
employment categories are not included in wage record data. Finally, there
are other issues related to how states count veterans for performance-
reporting purposes that VETS needs to consider as they finalize their
performance- reporting requirements. In our report, we made several
recommendations to VETS regarding their proposed performance measurement
system that could strengthen VETS? efforts to effectively measure the
performance of its programs.
VETS administers national programs intended to (1) ensure that veterans
receive priority in employment and training opportunities from the
employment service; (2) assist veterans, reservists, and National Guard
members in securing employment; and (3) protect veterans? employment rights
and benefits. The key elements of VETS? services include enforcing veterans?
preference and reemployment rights and securing employment and training
services. VETS? programs are among those federal programs whose services
have been affected by WIA and other legislative changes aimed at
streamlining services and holding programs accountable for their results.
VETS carries out its responsibilities through a nationwide network that
includes representation in each of Labor?s 10 regions and staff in each
state. The Office of the Assistant Secretary for Veterans? Employment and
Training administers VETS? activities through regional administrators and a
VETS director in each state. These VETS staff are the link between VETS
Background
VETS? Programs
Page 3 GAO- 01- 757T
and the states? employment service system, which is overseen by Labor?s
Employment and Training Administration (ETA). VETS funds two primary
veterans? employment assistance grants to states- the Disabled Veterans?
Outreach Program (DVOP) and the Local Veterans? Employment Representatives
(LVER). Fiscal year 2001 appropriation for VETS was about $212 million,
including $81.6 million for DVOP specialists (DVOPS) and $77.2 million for
LVER staff. 2 These funds paid for 1,327 DVOP positions and 1,206 LVER
positions.
The DVOP and LVER programs provide employment and training opportunities
specifically for veterans. A key responsibility of a DVOP is to develop job
and job training opportunities for veterans through contacts with employers,
especially small- and medium- size private sector employers. LVERs are to
provide program oversight of local employment service offices to ensure that
veterans receive maximum employment and training opportunities from the
entire local office staff. 3 In addition, DVOPS and LVERs traditionally have
provided services that include locating veterans who need services and
providing case management for those veterans in need of more intensive
services. Increasingly, however, veterans are accessing services on their
own, through tools such as internet- based job listings or resume writing
software.
As part of the DVOP and LVER grant agreements, states must ensure that
veterans receive priority at every point where public employment and
training services are available. The DVOP and LVER programs give priority to
the needs of disabled veterans and veterans who served during the Vietnam
era, and state employment service systems are expected to give priority to
veterans over nonveterans. To monitor the states? programs, VETS has been
using a set of measures that evaluates states? performance in five
dimensions: (1) veterans placed in training, (2) those receiving counseling,
(3) those receiving services, (4) those entering employment, and (5) those
obtaining federal contractor jobs. These measures primarily count the number
of services that veterans receive and compare the totals with similar
services provided to nonveterans. To ensure priority service
2 In addition to funds for DVOPS and the LVERs, VETS? fiscal year 2001
appropriation included funds for the Homeless Veteran Program and Veterans
Workforce Investment Program.
3 The roles of the DVOPS and LVERs have been separately defined in two
statutes. LVERs were first authorized under the original GI bill (the
Servicemen?s Readjustment Act of 1944) and DVOP specialists were authorized
by the Veterans? Rehabilitation and Education Amendments of 1980.
Page 4 GAO- 01- 757T
to veterans, VETS expects levels of performance for services provided to
veterans to be higher than levels for nonveterans. For example, veterans and
other eligibles must be placed in or obtain employment at a rate 15 percent
higher than that achieved by nonveterans.
To report on performance, VETS currently relies on the Employment and
Training Administration?s 9002 system to aggregate data reported by states
on veterans and nonveterans who register with state Employment Services (ES)
offices, track the services provided to them (such as counseling or job
referral), and gather information on their employment outcomes. The 9002
system also collects information such as the registrants? employment status,
level of education (e. g., high school, postsecondary degree/ certificate),
and basic demographic information, such as age and race.
Over the past several years, the Congress has taken steps to streamline and
integrate services provided by federally funded employment and training
programs. WIA, which the Congress passed in 1998, requires states and
localities to use a one- stop center structure to provide access to most
employment and training services in a single location. WIA requires about 17
categories of programs, including VETS and ES programs, to provide services
through the one- stop center. While DVOP and LVER staff are required to
provide assistance only to veterans, it is unclear how their services will
be integrated at the one- stop centers. However, according to VETS
officials, agreements made with each state on planned services to veterans
now include provisions on how DVOPS and LVERs will be integrated into the
one- stop delivery system.
In addition to changing the way services are provided, programs are now
increasingly held accountable for their results. Through the Government
Performance and Results Act of 1993 (GPRA), the Congress seeks to improve
the efficiency, effectiveness, and public accountability of federal agencies
as well as improve congressional decision making. GPRA does so, in part, by
promoting a focus on what the program achieves rather than tracking program
activities. GPRA outlines a series of steps in which agencies are required
to identify their goals, measure performance, and report on the degree to
which those goals were met. Legislative and Regulatory
Changes Affecting VETS
Page 5 GAO- 01- 757T
To address the goals of GPRA and in response to recommendations by us and
other groups, such as the Congressional Commission on Servicemembers and
Veterans Transition Assistance, 4 VETS is currently developing a new system
to measure the performance of its programs. Over the last several years,
VETS conducted pilot programs in about eight states that tested some new
performance measures and the use of new data to support these measures. VETS
officials told us that they anticipate implementing their new performance
measurement system in program year 2001.
VETS? proposed performance measures include an (1) enteredemployment rate,
(2) employment rate following staff- assisted services, (3) employment
retention rate, and (4) increase in the number of federal contractor job
openings listed. These measures are an improvement over current ones, but
certain aspects of these measures raise concerns that VETS needs to address.
The proposed performance measures improve accountability because they place
more emphasis on employment- related outcomes by eliminating process-
oriented measures- measures that simply track services provided to veterans.
Current process measures that VETS eliminated from the proposed performance
system include the number of veterans referred to counseling, the number
placed in training, and the number receiving certain other services, such as
job referrals. As we noted in past reports, these process- oriented measures
are activity- and volume- driven and focus efforts on the number of services
provided, not on the outcomes veterans achieve. 5 These measures offer
states little incentive to provide services to those veterans who are only
marginally prepared for work and who may need more intensive services
requiring more staff time. The VETS? proposal still includes one process-
oriented measure that simply reflects the percentage increase in the number
of federal contractor job openings listed with the public labor exchange but
adds two outcome- oriented
4 See Report of the Congressional Commission on Servicemembers and Veterans
Transition Assistance, January 14, 1999, Congressional Commission on
Servicemembers and Veterans Transition Assistance, Arlington, VA. 5 See
Veterans? Employment and Training Service: Focusing on Program Results to
Improve Agency Performance (GAO/ T- HEHS- 97- 129, May 7, 1997) and
Veterans? Employment and Training Service: Better Planning Needed to Address
Future Needs (GAO/ T- HEHS- 00- 206,
Sept. 27, 2000). VETS? Proposed
Measures Hold Promise, but Some Concerns Remain
Proposed Measures Are an Improvement Over the Current Ones
Page 6 GAO- 01- 757T
measures- job retention after 6 months and the employment rate following
staff- assisted services. 6 The VETS? proposal also retains an outcome
measure that is in the current system- the entered- employment rate. (See
table 1.)
Table 1: VETS? Current Performance Measures Compared With Proposed Measures
Current measures Processoriented Outcomeoriented
Entered- employment rate: The percentage of all registered veterans who were
placed in or obtained employment. X Number of veterans placed in training: A
count of the veterans placed in training. X Number of veterans receiving
counseling: A count of the veterans who received counseling services. X
Number of veterans receiving some reportable service: A count of the
veterans who received at least one reportable service. X Federal contractor
jobs filled by Vietnam and special disabled veterans: A count of the
veterans who were placed in jobs listed on the federal contractor job list.
X
Proposed measures Processoriented Outcomeoriented
Entered- employment rate: The percentage of all registered veterans who were
placed in or obtained employment. X Employment rate following receipt of
staff- assisted services: The percentage of registered veterans who are
employed after receiving some form of staff- assisted labor exchange
services.
X Employment retention rate at 6 months: Of the veterans who had entered
employment following registration, the percentage of those who continued to
earn wages 6 months after entering employment.
X Federal contractor job openings listed with the public labor exchange: The
percentage increase in the number of federal contractor job openings listed
annually with the public labor exchange from one program year to the next.
X Source: Veterans? Employment and Training Service, Department of Labor.
6 VETS considers this measure as process- oriented ?with an emphasis on
outcomes.? However, for this report, we classified the measure as outcome-
based because it reports an employment rate rather than only reporting a
count of services.
Page 7 GAO- 01- 757T
The proposed performance measures also improve the way VETS establishes the
level of performance that states are expected to achieve. VETS no longer
requires states to compare the level of services provided to veterans with
those provided to nonveterans. 7 In past reports, we have pointed out that
the use of these relative standards results in states with poor levels of
service to nonveterans being held to lower standards for service to veterans
than states with better overall performance. 8 Under the proposed system,
VETS will negotiate performance levels annually with each state based on
that state?s past performance, using guidelines similar to those used for
WIA. 9 VETS will also be able to adjust these levels based on economic
conditions within each state, such as the unemployment rate, the rate of job
creation or loss, or other factors.
The proposed performance measures are also similar to those established
under WIA, making it easier for service providers to achieve WIA?s goal of
integrating and streamlining employment and training services. In the
current environment, many of the programs that provide services through the
one- stop centers have their own unique performance measures and program
definitions, requiring multiple systems and multiple data collection efforts
to track a single client. In the proposed system, VETS is trying to align
its performance measures with those of WIA. Two of the five proposed
measures- entered- employment rate and employment retention- are nearly
identical to WIA?s and to those proposed for ES. 10 If VETS aligns the
measures with those of WIA and ES, local offices will be more readily able
to establish integrated data systems that will minimize the data collection
burden on service providers and clients.
7 While states will no longer be required to compare the level of services
given to veterans and nonveterans, VETS is required to report annually to
the Congress on the job placement rate of veterans compared with the rate
for nonveterans. 38 U. S. C. sect. 4107.
8 See GAO/ T- HEHS- 97- 129. 9 VETS is planning to use WIA?s negotiation
process to establish expected performance levels for labor exchange
services. VETS proposes that states use 2 years of data if possible, but not
less than 1 year in determining trends for performance and factors that may
influence performance.
10 See GAO- 01- 580 for a comparison of VETS, ES, and WIA performance
measures.
Page 8 GAO- 01- 757T
While the proposed performance measures are an improvement over those
currently in place, there are issues with these measures that VETS should
address. First, a comparison of the performance measures with the strategic
plan indicates that VETS is sending a mixed message to states about what
services to provide and to whom. The strategic plan suggests that states
focus their efforts on providing staff- assisted services to veterans,
including case management. Yet, none of the proposed measures specifically
gauges whether more staff- intensive services are helping veterans get jobs.
VETS? proposal includes a measure that tracks employment outcomes following
staff- assisted services. However, this measure is broadly defined, and the
list of staff- assisted services includes nearly all services provided to
veterans. 11 This makes the outcomes achieved for the staff- assisted
measure nearly identical to those reported for the more general ?entered-
employment rate.? In addition, as VETS has defined it, staff- assisted
services include many services that might not be considered ?intensive,?
such as referral to a job and job search activities. Because the definition
is so broadly defined, a veteran who only attended a job search workshop
would be counted the same as a veteran who received more intensive services,
such as testing and employability planning. Both would be counted in the
more general entered- employment rate measure, as well as the staff-
assisted service measure. A stricter definition for staff- assisted services
that includes only those services that are generally considered staff-
intensive would allow VETS to more accurately assess the success of those
services and help to clarify the goals of the program.
Second, VETS is sending a mixed message about which groups of veterans to
target for services. As we noted in past reports and testimonies, VETS has
inconsistently identified various ?targeted? groups of veterans it plans to
help. 12 In its strategic plan, VETS identifies two broad veterans groups
that should be targeted to receive special attention-( 1) disabled veterans
11 VETS uses the ETA definition of staff- assisted services. Staff- assisted
services include (a) referral to a job; (b) placement in training; (c)
assessment services, including an assessment interview, testing, counseling,
and employability planning; (d) career guidance; (e) job search activities,
including resume assistance, job search workshops, job finding clubs,
providing specific labor market information and job search planning; (f)
federal bonding program; (g) job development contacts; (h) tax credit
eligibility determination; (i) referral to other services, including skills
training, educational services, and supportive services; and (j) any other
service requiring expenditure of time. Application taking and/ or
registration services are not included as staff- assisted services.
12 See GAO/ T- HEHS- 00- 206. Concerns Remain That
VETS Should Address
Page 9 GAO- 01- 757T
and (2) all veterans and other eligible persons. And consistent with this,
VETS proposes that expected performance levels be negotiated separately for
each of these same two groups. Yet, the strategic plan also suggests that,
when providing services to all veterans, special attention should be given
to meeting the needs of certain other target groups, some of which might
require more intensive services to become employed. The groups targeted for
special attention include (1) veterans who have significant barriers to
employment, (2) veterans who served on active duty during a war (or campaign
or expedition in which a campaign badge has been authorized), and (3)
veterans recently separated from military service. In reviewing VETS?
proposed measures and the plan for negotiating performance levels, it is
unclear what steps VETS will take to ensure that DVOPS and LVERs are
provided ample opportunity and encouragement to focus attention on the
portion of the ?all veterans? group who may require more staff time to be
successful in getting a job.
Last, VETS? proposal also continues to include a performance measure related
to federal contractor job openings listed with the state?s ES office.
However, in its proposal, VETS has changed the measure. Under the current
system, VETS tracks the number of Vietnam- era and special disabled veterans
who were placed in jobs listed by federal contractors- an outcome measure.
Now, under the proposed system, VETS will track the increase in the number
of federal contractor jobs listed with the state?s ES office- a process-
oriented measure. 13 This new measure ultimately holds DVOPS and LVERs
accountable for the number of federal contractors in a given state or local
area, not for veteran placements with those contractors. The presence of
federal contractors in a given state or local area is unpredictable and is
determined by the federal agencies awarding contracts. Furthermore,
according to state officials that we talked with, the federal contractor
measure should be eliminated altogether because it is the responsibility of
contractors to list their job openings. In addition, it is Labor?s Office of
Federal Contract Compliance
13 Any contractor or subcontractor with a contract of $25,000 or more with
the federal government must take affirmative action to hire and promote
qualified special disabled veterans, veterans of the Vietnam- era, or any
other veterans who served on active duty during a war or in a campaign or
expedition for which a campaign badge has been authorized. Contractors and
subcontractors with job openings, other than executive or top management
jobs, must list them with the nearest state employment office. Veterans
cited above receive priority for referral to federal contractor job openings
listed at those offices.
Page 10 GAO- 01- 757T
that is responsible for ensuring that all companies conducting business with
the federal government list their jobs with state ES offices and take
affirmative action to hire qualified veterans.
Consistent with WIA and ES, VETS is proposing that all states use UI wage
records, which contain the earnings of each employee reported quarterly to
state agencies, 14 to identify veterans who get jobs. While these data will
greatly improve the comparability and reliability of the proposed
performance measures, they will also bring some challenges that VETS needs
to address.
Using a single data source will help to standardize the way in which states
collect data on veterans, thereby making it easier to compare performance
across states. Currently, states are using various data sources for
performance- reporting purposes. While almost all of the states in our
review used a combination of data sources to determine whether or not a
veteran got a job, most of the states relied substantially on one data
source, but that source differed among states. For example, in program year
1999
7 of the 15 states that we contacted relied to a large extent on wage
record data to determine whether a veteran got a job or not;
7 others relied, for the most part, on telephone calls and letters to
veterans and employers to determine a veteran?s employment status; and
1 state relied primarily on its new hire database for employment data. 15
14 Each calendar quarter, employers in a state provide wage information on
their employees to their state?s UI agency or some other state agency. The
information contained in wage records varies from state to state. However,
all wage records contain at least the following information: the calendar
quarter that the wages were reported in, the employee?s social security
number, wages paid to the employee in that quarter, and employer
information.
15 The Personal Responsibility and Work Opportunity Reconciliation Act of
1996 established the National Directory of New Hires and State Directories
of New Hires. The National Directory is maintained by the Social Security
Administration on behalf of the Department of Health and Human Services,
Office of Child Support Enforcement. States maintain their own State
Directories of New Hires and generally supply data for the National
Directory. Information in these directories includes: new hire information,
such as name, address, and social security number of the employee and the
name, address, federal identification number of the employer; in some
states, wage information; and UI claim information. Proposed Data Source
for New Measures Is an Improvement but Will Bring Some Challenges
Proposed Data Source Will Help to Ensure Comparability and Reliability
Across States
Page 11 GAO- 01- 757T
In addition to making state data more comparable, we found evidence that
states currently using wage records have been able to better identify those
veterans who get jobs after receiving services. A recent study found that UI
wage records more accurately identified how many veterans in the state of
Maryland got jobs after receiving DVOP, LVER, or ES services. 16 In
addition, most states in our review that are now using UI wage records,
either as their primary data source or to augment other data sources,
reported higher employment rates in program year 1999 for veterans they
served than that year?s national average of 30 percent. 17 By comparison,
all but one of the states that relied either on manual follow- up or the new
hire database reported an employment rate below the national average.
Another benefit of using UI wage records is that staff assisting veterans
will be relying on data already available rather than collecting additional
information from veterans or employers. Relying on these already reported
data would require less staff time from DVOP, LVER, and ES staff, freeing
them to focus more on providing job- related services to veterans. State
officials told us that relying on manual follow- up, such as telephone
calls, has been labor- intensive and has diverted staff attention away from
providing appropriate assistance to veterans.
While UI wage records offer advantages over the current data collection
system, some challenges need to be addressed. First, states should find ways
to identify interstate job placements. Because the UI wage record system
resides within each state, states generally do not have access to wage
records from other states, making it difficult to track individuals who
receive services in one state but get a job in another. Currently, there is
no national system in place that facilitates data sharing among states.
However, in response to WIA requirements, states are developing an
interstate UI wage record information sharing system, known as the Wage
16 Using UI wage records, this study tracked veterans who registered with
the Maryland Job Service during program year 1997 and found an entered-
employment rate that ranged from 65 percent to 82 percent, depending on the
way the study defined a registrant. In that same program year, Maryland
reported to VETS an entered- employment rate of 31 percent, which was based
on staff telephoning veterans and employers to verify employment. See
Proposed New Entered Employment Patterns of Veteran Wagner- Peyser
Registrants in the State of Maryland, by Robert Cook, BETAH Associates; and
Edward Davin and Karin
Willner, DynCorp (Apr. 12, 2000). 17 See GAO- 01- 580 for a list of all
states and their respective entered- employment rates for program years
1996- 99. UI Data Presents Some
Challenges
Page 12 GAO- 01- 757T
Record Interchange System (WRIS). The system is designed to minimize the
burden on state unemployment insurance programs in responding to requests
for wage record data, to ensure the security of the transactions involving
individual wage records, and to produce the results at a low cost per
record. In addition, some states have entered into agreements with
neighboring states to share wage information in support of WIA. These
efforts should help VETS as well.
Second, states should find ways to identify those veterans finding jobs in
categories not covered by UI wage records. According to Labor, UI wage
records include data on about 94 percent of workers. Certain employment
categories are not included in these data, such as self- employed persons,
most independent contractors, military personnel, federal government
workers, railroad employees, some part- time employees of nonprofit
institutions, and employees of religious orders. Therefore, the UI system
will not be able to track and count veterans who get these types of jobs.
This is an issue for WIA as well, and states are beginning to assess the
extent to which this issue will affect their ability to accurately determine
the outcome of WIA- funded programs.
There are other issues not related to the use of UI wage records that VETS
should consider as it finalizes its performance- reporting requirements.
VETS? proposed performance system does not standardize how states report
veterans or nonveterans who use self- service activities, making it
difficult to reliably assess nationwide performance. In an environment in
which self- service is becoming more common, we found that states vary in
whether they register veteran job seekers who access self- service tools,
such as internet- based job listings or resume writing software. For
example, some states allow job seekers greater access to job listings
without requiring that they register, while others have more restrictions on
who can access job lists. Table 2 shows how such differences can affect
entered- employment rates. In this example, 100 veterans enter the
employment service for assistance. In both cases, 40 veterans ultimately get
jobs after receiving identical services. In one case, the placement rate is
40 percent and in the other, 50 percent- a 10- percentage point difference.
This difference results from counting all job seekers in one case and only
those requiring staff assistance in the other. As a result of the different
ways states currently count veterans and report outcomes, the entered-
employment rate measure is not consistently calculated across states, and
nationwide comparisons are misleading. Other Measurement Issues
Affect Comparability of States? Performance Data
Page 13 GAO- 01- 757T
Table 2: A Comparison of Entered- Employment Rates by Registration Policy
All veterans required to register Veterans accessing self- service do not
have to register
Veterans registered
Number of veterans who
get jobs Number of
veterans with jobs counted
in enteredemployment rate Veterans
registered Number of
veterans who get jobs
Number of veterans with
jobs counted in enteredemployment
rate
40 veterans use selfservice 40 10 10 40 veterans
use selfservice 0 100
60 veterans require staff assistance
60 30 30 60 veterans require staff
assistance 60 30 30
Total 100 40 40 Total 60 40 30 Reported Entered- Employment Rate: 40/ 100 =
40% Reported Entered- Employment Rate: 30/ 60 = 50%
Source: GAO analysis.
VETS? proposed performance system does not standardize how long a veteran or
nonveteran remains registered after seeking services for performance-
reporting purposes. We found that states differ in how long they keep
veterans registered. This difference makes the calculation of the entered-
employment rate (i. e., the number of veterans that get jobs) different from
state- to- state and nationwide comparisons unreliable. Many of the states
we contacted count individuals as registered who have received a service in
the last 6 months. However, two states only count those as registered who
have received a service in the last 3 months, while two others count only
those who received a service in the last 2 months. And in one state, anyone
who has received a service from the state?s employment office since 1998 is
counted as a registrant when determining the entered- employment rate.
VETS has proposed changes to its performance measurement system that will
move VETS closer to implementing an effective accountability system.
However, additional changes are needed so that VETS can effectively
determine whether its programs and services are fulfilling its mission. VETS
continues to send a mixed message to states about what services to provide
and to whom. In addition, two of the proposed measures- the entered-
employment rate and the employment rate following staff- assisted services-
may provide nearly identical results, and neither helps VETS to monitor
whether more intensive services are being provided to veterans or whether
these services are successful. Furthermore, through its planning documents
and proposed performance measures, VETS continues to Conclusions and
Recommendations
Page 14 GAO- 01- 757T
inconsistently identify the groups of veterans that it wants states to help.
Our report recommended that the Secretary of Labor direct VETS to redefine
staff- assisted services to include only those that may be considered staff
intensive, such as case management, so that VETS can evaluate these
services. We also recommended that VETS clearly define its target
populations so that staff assisting veterans know where to place their
priorities. VETS acknowledged that its current strategic plan sends a mixed
message to the states about which groups of veterans staff should target for
special attention and noted that it is revising its planning documents to
reflect a more consistent message. VETS disagreed, however, with our
recommendation for a revised definition of the performance measure related
to staff- assisted services.
VETS also maintains a measure related to federal contractors that does not
focus on whether veterans get jobs but instead counts how many job openings
are listed by federal contractors. In addition, state officials told us that
it is the responsibility of the contractors, not DVOPS and LVERs, to list
their job openings. Our report recommended that VETS eliminate this measure.
While VETS disagreed with this recommendation, it agreed to reconsider the
suitability of this specific measure after public comments have been
received.
Finally, because VETS allows states to decide which veterans to include in
its performance reports, data across states are inconsistent and state-
tostate comparisons are unreliable. Our report recommended that VETS
establish and communicate guidelines that standardize how to count veterans
for performance- reporting purposes so that VETS will be able to assess
program performance nationwide. VETS noted that it is working with ETA to
determine how states can uniformly report veterans and nonveterans that use
self- service activities and that the revised ETA 9002 report will provide
instructions on how long individuals remain registered in the system.
Mr. Chairman, this concludes my prepared statement. I will be happy to
answer any questions that you or other members of the Subcommittee may have.
For future contacts regarding this testimony, please contact Sigurd R.
Nilsen at (202) 512- 7215. Key contributors to this testimony were Dianne
Blank, Elizabeth Morrison, and Amanda Ahlstrand. GAO Contacts and
Acknowledgments
(130045)
*** End of document. ***