[Federal Register Volume 67, Number 78 (Tuesday, April 23, 2002)]
[Notices]
[Pages 19998-20004]
From the Federal Register Online via the Government Publishing Office [www.gpo.gov]
[FR Doc No: 02-9918]



[[Page 19997]]

-----------------------------------------------------------------------

Part XI





Department of Labor





-----------------------------------------------------------------------



Veterans' Employment and Training Service



-----------------------------------------------------------------------



Service to Veterans; Final Performance Measures for State Employment 
Security Agencies; Notice

  Federal Register / Vol. 67, No. 78 / Tuesday, April 23, 2002 / 
Notices  

[[Page 19998]]


-----------------------------------------------------------------------

DEPARTMENT OF LABOR

Veterans' Employment and Training Service


Service to Veterans; Final Performance Measures for State 
Employment Security Agencies

AGENCY: Veterans' Employment and Training Service, Labor.

ACTION: Final Notice.

-----------------------------------------------------------------------

SUMMARY: By law, the Assistant Secretary for Veterans' Employment and 
Training (ASVET) is required to establish performance standards for the 
provision of services to veterans by State Employment Security Agencies 
(SESA's). The ASVET, in turn, is required to report on these results in 
the Veterans' Employment and Training Service's (VETS) Annual Report to 
Congress. This document communicates the establishment of final 
performance measures for the provision of services to veterans by the 
public labor exchange, including those services provided by Local 
Veterans' Employment Representative (LVER) and Disabled Veterans' 
Outreach Program (DVOP) staff. VETS' performance measurement system for 
the public labor exchange consists of three measures: (1) Veteran Job 
Seeker Entered Employment Rate, (2) Veteran Job Seeker Employment 
Retention Rate at Six Months, and (3) Veteran Job Seeker Entered 
Employment Rate Following Receipt of Staff-Assisted Services. These 
measures are to be calculated for two categories of veterans: (1) 
Veterans and Eligible Persons and (2) Disabled Veterans. VETS 
establishes these measures in light of comments received in response to 
proposed performance measures for services to veterans by the public 
labor exchange, as published in the Federal Register on May 31, 2001.

EFFECTIVE DATE: These performance measures for the public labor 
exchange service to veterans will become effective July 1, 2002.

ADDRESS: All comments received during the comment period following the 
publication of the proposed labor exchange measures for services to 
veterans (66 FR 29602 et seq., May 31, 2001) are available for public 
inspection and copying during normal business hours at the Veterans' 
Employment and Training Service, U.S. Department of Labor, 200 
Constitution Avenue NW., Room S-1316, Washington, DC 20210.

FOR FURTHER INFORMATION CONTACT: Robert Wilson, 202-693-4719 (voice), 
or (800) 670-7008 (TTY/TDD for the hearing impaired), or E-mail: 
[email protected].

SUPPLEMENTARY INFORMATION:

I. Authority

    VETS establishes performance measures for the provision of services 
by the public labor exchange under the following authority:

A. Title 38, United States Code (U.S.C.), Chapter 41--Job Counseling, 
Training, and Placement Services for Veterans, Sec. 4107(a)(1)

    The Secretary shall establish administrative controls for the 
following purposes: To insure that each eligible veteran, especially 
veterans of the Vietnam era and disabled veterans and each eligible 
person, who requests assistance under this chapter (38 USCS 4100 et 
seq.) shall promptly be placed in a satisfactory job, or job 
training opportunity or receive some other specific form of 
assistance designed to enhance such veteran's and eligible person's 
employment prospects substantially, such as individual job 
development or employment counseling services.

B. Title 38, U.S.C., Chapter 41--Job Counseling, Training, and 
Placement Services for Veterans, Sec. 4107 (b)

    The Secretary shall establish definitive performance standards 
for determining compliance by the State public employment service 
agencies with the provisions of this chapter (38 USCS 4100 et seq.) 
and chapter 42 of this title (38 USCS 4211 et seq.).
    These final performance measures are separate from the reporting 
requirements of section 4107 (c) of Title 38, U.S.C., and they do not 
negate these reporting requirements. Section 4212 of Title 38, U.S.C. 
requires entities awarded Federal contracts or subcontracts of $25,000 
or more to take affirmative action to employ and advance in employment 
qualified Special Disabled Veterans, Vietnam-era Veterans, Recently 
Separated Veterans, (Pub. L. 106-419 added Recently Separated Veterans 
to the class of veterans receiving emphasis under Federal Contracts) 
and Campaign Veterans (any other veterans who served on active duty 
during a war or in a campaign or expedition for which a campaign badge 
or expeditionary medal has been authorized). Federal contractors and 
subcontractors are required by law and regulation to list virtually all 
job openings with their local SESA office. The SESA's, through the 
public labor exchange system, are required to provide priority 
referrals of qualified targeted veterans to these Federal contractor 
openings.

II. VETS Performance Measures

A. Background

    VETS developed its performance measurement system based upon a 
series of tests of proposed measures and meetings with various 
stakeholders. Beginning in September 1998, and continuing through the 
Summer of 1999, VETS reviewed current performance measurements for the 
public labor exchange and DVOP and LVER programs. Following the 
finalization of the performance measures for the Workforce Investment 
Act (WIA) in 1999, and the publication of recommendations of the United 
States Employment Service's (USES) workgroup on performance measures 
for the public labor exchange system (65 FR 49708 et seq., Aug. 14, 
2000), VETS developed five proposed measures that were consistent with 
these related workforce development programs. These measures were: 
Entered Employment Rate, Employment Retention Rate at Six Months, 
Earnings Gain, Employment Rate Following Receipt of Staff-Assisted 
Services by Wagner-Peyser Staff, and Entered Employment Rate Following 
Referral to a Federal Contractor.
    During July through September 2000, VETS conducted a beta test of 
its proposed performance measures in six (6) States. Based on the 
results of this beta test, the final recommendations of the USES 
workgroup on performance measures for the public labor exchange, and 
through coordination with the USES on the Revised Employment and 
Training Administration (ETA) 9002 Reports and Revised Employment and 
Training (ET) Handbook 406 (ETA 9002 Data Preparation Handbook), VETS 
published a set of four proposed performance measures in the Federal 
Register (66 FR 29602 et seq., May 31, 2001). These measures were 
Veteran Job Seeker Entered Employment Rate, Veteran Job Seeker 
Employment Retention Rate at Six Months, Veterans' Employment Rate 
Following Receipt of Staff-Assisted Services, and Federal Contractor 
Job Openings Listed with the Public Labor Exchange. The first three of 
these measure were proposed to apply to all Veterans and to Disabled 
Veterans. The Federal Register Notice also provided a general framework 
for establishing expected levels of performance for each of these 
measures.
    During the Summer of 2001, VETS reviewed and analyzed the comments 
received in response to the publication of the proposed performance 
measures. In addition, VETS considered reviews and comments from 
various sources on its proposed measures as well as other related 
measures, and guidelines. VETS considered the findings and 
recommendations in a report published in May 2001, by the General 
Accounting Office on the VETS proposed

[[Page 19999]]

performance measurement system. VETS then considered commentary on the 
proposed measures received from SESA representatives who attended a 
series of four Regional Planning and Management Workshops that VETS 
hosted across the country earlier in 2001. VETS also reviewed the final 
performance measures for the public labor exchange established by the 
ETA on July 1, 2001, and the comments received by USES for its 
publication of the draft ET Handbook 406 and VETS 200 Reports. Based on 
these considerations, VETS developed its final performance measures of 
services to veterans by the public labor exchange.
    The final measures are consistent with Title 38, U.S.C., the 
requirements of the Government Performance and Results Act (GPRA) of 
1993, VETS'' Strategic Plan and Annual Plan, and the performance 
measurement systems established by the WIA and the USES.

B. Response to Comments

    In response to the four proposed VETS performance measures 
published in the May 31, 2001, Federal Register, VETS received 31 sets 
of comments distributed as follows: representatives from state agencies 
(17), Disabled Veterans' Outreach Program Specialists and Local 
Veterans' Employment Representatives (9), the National Association of 
State Workforce Agencies, USES, an ETA Regional Office (3), and current 
and former employees of VETS (2). The comments are discussed at length 
as follows:
(1) Veteran Job Seeker Entered Employment Rate
    VETS initially proposed the Veteran Job Seeker Entered Employment 
Rate defined as:

    Of Wagner-Peyser Act funded labor exchange applicants who are 
veterans and who in the first or second quarter following 
registration (Q+1 or Q+2), earned wages from a 
new or different employer than that from which the applicant earned 
wages in the quarter prior to registration (Q-1), divided 
by the number of applicants registered during the measurement 
period. Those applicants who earned wages in the first or second 
quarter following registration (Q+1 or Q+2), 
solely from the same employer from which wages were earned in the 
quarter prior to registration (Q-1), are excluded from 
the measure.

    Comment: Five respondents supported the measure as proposed. Eight 
respondents stated concern that the two quarter period to follow-up 
veteran registrants' employment outcomes after the registration dates 
will not be sufficient for those with multiple barriers to employment 
or who are placed in training. Six respondents suggested that veterans 
who are enrolled in case management should be excluded from the 
denominator for the Veteran Job Seeker Entered Employment Rate because 
they are likely to require extensive services over a period of time 
that extends beyond two outcome follow-up quarters (Q+1 and 
Q+2). Similarly, four respondents suggested that veterans 
who are enrolled in long-term training should be excluded from the 
denominator for the Veteran Job Seeker Entered Employment Rate because 
their employment outcomes are not likely to appear in the first or 
second quarter after registration.
    Response: To maintain consistency with the final measures for the 
public labor exchange, as published by the USES in the May 31, 2001, 
Federal Register, VETS will retain the proposed definition where 
outcomes are determined in the two quarters following the quarter of 
registration. To ensure consistency with the performance measures for 
the public labor exchange, VETS will use the registration policies 
defined in the ET Handbook 406. VETS and USES have jointly discussed 
the issue of State systems not counting registrants while they are in 
long-term training or are receiving intensive services such as case 
management. In order to develop a broad, overall measure of the 
effectiveness of participation in the public labor exchange system, 
VETS and USES agree that states are not to exclude from the applicant 
counts any individuals engaged in long term training or case 
management. VETS believes the measure of Veteran Job Seeker Entered 
Employment Rate will help capture the success of the public labor 
exchange efforts to assist veteran job seekers in achieving the desired 
outcome of entering employment.
(2) Veteran Job Seeker Employment Retention Rate at Six Months
    VETS initially proposed the Veteran Job Seeker Employment Retention 
Rate at Six Months defined as:

    Of those Wagner-Peyser Act labor exchange applicants age 19 and 
older at the time of registration who are veterans, and who in the 
first or second quarter following registration (Q+1 or 
Q+2), earned wages from a new or different employer than 
that from which the applicant earned wages in the quarter prior to 
registration (Q-1); those who also continue to earn wages 
in the third or fourth quarter (Q+3 or Q+4) 
respectively, following registration, divided by the number who 
earned wages in the first or second quarter after registration 
(Q+1 or Q+2 ).

    Comment: Six respondents supported the measure as proposed. Four 
respondents asked for further clarification about why this measure 
includes the applicants aged 19 years old.
    Response: To maintain consistency with the public labor exchange 
and the similar measure for Title 1 of WIA, the VETS employment 
retention measure will follow the definition and specifications set 
forth for the public labor exchange employment retention measure. The 
exclusion of registrants under the age of 19 at the time of 
registration will remove from the equation a large portion of those 
registered job seekers who most likely are seeking only short term 
employment.
    Comment: Five respondents stated that the measure is beyond the 
control of the public labor exchange because several factors may 
influence employment retention. Some of the possibilities include 
seasonal nature of some occupations and changes in the economy of a 
particular area.
    Response: VETS acknowledges that the employment retention measure 
is blind to labor market conditions. In conjunction with the USES, VETS 
will be developing methods to adjust for economic conditions and 
characteristics of registered veteran job seekers to use in 
establishing performance goals and for interpreting final performance 
levels compared to the rates established in the baseline year.
    Comment: One respondent noted that the measure, as originally 
proposed, does not measure the same group of veterans as the Veteran 
Job Seeker Entered Employment Rate measure.
    Response: We agree that the Veteran Job Seeker Employment Retention 
Rate at Six Months measures outcomes for a different pool of applicants 
than those measured in the Entered Employment Rate. The Employment 
Retention Rate measures outcomes for job seekers who are age 19 or 
older at the time of registration while the Entered Employment Rate 
measures outcomes for applicants of all ages. The intent of the 
proposed Employment Retention Rate measure is to determine employment 
retention outcomes for those most likely to be seeking long-term 
employment, excluding youth, who are most likely seeking short term 
employment.
    Comment: Another respondent noted that the standard does not 
measure movement from part-time to full-time work. Some respondents 
also noted that initial placements are often temporary and seasonal.
    Response: VETS supports the measure of Veteran Job Seeker 
Employment Retention Rate at Six Months as a

[[Page 20000]]

measure that is consistent with those of other workforce development 
programs. The intent of the measure also supports VETS' vision of 
minimizing underemployment, maximizing career employment opportunities, 
and improving labor market status of veteran job seekers. With respect 
to the comment about temporary and seasonal work, it is important to 
note that this measure assesses retention in employment, not job 
retention with a specific employer. To be counted in this measure, the 
veteran job seeker must retain some form of employment in the two 
quarters following initial entry to employment, not necessarily 
continued employment with the same employer.
    Comment: A respondent questioned the effect that the emphasis on 
self-service and automated systems will have on this standard.
    Response: The measure is not intended to specifically assess 
effectiveness of the use of self-service tools by individuals. 
Individuals using only self-service tools will not be counted unless a 
State has defined its registration policy to include these individuals 
in the registered applicant pool. For States that do register these 
individuals, the measure will help capture the quality of staff-
assisted services, facilitated self-help services, and self-services 
that provide job seekers with resources to secure and maintain 
continued employment.
    Comment: Another respondent also suggested that the measure will be 
flawed if it fails to take into account customer satisfaction.
    Response: The measure of Veteran Job Seeker Employment Retention 
Rate at Six Months does not specifically include a customer 
satisfaction component as that type of information is intended to be 
collected separately. The public labor exchange performance measures 
for customer satisfaction will serve as a broad indicator of how well 
the public labor exchange is serving all job seekers including 
veterans.
(3) Veterans' Employment Rate Following Receipt of Staff-Assisted 
Services
    VETS initially proposed the Veterans' Job Seeker Employment Rate 
Following Receipt of Staff-Assisted Services defined as:

    Of the Wagner-Peyser Act applicants who are veterans, who 
registered in a quarter, (Q0), and who received some form 
of staff-assisted services from public labor exchange staff, the 
number who are employed by the end of the first or second quarter 
after registration, (Q+1 or Q+2).

    Comment: Five respondents supported a performance measure for 
Veterans' Job Seeker Employment Rate Following Staff-Assisted Services. 
Several respondents discussed the definition of staff-assisted 
services, the ETA policies for States to define registration, and the 
potential impacts of these two issues on this measure. Four respondents 
suggested that the staff-assisted services should be more specifically 
defined. Two respondents noted that it is possible that outcomes 
following this measure are likely to be similar to those of the Veteran 
Job Seeker Entered Employment Rate because States may choose to 
implement a registration policy where only those job seekers who 
receive staff-assisted services are actually registered. Two 
respondents suggested that states should have greater flexibility to 
define staff-assisted services. One respondent recommended that 
referrals to a job not be considered a staff-assisted service because 
this activity is not staff-intensive. One respondent recommended not 
implementing this measure unless VETS provides additional funding for 
programming. Finally, two other respondents suggested that the quality 
of staff-assisted services can be measured through the Veteran Job 
Seeker Employment Retention Rate After Six Months measure.
    Response: The comments about this measure pertain to two issues: 
registration policies and definition of staff-assisted services. The 
new registration policy for the public labor exchange requires that 
anyone receiving staff-assisted services be registered. Under the WIA, 
States do have the option to register those using only self-service 
tools. For States that exercise the option to not register self-service 
customers, VETS acknowledges that the Veterans' Employment Rate 
Following Receipt of Staff-Assisted Services measure may parallel the 
entered employment measure. To maintain consistency, and to ensure that 
any veteran in any State who received services from the public labor 
exchange will be counted by the VETS performance measures, VETS will 
maintain the Veterans' Employment Rate Following Receipt of Staff-
Assisted Services measure (this measure will be renamed as Veteran Job 
Seeker Entered Employment Rate Following Receipt of Staff-Assisted 
Services).
    The elements of the staff-assisted services were developed in 
consultation with the USES workgroup of State representatives, 
officials from the National Association of State Workforce Agencies, 
and staff from VETS and ETA National and Regional offices. The 
workgroup revisited the definition and specifications for staff-
assisted services to ensure that uniformity can be achieved among 
States. Therefore, the elements of staff-assisted services have been 
defined in ways that all States can reasonably apply the definition of 
staff-assisted services to their specific workforce development 
programs and systems.
    VETS believes that this measure will provide effective data 
regarding the outcomes of more intensive public labor exchange services 
to veterans. For example, a veteran job seeker receiving staff-assisted 
services may require a multitude of services any one of which, or 
combination thereof, may require extensive staff time. Thus, measures 
of entered employment outcomes after receipt of staff-assisted services 
will provide an indication of the quality of those services. VETS 
encourages States to collect and analyze more detailed information on 
staff-assisted services, as each State deems necessary for its 
management purposes.
    Comment: Two respondents suggested that VETS consider ``weighting'' 
services so that those Services that are more staff-intensive (such as 
case management) are given additional weight, compared to those that 
are less intensive.
    Response: VETS is exploring the development of a weighted 
measurement system that would further encourage the provision of 
intensive services to veterans by the public labor exchange system.
    Comment: Two respondents suggested that case management by DVOP or 
LVER staff specifically should be emphasized in this measure.
    Response: VETS is focusing its guidance on DVOP services to address 
the needs of those veteran job seekers, particularly disabled veterans, 
who could benefit from case management and other employment development 
activities by DVOP staff. VETS is currently developing specific 
performance measures as well as prototype performance standards that 
would apply to DVOP and LVER staff activities.
    Comment: Another respondent suggested that restricting measurement 
for this measure to only two quarters may not fully capture the entered 
employment outcome, and therefore three follow-up quarters would be 
preferable.
    Response: VETS believes that consistency between this measure and 
other entered employment measures for the public labor exchange will 
help provide broad, overall measures of the effectiveness of the public 
labor exchange system in serving veterans.

[[Page 20001]]

Therefore, VETS and ETA agree that this measure should use data from 
the two quarters following registration as the period for determining 
entered employment outcomes.
(4) Federal Contractor Job Openings Listed With the Public Labor 
Exchange
    VETS initially proposed the Federal Contractor Job Openings Listed 
with the Public Labor Exchange be defined as:

    The percentage increase in the number of Federal contractor job 
openings listed annually with the public labor exchange, relative to 
the number listed in the previous Program Year (PY).

    Comment: VETS received 27 comments on the proposed measure for 
Federal Contractor Job Openings Listed (FCJL) With the Public Labor 
Exchange. Listing of jobs by Federal contractors with the public labor 
exchange is not under the control of labor exchange staff. Four 
respondents noted that listings of contractors and subcontractors 
available to staff are not complete. Seven respondents noted that 
enforcement of the FCJL program is the responsibility of the Office of 
Federal Contract Compliance Programs. Nine respondents noted that 
contractors can meet their Federal contractor award requirements by 
listing job openings with America's Job Bank, not the public labor 
exchange. New Federal contracts in any given State do not necessarily 
imply new job openings. Firms require new contracts to maintain their 
current workforce. The quantity of new Federal contractor job listings 
is determined by the amount and location of Federal contracting. No 
State is guaranteed a quarter to quarter increase in the number of 
Federal contracts that will be awarded in its jurisdiction.
    Response: VETS has chosen not to use the proposed FCJL measure as a 
performance standard for the public labor exchange. Since VETS is 
mandated by section 4212 of Title 38, U.S.C. to report to Congress on 
Federal contractor listing and hiring activity, States will still be 
required to submit data on the number of Federal contractor job 
listings received, the number of Federal contractors listing jobs, the 
number of veterans referred to FCJL jobs, and on the number of veterans 
placed in FCJL jobs.
(5) General Comments
    Comment: Additionally, VETS received a number of comments related 
to performance measures in general. Several respondents raised concerns 
about registration policies. Four respondents requested clarification 
of the definitions of (1) registration, (2) renewal of registration, 
and (3) reporting periods, as they relate to the rolling four quarter 
concept. One respondent expressed concern that under the self-service 
concept, the number of veteran job seekers will drop significantly. 
Another respondent suggested that in some States, the registration 
policy will create situations where the only veteran job seekers who 
will be registered are those who receive intensive services offered 
principally by staff funded under WIA grants, without regard to the 
services provided by staff funded under Wagner-Peyser or DVOP or LVER 
grants. As a result of these registration policies, the labor exchange 
services to veterans will be evaluated not on the general veteran 
population served by the public labor exchange, but by those veterans 
receiving intensives services. Similarly, two respondents expressed 
concern that the new measures will not measure the success or failure 
of the self-service and automated systems, unless all veterans who use 
self-service are registered. Two respondents voiced concerns that 
without a National, uniform policy for registration, data will not be 
available for State-to-State comparisons that are valid or reliable. 
However, another respondent encouraged VETS to allow States to 
establish their own registration policies for self-service. Another 
respondent asked for clarification that, as the public labor exchange 
measures are modified, VETS measures will also be modified to maintain 
consistency.
    Response: States have the option of registering self-service 
customers. As a result, VETS, in conjunction with the USES, has decided 
not to implement a policy that will require registration or establish 
mandatory performance measures for users of self-service tools. To 
maintain consistency with the performance measures for the public labor 
exchange, VETS will use the registration policies defined in the ET 
Handbook 406. A veteran job seeker customer is counted as registered 
during the quarter in which registration occurs (registration quarter) 
and the subsequent three quarters. Registration is the date of 
registration or re-registration, and the registration year is the 
quarter of registration plus the following three quarters. A veteran 
job seeker who engages in a labor exchange activity after a 
registration year expires will begin a new registration year, and will 
once again be eligible to be counted in the measures.
    For purposes of reporting, the rolling four quarter period concept 
will be used. As specified in the ET Handbook 406, the ETA 9002A 
(quarterly services report for all applicants), and the ETA 9002B 
(quarterly services report for veteran applicants) will provide data on 
persons who either registered or received services within the four 
quarter reporting period. The 9002C (quarterly outcome report for all 
applicants), and 9002D (quarterly outcome report for veteran 
applicants) will report the outcomes available on services provided to 
all registered job seekers, and to veteran job seekers for the four 
quarter reporting period. The rolling four quarter reporting period 
eliminates the concept of a carry-over registration from one program 
year to the next and provides uniformity for registration across 
States.
    As the performance measurement system for the public labor exchange 
is modified, VETS will continue to coordinate with USES to ensure as 
much consistency as possible between the VETS measures and the ETA 
measures for the public labor exchange.
    Comment: Other respondents expressed concern about the use of wage 
records, specifically the delay in the availability of wage data and 
the difficulty in obtaining access to wage data for Federal employees, 
military employees, interstate data, and occupations where employers do 
not regularly submit wage data to State agencies. One respondent 
recommended that consideration be given to adding a line item to the 
United States Department of Labor budget to cover the costs of States' 
use of the Wage Record Interchange System. Two respondents voiced 
concerns about the additional costs of reporting that are not borne by 
the DVOP and LVER grants. A final respondent suggested revisiting the 
economic model used by the Job Training Partnership Act where 
considerations such as unemployment rates and labor market conditions 
are factored into the results to evaluate outcomes.
    Response: The three final performance measures rely heavily on wage 
record data for calculation. VETS acknowledges that the time lags and 
lack of coverage of all employers that are associated with Unemployment 
Insurance (UI) wage record data will pose significant challenges as 
States transition to this new way of measuring the performance of the 
public labor exchange's services to veterans. VETS continues to support 
the use of wage record data for performance measurement of the public 
labor exchange's services to veterans. Use of UI wage records is 
consistent with the performance measurement systems established for WIA 
and for the public labor exchange. The use of wage record information 
will ease the burden of administrative follow-up inherent in the 
current reporting system for the public

[[Page 20002]]

labor exchange. In conjunction with the USES, VETS will be developing 
methods to adjust for economic conditions and characteristics of 
registered veteran job seekers, to use in establishing performance 
goals and for interpreting final performance levels compared to the 
rates established in the base line year. In addition, VETS is working 
with USES to develop data validation procedures to support quality 
control in performance measurement and data collection.
    Comment: One respondent requested further clarification about the 
decision not to pursue the earnings gain measure that VETS tested with 
historical data in six States in the summer of 2000.
    Response: During our analysis of the test of proposed measures in 
the Summer of 2000, VETS determined that results of the test did not 
sufficiently demonstrate that States could produce this measure validly 
and reliably due to differences in the way the States processed the 
data for this measure. In addition, VETS determined that the earnings 
gain measure was not an effective measure for the public labor exchange 
since there is no control over which jobs applicants choose. Thus, VETS 
decided not to use earnings gain as a performance measure.
    Comment: Other comments focused on issues pertaining to the 
implementation of the performance measures. Three respondents asked for 
clarification that the performance measures are to be applied at the 
State level, and that States should have the authority to negotiate and 
establish standards at the sub-State level with local workforce 
investment boards. Another respondent asked for clarification of 
whether the five percent variance will be for the total standards or 
the negotiated performance standards. A final respondent asked for 
clarification about the time frames for establishing expected 
performance levels, when performance levels may be renegotiated, and 
the impact of receiving an incentive or renegotiating downward the 
initial performance levels.
    Response: The VETS performance standards are intended to measure, 
at the State level, the effectiveness of services provided to veterans. 
The State Directors of Veterans' Employment and Training (DVETs) and 
representatives from the State agencies will negotiate the expected 
levels of performance, based on past performance by the entire public 
labor exchange system in their State. State agencies retain the 
authority to negotiate and establish performance standards with the 
local workforce investment areas in their State. The five percent 
variance applies to each of the negotiated performance standards, 
developed through the use of two years of data if possible, but not 
less than one year, to establish baseline data for each performance 
measure. In the absence of established baseline data, negotiations 
between DVETs and State agencies will form the basis of performance 
standards for the transition year of PY 2002. Data from PY 1999 and PY 
2000 should serve as the basis for establishing performance levels for 
PY 2002. In addition, VETS is seeking several States willing to pilot 
new performance standards based on data for the past two program years 
replicating outcomes on the proposed performance measures.
    Comment: One respondent also stated concerns about the implications 
that VETS might withhold grant funds to support incentives for States 
with ``exemplary performance''. Another respondent suggested that VETS 
include incremental incentives in the definition of performance 
standards.
    Response: VETS is exploring the feasibility of initiating an 
incentive program for exemplary performance in service to veterans by a 
State's public labor exchange system. Should this incentive program be 
implemented, VETS does not intend to withhold any grant funds from 
State agencies to support this program.
    Comment: One respondent suggested that the baselines for each 
measure should not be developed by each State in negotiation with the 
DVET. Rather, VETS should establish a National formula and time frames 
to develop a year of baseline data. Another respondent stated that the 
process for negotiating baselines using prior data seems unnecessary, 
and that the first cycle of data following July 2002 (PY 2002) should 
be the basis for States' negotiations with DVETs.
    Response: VETS intends for each State to develop baseline data 
based on their past performance. This allows for consideration of 
economic conditions, new business start-ups, State legislation, and 
other factors which might impact on an individual State's performance. 
Thus, as with the negotiations for performance measures under Title I 
of WIA, States are given the flexibility to develop their own baseline 
data.
    Comment: One respondent suggested that comparisons be restricted to 
comparing veteran outcomes against veterans served, rather than 
comparing veterans against the total population of job seekers served 
by the public labor exchange. The respondent also suggested counting 
the number of transactions and the number of times an individual is 
served.
    Response: Beginning in PY 2002, comparisons of services to veterans 
versus service to other groups will not be used as a quantitative 
performance measure. The new VETS performance measures do not include 
comparisons of veteran job seeker services or outcomes to those of 
other populations served by the public labor exchange. The intent of 
the new measures is to encourage a State's public labor exchange system 
to improve its services to veterans, as compared to its previous years' 
performance. These improvements will be demonstrated by the three 
outcome measures: Veteran Job Seeker Entered Employment Rate, Veteran 
Job Seeker Employment Retention at Six Months, and Veteran Job Seeker 
Entered Employment Rate Following Receipt of Staff-Assisted Services, 
rather than counts of the number of services provided. Counts of 
services provided are more process-oriented, and do not reflect VETS' 
emphasis on outcomes.
    Comment: One respondent asked for clarification about the 
definition of Disabled Veteran and Special Disabled Veteran, if the 
registrant did not serve over 180 days.
    Response: The VETS performance measures and other reporting 
requirements for services to veterans by the public labor exchange will 
use the legal definition of veteran as established by section 4211 of 
Title 38, U.S.C. As this relates to disabled veterans, if the applicant 
did not serve on active duty for over 180 days, then he/she must have 
been discharged or released with other than a dishonorable discharge, 
and have been discharged or released because of a service-connected 
disability.
    Comment: One respondent noted that the VETS measures do not 
distinguish among the One-Stop partners, public labor exchange staff, 
or other employment security staff such as DVOP or LVER staff who serve 
veteran job seekers. Additionally, the respondent asked for 
clarification on how States should determine which VETS 200 Report is 
credited with service in the event that an LVER and a DVOP both serve 
the same individual. Another respondent stated that since only two 
measures are exactly consistent with the public labor exchange 
measures, States will have a more difficult time integrating veterans 
services into the entire One-Stop system.
    Response: The intent of the VETS performance standards is to 
measure the services to veterans by the entire public labor exchange 
system. This system includes One-Stop partners, public labor exchange 
staff, and DVOP and

[[Page 20003]]

LVER staff. The measures are not designed to distinguish which 
particular program provided services to veteran job seekers. The 
specifications for generating the Veteran Job Seeker Entered Employment 
Rate measure and the Veteran Job Seeker Employment Retention Rate at 
Six Months measure are exactly the same as the public labor exchange 
measures, with the exception that the VETS measures restrict the 
applicant pool to veterans. The measure of Veteran Job Seeker Entered 
Employment Rate Following Receipt of Staff-Assisted Services is an 
outcome that the USES is not measuring. This, however, is an outcome 
measure on services that are to be provided to veterans by the entire 
public labor exchange system. The revised ETA 9002B and D Reports are 
proposed to capture the necessary information about receipt of staff-
assisted services and outcomes following these staff-assisted services. 
VETS, in conjunction with USES, has made every effort to minimize 
reporting burdens incurred by States while still requesting sufficient 
information to demonstrate that veteran job seekers receive suitable 
services from the public labor exchange. The specification for the 
revised VETS 200C Report provides procedures for generating reports of 
unduplicated counts of services to veteran job seekers by DVOP and LVER 
staff.

C. VETS Performance Measures

    We establish three performance measures for the provision of 
services to veterans by the public labor exchange:
     Veteran Job Seeker Entered Employment Rate.
     Veteran Job Seeker Employment Retention Rate at Six 
Months.
     Veteran Job Seeker Entered Employment Rate Following 
Receipt of Staff-Assisted Services.
    The VETS performance measures apply to public labor exchange 
services provided to veterans as part of the One-Stop delivery systems 
implemented by the States. These services include those provided by 
DVOP and LVER staff and other public labor exchange staff employed 
through funds under the Wagner-Peyser Act. At their discretion, States 
may include other publicly-funded labor exchange services in this 
measurement system for services to veterans.
    The VETS performance measures apply to all individuals who meet the 
definition of eligible veteran or other eligible, as established by 
sections 4101 and 4211 of Title 38, U.S.C. An eligible veteran is a 
person who served on active duty for a period of more than 180 days and 
was discharged or released therefrom with other than a dishonorable 
discharge; or was discharged or released from active duty because of a 
service-connected disability; or a member of a reserve component who 
served on active duty during a period of war or in a campaign or 
expedition for which a campaign badge is authorized and was discharged 
or released with other than a dishonorable discharge.
    Other eligible persons must fit the following criteria, established 
in section 4101(5) (A), (B), and (C) of Title 38, U.S.C.:

--The spouse of any person who died of a service-connected disability;
--The spouse of any member of the Armed Forces serving on active duty, 
who at the time of application, is listed in one or more of the 
following categories and has been so listed for a total of more than 
ninety days: (i) Missing in action, (ii) captured in the line of duty 
by a hostile force, (iii) forcibly detained or interned in the line of 
duty by a foreign government of power; or
--The spouse of any person who has a total disability permanent in 
nature resulting from a service-connected disability.
    A veteran job seeker is counted as a registered veteran job seeker 
during the quarter in which the registration occurs (registration 
quarter), and the subsequent three quarters. This four quarter period 
constitutes the registration year. A registered veteran job seeker who 
receives services during the fourth quarter after the registration 
quarter will begin a new registration year, or will be considered re-
registered. This veteran job seeker will be counted again during each 
of the four reporting periods covering the subsequent registration 
year. If the veteran job seeker's registration year lapses, and after 
some time he or she returns to the public labor exchange, that job 
seeker would begin a new registration year.
    The VETS performance measures are defined below.
(1) Veteran Job Seeker Entered Employment Rate (VJSEER)
    The count of registered job seekers who are veterans and who, in 
the first or second quarter following the registration quarter 
(Q+1 or Q+2), earned wages from a new or 
different employer than that from which the registered veteran job 
seeker earned wages in the quarter prior to registration 
(Q-1), divided by the difference between the count of 
veteran job seekers who registered or re-registered with the labor 
exchange during any of the previous four calendar quarters and the 
count of any of those veteran job seekers whose wages earned in the 
first and second quarter following registration were exclusively with 
the same employer from which wages were earned in the quarter prior to 
registration.
    This measure contains the following elements.
    Entered Employment with a New Employer: To be counted as 
successfully entering employment, the veteran applicant must, in the 
first or second quarter following the quarter of registration 
(Q+1 or Q+2), earn wages from a different 
employer than from whom he/she earned wages in the quarter prior to 
registration (Q-1).
    Registered Veteran Job Seeker: Veteran job seekers are considered 
to be registered for a four quarter period beginning with their 
registration quarter and the subsequent three quarters (registration 
year). Veteran job seekers who engaged in a labor exchange activity 
after their registration year has expired, will be re-registered and 
will then begin a new registration year.
    Quarter of Registration: The calendar quarter in which a veteran 
job seeker completed an initial registration with the public labor 
exchange (Q0) or in which a previously registered job seeker 
began a new registration year.
    A veteran job seeker may be employed or unemployed at the time of 
registration. The key factor that determines whether the veteran job 
seeker is counted in the entered employment rate is if he/she earns 
wages with a new employer in the follow-up quarters after the 
registration quarter. A successful employment outcome is determined by 
comparing the Employer Identification Numbers (EIN) of registered 
veteran job seekers' employers prior to and following registration 
based on information contained in the UI wage record database, the 
State Directory of New Hires database, or other available records. 
Veteran job seekers who remain employed exclusively with the same 
employer(s) (those found in Q-1) during the measurement period are 
excluded from the calculation.
(2) Veteran Job Seeker Employment Retention Rate at Six Months (VJSERR)
    The count of the number of registered veteran job seekers who are 
veterans age 19 and older at the time of registration, who in the first 
or second quarter following registration (Q+1 or 
Q+2), earned wages from a new or different employer than 
that from which the applicant earned wages in the quarter prior to 
registration (Q-1), and who also continued to earn wages in 
the second

[[Page 20004]]

quarter following the quarter in which they entered employment 
(Q+3 or Q+4), divided by the number of veteran 
job seekers who entered employment during the reporting period.
    This measure contains the following elements.
    Entered Employment with a New Employer (age 19 and older): The 
base, or the denominator, of the VJSERR measure is the number of 
veteran job seekers age 19 and older at the time of registration who 
enter employment with a new employer in the first or second quarter 
after the quarter of registration. The process for determining entered 
employment is described above in the VJSERR section.
    Retained Employment Two Quarters after Entered Employment with a 
New Employer (age 19 and older): To be counted in this measure, veteran 
job seekers must earn wages in the second quarter following the quarter 
in which they entered employment with a new employer. If a veteran job 
seeker enters employment in the first quarter after registration, then 
this measure will check to see if he/she is still earning wages in the 
third quarter after the quarter of registration. If the veteran job 
seeker enters employment in the second quarter following the quarter of 
registration, then this measure will check to see if he/she is still 
earning wages in the fourth quarter after registration.
    Successful employment retention is recorded for veteran job 
seekers, age 19 and older at the time of registration, who are 
determined to have entered employment according to the entered 
employment rate measure, and who earned wages with any employer in the 
second quarter following the quarter in which they were first 
determined to have entered employment.
(3) Veteran Job Seeker Entered Employment Rate Following Receipt of 
Staff-Assisted Services (VERS)
    The count of registered job seekers who are veterans and who, in 
the quarter of registration (Q0) or in the first or second 
quarters following the quarter of registration (Q+1 or 
Q+2) received staff-assisted services and who in the first 
or second quarter following the registration quarter (Q+1 or 
Q+2) earned wages from a new or different employer than that 
from which the registered job seeker earned wages in the quarter prior 
to registration (Q-1); divided by the difference between the 
count of job seekers who are veterans who registered or re-registered 
with the labor exchange during any of the previous four calendar 
quarters and who in the first or second quarter following registration 
(Q+1 or Q+2) received staff-assisted services, 
and the count of any of those same job seekers whose wages earned in 
the first or second quarter following registration (Q+1 or 
Q+2) were exclusively with the same employer from which 
wages were earned in the quarter prior to registration 
(Q-1).
    This measure contains the following elements.
    Received Staff-Assisted Services: The elements of staff-assisted 
services can be found in the Revised ET Handbook 406. These elements 
were developed in consultation with the USES workgroup of State 
representatives, officials from the National Association of State 
Workforce Agencies, and staff from VETS and ETA National and Regional 
offices. Draft Specifications for the ET 406 Handbook were published in 
the Federal Register on June 6, 2001 (66 FR 30487 et seq.).
    Entered Employment with a New Employer: To be counted as 
successfully entering employment, the veteran applicant must meet the 
entered employment criteria as described in measure (1) above, and have 
received staff-assisted services during the measurement period 
(Q0-Q+2 ), as reported on the ETA 9002 Reports.

D. Levels of Performance and Rules for Application

    VETS will use the WIA Title I framework (published in Training and 
Employment Guidance Letter No. 8-99) which was also used for 
negotiating and setting expected performance levels for public labor 
exchange services. Accordingly, States, in conjunction with their DVET, 
will develop baseline data for each of the measures based on historical 
data, analyze the baseline data, and propose performance levels for 
each measure based on that analysis. Each State will negotiate with its 
DVET to obtain mutually agreed upon expected levels of performance.
    In developing baseline data, States should use two years of data if 
possible, but not less than one year in determining trends for 
performance and factors which may influence performance. In 
establishing expected performance levels for each measure, factors 
beyond the control of the State are also to be considered. When 
submitting their proposed performance levels, States should be prepared 
to provide support for their proposed levels by providing baseline 
performance data, the methodology for developing baseline data, and a 
description of data sources.
    The Regional Administrator for Veterans' Employment and Training 
(RAVET) will review the negotiated levels of performance as submitted 
through the DVET and will compare the expected performance levels with 
the National averages, baseline information from other States, and the 
negotiated levels of performance established for other States, while 
taking into account factors including differences in economic 
conditions and other factors as discussed above. The RAVET will analyze 
the quality of the data presented by States, including the relevance of 
the data, the source of the data, the time period from which the data 
were drawn, and if the data are part of a trend or anomalous. 
Established GPRA Annual Performance Plan goals for relevant measures 
will also be an important part of the Regional review and negotiation 
of performance levels. When the RAVET's analysis is completed, if need 
be, there will be the opportunity, through the DVET, for negotiations 
with the State to obtain mutually agreed upon expected levels of 
performance. Provisions will also be made for renegotiation of 
performance levels if circumstances arise that result in a significant 
change in the factors used to establish the original levels. It is 
understood that either a State or VETS may elect to renegotiate 
performance as new information becomes available. Factors which will be 
considered for making changes include those discussed above.
    During the first year of implementation of the new performance 
measures in PY 2002, performance will be reviewed as follows: States 
will be held harmless from any consequences of failing to meet their 
performance goals. Actual performance for each program year will be 
compared to negotiated performance levels. If a State's actual 
performance varies from the expected performance level by minus two 
percent or more, VETS will have the option of renegotiating new 
performance levels with the State. VETS will offer technical assistance 
as well as giving consideration to external factors affecting 
performance levels. A negative variation of five percent or more would 
result in the requirement of a State Corrective Action Plan (CAP) to 
rectify the situation. Failure to submit or comply with a CAP could 
become the basis for sanctions.

    Signed at Washington, DC, this 16th day of April.
Frederico Juarbe Jr.,
Assistant Secretary, Veterans' Employment and Training Service.
[FR Doc. 02-9918 Filed 4-22-02; 8:45 am]
BILLING CODE 4510-79-P