[Federal Register Volume 66, Number 105 (Thursday, May 31, 2001)]
[Notices]
[Pages 29648-29653]
From the Federal Register Online via the Government Publishing Office [www.gpo.gov]
[FR Doc No: 01-13611]



[[Page 29647]]

-----------------------------------------------------------------------

Part III





Department of Labor





-----------------------------------------------------------------------



 Employment and Training Administration



-----------------------------------------------------------------------



United States Employment Service and America's Labor Market Information 
System: Labor Exchange Performance Measures; Notice

  Federal Register / Vol. 66, No. 105 / Thursday, May 31, 2001 / 
Notices  

[[Page 29648]]


-----------------------------------------------------------------------

DEPARTMENT OF LABOR

Employment and Training Administration


United States Employment Service and America's Labor Market 
Information System; Labor Exchange Performance Measures

AGENCY: Employment and Training Administration, Labor.

ACTION: Notice.

-----------------------------------------------------------------------

SUMMARY: This notice announces the establishment of final performance 
measures for the public labor exchange administered as part of the One-
Stop delivery systems of the States. We established these performance 
measures based on comments received in response to proposed labor 
exchange performance measures previously published in the Federal 
Register. This notice discusses the comments received and our response 
to the comments. Three of the performance measures apply to job seekers 
registered with the labor exchange: job seeker entered employment rate; 
job seeker employment retention rate at six months; and job seeker 
customer satisfaction. One performance measure applies to employers 
receiving services provided through the One-Stop delivery system: 
employer customer satisfaction.

DATES: These labor exchange performance measures will become effective 
July 1, 2001.

ADDRESSES: All comments received during the comment period following 
the publication of the proposed labor exchange performance measures (65 
FR 49708, et seq., Aug. 14, 2000) are available for public inspection 
and copying during normal business hours at the Employment and Training 
Administration, Office of Career Transition Assistance, Division of 
U.S. Employment Service & ALMIS, 200 Constitution Avenue, NW., Room C-
4514, Washington, DC 20210.

FOR FURTHER INFORMATION CONTACT: Gay Gilbert, 202-693-3046 (voice) 
(this is not a toll-free number), or e-mail: [email protected].

SUPPLEMENTARY INFORMATION:
    Paperwork Reduction Act: As required by the Paperwork Reduction Act 
of 1995 (44 U.S.C. 3507(d)), these performance measures are being 
submitted to the Office of Management and Budget (OMB) for review. 
Affected parties do not have to comply with the information collection 
requirements contained in this document until we have published in the 
Federal Register the control number assigned by the Office of 
Management and Budget. Publication of the control number notifies the 
public that OMB has approved these performance measures under the 
Paperwork Reduction Act.

I. Authority

    Labor exchange performance measures are established under the 
following authority:

A. Wagner-Peyser Act sec. 3(a), 29 U.S.C. 49b(a)

    The Secretary shall assist in coordinating the State public 
employment services throughout the country and in increasing their 
usefulness by developing and prescribing minimum standards of 
efficiency, assisting them in meeting problems peculiar to their 
localities, promoting uniformity in their administrative and 
statistical procedure, furnishing and publishing information as to 
opportunities for employment and other information of value in the 
operation of the system, and maintaining a system for clearing labor 
between the States.

B. Wagner-Peyser Act sec. 3(c)(2), 29 U.S.C. 49b(c)

    The Secretary shall--
* * * * *
    (2) assist in the development of continuous improvement models for 
such nationwide system that ensure private sector satisfaction with the 
system and meet the demands of job seekers relating to the system.

C. Wagner-Peyser Act sec. 7(b), 29 U.S.C. 49f(b)

    Ten percent of the sums allotted to each State pursuant to section 
49e of this title shall be reserved for use in accordance with this 
subsection by the Governor of each such State to provide--(1) 
performance incentives for public employment service offices and 
programs, consistent with performance standards established by the 
Secretary, taking into account direct or indirect placements (including 
those resulting from self-directed job search or group job search 
activities assisted by such offices or programs), wages on entered 
employment, retention, and other appropriate factors.

D. Wagner-Peyser Act sec. 10(c), 29 U.S.C. 49i(c)

    Each State receiving funds under this Act shall--
    (1) make such reports concerning its operations and expenditures in 
such form and containing such information as shall be prescribed by the 
Secretary, and
    (2) establish and maintain a management information system in 
accordance with guidelines established by the Secretary designed to 
facilitate the compilation and analysis of programmatic and financial 
data necessary for reporting, monitoring and evaluating purposes.

E. Wagner-Peyser Act sec. 13(a), 29 U.S.C. 49l(a)

    The Secretary is authorized to establish performance standards for 
activities under this Act which shall take into account the differences 
in priorities reflected in State plans.

F. Wagner-Peyser Act sec. 15(e)(2)(I), 29 U.S.C. 49l-2(e)(2)(I)

    (e) State responsibilities.--
* * * * *
    (2) Duties.--In order to receive Federal financial assistance under 
this section, the State agency shall--
* * * * *
    (I) utilize the quarterly records described in section 2871(f)(2) 
of this title to assist the State and other States in measuring State 
progress on State performance measures.

II. Labor Exchange Performance Measures

A. Background

    We initiated the development of a performance measurement system 
for the public labor exchange in early 2000 with the formation of a 
workgroup in collaboration with the Interstate Conference of Employment 
Security Agencies (ICESA)--now the National Association of State 
Workforce Agencies (NASWA). This workgroup consisted of representatives 
from fifteen State agencies, ICESA, the Veterans' Employment and 
Training Service (VETS), and the ETA Regional and National Offices. The 
workgroup met three times during 2000 to develop recommendations for a 
labor exchange performance measurement system, to include specific 
labor exchange performance measures.
    Based on recommendations the workgroup developed during its first 
two meetings, we published a set of five proposed labor exchange 
performance measures in the Federal Register (65 FR 49708 et seq., Aug. 
14, 2000). These measures were: employer customer satisfaction; job 
seeker customer satisfaction; employment rate; entered employment rate; 
and employment retention rate at six months. We also published a 
framework for establishing expected levels of performance for each of 
these measures.

[[Page 29649]]

    During its third meeting, the workgroup reviewed and analyzed all 
comments received on the proposed labor exchange performance measures. 
The review and analysis led to the final set of performance measures 
presented in this document. The workgroup also provided substantial 
input that will lead to revised ETA 9002 Reports, where the results of 
the performance measures will be reported, and a revised ET Handbook 
No. 406 (ETA 9002 Data Preparation Handbook), containing data 
collection and reporting instructions. Finally, the workgroup 
recommended that the proposed procedures for establishing expected 
levels of performance be revised, based on the comments received. 
Further information about reporting on labor exchange services and 
performance measures, and methods for establishing expected levels of 
performance will be published in separate notices.

B. Response to Comments

    We received twenty-five sets of comments in response to the five 
proposed labor exchange performance measures published in the August 
14, 2000, Federal Register. Representatives from twenty State agencies 
(three agencies provided two separate sets of comments), a private 
researcher, and VETS provided comments. We considered these comments 
and the recommendations of the workgroup in establishing the final 
labor exchange performance measures. The comments are discussed at 
length as follows:
(1) Employer Customer Satisfaction
    We proposed to adopt the results of the employer customer 
satisfaction survey administered under Title I of the Workforce 
Investment Act of 1998 (WIA) using the American Customer Satisfaction 
Index (ACSI) methodology for employers receiving services involving 
significant personal contact with One-Stop staff to reflect employer 
satisfaction with One-Stop services (see 65 FR 49709-49711).
    Seven commenters supported the use of the employer customer 
satisfaction survey as stated, with two commenters specifically 
supporting the concept of the combined WIA/labor exchange survey of 
employers receiving services through the One-Stop centers that involved 
significant personal contact with staff.
    Several commenters commented on the employer survey. These comments 
included general concern about the wording of the lead-in before the 
questions, the wording of the second and third questions, the degree of 
flexibility allowed States to add additional questions, and the level 
of specificity of the questions.\1\ One commenter suggested that an 
eleven-point scale (0-11) should be used so that the number ``five'' 
would be the mid-point, whereas the proposed ten-point scale does not 
have a mid-point. Another commenter pointed out that the terms 
``completion of service'' and ``30-60 days after a job order has been 
listed'' were used as if they meant the same thing in indicating when 
the survey should be administered to an employer. In fact, services 
could have been provided to one employer over a 6-8 month period before 
being completed, although a job order could have been listed by another 
employer after only one phone call. Such employers would have vastly 
differing experiences with the labor exchange. Other commenters 
questioned what provisions would be made to administer the survey to 
non-English-speaking employers, and why the decision was made to use a 
telephone versus a mail survey. Still others expressed concerns about 
the cost of the survey--specifically whether provisions would be made 
for sharing the cost with Title I of WIA.
---------------------------------------------------------------------------

    \1\ The three mandatory questions originally proposed for the 
job seeker customer satisfaction survey are the following:
    (1) Utilizing a scale of 1 to 10 where ``1'' means ``Very 
Dissatisfied'' and ``10'' means ``Very Satisfied'' what is your 
overall satisfaction with the service(s) provided from ____?
    (2) Considering all of the expectations you may have had about 
the services, to what extent have the services met your 
expectations? ``1'' now means ``Falls Short of Your Expectations'' 
and ``10'' means ``Exceeds Your Expectations.''
    (3) Now think of the ideal service(s) for people in your 
circumstances. How well do you think the service(s) you received 
compare with the ideal service(s)? ``1'' now means ``Not Very Close 
to Ideal'' and ``10'' now means ``Very Close to the Ideal.''
---------------------------------------------------------------------------

    Two commenters did not support the concept of a combined employer 
survey for both Title I of WIA and the labor exchange, while another 
commenter found it improper to hold programs funded under Title I of 
WIA accountable for Wagner-Peyser Act labor exchange performance. Other 
comments related to the response rate, with some commenters stating 
that the proposed fifty percent response rate was too high and not 
achievable, and another stating that employers do not want to be 
bothered with surveys. Another commenter noted that the Office of 
Management and Budget (OMB) guidelines state that erroneous inferences 
are possible when response rates are in the 50%-75% range. Finally, a 
commenter sought definitions of the terms ``substantial service'' and 
``labor exchange.''
    Response: We agree that the commenters identified some problems 
with the proposed employer customer satisfaction survey. We have, 
however, decided to use a revised version of the proposed employer 
customer satisfaction survey to measure employer satisfaction with 
services involving substantial personal contact with One-Stop staff. 
Using a single instrument to measure customer satisfaction with One-
Stop employer services provides an overarching measure of the One-Stop 
system. Specifications for the final employer customer satisfaction 
measure for employers receiving services involving significant personal 
contact with One-Stop staff can be found in TEGL 14-00, March 5, 2001.
    The ACSI methodology is being adopted to be consistent with Title I 
of WIA. The ACSI is the most widely used index currently in practice. 
It is used extensively in the business community, including at over 150 
Fortune 500 companies, and in many European countries. Twenty-nine 
agencies of the Federal government have used the ACSI. In addition, it 
has been used twice in the past four years to assess customer 
satisfaction for ETA's Quality Initiative--the Enterprise. The ACSI 
will allow the workforce investment system and particularly, the public 
labor exchange, to not only look at performance within the system, but 
also to be able to gain perspective on the workforce investment 
system's performance by benchmarking against outside organizations and 
industries. The ACSI also has a history of usefulness in tracking 
change in customer satisfaction over time, making it an ideal way to 
gauge States' progress in continuously improving performance.
    This survey approach captures common customer satisfaction 
information that can be aggregated and compared at a State and national 
level. The survey will be administered using a set of three required 
questions that will form a customer satisfaction index. The ACSI score 
is obtained by combining scores from three specific questions that 
address different dimensions of customers' experience. In order for the 
ACSI survey to yield meaningful results under the prescribed 
methodology, we cannot significantly modify the survey questions or the 
associated scale and must administer the survey via telephone.
    We will publish instructions for reporting employer customer 
satisfaction scores as part of the labor exchange performance 
measurement system as part of a revised version of ET Handbook No. 406 
(ETA 9002 Data Preparation Handbook). These

[[Page 29650]]

instructions will address many of the comments about the details of 
survey administration.

(2) Job Seeker Customer Satisfaction

    We proposed a job seeker customer satisfaction measure that would 
follow the WIA methodology, but which would apply distinctly to job 
seekers registered with the labor exchange (see 65 FR 49711-49712).
    We received a large number of comments on the job seeker customer 
satisfaction measure. Many of these comments were similar to those 
provided on the employer customer satisfaction survey. Four commenters 
supported the measure as proposed. Several commenters expressed concern 
about coordinating the labor exchange job seeker survey with the WIA 
participant survey for the several reasons discussed below. Some 
commenters were concerned about being able to identify which 
individuals were surveyed under Title I of WIA so that they would not 
be surveyed again by the labor exchange survey. Others were concerned 
about relating the results of the survey specifically to labor exchange 
services if queried job seekers had received services from multiple 
One-Stop partner programs, and how those queried would be able to 
identify whether they were responding about their satisfaction with 
labor exchange services or other One-Stop services. One commenter 
suggested combining the labor exchange survey with the WIA survey and 
sorting according to the services received by the job seeker. Another 
commenter suggested that those job seekers required to register by 
State law or policy should be excluded from the survey, so that the 
survey would only include those voluntarily registering. Another 
commenter suggested that some job seekers might be deterred from 
registering with the labor exchange by the prospect of being surveyed.
    A number of commenters cited concerns about the ACSI methodology. 
One suggested that an eleven-point scale (0-11) should be used so that 
the number ``five'' would be the mid-point, whereas the proposed ten-
point scale does not have a mid-point. Others wanted more information 
to be provided about the ACSI weights, and questioned the quality and 
reliability of the ACSI methodology. Two commenters wanted 
clarification on the timing of the survey, and one of the two 
additionally suggested that the survey be conducted within a set 
timeframe after receipt of service rather than after registration, as 
was proposed. Additional comments centered on the cost of the survey 
and questioned the use of a telephone methodology. Among the concerns 
cited with the telephone methodology were the lack of telephone service 
in rural areas and difficulty in properly administering the survey to 
non-English speaking individuals.
    Finally, a number of commenters were concerned about the questions 
contained in the proposed survey.\2\ One commenter suggested 
eliminating the third mandatory question and rephrasing the second to 
read: ``How did the services you received meet the expectations that 
you had?'' Others suggested that the questions were too broad and did 
not provide enough specific information on how to improve the labor 
exchange system. Still others wanted more guidance on what part of the 
survey could be modified--particularly whether the lead-in section 
before the first question could be modified.
---------------------------------------------------------------------------

    \2\ The three mandatory questions originally proposed for the 
job seeker customer satisfaction survey are the following:
    (1) Utilizing a scale of 1 to 10 where ``1'' means ``Very 
Dissatisfied'' and ``10'' means ``Very Satisfied'' what is your 
overall satisfaction with the service(s) provided from ____?
    (2) Considering all of the expectations you may have had about 
the services, to what extent have the services met your 
expectations? ``1'' now means ``Falls Short of Your Expectations'' 
and ``10'' means ``Exceeds Your Expectations.''
    (3) Now think of the ideal service(s) for people in your 
circumstances. How well do you think the service(s) you received 
compare with the ideal service(s)? ``1'' now means ``Not Very Close 
to Ideal'' and ``10'' now means ``Very Close to the Ideal.''
---------------------------------------------------------------------------

    Response: The purpose of the job seeker customer satisfaction 
measure is to gauge the satisfaction of registered job seekers with the 
labor exchange. We are adopting the ACSI methodology to measure job 
seeker customer satisfaction to be consistent with Title I of WIA and 
for the reasons described in the previous section on employer customer 
satisfaction.
    Since the ACSI trademark is property of the University of Michigan 
and the Claes Fornell International Group (CFI), we will be modifying 
our existing license agreement with the University of Michigan to allow 
States to use the ACSI for a Statewide sample of job seekers. This 
sample will be in addition to the sample of employers and WIA, Title I 
participants already being surveyed under our current license 
agreement.
    In the near future, we will publish detailed instructions for 
administering the job seeker customer satisfaction survey and reporting 
the job seeker customer satisfaction scores in the ET Handbook No. 406 
(ETA 9002 Data Preparation Handbook) which will be filed with OMB as 
part of a Paperwork Reduction Act of 1995 (PRA) package.
(3) Employment Rate
    ETA initially proposed an employment rate defined as:

    All Wagner-Peyser Act labor exchange applicants who registered 
in quarter Q0 and who earned wages in quarter 
Q1 or Q2 after registration, divided by the 
number of Wagner-Peyser Act labor exchange applicants who registered 
in quarter Q0.

    Six commenters supported the employment rate measure as proposed. 
One commenter questioned the concept of treating continued employment 
with the same employer as a positive outcome. This commenter suggested 
that a better outcome measure would be one that was previously 
considered by the workgroup, but not proposed (as noted in the August 
14, 2000 Federal Register), in which employment with a different 
employer following registration with the labor exchange would be 
counted as a positive outcome. Two commenters suggested that the 
employment rate measure did not add value and that it might be 
confusing due to its inconsistency with the performance measures for 
Title I of WIA. One commenter stated that measuring the number of job 
seekers who continue to be employed after receiving labor exchange 
services is difficult given the proposed method of measurement. 
Finally, one commenter raised concerns about including individuals 
using labor exchange services, but who do not have a goal of obtaining 
employment in the measure.
    Response: We have decided not to use the employment rate as a 
performance measure. However, we did reconsider the measurement concept 
previously reviewed by the workgroup in which employment with a 
different employer following registration with the labor exchange would 
be counted as a positive outcome and incorporated this concept into the 
job seeker entered employment rate.
(4) Entered Employment Rate
    We initially proposed an entered employment rate defined as:

    Of those Wagner-Peyser Act labor exchange applicants who were 
not employed upon registration in quarter Q0: The number 
who earned wages in quarter Q1 or Q2 after 
registration, divided by the number who registered in quarter 
Q0.

    Four commenters supported the entered employment rate measure as 
proposed. Several commenters noted that the measure does not capture 
the employment outcomes of people changing jobs, or those who make a

[[Page 29651]]

transition from part-time to full-time employment or vice versa. Some 
commenters suggested that a similar measure should apply specifically 
to unemployment insurance (UI) claimants. One commenter suggested that 
the measure should apply only to job seekers who received staff-
assisted services. Finally, two commenters suggested that the labor 
exchange methodology should not differ from that employed for Title I 
of WIA.
    Response: We have refined the job seeker entered employment rate to 
account for the employment outcomes of people who change jobs, rather 
than limiting it to those unemployed at registration. In the final 
measure, job seekers who, in the first or second quarter following the 
beginning of their registration year, become employed by a different 
employer than that by which they were employed the quarter prior to 
registration would be counted as experiencing successful employment 
outcomes. Thus, job seekers who make a transition from part-time to 
full-time employment, or full-time to part-time employment, would be 
counted as experiencing a successful employment outcome if the 
transition was based on employment with a new employer. Additionally, 
job seekers who are not employed at the beginning of the registration 
year will be counted as having entered employment if they become 
employed in the first or second quarter after the beginning of their 
registration year.
    The job seeker entered employment rate differs from that 
established for Title I of WIA because of the inherent differences 
between labor exchange services and the core, intensive, and training 
activities provided under Title I of WIA. The entered employment rate 
for Title I of WIA applies only to participants who have been 
determined to have exited the program after having completed receipt of 
workforce investment services. Also, the measure for Title I of WIA 
only applies to participants who are unemployed at the time of 
registration. The job seeker entered employment rate applies to all job 
seekers who register with the labor exchange to aid them in their 
search for work, regardless of whether they are employed or unemployed 
at the time of registration. Also, because the labor exchange system is 
not structured around the concept of exiting from service receipt, we 
deemed it impractical to establish such a concept for labor exchange 
performance measurement.
(5) Employment Retention Rate at Six Months
    We proposed an employment retention rate at six months measure 
defined as:

    Of those Wagner-Peyser Act labor exchange applicants who 
registered in quarter Q0 and who earned wages in quarter 
Q1 or Q2 after registration: the number who 
also earned wages in the second quarter following the quarter in 
which earned wages were first recorded, divided by the number who 
earned wages in quarter Q1 or Q2.

    Two commenters supported the measure in its proposed form. One 
suggested that the measure should be the same as the WIA retention 
measure. Many commenters provided comments on how the measure might be 
modified. Seven commenters stated that, as proposed, the results of the 
measure might be impacted by seasonal employment, or individuals who do 
not have a desire to work year-round, for example students, seasonal 
farmworkers, and those employed in certain seasonal industries, such as 
agriculture and tourism. Seven commenters questioned whether retention 
in employment should be established as a goal for the labor exchange 
and whether the labor exchange has the capacity to impact retention in 
employment. Two commenters cited concerns about the lag time before 
information would become available due to the use of wage records as a 
data source, and one commenter did not believe that total wages earned 
in a quarter from multiple employers would provide valuable information 
for performance measurement purposes. Finally, two commenters suggested 
that the retention measure should only apply to job seekers who receive 
staff-assisted services.
    Response: We have decided to retain the job seeker employment 
retention rate at six months measure for the labor exchange to parallel 
the employment retention rate at six months measure for Title I of WIA. 
However, the labor exchange retention measure builds upon the job 
seeker entered employment rate for consistency. Like the retention rate 
measure for Title I of WIA, the job seeker retention measure only 
applies to those who were determined to have entered employment 
according to the respective program's entered employment rate. To 
account for a portion of those registered job seekers who may not be 
likely to be in the labor force year-round, individuals under the age 
of 19 at the time of registration are excluded from the measure. This 
excludes many individuals who are students and is consistent with Title 
I of WIA in that there is no employment-specific retention rate measure 
for the younger youth (age 14-18) program. The job seeker retention 
measure is blind to conditions of the labor market. Thus, we will be 
developing methods to adjust for economic conditions and the 
characteristics of registered job seekers to use in adjusting 
performance goals and for interpreting final performance levels. Such 
methods are still being considered and will be addressed in a future 
notice. Like the measure for Title I of WIA, there is a substantial 
delay between when a job seeker registers with the labor exchange and 
when wage record information will become available for calculation of 
the measure. However, we believe that the benefit to be gained from 
this measure for program oversight outweighs the drawbacks associated 
with the need to wait until data become available.
    We support the measure of employment retention at six months for 
the labor exchange as a measure that is consistent with those of other 
workforce development programs. An employment retention measure helps 
capture the quality of staff-assisted services such as referrals to 
employment, job search workshops, career guidance, and other services 
provided by labor exchange staff. Quality self-services and facilitated 
self-help services also provide job seekers with resources to maintain 
continued employment. Examples include job seekers who may enter 
temporary or short-term employment, but who pursue continued employment 
based in part on their experience with the labor exchange. The measure 
also can account for those who return to the labor exchange for 
assistance in finding their next job following a spell of temporary or 
short-term employment. In addition to serving as a point of entry into 
the One-Stop system, the labor exchange also fulfils an important 
function in assisting job seekers in entering and maintaining 
employment as they exit One-Stop partner programs.
(6) General Comments
    Additionally, we received a number of comments on issues related to 
the performance measures in general. A number of these comments were 
requests for clearer definitions of such terms as ``registration,'' 
``labor exchange,'' and ``satisfaction.'' With regard to registration, 
some commenters wanted clarification on how the measures would apply to 
job seekers using self-services, and asked whether additional measures 
would be developed exclusively for users of self-

[[Page 29652]]

services. Others were concerned that registration policies may differ 
from State to State, because some States only register job seekers who 
receive staff-assisted services while other States register job seekers 
who utilize self-services, facilitated self-help services, and/or 
staff-assisted services.
    Response: Job seekers who receive staff-assisted services funded 
under the Wagner-Peyser Act will be counted as registered job seekers, 
as will individuals who are required to register with the labor 
exchange according to State law or policy. State agencies may establish 
their own policies about whether to register job seekers using self-
services or facilitated self-help services. At this time, we have 
decided not to implement a policy that will require registration or 
establish mandatory performance measures for users of self-services or 
facilitated self-help services provided by the labor exchange. We will 
continue to use the term--labor exchange--in the same way it is used in 
the Wagner-Peyser Act (29 U.S.C. 49). At this time, we are not formally 
defining the term, ``satisfaction,'' but are using that term in the 
same context to which it is referred in the ACSI methodology currently 
approved by OMB for use under Title I of WIA.
    There were other comments about the use of wage records, 
specifically, the delay in the availability of wage data and the 
difficulty in obtaining access to wage data for federal employees and 
military personnel. Several commenters wanted to ensure that procedures 
would be put in place to establish baseline data for setting 
performance goals and adjusting the measures to take into account such 
factors as economic conditions and the characteristics of the 
population served. One commenter suggested that the measures should be 
constructed to control for what otherwise would have happened to the 
registered job seekers, had they not registered with or used the labor 
exchange.
    Response: We have established two performance measures that rely 
heavily on wage record data for calculation. We support the use of wage 
record data for performance measurement for the labor exchange to 
maintain consistency with the performance measurement system for Title 
I of WIA, and to ease the burden of administrative follow-up inherent 
in the current reporting system for the labor exchange. We are 
currently in the process of developing data validation procedures to 
support quality control in performance measurement and data collection. 
Data validation procedures will apply to the wage record information 
that is used for the labor exchange performance measures, as well as to 
administrative records used to identify job seekers and employers. In 
addition, we expect that the Wage Record Interchange System (WRIS) will 
provide State agencies with a tool to enhance the availability of 
employment outcome data used to indicate entry into employment and 
retention.
    We acknowledge that many factors outside the control of the labor 
exchange will impact the prospects of registered job seekers entering 
and/or retaining employment. Although we have not yet developed methods 
to account for such factors, we plan to consider local and regional 
economic conditions and the socioeconomic characteristics of registered 
job seekers as performance goals are established and as the success in 
meeting such goals is evaluated. We will publish methods for 
establishing and adjusting performance goals in a future notice.
    Finally, we received a number of suggestions for additional 
measures. Several commenters suggested additional employer measures, 
including a comparison of the number of employers receiving services 
compared to the total number of employers in the State, and a 
comparison of the number of job openings listed with the State agency 
compared to the total number of new hires occurring in a State. Other 
proposed measures included cost per entered employment, a measure of 
the length of it takes a job seeker to enter employment after 
registering with the labor exchange, and a measure relating the value-
added of the labor exchange to its cost.
    Response: We acknowledge that only one labor exchange performance 
measure applies specifically to employers, and that this is a measure 
of employers satisfaction with the One-Stop system in general, rather 
than with the labor exchange program in particular. We will continue to 
investigate additional techniques to assess the performance of the 
labor exchange in providing services to its employer customers. We will 
consider the measures suggested above as possibilities for any 
enhancements to the measurement strategy for employer services in 
particular and the labor exchange in general. In the meantime, States 
are free to adopt additional measures that they believe will enhance 
the delivery of labor exchange services.

C. Labor Exchange Performance Measures.

    We establish four performance measures for the public labor 
exchange:

 Job Seeker Entered Employment Rate
 Job Seeker Employment Retention Rate at Six Months
 Job Seeker Customer Satisfaction
 Employer Customer Satisfaction

    The labor exchange performance measures apply to public labor 
exchange services provided as part of the One-Stop delivery systems of 
the States. This includes labor exchange services provided to job 
seekers and employers under the Wagner-Peyser Act, and to veterans by 
Disabled Veterans' Outreach Program (DVOP) and Local Veterans' 
Employment Representative (LVER) staff under VETS programs, as 
specified in Title 38 of the U.S.C. Individual States may include other 
publicly-funded labor exchange services in the labor exchange 
performance measurement system at their discretion.
    The labor exchange performance measures apply to all individuals 
who are registered job seekers with the public labor exchange, and to 
employers who receive substantial service involving personal contact 
with One-Stop staff. At a minimum, State agencies must request the 
following information from job seekers during registration: name, 
contact information, social security number, ethnicity, race, veteran 
status, age, gender, employment status, educational attainment, 
disability status, and migrant and seasonal farmworker status. Job 
seekers may be registered upon contacting the labor exchange through 
the One-Stop delivery system or as required by State law or policy; 
however, job seekers receiving staff-assisted services funded under the 
Wagner-Peyser Act must be registered. Job seekers who use self-services 
or facilitated self-help services also may be registered, but 
registration is not required for receipt of these services.
    A job-seeking customer is counted as a registered job seeker during 
the quarter in which registration occurs (registration quarter) and the 
subsequent three quarters. This four quarter period constitutes the 
registration year. A registered job seeker who receives services during 
the fourth quarter after the registration quarter will begin a new 
registration year or be considered re-registered. Such a job seeker 
would then be counted again as a registered job seeker during each of 
the four reporting periods covering that registration year. If a job 
seeker's registration year elapses, and after some time he or she 
returns to the labor exchange, that job seeker would begin a new 
registration year.
    The labor exchange performance measures are defined as follows:
(1) Job Seeker Entered Employment Rate (JSEER)

[[Page 29653]]

[GRAPHIC] [TIFF OMITTED] TN31MY01.001

    Elements of the measure are defined as follows:
    Entered Employment with a New Employer: The number of registered 
job seekers who, in the first or second quarter following the 
registration quarter, earned wages from a new employer if the job 
seeker was previously not employed, or earned wages from a different 
employer than that from which the registered job seeker earned wages in 
the quarter prior to the registration quarter if the job seeker was 
previously employed.
    Registered Job Seekers: Job seekers who registered with the labor 
exchange during the registration quarter; job seekers who were re-
registered after their registration year expired; job seekers who were 
not formally re-registered, but who engaged in a labor exchange 
activity after their registration year expired.
    Employed or Re-employed with Same Employer: Those job seekers whose 
only wages earned in the first and second quarter following 
registration were exclusively with the same employer from which wages 
were earned in the quarter prior to the registration quarter.
    Registration Quarter: The calendar quarter in which a job seeker 
completed an initial registration with the labor exchange or in which a 
previously registered job seeker began a new registration year.
    According to this measure, a successful employment outcome is 
recorded for a job seeker who enters employment with a new employer, 
whether the job seeker was employed or unemployed at the time of 
registration. This outcome is determined by comparing the employer 
identification numbers (EIN) of registered job seekers' employers prior 
to and following registration based on information contained in the UI 
wage record database, the State Directory of New Hires (SDNH) database, 
or other available records. An unsuccessful outcome is recorded for a 
job seeker who does not enter employment with a new employer during the 
measurement period. Job seekers who remain employed exclusively with 
the same employer during the measurement period are excluded from the 
calculation.
(2) Job Seeker Employment Retention Rate at Six Months (JSERR)
[GRAPHIC] [TIFF OMITTED] TN31MY01.002

    Elements of the measure are defined as follows:
    Retained Employment Two Quarters after Entered Employment with a 
New Employer (age 19 and over): The number of registered job seekers 
age 19 and older at the time of registration who earned wages in the 
second quarter following the quarter in which they Entered Employment 
with a New Employer.
    Entered Employment with New Employer (age 19 and over): The number 
of registered job seekers age 19 and older at the time of registration 
who, in the first or second quarter following the registration quarter, 
earned wages from a new or different employer than that from which the 
registered job seeker earned wages in the quarter prior to registration 
quarter.
    According to this measure, a successful employment retention 
outcome is recorded for job seekers, age 19 and over at the time of 
registration, who are determined to have entered employment according 
to the job seeker entered employment rate measure, and who earned wages 
with any employer in the second quarter following the quarter in which 
they first were determined to have entered employment.
(3) Job Seeker Customer Satisfaction
    The job seeker customer satisfaction measure for the public labor 
exchange is patterned after the participant customer satisfaction 
survey for Title I of WIA; it requires the use of the ACSI methodology. 
This methodology is published at TEGL 14-00, March 5, 2001. A sample of 
registered job seekers are surveyed between 60 and 90 days following 
the date of registration with the labor exchange. State agencies will 
be able to exercise some discretion in how they administer the survey, 
so long as the ACSI methodology is followed. Possibilities might range 
from surveying a population of registered job seekers in a distinct 
survey, to coordinating the job seeker customer satisfaction survey 
with the WIA participant customer satisfaction survey or any customer 
satisfaction survey that might be administered by VETS or another One-
Stop partner program. In all cases, the ACSI methodology must be 
followed. We support the concept of common measurement techniques for 
services provided as part of the One-Stop delivery system and intend to 
provide States with the broadest opportunity to coordinate surveys of 
One-Stop customers' satisfaction.
    We are currently engaging in discussions with the University of 
Michigan to use the ACSI for the labor exchange job seeker customer 
satisfaction measure. We will publish the specifications for the job 
seeker customer satisfaction survey, including the required questions 
and the survey methodology, as part of the ET Handbook No. 406 (ETA 
9002 Data Preparation Handbook).
(4) Employer Customer Satisfaction
    The public labor exchange will adopt the results of the ACSI survey 
administered under Title I of WIA to measure employer satisfaction with 
One-Stop employer services. Accordingly, States should administer only 
one survey of employers to measure their satisfaction with One-Stop 
employer services to meet both the WIA and the labor exchange employer 
customer satisfaction measurement requirements. Specifications for the 
employer customer satisfaction survey are described in TEGL 14-01, 
March 5, 2001.

    Signed at Washington, D.C., this 23rd day of May, 2001.
Raymond J. Uhalde,
Deputy Assistant Secretary for Employment and Training.
[FR Doc. 01-13611 Filed 5-30-01; 8:45 am]
BILLING CODE 4510-30-P