[Federal Register Volume 65, Number 157 (Monday, August 14, 2000)]
[Notices]
[Pages 49708-49714]
From the Federal Register Online via the Government Publishing Office [www.gpo.gov]
[FR Doc No: 00-20544]



[[Page 49707]]

-----------------------------------------------------------------------

Part IV





Department of Labor





-----------------------------------------------------------------------



Employment and Training Administration



-----------------------------------------------------------------------



Labor Exchange Performance Measurement System; Notice

  Federal Register / Vol. 65, No. 157 / Monday, August 14, 2000 / 
Notices  

[[Page 49708]]


-----------------------------------------------------------------------

DEPARTMENT OF LABOR

Employment and Training Administration


Labor Exchange Performance Measurement System

AGENCY: Employment and Training Administration, Labor.

ACTION: Notice and request for comments.

-----------------------------------------------------------------------

SUMMARY: This notice announces and requests comments on two components 
of a proposed labor exchange performance measurement system. A set of 
performance measures are proposed for measuring the performance of the 
public employment service in providing effective labor exchange 
services to employers and job seekers as part of the One-Stop delivery 
systems established by the States. A set of procedures also are 
proposed for State agencies and ETA to employ in establishing expected 
levels of performance to assure the delivery of high quality labor 
exchange services. These proposed labor exchange performance measures 
and procedures for establishing expected levels of performance will be 
key components of a comprehensive performance accountability system 
being developed for the employment service.

DATES: Comments on these proposed labor exchange performance measures 
and procedures for establishing expected levels of performance must be 
received by the U.S. Department of Labor on or before October 13, 2000. 
Late-filed comments will be considered to the extent possible.

ADDRESSES: Send comments to Timothy F. Sullivan, Chief, Division of 
United States Employment Service & ALMIS, U.S. Department of Labor, 200 
Constitution Avenue NW., Room C-4514, Washington, DC 20210, Facsimile: 
202-208-5844, E-mail: [email protected].

FOR FURTHER INFORMATION CONTACT: Timothy F. Sullivan, 202-219-5257, E-
mail: [email protected].

SUPPLEMENTARY INFORMATION:

I. Authority

    Components of a labor exchange performance measurement system are 
proposed under the following authority:

A. Wagner-Peyser Act Sec. 3(a), 29 U.S.C. 49b(a)

    The Secretary shall assist in coordinating the State public 
employment services throughout the country and in increasing their 
usefulness by developing and prescribing minimum standards of 
efficiency, assisting them in meeting problems peculiar to their 
localities, promoting uniformity in their administrative and 
statistical procedures, furnishing and publishing information as to 
opportunities for employment and other information of value in the 
operation of the system, and maintaining a system for clearing labor 
between the States.

B. Wagner-Peyser Act Sec. 3(c)(2), 29 U.S.C. 49b(c)

    The Secretary shall--(2) assist in the development of continuous 
improvement models for such nationwide system that ensure private 
sector satisfaction with the system and meet the demands of job seekers 
relating to the system.

C. Wagner-Peyser Act Sec. 7(b), 29 U.S.C. 49f(b)

    Ten percent of the sums allotted to each State pursuant to section 
6 shall be reserved for use in accordance with this subsection by the 
Governor of each such State to provide--(1) performance incentives for 
public employment service offices and programs, consistent with 
performance standards established by the Secretary, taking into account 
direct or indirect placements (including those resulting from self-
directed job search or group job search activities assisted by such 
offices or programs), wages on entered employment, retention, and other 
appropriate factors.

D. Wagner-Peyser Act Sec. 10(c), 29 U.S.C. 49i(c)

    Each State receiving funds under this Act shall--
    (1) make such reports concerning its operations and expenditures in 
such form and containing such information as shall be prescribed by the 
Secretary, and
    (2) establish and maintain a management information system in 
accordance with guidelines established by the Secretary designed to 
facilitate the compilation and analysis of programmatic and financial 
data necessary for reporting, monitoring and evaluating purposes.

E. Wagner-Peyser Act Sec. 13(a), 29 U.S.C. 49l(a)

    The Secretary is authorized to establish performance standards for 
activities under this Act which shall take into account the differences 
in priorities reflected in State plans.

F. Wagner-Peyser Act Sec. 15(e)(2)(I), 29 U.S.C. 49l-2(e)

    (e) State responsibilities.--
    (2) Duties.--In order to receive Federal financial assistance under 
this section, the State agency shall--
    (I) utilize the quarterly records described in section 136(f)(2) of 
the Workforce Investment Act of 1998 (29 U.S.C. 2871(f)(2)) to assist 
the State and other States in measuring State progress on State 
performance measures.

II. Labor Exchange Performance Measurement System

    The Employment and Training Administration (ETA) is establishing a 
comprehensive performance measurement system for the public labor 
exchange. This process consists of three tasks: (1) Developing a set of 
labor exchange performance measures, (2) developing procedures for 
establishing expected levels of performance that State agencies and ETA 
can use for assuring the delivery of high quality labor exchange 
services, and (3) revising the data collection procedures and reporting 
requirements applicable to the public labor exchange.
    In February 2000, ETA convened a workgroup to begin the development 
of a comprehensive performance measurement system for the public labor 
exchange. This workgroup was formed in collaboration with the 
Interstate Conference of Employment Security Agencies (ICESA), and is 
comprised of representatives from fifteen State agencies, ICESA, the 
Veterans' Employment and Training Service (VETS), and the ETA regional 
and national offices. Representatives from America's Workforce 
Technology Solutions (AWTS) and Social Policy Research (SPR) 
Associates, Inc. provided technical support to the workgroup, but did 
not participate in the process of making final recommendations. The 
workgroup met once in the winter and once in the spring of 2000.
    By the conclusion of the spring 2000 meeting, the workgroup had 
identified and defined a set of recommended performance measures for 
the public labor exchange. It also had developed recommended procedures 
for State agencies and ETA to employ in establishing expected levels of 
performance for the labor exchange and for assuring the delivery of 
high quality labor exchange services.
    This notice announces and requests comments on a proposed set of 
performance measures to be used to measure the performance of the 
public employment service in providing effective labor exchange 
services to employers and job seekers as part of the One-Stop delivery 
systems. It also announces and requests comments on a

[[Page 49709]]

proposed set of procedures for State agencies and ETA to use in 
establishing expected levels of performance.

A. Labor Exchange Performance Measures

    Based on recommendations of the labor exchange performance 
measurement system workgroup, ETA proposes the following performance 
measures for the public labor exchange:

 Employer Customer Satisfaction
 Job Seeker Customer Satisfaction
 Employment Rate
 Entered Employment Rate
 Employment Retention Rate at Six Months
1. Identification and Selection of Performance Measures
    During the first meeting, the workgroup followed a methodological 
approach in developing performance measures to recommend for the labor 
exchange. This consisted of describing the value that implementation of 
a performance measurement system would have for the public labor 
exchange and identifying concerns that might arise out of such a 
system; identifying the labor exchange outcomes that should be 
measured; and identifying qualities that are characteristic of good 
performance measures.
    The workgroup identified a number of sound reasons for establishing 
a performance measurement system for the labor exchange. Key among 
these are that performance measures are essential for program managers 
to monitor the effectiveness of service delivery, and that performance 
information is of paramount importance to the Congress, State 
legislatures, the business community, and the general public as a means 
of assessing the value of the public labor exchange. The workgroup also 
noted that funds for the labor exchange are budgeted and appropriated, 
in part, based on such information and the message it conveys regarding 
the effectiveness of labor exchange service delivery. Additionally, 
performance reporting is required under the Government Performance and 
Results Act (GPRA) and it is important that a common system of 
measurement be developed so that performance reports of the various 
State agencies can be aggregated for reporting at the Federal level in 
support of GPRA requirements.
    The workgroup recommended that the labor exchange performance 
measurement system contain procedures for setting expected levels of 
performance. These procedures should take into account the many 
differences between the States, such as labor market conditions and 
variations in how the States administer their programs under the 
Wagner-Peyser Act. The workgroup decided that demographic 
characteristics of the population served, such as age, race, ethnicity, 
and sex should not be considered when negotiating expected levels of 
performance. This is consistent with the requirement of providing 
universal access to job seekers. The workgroup also observed that 
timely and reliable data are essential to an effective performance 
measurement system. Finally, the group recognized that the labor 
exchange performance measurement system must not be developed in a 
vacuum. The performance measurement systems developed for related 
workforce development programs, such as the one currently being 
implemented for the Workforce Investment Act of 1998 (WIA) and those 
currently in use by the various States, should be taken into account in 
the development process.
    The events that naturally result from the labor exchange carrying 
out its roles and responsibilities in providing services to its 
customers can be termed outcomes. There are two types of outcomes: end 
outcomes and intermediate outcomes. End outcomes represent the final 
objectives of the labor exchange, while intermediate outcomes represent 
accomplishments that lead to achieving the final objectives. The 
workgroup identified the following as key end outcomes for customers of 
the labor exchange:

 Job placements
 Entry into employment
 Shorter duration of unemployment
 Steady employment and income
 Customer satisfaction

    The workgroup identified the following as key intermediate outcomes 
for primary customers:

 Quality job matches
 Knowledge of career and labor market information
 Qualified applicants
 Access to qualified applicants
 Access to job openings

    The workgroup also identified characteristics of a good performance 
measurement system and used these as it considered and then recommended 
performance measures for the labor exchange. A good performance 
measurement system must be comprehensive and, to the extent possible, 
measure the primary end outcomes of the labor exchange. It must consist 
of a limited number of simple and easy to understand measures, and must 
yield timely information for management purposes. The performance 
measurement system must be developed such that it is objective and non-
manipulative in order to avert unintended consequences. Data necessary 
for input to the performance measures must be readily available and 
collected at a reasonable cost. The system must take into account the 
uniqueness of the States and how each operates somewhat differently, 
while also allowing for uniform measurement across the States so that 
the aggregation of State performance information will be meaningful at 
the national level. The system also should allow for the measures to be 
applied at the sub-State level if so desired by the various States. 
Finally, the system should be consistent, to the extent possible, with 
related workforce development programs.
2. Proposed Labor Exchange Performance Measures
    Five performance measures are proposed for the public labor 
exchange based on recommendations of the labor exchange performance 
measurement system workgroup. In its deliberations, the workgroup 
considered a wide range of options as potential measures of 
performance. The workgroup agreed by consensus to recommend two 
customer satisfaction measures, an employment rate measure and an 
entered employment rate measure. A substantial majority of the 
workgroup also supported the employment retention rate at six months 
measure. These recommended labor exchange performance measures are 
consistent with the aforementioned characteristics of good performance 
measures.
    What follows are operational definitions of the proposed labor 
exchange employer and job seeker performance measures, and the 
rationale for recommending them:

a. Employer Measure

Employer Customer Satisfaction
    It is proposed that the results of the American Customer 
Satisfaction Index (ACSI) which will be used to measure employer 
customer satisfaction under WIA also be used to measure employers' 
satisfaction with labor exchange services. Under this proposal, one 
survey will be conducted by the States to measure employer customer 
satisfaction with both WIA services and Wagner-Peyser Act labor 
exchange services. Specifications for the employer customer 
satisfaction survey are described in Training and Employment Guidance 
Letter (TEGL) 7-99, pp. 36-40, issued by ETA on March 3, 2000.

[[Page 49710]]

    Adopting the WIA employer employer customer satisfaction measure 
for the labor exchange is proposed because the employer population from 
which the sample is drawn consists of employers who received a 
substantial service involving personal contact with One-Stop staff. 
Labor exchange staff provide a substantial portion of such services and 
the WIA employer satisfaction measure depicts, to a great extent, the 
satisfaction of employers with labor exchange services. Using a common 
measure to obtain information on employer customer satisfaction for 
both WIA and the labor exchange supports the integration of the labor 
exchange into the One-Stop delivery system. It also emphasizes the 
importance of providing high quality services to employers, a focus of 
the One-Stop delivery system. State Wagner-Peyser Act agencies will 
need to coordinate with State WIA agencies to obtain the results from 
the employer customer satisfaction survey and, if they so desire, to 
add additional questions to the survey instrument.
    Using a uniform telephone methodology, each State must survey up to 
1000 employers each year to obtain at least 500 completed surveys 
(except for States that serve less than 1000 employers, in which case, 
all employers served must be surveyed). The surveys should be conducted 
on a rolling basis throughout the program year. To obtain sufficient 
numbers, smaller States will need to survey on an ongoing basis. 
Employers should be contacted within 60 days of the completion of the 
service or 30-60 days after a job order has been listed where no 
referrals have been made. The employer customer satisfaction score is a 
weighted average of employer ratings on each of three questions 
regarding overall satisfaction, and is reported on a 0-100 point scale. 
The score is a weighted average, not a percentage.
What Questions Will Be Asked in the Survey?
    The survey will be conducted by telephone. The proposed lead-in can 
be modified to suit the individual needs of the State and the program 
names recognizable for their population. The lead-in provided below is 
a model to be used as guidance. However, the numbered questions must 
remain as stated.

    My name is __________ with XXXXX and I am conducting a survey 
for the XXXX XXXXX. I would like to speak to Ms./Mr. __________.
    Are you the Ms./Mr. __________ who (describe the service 
received).
    I would like to ask you some questions about your recent 
experience with __________. Our purpose is to learn from you how to 
improve programs and services offered to employers. The survey 
should take about XX minutes to complete.
    (1) Utilizing a scale of 1 to 10 where ``1'' means ``Very 
Dissatisfied'' and ``10'' means ``Very Satisfied'' what is your 
overall satisfaction with the service(s) provided from __________?

 
     Very                                                                                                                Very
 Dissatisfied                                                                                                          Satisfied      DK          REF
 
          1             2            3            4            5            6            7            8           9          10          11          12
 

    (2) Considering all of the expectations you may have had about 
the services, to what extent have the services met your 
expectations? ``1'' now means ``Falls Short of Your Expectations'' 
and ``10'' means ``Exceeds Your Expectations.''

 
 Falls Short
      of                                                                                                               Exceeds        DK          REF
 Expectations                                                                                                       Expectations
 
          1             2            3            4            5            6           7           8           9           10           11          12
 

    (3) Now think of the ideal service(s) for people in your 
circumstances. How well do you think the service(s) you received 
compare with the ideal service(s)? ``1'' now means ``Not Very Close 
to Ideal'' and ``10'' now means ``Very Close to the Ideal.''

 
 Not Close                                                                                                           Very Close
 To Ideal                                                                                                             To Ideal        DK          REF
 
         1           2            3            4            5            6            7            8            9           10           11          12
 

Definition of Terms

    Sample. A group of cases selected from a population by a random 
process where everyone has an equal probability of being selected.
    Response rate. The percentage of people who have valid contact 
information who are contacted and respond to all the questions on 
the survey.
    DK. Don't Know.
    REF. Refused to answer.
The Calculation
    The overall score for the American Customer Satisfaction Index 
(ACSI) Measure is accomplished by calculating the weighted average of 
the raw scores for each of the customer satisfaction questions given by 
each respondent. The weighted average score is then transformed to an 
index reported on a 0-100 scale. The aggregate index score is simply 
the weighted average of each case's index score.
    The ACSI trademark is proprietary property of the University of 
Michigan. The Department has established a license agreement with the 
University of Michigan that will allow States the use of the ACSI for a 
Statewide sample of employers (and WIA participants). States that want 
to use the ACSI for measuring customer satisfaction for each local area 
will have to establish an independent contract with the University of 
Michigan. States may also contract with CFI Group for additional 
assistance in measuring, analyzing, and understanding ACSI data. 
Procedures for contracting with the CFI Group are being developed and 
will be issued when finalized.

    Notes: CFI Group will provide the actual weights given for (W1), 
(W2), and (W3) below. (It has yet to be determined how the weights 
will be distributed to the States). In calculating respondent level 
index scores, round to two decimal points.

    When calculating the average index score, round to the nearest 
whole number. For any case, the general formula for calculating the 
index score is given as:

Index Score = {[(Q1)(W1) + (Q2)(W2) + (Q3)(W3)] -1}  x  11.111 where:

Q1 = raw score on question #1
Q2 = raw score on question #2
Q3 = raw score for question #3
W1 = weight for question #1

[[Page 49711]]

W2 = weight for question #2
W3 = weight for question #3

    Example:
    If the respondent answers were 5, 8, and 9 respectively for each of 
the three customer satisfaction questions, and the weights for each of 
the three questions were .4, .2, and .4* respectively the calculation 
for the respondent's index score would be as follows:

{[(5)(.4) + (8)(.2) + (9)(.4)]--1}  x  11.111 = {[7.2]-1}  x  11.111 = 
68.89

    If two more respondents whose raw scores on the three questions 
were 6, 10, and 6 and 9, 6, and 7 respectively, using the same weights 
listed above, those two respondent's index scores would be: 64.44 and 
73.33. To calculate the aggregate index score, simply average the 
individual respondent's index scores and round to the nearest whole 
number as follows:

(68.89 + 64.44 + 73.33) /3 = 69

    * These weights are examples only, CFI group will provide the 
actual weights.
    The workgroup considered other options for employer measures 
including a job order or job opening fill rate, an employer market 
penetration rate, and a measure of employer use of labor exchange 
services. The workgroup was unable to identify definitions for these 
types of employer performance measures that were consistent with the 
identified characteristics of a good performance measure. While 
employer customer satisfaction is the only proposed performance measure 
for employers, ETA will research the development of possible additional 
employer performance measures for future consideration.

b. Job Seeker Measures

    For job seekers, measures will be used to account for performance 
of the public labor exchange with respect to all applicants who 
register, subject to the criteria contained in the definition of each 
measure. The universe of job seekers will consist of an unduplicated 
count of job seekers who register, or who renew or reactivate their 
registration, during the applicable program year.
    Including all registrants in the universe is proposed because it 
maintains consistency with the concept of providing universal access to 
labor exchange services. The measurement system is designed to capture 
the employment outcomes of all those who request access to labor 
exchange services through registration. This allows for measuring the 
outcomes of all labor exchange services that are made available to job-
seeking applicants. It also maintains consistency with the criteria 
described in section 7(b) of the Wagner-Peyser Act for performance 
standards to be established by the Secretary that take into account 
entry into employment resulting from either self-directed job search 
activities or staff-assisted job search activities.
    The job seeker customer satisfaction measure will rely on telephone 
survey data for outcome information, while the employment rate, entered 
employment rate, and employment retention rate at six months outcome 
measures will rely on unemployment insurance (UI) wage records as a 
primary data source. State agencies also will retain the option to use 
data obtained from administrative follow-up, the method of data 
collection currently used by many State agencies, to supplement the 
wage record information. The use of wage record information will allow 
for more reliable and comprehensive collection of employment outcome 
data at a lower cost than methods currently used by many State 
agencies. The advent of the Wage Record Interchange System (WRIS) will 
provide State agencies with an additional resource for obtaining wage 
records from other State agencies to use in tracking the outcomes of 
job seekers who have migrated across State lines.
    The measurement period will consist of the four quarters comprising 
a program year. Performance outcomes will be attributable to the 
program year in which the outcome occurs, whether the job seeker 
registered with the labor exchange in that program year or the previous 
program year. This will require reporting in the numerator, the total 
number of job seekers who achieve the expected outcomes during the 
appropriate measurement quarters, and reporting in the denominator, the 
total number of registered job seekers who could have achieved the 
expected outcomes (i.e. employment or retention) during the measurement 
quarters.
    The aforementioned criteria apply, as appropriate, to the following 
job seeker labor exchange performance measures:
Job Seeker Customer Satisfaction
    ETA proposes to implement a job seeker customer satisfaction 
measure that mirrors the WIA participant customer satisfaction survey 
and uses the ACSI methodology. Specifications for the labor exchange 
job seeker customer satisfaction survey are as follows:
    The job seeker customer satisfaction score is a weighted average of 
participant ratings on each of three questions regarding overall 
satisfaction, and is reported on a 0-100 point scale. The score is a 
weighted average, not a percentage.
Who Will Be Surveyed?
    All labor exchange applicants who register with the labor exchange 
are eligible to be chosen for inclusion in the random sample.
How Many Must Be Surveyed?
    A sample of 250 will be taken from the job seeker applicants who 
register with the labor exchange in each quarter. Five hundred 
completed job seeker surveys must be obtained each year for calculation 
of the measure. A completed job seeker survey is defined as a survey in 
which all three questions regarding overall satisfaction have been 
answered. The response rate from the sample with valid contact 
information must be a minimum of 50 percent. The standard of 500 from a 
sample of the whole population of customers provides accuracy such that 
there is only a 5 in 100 chance that the results would vary by more 
than 5 points from the score obtained from surveying the 
whole population.
How Should the Survey Be Conducted?
    The responses are obtained using a uniform telephone methodology. 
The rationale for only using telephone surveys include: the 
comparability of the measure for assessing performance levels is most 
reliably obtained with a telephone survey; telephone surveys are easily 
and reliably administered; and defining procedures for mailed surveys 
is more difficult than defining procedures for telephone surveys. 
Estimates of the cost of telephone surveys nationwide average $15 per 
completed survey. Since States will need to complete 500 job seeker 
surveys, the cost is estimated at about $7,500 per State per year.
When Should the Survey Be Conducted?
    The surveys should be conducted on a rolling basis within the 
timeframe indicated below for job seekers. To obtain sufficient 
numbers, smaller States will need to survey on an ongoing basis. Job 
seekers should be contacted within 60-90 days of the date of 
registration, or renewal or reactivation.
What Questions Will Be Asked in the Survey?
    The survey will be conducted by telephone and the following lead-in 
will be used at the beginning of the interview. The lead-in can be 
modified to suit the individual needs of the State and the names for 
program services recognizable for their population. The

[[Page 49712]]

lead-in provided below is a model to be used as guidance. The numbered 
questions must remain as stated.

    My name is __________ with XXXXX and I am conducting a survey 
for the XXXX XXXXX. I would like to speak to Ms./Mr. __________.
    Are you the Ms./Mr. __________ who was looking for a job a few 
months ago?
    I would like to ask you some questions about your recent 
experience looking for a job. Our purpose is to learn from you how 
to improve programs and services offered to people in XXX. The 
survey should take about XX minutes to complete. First I am going to 
read a list of services you may have received. Indicate as I read 
them those you recall receiving during the period in which you were 
seeking employment and/or training at the XX center.

 A thorough assessment of your needs
 Assistance about finding a job
 Assistance to develop an individual employment plan
 Assistance to decide about the best training to take
 Assistance from someone to support you during your job 
search or training
 Use of electronic job search tools (e.g. America's Job 
Bank, Internet tools)

(States may modify the list of services as appropriate for the labor 
exchange)

    Did you get any other help or services that I have not 
mentioned? (specify)
    (1) Utilizing a scale of 1 to 10 where ``1'' means ``Very 
Dissatisfied'' and ``10'' means ``Very Satisfied'' what is your 
overall satisfaction with the services provided from __________?

 
     Very                                                                                                                Very
 dissatisfied                                                                                                          satisfied      DK          REF
 
          1             2            3            4            5            6            7            8           9          10          11          12
 

    (2) Considering all of the expectations you may have had about the 
services, to what extent have the services met your expectations? ``1'' 
now means ``Falls Short of Your Expectations'' and ``10'' means 
``Exceeds Your Expectations.''

 
 Falls short
      of                                                                                                               Exceeds        DK          REF
 expectations                                                                                                       expectations
 
          1             2            3            4            5            6           7           8           9           10           11          12
 

    (3) Now think of the ideal program for people in your 
circumstances. How well do you think the services you received compare 
with the ideal set of services? ``1'' now means ``Not very close to the 
Ideal'' and ``10'' means ``Very close to the Ideal.''

 
 Not close                                                                                                           Very close
 to ideal                                                                                                             to ideal        DK          REF
 
         1           2            3            4            5            6            7            8            9           10           11          12
 

    The same ASCI calculation is used for the job seeker customer 
satisfaction measure as was described above for the employer customer 
satisfaction measure. The Department is currently engaging in 
discussions with the University of Michigan to make arrangements to use 
the ACSI for the labor exchange job seeker customer satisfaction 
measure. As the population of job seekers registering with the labor 
exchange is different from the population of participants exiting WIA 
services, a separate survey is required to adequately gauge the 
satisfaction of job seeker customers.
    State agencies will have flexibility in modifying the lead-in to 
the questionnaire to suit their particular needs and also may add 
additional questions, as long as the three questions presented above 
remain the same and are the initial three questions in the survey. 
Since there likely will be a number of individuals who both register 
with the labor exchange and who exit WIA, State agencies are requested 
to coordinate these survey efforts to eliminate the possibility of 
individuals being surveyed twice.
Employment Rate
    The proposed employment rate performance measure is defined as:
    All Wagner-Peyser Act labor exchange applicants who registered in 
quarter Q0 and who earned wages in quarter Q1 or 
Q2 after registration, divided by the number of Wagner-
Peyser Act labor exchange applicants who registered in quarter 
Q0.
    This performance measure reports on the employment outcomes that 
may be attributable to the labor exchange services made available to 
all applicants. Including all applicants in the measurement population 
supports the concept of providing universal access to labor exchange 
services by establishing accountability for the employment outcomes of 
all job seekers provided access to labor exchange services. This 
includes new entrants to the labor market, job seekers who are not 
employed, and incumbent workers.
    This performance measure uses a period of two quarters to look for 
entry into employment because two quarters can be considered an 
appropriate length of time in which to expect a positive employment 
outcome for those provided access to labor exchange services. This 
period of time is also similar to the 26-week maximum period of 
eligibility for unemployment insurance (UI) benefits, which is deemed 
to be an appropriate period of time for UI claimants, a key labor 
exchange customer group, to obtain suitable employment.
    In addition to entry into employment with a new employer, this 
measure also recognizes as a positive outcome, the job seeker who is 
employed at the time of registration with the labor exchange, and who 
during the next two quarters remains employed with the same employer. 
In such instances, if this job seeker registered with the labor 
exchange, he or she had some inclination to continue being employed, 
either with the same or a different employer. For the job seeker who 
remained with the same employer, available labor exchange services, 
such as job counseling and labor market information, may have enabled 
that job seeker to assess his or her employment situation by surveying 
the labor market (i.e. available jobs, availability of transportation, 
wage rates, training requirements, etc.). That job seeker's continued 
employment, with the same employer, may in part be attributable to a 
rational decision to maintain his or her employment situation based on 
such labor exchange services.

[[Page 49713]]

    The workgroup considered including only as a positive outcome for 
this measure, entry into employment by those not employed and entry 
into employment with a different employer by those currently employed. 
While such a definition was a strong candidate for a labor exchange 
performance measure, members of the workgroup deemed the difficulty of 
distinguishing employment with one employer from that with another as 
too burdensome for performance measurement purposes, and thus this 
option was not recommended to ETA. Such a performance measure would 
have required each State agency to conduct a match to determine whether 
the Federal Employer Identification Number (FEIN) on a job seeker's 
wage records was with a different employer in the measurement quarters 
(Q1 or Q2) than it was in the registration 
quarter (Q0). Several members of the workgroup expressed 
concern that this was not feasible at a reasonable cost. Others pointed 
out that it would also exclude job seekers who obtained a new position 
with the same employer.
    Finally, a considerable number of workgroup members suggested 
limiting the measurement period to a single quarter following 
registration, rather than two quarters as proposed for the job seeker 
measures. While this would enhance consistency with the WIA core 
measures and might support the more timely delivery of services to job 
seekers, the workgroup ultimately decided to use two quarters, 
acknowledging that the benefits of allowing two quarters to record 
employment outcomes outweighed these other concerns.
Entered Employment Rate
    The proposed entered employment rate performance measure is defined 
as:
    Of those Wagner-Peyser Act labor exchange applicants who were not 
employed upon registration in quarter Q0: The number who 
earned wages in quarter Q1 or Q2 after 
registration, divided by the number who registered in quarter 
Q0.
    Within the universe of all applicants, this performance indicator 
measures the employment outcomes of the job seeker population that is 
not employed at the time of registration. The rationale for using a 
time period of two quarters for this performance measure is the same as 
that described above for the employment rate measure. The entered 
employment rate measure is proposed out of recognition that it is 
important to obtain employment outcome information specifically on job 
seekers who are not employed when registering with the labor exchange.
    It is acknowledged that there are some subtle distinctions between 
the entered employment rate measure for labor exchange job seekers and 
the entered employment rate measure for WIA participants, (i.e., using 
registration as the trigger for the measurement period for the labor 
exchange, rather than exit, as is done for WIA; and using two quarters 
as the measurement period for the labor exchange, rather than one, as 
is the case for WIA). The entered employment rate measure for the labor 
exchange is recommended, as defined above, since the nature of the 
labor exchange services provided to job seekers are different than the 
services provided under WIA. Many WIA participants need core, 
intensive, and training services in order to become job ready, while 
the preponderance job seekers who avail themselves of labor exchange 
services are at the stage of actively seeking work.
    The workgroup also considered including in the definition of the 
entered employment rate measure, the outcomes of currently employed job 
seekers who enter into new employment with a different employer. For 
the same reasons as indicated for the employment rate measure, the 
workgroup did not recommend defining the entered employment rate 
measure to include this group of job seekers.
Employment Retention Rate at Six Months
    The proposed employment retention rate at six months performance 
measure is defined as:
    Of those Wagner-Peyser Act labor exchange applicants who registered 
in quarter Q0 and who earned wages in quarter Q1 
or Q2 after registration: the number who also earned wages 
in the second quarter following the quarter in which earned wages were 
first recorded, divided by the number who earned wages in quarter 
Q1 or Q2.
    This performance measure recognizes as a positive employment 
outcome employment in any job two quarters following the employment 
that is recorded in an initial job during quarter Q1 or 
Q2 after registration. In recommending this performance 
measure, the workgroup acknowledged that while many job seekers 
register with the labor exchange to search for and find a job that 
results in lasting employment, others may use labor exchange services 
to assist them in acquiring temporary employment or a series of short-
term jobs. Services such as job search workshops, resume assistance, 
job finding clubs, job counseling, and even self-services are 
activities that have a lasting effect on job seekers and can contribute 
to a job seeker retaining employment in his or her current job, or 
entering and retaining employment in a subsequent job.
    The labor exchange employment retention rate measure provides a 
degree of consistency with the WIA performance measurement system, 
which also includes an employment retention measure. Both rely on the 
assumption that the vast majority of individuals seeking the services 
provided by the respective programs possess an inherent desire to 
maintain employment during the Short- and medium-term. However, a small 
number of seasonal workers, such as students and some farmworkers, may 
desire to work only sporadically throughout the year. Recognizing that 
such workers are only a small fraction of all job seekers and that it 
is important for the labor exchange to be able to monitor the 
employment outcomes of job seekers beyond their initial entry into 
employment, the employment retention rate at six months measure is 
recommended as one that provides valuable information on the medium-
term employment outcomes of the job seekers who register with the labor 
exchange.

B. Procedures for Establishing Expected Levels of Performance

    In accordance with the recommendation from the workgroup, the WIA 
Title I framework will be used for negotiating and setting expected 
performance levels for labor exchange services. This means that States 
will develop baseline data for the measures, analyze the baseline data, 
and propose performance levels for each measure based on that analysis. 
After providing the required information to the appropriate ETA 
Regional Office, States will negotiate with the region to obtain 
mutually agreed upon expected levels of performance. In developing 
baseline data, States should use two years of data if possible, but not 
less than one year in determining trends for performance and factors 
which may influence performance. For the customer satisfaction 
measures, States should look at experience thus far under WIA and any 
other survey instruments they have previously used. In establishing 
expected performance levels for each measure, factors beyond the 
control of the State are also to be considered. The specific steps for 
setting expected levels are as follows:

[[Page 49714]]

Baseline Performance
    Baselines for each of the measures will be developed by each State 
and will be a key factor used to determine the expected level of 
performance that is negotiated with ETA. Baselines are intended to give 
an indication of the past outcomes of a performance measure. For 
performance negotiations to be data-driven and reality based, the 
development of baselines is a critical aspect of the negotiation 
process.
State Expected Levels of Performance
    As part of the 5 year State Plan submitted to the Department of 
Labor, each State will propose expected levels of performance for the 
next three program years (PY 2001-2003). States should be prepared to 
provide support for their proposed levels by providing information on 
how baseline performance levels were developed and providing other 
information they believe may affect performance. States will include in 
their plan expected levels of performance levels for each measure. In 
addition, States will provide the baseline performance data and a 
description of any other factors, such as economic conditions, that 
contributed to the establishment of the expected performance levels. 
States also should include the methodology for developing the baseline 
data, a description of data sources and appropriate factors used to 
project expected levels of performance.
    In recommending factors to be considered, the workgroup explicitly 
excluded applicant characteristics and types of services provided. The 
labor exchange is viewed as an agency offering universal access to all 
job seekers, with basically the same set of services provided across 
all States. Allowing for differences in applicant characteristics might 
have the unintended consequence of favoring service provision to some 
applicant groups over others. With respect service mix, this is not an 
appropriate factor for adjusting expected levels of performance, 
because this is within the full control of the State agency.
    Examples of possible factors to consider in negotiating expected 
levels of performance are: economic conditions such as the unemployment 
rate, the rate of job creation/loss, new business start-ups; community 
factors such as availability of transportation and daycare; pursuit of 
new or enhanced employer partnerships; other factors such as State 
legislation or policies which might impact performance; and natural 
disasters. This list is not intended to be prescriptive or exhaustive, 
but to suggest the kinds of information that might be considered in the 
negotiation process.
Negotiation of Expected Levels of Performance
    The Regional Office will review the information contained in the 
State plan and will compare the expected performance levels with the 
national averages, baseline information from other States, and the 
negotiated levels of performance established for other States, taking 
into account factors including differences in economic conditions and 
other factors as discussed above. In addition, the Regional Office will 
analyze the quality of the data presented by States, including the 
relevance of the data, the source of the data, the time period from 
which the data were drawn, and if the data are part of a trend or 
anomalous. Established GPRA Annual Performance Plan goals for relevant 
measures will also be an important part of regional review and 
negotiation of performance levels. When the Regional Office finalizes 
its analysis, there will be negotiations with the State to obtain 
mutually agreed upon expected levels of performance.
    Similar to WIA, provision will be made for renegotiation of 
performance levels if circumstances arise that result in a significant 
change in the factors used to establish the original levels. It is 
understood that either a State or the regional office may elect to 
renegotiate performance as new information becomes available. Factors 
which will be considered for making changes include those discussed 
above.
    Expected performance levels may, depending on the factors to be 
considered, be renegotiated for any one or all three years of the 
performance period. States initiating the renegotiation will prepare a 
modification to the approved State plan and submit it to the regional 
office. The negotiation process described above will then be followed. 
In cases where the change is initiated by the region, States will be 
asked to prepare an amendment to the approved plan. Once the amendment 
is submitted to the regional office, the established negotiation 
process will then be followed.

C. Rules for Application

    Actual performance for each program year will be compared to 
negotiated performance levels. For a State to be designated as 
``exemplary,'' expected levels for all measures must be achieved or 
exceeded for all measures. Actions that may be taken in the case of 
``exemplary'' performance by States include: formal recognition by the 
Department of Labor through letters to Governors and publication of 
results in an annual report; recognizing and publicizing practices that 
foster good performance through publication on Internet websites; and 
the election by States to use their Wagner-Peyser Act 7(b) funds to 
provide performance incentives for public employment service offices 
and programs. States and regional offices should analyze performance 
information on an ongoing basis and, where performance is not achieving 
expected levels, work together to develop corrective action plans--
including the provision of any training or technical assistance that 
may be required.

    Signed at Washington, DC, this 24th day of July, 2000.
Ray Bramucci,
Assistant Secretary for Employment and Training.
[FR Doc. 00-20544 Filed 8-11-00; 8:45 am]
BILLING CODE 4510-30-P