[Federal Register Volume 63, Number 113 (Friday, June 12, 1998)]
[Notices]
[Pages 32564-32578]
From the Federal Register Online via the Government Publishing Office [www.gpo.gov]
[FR Doc No: 98-15754]



[[Page 32563]]

_______________________________________________________________________

Part V





Department of Labor





_______________________________________________________________________



Employment and Training Administration



_______________________________________________________________________



United States Employment Service; Labor Exchange Performance Measures; 
Notice

  Federal Register / Vol. 63, No. 113 / Friday, June 12, 1998 / 
Notices  

[[Page 32564]]


-----------------------------------------------------------------------

DEPARTMENT OF LABOR

Employment and Training Administration


United States Employment Service; Labor Exchange Performance 
Measures

AGENCY: Employment and Training Administration.

ACTION: Notice and request for comments.

-----------------------------------------------------------------------

SUMMARY: In response to the requirements of the Government Performance 
and Results Act (GPRA) of 1993, the national call for government 
programs to be more accountable and results-oriented, the Department of 
Labor (DOL) Employment and Training Administration (ETA) launched a 
project called the Workforce Development Performance Measures 
Initiative (WDPMI) to develop a menu of key performance measures for 
use in the workforce development system. Due to the absence of 
established key performance indicators for the public labor exchange 
program that measures both the self-service and staff-assisted service 
options now available, the United States Employment Service (USES) 
initiated a project at about the same time to work cooperatively with 
States in developing program-specific performance measures. This 
Federal-State workgroup also was charged with ensuring that the labor 
exchange performance measures it proposed would be compatible with and 
complementary to the overall WDPMI performance measures being 
developed.
    The Labor Exchange Performance Measures workgroup prepared a 
discussion draft issues paper entitled ``America's Labor Exchange 
Performance Measures'' that identified a list of potential measures. 
This paper was shared informally among the State Employment Security 
Agencies during a period beginning on September 10, 1997, until October 
31, 1997. More than 20 States provided reactions and feedback on the 
proposed measures. The Labor Exchange Performance Measures workgroup 
considered these comments and in subsequent discussions refined the 
potential program measures.
    This Federal Register Notice (FRN) provides a description of the 
conceptual framework within which public labor exchange services are 
delivered, and requests comment from interested parties and USES' 
stakeholders on proposed performance measures for labor exchange 
services. The Department of Labor also is interested in comments on the 
appropriate number of measures, and whether the proposed measures take 
into account the full range of services and service options in the 
modern labor exchange. In addition, the Department is interested in 
learning about other measures that State labor exchange agencies have 
found useful for management and continuous improvement purposes.
    There currently are two bills being considered by the Congress 
(H.R. 1385 and S. 1186) which will provide the framework for the 
Nation's workforce development system. Among the provisions of the 
bills is a requirement that the Wagner-Peyser funded labor exchange 
program functions are provided through the One-Stop system (called 
``full-service'' in H.R. 1385).
    These proposed measures are a starting point for development of 
comprehensive measures for the labor exchange function of the emerging 
workforce development system. This FRN does not address the data 
elements needed to produce the performance measures nor proposes 
specific changes to the ETA reporting requirements. That will be the 
subject of a subsequent notice.

DATES: Comments on the proposed Labor Exchange function performance 
measures must be received by the U.S. Department of Labor on or before 
July 27, 1998. Late-filed comments will be considered to the extent 
possible.

ADDRESSES: Comments must be filed in Room N4470, U.S. Department of 
Labor, Employment and Training Administration, United States Employment 
Service, 200 Constitution Avenue NW., Washington, DC 20210.

FOR FURTHER INFORMATION CONTACT: John R. Beverly, III, United States 
Employment Service, U.S. Department of Labor, 200 Constitution Avenue 
NW., Room N4470, Washington, DC 20210, Tel. 202-219-5257, Fax 202-219-
6643, E-mail [email protected]

SUPPLEMENTARY INFORMATION:

Contents

I. Authority

II. Introduction

A. Mission of the US Department of Labor
B. Role of the Public Labor Exchange

III. Performance Measures Development Process

A. Labor Exchange Performance Measures Workgroup
B. Models for Service in the Modern Labor Exchange
    1. Self-Service strategy
    2. Facilitated Self-Help strategy
    3. Staff-Assisted strategy
C. Labor Exchange Performance Measurement Strategy
D. Initial Comments on the Draft for Discussion Issues Paper

IV. Definitions

A. Job Search Assistance
B. Facilitated Self-help
C. Self-help Tools
D. Staff Assisted Labor Exchange Services
E. Job Listing Services
F. Referral Related Action
G. Business Assistance Services
H. Entered Employment
I. Entered Employment Rate
J. Cost per Entered Employment

V. Proposed Labor Exchange Performance Measures

A. For Self-Service Strategies
    1. Holdings
    2. Users
    3. Transactions
B. For Facilitated Self-help Strategies
    1. Number of Job Seekers Using Self-Help Service
    2. Customer Satisfaction
C. For Staff-assisted Service Strategies
    Measures of Job Seeker Customer Services
    1. Entered Employment Rate
    [Option] Job Development Entered Employment Rate
    Measures of Employer Customer Services
    2. Job Listing Return Business Rate
    3. Business Assistance Service Return Business
    4. Referral Response Time
    5. Average Time Lapse to Successful Referral
    6. Job Order Fill Rate
D. System Measures
    1. Cost Per Entered Employment
    2. Duration of Benefits Compensated

I. Authority

    Labor exchange performance measures will be implemented under the 
following authority:

A. Wagner-Peyser Act section 10 (c), 29 U.S.C. 49i(c)

(c) Reports
    Each State receiving funds under this Act shall (1) make such 
reports concerning its operations and expenditures in such form and 
containing such information as shall be prescribed by the Secretary, 
and (2) establish and maintain a management system in accordance with 
guidelines established by the Secretary designed to facilitate the 
compilation and analysis of programmatic and financial data necessary 
for reporting, monitoring and evaluating purposes.

B. Government Performance and Results Act of 1993

    Purposes--The purposes of this Act are to improve the confidence of 
the American people in the capability of the Federal Government, by 
systematically holding Federal agencies accountable for achieving 
program results; --improve Federal program effectiveness and public 
accountability by promoting a new focus on results,

[[Page 32565]]

service quality, and customer satisfaction.

II. Introduction

A. Mission of the US Department of Labor

    The 1998 USDOL Strategic Performance Plan states that it is the 
mission of the Department of Labor to foster and promote the welfare of 
the job seekers, wage earners and retirees of the United States by 
improving their working conditions, advancing their opportunities for 
profitable employment, and protecting their retirement, health, and 
other benefits.
    In carrying out a part of this mission, the Department, through the 
Employment and Training Administration, administers a variety of 
Federal programs to help employers find workers, to help job seekers 
find employment, to provide unemployment insurance benefits and 
reemployment services to jobless workers, to track changes in 
employment and to provide job training to meet the changing workplace 
skill needs.

B. Role of the Public Labor Exchange in the USDOL Mission

    In a May 1, 1933, report to the Senate from the Education and Labor 
Committee, and a May 22, 1933, report to the House of Representatives 
from the Labor Committee, the stated purpose of the public labor 
exchange is:
    (1) To foster, promote and develop the welfare of the wage earners 
of the United States including juniors (youth) regularly employed;
    (2) To improve their working conditions;
    (3) To advance their opportunities for profitable employment by 
regularly collecting, furnishing, and publishing employment information 
as to opportunities for employment;
    (4) For maintaining a system for clearing labor between the several 
States; and
    (5) For cooperating and coordinating the public employment offices 
throughout the country.
    On June 6, 1933, the Wagner-Peyser Act (29 U.S.C. 49 et seq.) 
became law and has since provided the basic funding support for the 
public labor exchange system.
    Over the 64 years of its history, the role of the public labor 
exchange has changed and evolved to meet the needs of the nation, 
employers and job seekers. Dedicated and hard working employees have 
labored in service to the United States from its earliest role in 
assisting the nation to recover from the depths of the Great Depression 
as part of the national recovery effort, through the efforts in support 
of the nation's economy by assisting private sector employers as their 
need for workers was revived. In the years that followed, the public 
labor exchange was an important part of the mobilization in support of 
war industries during World War II and the return to a post war 
economy. During that time the federal/State partnership was 
restructured with a redirection of effort to support private sector 
employers' workforce needs. Through the 1960's and 70's the national 
system of Wagner-Peyser funded labor exchange offices was instrumental 
in advancing a variety of social programs designed to expand 
participation and conditions of employment for several target groups 
including jobless workers, the economically disadvantaged and those who 
had particular difficulty in gaining access to employment 
opportunities.
    In 1986, the Secretary of Labor conducted a series of four public 
hearings at which 138 individuals testified and 562 submitted written 
testimony representing a cross section of the stakeholders of the 
nation's public labor exchange, including business and industry, the 
unemployed, target group advocates (e.g. veterans etc.), labor unions, 
State governments, employees and the training and education 
communities. The intent was to assess the current role of the labor 
exchange and to secure input on the appropriate role for the Wagner-
Peyser Act funded labor exchange in the 21st century.
    The vast majority of respondents (97.6%) indicated that the 
nationwide public labor exchange system was viable, and should be 
maintained and improved. Many of the respondents highlighted the need 
to streamline the system and to return to providing basic labor 
exchange activities. The list which follows represents the most 
frequently identified services in their order of priority:
     Intake, general assessment, and referral to jobs, training 
or re-training opportunities, and/ or support services;
     Labor market and occupational information;
     Preliminary screening of applicants for employers' job 
orders;
     Basic skill and/or aptitude testing;
     Direct placement (includes job referral);
     Job Search Assistance, including training and tips on 
interviewing, etc.; and
     Specialized recruitment for employers with large staffing 
requirements and workforce needs.
    The majority of commentators indicated that the public labor 
exchange services should be available free to all employers and job 
seekers, and that the role of the federal, State and local governments 
and the private sector should be maintained as they were at the time of 
the review and are today.
    In addition to responding to specific questions posed in the FRN 
that announced the hearings, the commentators also offered other 
recommendations including:
     The public image of the labor exchange should be improved 
to promote use of the available services,
     There be an increased use of automation and technology to 
provide linkage among states and localities and related programs to 
assist job matching, and to provide more timely and accurate labor 
force projections,
     DOL develop better standards to evaluate performance and 
incentive systems,
     Staffing levels be increased,
     Data collection and reporting systems be enhanced,
     Staff training be increased,
     Better job skills assessment tools be developed,
     Labor market information be improved,
     Federal support for State-level program integration 
efforts be increased, and
     Alternative funding sources to finance operations be 
found.
    In the 12 years since the re-examination, the public labor exchange 
system has been streamlined. Revitalized operations now provide service 
using flexible tiered service delivery strategies. The public labor 
exchange has significantly increased the use of automation and 
technology to provide service and information, and implemented many of 
the changes and improvements which were suggested. In cooperation with 
the Unemployment Insurance Service, the public labor exchange also 
developed a system of worker profiling and re-employment services for 
Unemployment Insurance (UI) claimants to speed their return to 
productive employment.
    The Wagner-Peyser funded public labor exchange has become a core 
component of the new Workforce Development System and is the universal 
access component of the nation's One-Stop Career Centers. For many 
business and job seeking customers, their only experience and contact 
with the Workforce Development System will be Wagner-Peyser funded 
services.
    The lack of additional financial investment in the public labor 
exchange may be traced, to some degree, to its failure to develop 
national labor

[[Page 32566]]

exchange performance measures. Performance data that have been 
collected and used provides an inaccurate picture of the overall 
success and value of the services provided. Moreover, these data fail 
to address the employer customer who is taxed to pay for the public 
labor exchange services.
    In this fiscal environment where labor exchange funding has either 
remained the same or declined, a good performance measurement system 
will help States and agencies manage programs and decide on the most 
appropriate mix of labor exchange services to offer.
    For the most part, previous measurement schemes were simple counts 
of service outputs, e.g., individuals referred to jobs, with the 
effectiveness of those outputs gauged through ratios made by comparing 
those counts to the total pool of persons registering with the labor 
exchange, e.g., 24% of all labor exchange applicants received a job 
referral. Moreover, measurement of the effectiveness of employer 
services were confined to a count of the number of job openings 
received by the public labor exchange as compared to the number of 
openings on which a placement action occurred, i.e., the job openings 
filled rate.
    The ETA 9002 reporting and data collection system has been the 
source of these program data. The authority to collect ETA 9002 data 
will expire on August 31, 1999. This sunset date offers an opportunity 
to re-examine how the labor exchange function is measured. It is DOL's 
hope to secure comment from all of the stakeholders including 
employers, wage earners, unions, and partner agencies in the emerging 
workforce development system, including the education, training, and 
public assistance communities on both the performance measures proposed 
herein and the conceptual framework within which those measures would 
operate. The comments received will be reviewed and adjustments will be 
made to the proposed measures as appropriate. Following this 
participatory process, another FRN will be issued to promulgate the key 
performance measures, to identify the data elements needed to produce 
those measures, and to establish State data reporting requirements.
    ETA recognizes its responsibility to continue to collect 
information on the race, age, sex, religion, ethnicity, disability 
status, and veteran status of the job seekers who register to use its 
services.

III. Performance Measurement Development Process

A. Labor Exchange Performance Measures Workgroup

    The USES Labor Exchange Performance Measures Initiative (LEPMI) 
Workgroup was composed of fifteen representatives drawn from State 
employment service agencies, USDOL regional and national offices, the 
Veterans' Employment and Training Service, and the Interstate 
Conference of Employment Security Agencies (ICESA). The members of the 
Labor Exchange Performance Measures Initiative (LEPMI) possessed on 
average over 20 years of experience in a variety of program areas and 
disciplines including tracking and monitoring program performance.
    The workgroup was convened in June 1997 to develop a menu of 
conceptual performance measures which could be adopted for the Labor 
Exchange function of the Workforce Development System. The Workgroup 
used guidelines similar to those used by the ETA Workforce Development 
System Performance Measures Initiative Policy Committee to assess 
alternative measures, and maintained a close coordination between that 
effort and its work. Several persons served on both workgroups. This 
close working relationship between the two complementary performance 
measures initiatives has continued.
    The Workgroup began by agreeing that the new performance 
measurement system should be directed toward measuring the outcomes of 
service delivery, whenever possible, and not the inputs or outputs of 
the processes used to provide service. The first task the LEPMI work 
group tackled was to identify the core services being provided in 
today's labor exchange. The workgroup identified the following as the 
core service categories of the public labor exchange: for Employers the 
core services identified were Job Listing, Business Assistance, and Job 
Matching and Initial Screening of Candidates for Employment; for Job 
Seekers the core services identified were Job Search Assistance, Job 
Development, and Facilitating Access to Self-Help Tools.
    There are a wide variety of activities which are included within 
the broad categories of service, examples of which can be seen in 
Figure #1 which follows this FRN.
    To develop performance measures, the workgroup benchmarked a 
variety of measurement strategies including the way in which the 
results of the public labor exchange has been measured in the past, 
workload-based measures similar to measures used in UI programs, 
measures of the effectiveness of training programs, approaches used to 
measure private sector business success and indicators of success in 
the public library system.
    The workgroup also agreed early in its deliberations to a set of 
four guiding principles:
    1. The Performance Measures proposed are intended to apply to the 
labor exchange function without regard to the administrative or program 
structure in which labor exchange services are delivered. 
Traditionally, the Labor Exchange function has been within the purview 
of State Employment Security Agencies (SESA's) nationwide. With the 
emergence of the Electronic Labor Exchange, worker profiling and 
intensive job search assistance activities under the Dislocated Worker 
programs, Welfare Reform, and One-Stop Career Center service delivery 
approaches, the Labor Exchange function is now being performed in a 
variety of agency and program models. These measures focus on the 
function not the program.
    2. The customer universe against which staff-assisted service 
outcomes are to be measured would be limited to those persons who have 
received staff-assisted services and not all persons registered with 
the public labor exchange, whether or not they are seeking work.
    There are several customer groups who use the public labor exchange 
for purposes other than immediate employment. These include but are not 
limited to applicants and recipients of Food Stamps, Home Relief or 
State General Assistance, Medicaid, temporary housing, and day care 
programs. Many of these programs require registration with the public 
employment service as part of the program application process.
    Other customers have an intermittent interest in pursuing 
employment. These can include applicants who are not in the labor 
market and those who are currently employed but interested in new job 
opportunities. Many of these individuals will periodically register for 
employment with the labor exchange. A segment of this group includes 
candidates for employment with prominent local firms that have 
developed exclusive recruitment agreements with the public labor 
exchange office. Some employers, such as auto industry manufacturing 
companies and major retail outlets, post signs outside of their plants 
and personnel offices which inform candidates that they hire candidates 
for employment who are referred by the

[[Page 32567]]

public labor exchange only, directing the job seekers to complete 
applications at a local labor exchange office.
    Mass recruitment control services are valued by many employers and 
result in registration of thousands of job seekers who are interested 
in working for only one of these major area employers. If any of the 
members of these customer groups fail to respond to other employer 
recruitments or job referral call-ins, and never return to the offices 
for job search assistance, they will ultimately be inactivated with no 
service. These customers are excluded from consideration when 
calculating the outcomes for labor exchange service because they did 
not receive and were not interested in staff-assisted service.
    3. Although the performance measures presented in this FRN are not 
segmented by target group, e.g., Welfare-to-Work, Migrant and Seasonal 
Farm Workers, etc., ETA encourages segmenting performance measures to 
allow a comparison among groups of customers. Measuring program 
performance based simply on overall entered employment outcomes, 
without taking into account the labor market characteristics of the 
various target groups of job seekers being serviced, may under-
represent the true performance of the public labor exchange system. 
Because public labor exchange services are available to all job seekers 
and all employers (universal access), its performance outcomes 
routinely include hard-to-place customer groups, e.g., potential UI 
Exhaustees, Public Assistance Recipients, Migrant/Seasonal Farm 
Workers, or Youth. However, current methodology does not account for 
the difficulty factors these groups may present. Segmentation of 
performance measures may offer insight into these overall outcomes. For 
example, an entered employment rate might be developed for Welfare-to-
Work job seekers which could be compared to the entered employment rate 
for all job seekers.
    ETA believes segmentation of outcomes for specific groups of 
customers can provide valuable insights into overall performance 
outcomes. This FRN does not address the data elements needed to produce 
the segmented outcome measures nor does it propose specific changes to 
ETA reporting. These will be the subject of a subsequent notice.
    4. Data gathering, to the extent possible, should be directly 
related to service delivery and the cost of data collection and 
reporting should be proportionate to the cost of the service provided. 
A performance measurement system developed in the current environment 
that has been shaped by the national effort to reduce the 
administrative paperwork burden and to focus on the results of 
government programs should not propose measures that cost substantially 
more to collect and report than the unit price for delivering the 
service.

B. Service Delivery Strategies in the Modern Labor Exchange

    To continue to meet customer needs for labor exchange services in a 
period of diminishing resources, States operating Employment Security, 
One-Stop, and Workforce Development programs have devised a variety of 
approaches and service delivery models which are significantly 
different from the traditional model of an ``Unemployment'' office that 
provided one-on-one staff-intensive interviews and individualized 
assistance to job seekers in finding jobs. This model, as the sole 
means delivery of labor exchange, is now largely extinct.
    Today, labor exchange services are typically provided using a 
tiered delivery system composed of three flexible and adaptive service 
strategies: ``Self-Service''; ``Facilitated Self-Help''; and ``Staff-
Assisted Service.'' Figure #2, which follows this FRN, shows this 
service model. Each strategy is designed to respond to differing 
service needs and differing service populations. It is the Department's 
expectation that State agencies providing labor exchange service will 
use each of the three service delivery strategies.
    This FRN represents the first effort to describe the tiered service 
delivery continuum, particularly the electronic Self-Service and 
Facilitated Self-Help strategies. Accordingly, more descriptive text 
has been devoted to the descriptions of the Self-Service and 
Facilitated Self-Help service strategies compared to the description of 
the traditional Staff-Assisted service strategy. This should not be 
interpreted as a preference for one strategy over another. Each has a 
necessary and appropriate role in labor exchange service delivery.
    1. In a Self-Service strategy, States make labor exchange resources 
available which customers can utilize independently, i.e., without 
staff intervention. Three current trends have helped drive the 
development of self-service strategies in today's public labor 
exchange.
    First, government is increasingly required to do more with less, 
and so must find ways to deliver its products and services more 
efficiently.
    Second, the labor market itself has changed, with unemployment 
being less associated with cyclical pressures and more with structural 
causes where increasing numbers of job seekers are considered 
dislocated workers. Studies have suggested that in the future job 
market most workers will change careers (not just jobs) several times 
during their work lives.
    Third, customer expectations have changed, and the public labor 
exchange system faces new demands from its customers regarding the 
quality of the services that are provided. Customers also expect to 
participate in the decisions involving what services are provided and 
how they are provided.
    This delivery mode places the customers in charge of the services 
they receive. The acceptance of self-service modes of service delivery 
is demonstrated by the increased number and availability of public 
access computers, automated teller machines, debit cards, and library 
swipe cards with bar-coded data, etc. There has also been a general 
movement within the United States toward placing more responsibility 
on, and expecting greater participation from, an individual who wishes 
to receive governmental services or benefits. In line with these 
trends, customers of the public labor exchange increasingly have made 
clear their preference for exercising informed choices in determining 
which products and services they receive and how they receive them.
    An additional and significant benefit of the self-service mode of 
service delivery is that self-service expands the capacity of the 
system beyond the limits of the available staff resources, thereby 
efficiently handling a wide variety of customer labor market and 
employment information needs.
    2. In the Facilitated Self-Help strategy, customers are provided 
access to self-help resources at the One-Stop Career Center or local 
labor exchange office, generally through a dedicated Resource Room. For 
many customers this is their first exposure to self-service tools and 
computer-based systems. In addition, one or more staff are assigned to 
assist customers who need help in using those resources. These staff 
interact with the customer, as needed, to facilitate the customer's job 
search using government-provided resources, e.g., personal computers, 
word processing and/or resume writing software, fax and copy machines, 
etc., and online access to the DOL-funded Internet-based tools, e.g., 
America's Job Bank (AJB), America's Talent Bank (ATB), America's Career 
InfoNet, etc. After being introduced to and assisted to use these 
Internet-based

[[Page 32568]]

self-service systems many customers will be able to use them in the 
future without assistance both in the One-Stop Center and in other 
locations with access to the Internet, e.g., schools, libraries, and at 
home. As funding for the traditional labor exchange or employment 
service has remained flat or declined, use of this mode of service 
delivery has increased. It continues to place the customers in charge 
of the services they wish to access, but provides the staff 
facilitation needed to ensure that the customers can obtain the service 
or information they require. Again, the cost for providing services 
using this strategy is less than under a Staff-Assisted service 
strategy, since the ratio of staff to customers is much higher.
    3. The Staff-Assisted strategy ranges from intensive one-on-one 
services where a staff person is assigned to a job seeker as a case 
manager, to those where a customer interacts with service staff in a 
small group setting. This is the more traditional mode of operation for 
delivery of public labor exchange services. The services most 
frequently delivered in this manner are assessment, intensive job 
search assistance, employment plan development, case management, 
counseling and vocational guidance, and job development. The expected 
output is a job referral and the expected outcome is entry to 
employment. Group services, such as job clubs and workshops on 
interviewing techniques, the world of work, and use of labor market 
information are also classified as staff-assisted services, although 
they are provided on a one-to-many service basis.

C. Discussion of Labor Exchange Performance Measurement Approaches

    DOL is proposing a different approach for measuring performance 
under each of these service strategies. The Department does this for 
two primary reasons. The first reason is that each strategy results in 
significantly different relative costs. Costs can be measured per unit 
of service provided or as an overall cost for providing access to 
resources. The costs to provide a service should be the primary 
determinant in deciding how much to invest to measure that service and 
how much customer information to collect. The more significant the 
service intervention, e.g., the higher the unit cost of that service, 
the greater the rationale for collecting data to justify the 
expenditure of public funds. For example it would not appear to be 
prudent to spend $10.00 pursuing a follow-up action to determine the 
outcome of a service that costs less than 25 cents to deliver. However, 
one should be willing to expend significant resources in determining 
whether a $5000 investment in training results in a long term payoff to 
the taxpayer.
    The following are examples of labor exchange services and resources 
that might routinely be provided in a self-service or facilitated self-
help mode and for which it might be difficult to document an employment 
outcome without a substantial data collection and follow up effort.
     Labor Market Information.
     Self-help pamphlets, magazines, newspapers, and reference 
books.
     Internet access for use of self-help job finding tools, 
e.g., AJB/ATB.
     Access to computer hardware and software, fax and copy 
machines, telephones, and other equipment in resource centers.
    Gaining Internet access to resources and tools such as these from a 
job seeker's home or from a community resource, such as a library or 
community center, may cost the public labor exchange system only a few 
pennies per transaction. First, because economies of scale are achieved 
in the development and maintenance of Internet-based systems and 
resources. Second, because the cost of the computer equipment, Internet 
service provider access, telecommunications software and hardware, 
supplies, electric power, and facilities are borne by the user or by 
another organization.
    This cost per transaction rises slightly when these latter costs 
are borne directly by the public employment service, such as when 
offered through a resource room. When these self-service resources are 
further augmented by making staff resources available to assist and/or 
instruct customers in using these tools and resources, the unit cost 
again rises.
    A similar case might be made for employer services. An employer may 
post an unsuppressed (broadcast) job opening directly to AJB over the 
Internet instructing interested parties to contact that employer 
directly. This is a very low cost self-service option.
    Employers may also request assistance from public labor exchange 
staff in placing their job openings on America's Job Bank. The cost of 
this service can increase further when the employer also asks the Labor 
Exchange to screen and refer applicants to that job order. The cost to 
provide this level of staff-assisted service may rise to the level 
where tracking outcome data is important.
    The second reason for pursuing different kinds of performance 
measures when using different service delivery strategies is that, for 
program outcome measures, one must be able to make a rational argument 
that there is a nexus between the service(s) provided and the outcome 
measured. For example, it might be difficult to argue that a single 
user session of browsing through labor market information in a self-
service mode over the Internet can be directly linked to a person 
obtaining a job 30 days later. However, should that same person come 
into the local public labor exchange office several times over the 
course of a few weeks and attend workshops, obtain help in his or her 
job search from the local office staff, and receive one or more job 
referrals, such a nexus can be reasonably concluded.
    Clearly, finding a way to gauge the results of self-service and 
facilitated self-help strategies presents a challenge to the system. 
This is particularly true since the Government Performance and Results 
Act (GPRA) became law. Under GPRA, government agencies must account to 
their investors (Congress and the American people) as to both how their 
programs perform and what results they achieved. The challenge is to 
find a means to satisfy this requirement where the service costs only a 
few pennies per transaction to deliver and the relationship to 
traditional program results is difficult to draw. As the level of 
service intervention increases, staff involvement grows, and dollar 
investment rises, the use of outcome and impact performance measures 
may become necessary to justify and explain the expenditure of public 
resources.
    DOL hopes that it has addressed these concerns in the performance 
measures which follow.
1. Measurement Approach for Self-Service Strategies
    A small ad-hoc task team was convened to come up with an approach 
to measuring the self-service components of the workforce development 
system. This team included several persons who were members of both the 
Workforce Development Performance Measure Initiative (WDPMI) and the 
Labor Exchange Performance Measures Initiative (LEPMI) workgroups. The 
team concluded that the following factors should be used to determine 
when outcomes measures would be appropriate for gauging the performance 
of a particular self-service or facilitated self-help service:
     There is a significant value-added provided by the 
service, i.e., the service is more than just an information exchange.

[[Page 32569]]

     The cost to collect and track outcomes is less than the 
cost per unit to provide the service.
     The service itself is intended to lead to an employment or 
other measurable outcome.
     The cost of providing the service rises to a point which 
indicates a need to justify the expenditure of public resources.
     The time lapse between receiving the service and the 
measurement of the outcome is short enough that the service can be 
considered to be causative of and/or contributory to that outcome.
    The Ad-hoc Task Team used these factors to help determine whether 
self-service and facilitated self-help strategies should be included in 
outcome measurement calculations. The first group of services 
considered were those which could be provided in a completely self-
service mode, requiring no staff assistance. These services included:
     accessing and using electronic labor exchange information 
services,
     completing online self-assessments,
     finding information, such as labor market information 
(LMI) or community services,
     online training, self improvement, and skill enhancement, 
and
     applying for services and benefits (e.g., using telephones 
for original and continued UI claims).
    The Ad-hoc Task Team agreed that customers receiving services in 
this fashion should not be included in outcome measurements, but rather 
services should be measured using output measures.
    In general, the Ad-hoc Task Team agreed with the performance 
measurement strategy which had been developed by the Labor Exchange 
Performance Measures workgroup. Under this strategy, measuring the 
value of self-service strategies, such as using DOL and State-developed 
Internet-based tools (America's Job Bank, America's Talent Bank, and 
America's Career InfoNet) should be analogous to measuring the value of 
services measured in public library systems. Using this concept, 
performance would be measured by counting the growth over time of:
     Holdings: the numbers of jobs and resumes that are 
maintained in its inventory;
     Usage: the number of times customers use its services; and
     Transactions: how many times specific kinds of services 
are used, e.g., referral information on job listings are accessed or 
downloads of resumes are requested.
    Figure #3, which follows this FRN, shows a model of this approach 
to measure program performance.
2. Measurement Approach for Facilitated Self-Help Strategies
    Under the facilitated self-help service strategy, staff assist 
customers to use self-help tools and resources. The services provided 
include:
     the self-service activities noted above;
     access to computer hardware, software, telephones, office 
equipment and other physical resources in a Labor Exchange office; and
     staff assistance in using any of the above services or 
resources.
    The approach to measurement of these services would be a 
combination of usage and customers' satisfaction. Usage would be 
measured as a simple count of the number of persons using resources 
rooms. This could be collected using swipe cards, tallies, sign-in 
sheets, and/or automated session counts. Customers using self-help 
services should not be included in outcome measurement until or unless 
they are provided more intensive staff-assisted service. Customer 
satisfaction would be used to look at the dimensions of service: 
accessibility, quality, timeliness, and security. This could be done by 
surveys that would not be reported nationally. These surveys would be 
locally developed, controlled, and used for continuous improvement 
projects at the local level.
3. Measurement Approach for Staff-Assisted Strategies
    Services delivered using a staff-assisted service strategy 
represent a value-added service. Measuring the performance of these 
value-added services requires addressing their employment-related 
outcomes for job seekers and employers. Additionally, measures should 
be adopted which can be used to assess the overall effectiveness and 
impact of the public labor exchange system.

D. Initial Comments on the Draft for Discussion Issues Paper

    The United States Employment Service in cooperation with the 
Interstate Conference of Employment Security Agencies (ICESA) released 
the issues paper, entitled, ``Draft for Discussion--America's Labor 
Exchange Performance Measures,'' in September, 1997. It provided a 
review of the rationale and a suggested conceptual framework for 
establishing a set of key performance measures of labor exchange 
services. Included was a discussion of a variety of alternative 
measures and the identification of 10 potential performance measures of 
staff-assisted job seeker and employer services. In addition, 
performance measures were suggested for the electronic labor exchange 
which counts holdings, usage, and transactions. Also included were two 
system measures. One was a cost-to-benefit measure that looks at unit 
cost per outcome, and the other was a measure of the impact of service 
on Unemployment Insurance benefits compensated.
    State Employment Security Agency Administrators were asked to 
provide written reactions to the suggested measures by October 31, 
1997. The ``Draft for Discussion'' issue paper was also made available 
on the Internet on the USDOL-supported ICESA Workforce ATM website 
(http://www.icesa.org/national/docs/LABOREX.HTML). This approach 
provided an opportunity for the public labor exchange system to have 
direct input to the initial development of the performance measures 
that are the subject of this FRN.
    Based on the comments received from States, and subsequent 
additional discussion and consideration by the workgroup, the number of 
job seeker measures has been reduced and several improvements have been 
made to other measures. In some cases, where different approaches were 
suggested and there was no clear-cut decision as to which approach was 
the better choice, more than one option has been included in this FRN 
for comment.
    Current system redesign efforts to enhance and integrate the 
features of America's Job Bank and America's Talent Bank systems will 
make it possible to collect and aggregate some data electronically 
without additional State data collection or transmission effort. It is 
the Department's intention to actively pursue electronic data 
collection approaches to the extent feasible.

IV. Definitions

    A. Job Search Assistance (JSA): Includes services currently defined 
in the ETA 9002 Data Preparation Handbook NO. 406 under the ``Received 
Some Reportable Service'' categories and in items 12 through 42. The 
service activities included are: job search workshops, job finding 
clubs, resume assistance, providing specific labor market information, 
job search plan development, job matching, job development, referral to 
jobs, vocational guidance, assessment interviews, testing, vocational 
counseling, federal bonding, and referral to other services

[[Page 32570]]

(including skills training, educational services and supportive 
services).
    B. Facilitated Self-help: Is a staff-assisted service in which 
staff facilitates customer access to a variety of in-office self-help 
job finding tools and resources. Support staff provides limited 
assistance on an as needed basis. Most job seekers require limited 
assistance. Normally, job-seeking customers are not required to 
register for service before they are allowed access to self-help tools 
or resource areas, rooms or centers.
    C. Self-help Tools: Include, but are not limited to, resource 
rooms, automated labor exchange system devices, Internet access, fax 
machines, telephones, photocopiers, personal computers, word 
processors, career and labor market information, and reference 
materials.
    D. Staff Assisted Labor Exchange Services: Include job search 
assistance, job development assistance and job referrals, and can be 
characterized as service which is conducted one-on-one or in small 
groups.
    E. Job Listing Services (JLS): Include activities performed on 
behalf of employers to assist them in filling their workforce needs. 
JLS includes but is not limited to services and activities such as job 
order taking, job order maintenance, referral follow-up and interview 
support, together with connecting activities including searching job 
seeker databases, transmitting resumes, and marketing job orders to the 
job seeker (applicant) pool, etc.
    F. Referral Related Action: Includes the following services and 
activities:
    (1) Referral of qualified candidates, or
    (2) Contact with the employer to review potential referrals, to 
develop a candidate recruitment strategy or adjust features of the job 
order which prevent identification of candidates for referral, i.e., 
salary, experience requirements, etc.
    G. Business Assistance Service (BAS): Includes, but is not limited 
to, providing employer education seminars, job and task analysis, 
providing local labor market information, and referral to other 
workforce development or economic development services or agencies.
    There are a number of employer service activities which are 
excluded from the definition of BAS. They are: periodic mass mailings 
and routine promotional mailings to increase employer job listings, 
providing labor market information via the Internet and employer 
contacts for the provision of other core services, e.g., job listing, 
job matching, initial screening or referral services.
    H. Entered Employment: The unduplicated count of job seekers 
(applicants) who enter employment by job placement or obtained 
employment. See page II-12 of ETA 9002 Data Preparation Handbook, ET 
Handbook NO. 406.
    (1) Job Placement: The hiring of a job seeker by a private or 
public employer after referral to a job by the Agency staff, or 
collocated or out-stationed staff in cooperation with the Agency, 
provided that the following conditions were fulfilled:
    (a) Prepared a job order prior to referral, except in the case of a 
job development contact on behalf of a specific job seeker,
    (b) Made prior referral arrangement with the employer,
    (c) Referred an individual who was not designated by the employer, 
and
    (d) Verified from a reliable source that the job seeker had entered 
work, and the placement was recorded in the agency data base.
    See page II-12 of ETA 9002 Data Preparation Handbook ET Handbook 
NO. 406.
    (2) Obtained Employment: Individuals who secure employment within 
the current quarter or the next completed quarter following the last 
staff-assisted job search assistance service that was partially funded 
by the Agency, such as:
    (a) Participating in job search activities,
    (b) Accepting a position resulting from the use of an agency-
sponsored automated labor exchange,
    (c) Receiving employment counseling or testing or development of an 
employability plan,
    (d) Receiving bonding assistance,
    (e) Terminating from a skill training program to which a job seeker 
was referred by the Agency, or
    (f) Receiving tax credit voucher, and receipt of verification from 
a reliable source, preferably the employer.
    See page II-13 of ETA 9002 Data Preparation Handbook ETA NO. 406. 
Entered Employment can be counted in the current quarter or the next 
completed quarter following the receipt of the last staff-assisted 
service (a maximum of 180 days).
    I. Entered Employment Rate (EER): Is the percentage of job-seekers 
securing employment after receiving Staff-Assisted Services.
    J. Cost per Entered Employment (CPEE): Is the cost of achieving the 
positive outcome of entry to employment following the provision of 
service. The CPEE is determined by dividing the total of those who 
entered employment by the total funding received for the federal 
Wagner-Peyser Act, and Veteran Services grants and State appropriations 
used for the Labor Exchange function during a program year (July 1-June 
30).

V. Proposed Labor Exchange Performance Measures

    Specific questions have been developed regarding these performance 
measures for which public comment is sought. Questions can be found at 
the end of sections A., C., and D. below, and they are numbered 
sequentially, 1 through 16. Commentators need not repeat questions as 
part of their response, but addressing the specific questions by number 
would be most helpful to this effort.

A. For Self-Service Strategies

    Performance measures will be output measures based on holdings, 
usage and transactions (much like a library) until customer-friendly, 
non-invasive sign-on and low cost follow-up approaches are developed to 
identify job-seekers in the electronic labor exchange.
    In developing these performance measures, the needs and interests 
of legislators, policy makers, program managers, budget planners, 
analysts, employers and other investors were considered. These output 
measures may be used for strategic planning, program management, 
continuous improvement and research. This information combined with 
customer satisfaction surveys and feedback could assist in the design 
of system improvements to meet customer needs.
    It is DOL's expectation that national data reporting will be 
produced by the AJB/ATB system, and will be aggregated and reported 
electronically. It is also DOL's expectation that these performance 
measures will not require additional state data collection.
    1. Holdings: This measure is a count of the number of employer job 
orders and the number of job seeker resumes in the AJB/ATB system. 
Continued growth in the number of job orders and resumes in the AJB/ATB 
system provides an indication of customer satisfaction and perceived 
value by the customers.
    2. Usage: This measure is a count of user sessions on the AJB/ATB 
system. A user is an individual who accesses the AJB/ATB system for any 
purpose. Some users of the system supply personal identification 
numbers and passwords; others are anonymous. A user session represents 
each single continuous access. This is different than a ``hit'', which 
measures the number of Internet server accesses (ie. computer to 
computer communications). Under this measure, we will count registered 
employers, registered job seekers and anonymous sessions. Enhancements 
to

[[Page 32571]]

the AJB/ATB system are underway to provide better information on 
anonymous sessions. This output measure will provide a gauge of how 
many customers are using these electronic labor exchange services over 
some defined period of time and whether the usage is growing, remaining 
static or declining.
    3. Transactions: This is an output measure of a higher level of 
interaction with the AJB/ATB system beyond a review of the information 
resources. For example, among the variety of transactions which could 
be collected electronically, two stand out. They are Job Seeker 
Referral Requests and Employer Resume Downloads.
    ``Job Seeker Referral Requests'' measures the number of job order 
referral instruction screens that are viewed by job seekers on AJB. 
This output measure is a count of the number of times job order 
referral instruction screens are viewed by job seekers on AJB after 
viewing a job order description. This measure identifies the number of 
users who have requested specific job referral information and are 
likely to respond to the employer's job opportunity. There are two ways 
in which this occurs: unsuppressed job orders and suppressed job 
orders. In unsuppressed job orders the employer contact information is 
broadcast and not hidden from the job seeker. In this case, the job 
seeker self refers. In suppressed job orders the employer contact 
information is not viewable or accessible through computers and the job 
seeker must be referred to the employer by the local labor exchange 
office holding the job order.
    ``Resumes Downloaded'' is a count of job seeker resumes that 
employers have selected from a list of job seekers, who met an employer 
defined search criteria, to obtain job seeker contact information.
Questions
    The Department specifically invites interested persons to provide 
comments, data, information and views concerning the following:
    Q1. Do these proposed measures adequately represent an employer's 
or job seeker's satisfaction with the Electronic Labor Exchange?
    Q2. How would you rate the importance of the three types of self-
service outputs (holdings, usage, and transactions)? Which would be 
most or least important for strategic planning, program management, 
etc.?
    Q3. What other ways would you suggest for measurement of self-help 
services?
    Q4. How do you currently or how would you suggest the national 
system collect customer satisfaction data and information?
    Q5. Will the proposed Self-Service Electronic Labor Exchange 
measures enhance State quality improvement initiatives?
    Q6. What specifically defined period of time would be most 
appropriate for self-service performance measurement: monthly, 
quarterly, or annually?

B. For Facilitated Self-Help Strategies

    Two Facilitated Self-help measures are being proposed: the number 
of users and customer satisfaction. These measures are intended to 
determine the extent to which customers value facilitated self-help 
services. The measures address this by answering two questions: do 
customers use the service and are customers satisfied with the 
services?
1. Number of Users of Self-Help Service
    The Number of Users of Self-Help Service is an output measure of 
the number of customers who access the self-help resources which 
includes the assistance of a knowledgeable staff member assigned to 
facilitate customer access to and use of the self-help tools. It is the 
intent that facilitated self-help provides service to a large number of 
job seekers with a minimum investment of staff time.
    The accumulation of a count of users can be as rudimentary as an 
office stroke tally or a sign-in log. It would not be necessary to 
track which self-help tools are used, although valuable information 
needed for the continuous improvement of facilitated self-help programs 
might be gained from a periodic assessment of which tools are most 
popular or useful to job seekers.
2. Customer Satisfaction
    Customer Satisfaction can be measured by conducting periodic formal 
telephone or mail surveys or by collecting customer in-person feedback 
using a structured approach at specified intervals. The measure could 
capture the frequency of use of self-help resources. Customers normally 
are not required to register to use the self-help tools in a labor 
exchange office or one-stop career center. These measures recognize the 
customer's use of and satisfaction with self-help resources.
    Facilitated Self-Help is a widely used and valid form of service. 
Self-Help service is consistent with the reality of shrinking 
resources, and relates directly to customers' demand. This measure can 
contribute to continuous improvement with input from the customer.

C. For Staff-Assisted Service Strategies

Measures of Job Seeker Customer Services
    Staff-Assisted Labor Exchange Services are the core activities of 
the modern Labor Exchange Office. The purpose of these performance 
measures is to determine the outcomes, effectiveness, and system impact 
of Staff-Assisted job seeker service. Performance will be evaluated 
through a combination of interrelated performance measures. These 
measures will be useful to legislators, employers, policy makers, 
agency administrators and program managers since they provide a tool 
for managing programs, baseline data for continuous quality 
improvement, and feedback that will allow States to respond to customer 
needs. These quantitative measures should be supplemented by 
qualitative measures of customer satisfaction and are not intended to 
be the sole measures of satisfaction.
    1. Entered Employment Rate (EER): The EER is the percentage of job 
seekers securing employment after receiving staff-assisted labor 
exchange services divided by the total number of job seekers who 
received staff-assisted labor exchange services. This measure uses as 
its denominator only those customers who receive staff-assisted 
service(s) and not the total number of applicants registered in the 
labor exchange office database. This performance measure looks at the 
effectiveness of the Job Search Assistance services that have 
employment as the expected outcome. This measure encourages an 
increased level of job order and job seeker follow-up, tracking and 
employer feedback. Service and entered employment data would be 
collected and reported by staff to the labor exchange reporting system. 
Unemployment Insurance wage records can be an additional data source 
used to conduct job seeker follow-up on entry to employment.
    [Optional Measure] Job Development Entered Employment Rate: Is an 
additional and optional measure of Job Search Assistance service. Job 
Development (JD) has been an effective tool to help individuals who 
have barriers to employment or difficulty in finding employment to find 
jobs. With the expected expansion in the role of the Labor Exchange in 
Welfare-to-Work programs, it has been suggested that a JD Entered 
Employment Rate would be valuable for program management and continuous 
improvement.
    Job Development Entered Employment Rate is the percentage of job 
seekers who entered employment after receiving Job Development referral

[[Page 32572]]

compared to the total number of job seekers who received JD referral.
    An outcome measurement of the pro-active solicitation of a job 
opportunity for an individual or group, this measure assesses the 
effectiveness of job development efforts in contributing to job seeker 
entry to employment. Job development is defined as development of a job 
opening for a job seeker or group of job seekers through direct contact 
with potential employers when no suitable job openings are currently 
listed. Job development contacts are currently reported within the 
category of ``Received Reportable Service'' in the ETA 9002 report.
    This performance measure is intended to create an incentive for 
staff to use their considerable knowledge of the labor market and 
employers' needs to increase the number and effectiveness of job 
development contacts and referrals on behalf of job seekers.
Measures of Employer Customer Services
    The variety of staff-assisted services that are made available to 
employers can be grouped into three categories: Job Listing Services, 
Job Matching, and Business Assistance service.
    The purpose of the proposed performance measures is to determine 
the extent to which employers' needs are being fulfilled in terms of 
the ``Timeliness,'' ``Quality,'' and ``Impact'' of our work. Staff-
assisted employer service performance should be evaluated by this 
combination of interrelated performance measures. The individual 
measures should not be viewed in isolation and these measures should be 
supplemented by qualitative measures of employer satisfaction.
    The proposed measures provide an indirect indication of employer 
customer satisfaction. They are not intended to be the sole measures of 
satisfaction. For example, the Return (Repeat) Business measure may be 
a good measure of employer satisfaction at the macro (state or 
national) level. However, federal contractors and others who are 
required to list their jobs with the Agency may or may not be satisfied 
employer customers.
    It is recognized that services provided to employers are not 
currently reported to the USDOL and may not now be collected by the 
labor exchange agency. These employer service measures may require 
States to develop new administrative reporting or program record 
keeping systems.
    2. Job Listing Return Business Rate: The number of employers who 
list more than one job order divided by the total number of employers 
who use the Staff-Assisted Job Listing Services during a reporting 
year. This outcome measure of repeat business serves as a means to 
determine employer satisfaction with the job listing service.
    The intent of this measure is to provide an incentive for service 
delivery staff to improve customer service and relationships including 
quick response time, quality referrals, and employer follow-up 
contacts. Most of the data needed for the Return Business measure is 
readily available with the possible exception that some States do not 
capture employer identification numbers on job orders. This measure 
would require that employer identification information be collected and 
reported for all job orders in the agency reporting and Job Bank 
system.
    Some States have expressed a preference for measuring job openings 
rather than job orders. The focus of this measure is employer 
satisfaction, not the volume of job openings or the number of job 
orders the labor exchange receives or an employer market penetration 
rate. Repeat business is a better indicator of employer satisfaction, 
and this measure is used in the private sector as an important 
indicator of success in service sector business enterprises.
    Some States and labor exchange staff were also concerned with the 
effect of employer direct job order entry into AJB and employer access 
to resumes on ATB and its impact on the number of job orders they would 
receive. In the past the number of job orders secured by a local office 
and listed in the State job bank has been a key local office 
performance measure in some States. Again, this measure represents the 
percent of Return Business which results from the employer's 
satisfaction with the Staff-Assisted service. Job orders entered 
directly by employers are captured in the electronic labor exchange 
measures of AJB and ATB holdings and are not lost to the state. An 
unintended consequence of this measure may be a counter-productive 
competition between a State's staff-assisted service component and the 
State's electronic labor exchange component. From a local office 
viewpoint, this measure may motivate staff members to discourage 
employers use of the job order self entry service options in AJB to 
assure that there is a high rate of repeat job listing business 
credited to the staff assisted service component. Enhancements to the 
AJB/ATB systems and report generators are currently underway as 
mentioned above. This labor exchange system performance issue will be 
addressed in future AJB/ATB versions. The Department would appreciate 
comments and suggestions for dealing with this potential consequence of 
measuring Repeat Employer Business in the staff-assisted service 
component of the labor exchange.
    Other States commented on the impact of having a large number of 
small employers or adverse economic conditions which would reduce the 
opportunity for return business during a reporting period as short as a 
year. DOL acknowledges the concern and would encourage comments which 
are based on a review of State Employment Service, Unemployment 
Insurance or Tax administrative data which demonstrate this phenomenon. 
DOL would also be interested to learn if this is a result of an agency 
policy to target small employers who have limited workforce replacement 
or expansion needs.
    Finally, even if the suggested defect in this performance measure 
is accurate, the Return Business measure would be useful for strategic 
planning, performance management and continuous improvement efforts. 
This measure is intended to be used by States for national performance 
reporting and state program management as opposed to being used for 
State-to-State comparisons.
    3. Business Assistance Service Return Business: In addition to Job 
Listing and Job Matching services, there are a number of other types of 
Staff-Assisted labor market information and services which employers 
require. These were defined in section IV. G. above. The provision of 
Business Assistance Services (BAS) is intended as a means of 
stimulating increased employer job listings and employer support for 
the Labor Exchange and Workforce Development System. The BAS Return 
Business Rate is the number of employers who utilize more than one BAS 
service divided by the total number of employers who use at least one 
Job Listing, Job Matching or Business Assistance staff-assisted 
service. The intent is to measure the level of employer customer 
satisfaction with BAS provided and to be an incentive to increase 
service to an important customer.
    4. Referral Response Time: Is an outcome measure of Job Matching 
Service (JMS) effectiveness, which combines the interests of job-
seekers and employers. The Job Order Response Time measure is the 
percentage of job orders for which referral-related action took place 
within three business days compared to the total number of job orders 
listed. This performance measure provides an incentive for quick 
response

[[Page 32573]]

to job orders by job-seeker referral or by direct referral-related 
contact with employers. Three business days have been identified as the 
generally accepted standard for referral and/or follow-up contact with 
employers who use the Job Listing Service (JLS).
    Some States indicated concerns regarding their ability to track 
employer follow-up contacts. It is recognized that current systems may 
have to be adjusted to capture this information. Although it is 
understood that not all job orders can have a quality referral made 
within three days, this measure encourages rapid response to the 
employer's need by the provision of other employer services such as, 
contact with the employer to review potential referrals, to develop a 
candidate recruitment strategy and/or to adjust features of the job 
order which prevent identification of candidates for referral, i.e., 
salary, experience requirements, etc.
    5. Average Time Lapse to Successful Referral: This outcome measure 
will provide a picture of how quickly labor exchange agencies respond 
to job orders with referral of a qualified candidate for employment. 
The performance measure is the average time lapse (in days) from the 
date an employer's job order is listed to the date of referral of the 
first agency-referred individual hired by that employer. The underlying 
presumption is that when the employer decides to hire one of the labor 
exchange referrals, the agency has met the employer's candidate 
qualification need.
    Not included in this calculation are job orders where the employer 
hires from another source. This removes from the equation situations 
where, although qualified candidates are referred, the employer decides 
to hire an equally qualified candidate from another source.
    This measure focuses on the time lapse to the date of job-seeker 
referral. An employer's hiring practices which may delay the offer of 
employment for many days or weeks does not affect the outcome of this 
measure. This measure provides a snapshot of the agency's referral 
practices in terms of responsiveness with a quality feature. It is 
intended to balance the rapid referral measure and mitigates the 
possibility that fast, but poor quality, job-seeker referrals will be 
the reaction to a rapid response performance measure.
    Job Order Fill Rate: Based on State comment, an additional measure 
of the Employer Satisfaction is proposed. The Job Order Fill Rate is 
the number of job orders for which a placement is made divided by the 
total number of suppressed job orders received within the reporting 
year. A job order for which the employer name, address, and other 
identification or contact information is hidden from the job seekers' 
view is considered to be a suppressed job order.
    Employers have the option of listing a job order in a suppressed or 
unsuppressed mode depending on the level of initial screening, and/or 
referral control service the employer wish the labor exchange to exert. 
Frequently employers are guided in the decision to suppress contact 
information based on the local availablity of candidates for employment 
who meet the skill, background and experience required to be successful 
candidates for employment by labor exchange staff.
    This outcome measure will provide a picture of how effectively 
agencies or States respond to these job listings with qualified 
candidates for employment. In this measure, the fact that the employer 
hires one of the job seekers referred by the agency is verification 
that the candidate was qualified for the job and that the employer was 
satisfied with the service provided.
    The intent of this measure is to provide an incentive to meet and/
or exceed employer customer expectations by referral of qualified 
candidates for employment. Not including in the denominator of this 
performance measure job orders which are listed in an unsuppressed or 
broadcast fashion avoids the possible unintended consequence that the 
labor exchange fill rate performance is measured against job orders 
where staff-assisted service is neither needed nor wanted by the 
businesses that want to broadcast available employment opportunities. 
The required data are currently being collected in many States and are 
available.
    In some States, a Job Opening Fill Rate performance measure has 
been used for many years as the approach to gauge the efficiency of 
local Labor Exchange offices and operations. In other States, the Job 
Opening Fill Rate has been discontinued because it has proven to be a 
disincentive to securing employer job orders in occupational areas 
where local management and staff would find it difficult to identify 
local and immediately available candidates. By including in the 
performance measure denominator only those job orders and openings 
which have the employer contact information suppressed, there will be 
less of an incentive to withhold job orders from the system or to 
artificially reduce the number of job openings. Based on labor market 
information, and in consultation with the employer, job orders that are 
not likely to be filled by a local candidate can be listed in an 
unsuppressed fashion and would therefore not count in the Fill Rate 
measure denominator.
Questions
    The Department specifically invites interested persons to provide 
comments, data, information and views concerning the following:
    Q7. Are there other services which your State or agency provides 
which have entry to employment as a goal which should be included in a 
list of Job Search Assistance services?
    Q8. Veteran service and other target group service programs 
continue to provide a higher level of one-on-one job-seeker service and 
job development assistance. Is the level of the job development 
assistance activity in your State or agency significant enough to 
require measurement of the Job Development Entered Employment Rate 
separately and apart from the Staff-Assisted Entered Employment Rate 
(EER)?
    Q9. What would be the impact of a requirement to collect or have 
the facilities necessary to associate an employer ID number with a job 
order?
    Q10. Is the proposed three business day standard for measuring 
referral-related follow-up contact with employers consistent with your 
agency or State approach?
    Q11. The Job Order Fill Rate measure is intended to provide an 
indication of employer satisfaction with staff-assisted labor exchange 
services. Are the unintended consequences of a fill rate performance 
measure serious enough to eliminate it from consideration as a program 
performance measure?
    Q12. Are there other services and/or service delivery approaches 
which should be included in the mix of services suggested under the 
Business Assistance Service definition?
    Q13. Will these proposed Staff-Assisted Labor Exchange service 
measures enhance State continuous quality improvement initiatives?

D. System Measures

1. Cost Per Entered Employment (CPEE)
    This is an outcome measure of the efficiency of the labor exchange 
function. The CPEE is calculated by dividing the total number of 
entered employment counts by the total funds allocated to the labor 
exchange function. Funding sources would include Wagner-Peyser Act, 
Veteran Employment and Training Service (VETS) allocations to States, 
and supplemental State funding provided for the labor exchange 
function. This measure will provide the context for the public labor 
exchange systems to

[[Page 32574]]

compare its key outcome--entered employment--against other workforce 
development components. For example, a 40% EER that costs only $300 per 
entered employment may seem more favorable when compared to an 80% EER 
when each EE costs $5000.
    Another intent of the measure is to promote efficiency by providing 
management with cost-of-operation information. This information could 
be used for strategic planning, allocation and distribution of scarce 
resources, as well as continuous quality improvement efforts.
    Administrative data sources are available to produce this system 
measure, and includes SESA administrative records and labor exchange 
job-seeker service records. In many States, Unemployment Insurance wage 
record data can be used to improve entered employment information.
    Funding data can be captured from the annual grant allocation 
documents for Wagner-Peyser and Veteran's Services. The information on 
State funding for the labor exchange function would have to be 
collected by canvassing the States to determine the level of additional 
State support. The cost should be calculated based on the Wagner-Peyser 
Program Year basis, which is consistent with other employment and 
training program funding cycles. The Veterans' program grants would 
have to be prorated since they are allocated based on a Federal Fiscal 
Year. A similar proration would be required for State funding which is 
not allocated on a program year basis.
    The CPEE measure is easy to understand and is generally accepted. 
These data are also useful for program management and the resource 
allocation process.
2. Duration of Benefits Compensated
    Employers are the primary customers of Workforce Development 
programs nationwide. To meet employers' workforce needs, public labor 
exchange agencies focus on job-seekers who possess a labor market 
attachment and marketable skills. In addition, Wagner-Peyser Act 
funding is provided to the public labor exchange, in part, to 
administer the work test for the State Unemployment Insurance 
compensation system and to provide job search and placement services 
for claimants. The effect and/or impact of services directed toward UI 
benefit customers has many dimensions which can be significantly 
affected by movement in the economy and local business conditions. 
However, there is a general belief that labor exchange service can have 
a direct and measurable impact on the duration of benefits.
    Two options are proposed to measure the impact of staff-assisted 
service on unemployment insurance benefit customers:
    The first option is a measure of duration of benefits compensated 
for claimants who receive services compared to prior year(s)'' 
duration. This measure can be adjusted based on economic conditions. 
This measure seeks to determine the impact of staff-assisted labor 
exchange services on the duration of benefits for claimants who 
received services. Data can be collected from Unemployment Insurance 
average duration data records (UI claims first payments/weeks 
compensated), labor exchange agency records, and local office data 
reporting. USDOL Unemployment Actuarial Unit data on business cycles 
and economic conditions could also be useful in explaining the impact 
of job-seeker services on claimants. Benchmark data would be available 
from previous State reporting.
    An easy-to-understand measure of the success of labor exchange 
services, this measure can be adjusted to consider State and local 
economic conditions. Most of the required data is easily accessible. 
This duration-of-benefits model could also be adapted to other groups 
such as Welfare-to-Work customers and Veterans, etc., and could be 
expressed as a reduction in the number of income transfer payments, as 
a trust fund savings, or as a part of a return on investment statement.
    The second option is a measure of the impact of Labor Exchange 
Services on the Duration of Benefits Compensated for UI Claimants 
Required to Search for Work (Work-Test Claimants) who receive Staff-
Assisted Labor Exchange services and enter employment compared to the 
average duration-of-benefits compensated to all claimants who are 
required to search for work.
    This outcome measure also seeks to determine the impact of staff-
assisted labor exchange service on the duration of benefits claimants. 
In this option, the group to be studied will consist of UI claimants 
who: (1) were not exempt from an active work search; (2) received 
staff-assisted labor exchange job-seeker services; and (3) entered 
employment.
    Staff-Assisted Labor Exchange services for claimant job-seekers 
includes job search assistance (JSA) and a variety of facilitated self-
help services enumerated in the definitions above in section IV. A, B, 
and C. Since the benefit customer job-seeker's identity is known, they 
are usually registered in the labor exchange system.
    A claimant who is required to search for work is a job-seeker who, 
as a condition of receiving Unemployment Insurance benefits, is 
required to perform an active work search. This criterion excludes UI 
benefit customers who are on a temporary layoff, required to secure 
employment through a union agent, partially unemployed and receive 
benefits under an approved partial benefit program, or exempt from work 
search requirements due to enrollment in an approved training program.
    This performance measure provides an incentive to deliver high 
quality JSA and other service to UI job seekers, to begin the provision 
of service early in the claim, to aggressively follow-up on job 
referrals, to track UI job-seekers' follow-through on work search 
plans, and to determine when the benefit claimant became re-employed.
    Administrative data are available for development of this measure 
of impact from Unemployment Insurance Wage Records (claims filed and 
average duration data), ES records (referral and service data elements 
in the current ETA 9002 compliant reporting systems and the Employment 
Security Systems Institute systems), and local office data reporting. 
In addition, federal and State benchmark databases which can be used to 
measure the State against its previous accomplishments are available.
    The measure can be used to show the effectiveness of service when a 
reduction in duration can be shown, and would be useful to legislators, 
employers, and policy makers. The impact or reduction in the duration 
of benefits performance measure could be:
    (a) translated into a reduction in the number of weeks compensated 
and a calculation of Unemployment Insurance Trust Fund savings using 
the State's average benefit rate, and
    (b) adapted for other target groups including Welfare-to-Work 
customers, Veterans, etc. In this case the result could be expressed as 
a reduction in the number of monthly income transfer payments, as a 
dollar savings in benefits using average benefit rate data, or as part 
of a return on investment statement.
Questions
    The Department specifically invites interested persons to provide 
comments, data, information and views concerning the following:
    Q14. Can a measure of UI Benefit duration provide a meaningful 
measure of the impact of labor exchange services on those claimants who 
receive staff-assisted services?
    Q15. Would the measurement, and comparison of the rate of UI claim

[[Page 32575]]

exhaustion be a better system measure of the labor exchange staff-
assisted service to UI benefit customers?
    Q16. Considering the measures proposed in this FRN, is there an 
optimal mix of performance measures which your State or agency would 
suggest?
    Finally, it should be noted that the Workforce Development 
legislation currently pending before the Congress (H.R. 1385 and S. 
1186) provides core performance measures that include, in addition to 
entered employment, retention in employment six months after entry and 
increases in earnings. Under the proposed legislation, these measures 
would apply to participants in the Workforce Development activities 
provided through the One-stop System, including labor exchange 
activities.
    We would appreciate receiving views on the appropriateness of these 
additional measures for measuring public labor exchange programs.

Paperwork Reduction Act

    The notice issued here is not subject to the requirements of the 
Paperwork Reduction Act of 1995 (44 U.S.C. 3501 et seq.) Because it 
contains no ``collection of information'' as defined in 44 U.S.C. 
3502(3).

    Signed at Washington, D.C., this 8th day of June 1998.
Raymond J. Uhalde,
Acting Assistant Secretary for Employment and Training.

BILLING CODE 4510-30-P

[[Page 32576]]

[GRAPHIC] [TIFF OMITTED] TN12JN98.016



[[Page 32577]]

[GRAPHIC] [TIFF OMITTED] TN12JN98.017



[[Page 32578]]

[GRAPHIC] [TIFF OMITTED] TN12JN98.018



[FR Doc. 98-15754 Filed 6-11-98; 8:45 am]
BILLING CODE 4510-30-C