[Federal Register Volume 69, Number 11 (Friday, January 16, 2004)]
[Notices]
[Pages 2630-2634]
From the Federal Register Online via the Government Publishing Office [www.gpo.gov]
[FR Doc No: 04-990]


-----------------------------------------------------------------------

DEPARTMENT OF LABOR

Employment and Training Administration


Proposed Information Collection Submitted for Public Comment and 
Recommendations: Data Validation Requirement for Employment and 
Training Programs

AGENCY: Employment and Training Administration (ETA), Labor.

ACTION: Notice.

-----------------------------------------------------------------------

SUMMARY: The Department of Labor, as part of its continuing effort to 
reduce paperwork and respondent burden,

[[Page 2631]]

conducts a pre-clearance consultation program to provide the general 
public and Federal agencies with an opportunity to comment on proposed 
and/or continuing collections of information in accordance with the 
Paperwork Reduction Act of 1995 (PRA95) (44 U.S.C. 3506(c)(2)(A)). This 
program helps to ensure that requested data can be provided in the 
desired format, the reporting burden (time and financial resources) is 
minimized, the collection instruments are clearly understood, and the 
impact of collection requirements on respondents can be properly 
assessed. The Employment and Training Administration (ETA) is 
soliciting comments on the establishment of a data validation 
requirement for the following employment and training programs: 
Workforce Investment Act (WIA) Title IB, Labor Exchange, Trade 
Adjustment Assistance (TAA), Migrant and Seasonal Farmworkers (MSFW), 
Native American Employment and Training, and Senior Community Service 
Employment Program (SCSEP).

DATES: Submit comments on or before March 16, 2004.

ADDRESSES: Send comments to: Gail Eulenstein, Performance and Results 
Office, Employment and Training Administration, U.S. Department of 
Labor, 200 Constitution Avenue, NW., Room N-5309, Washington, DC 20210; 
telephone: (202) 693-3013 (this is not a toll-free number); fax: (202) 
693-3991; e-mail: [email protected].

FOR FURTHER INFORMATION CONTACT: Gail Eulenstein, Performance and 
Results Office, Employment and Training Administration, U.S. Department 
of Labor, 200 Constitution Avenue, NW., Room N-5309, Washington, DC 
20210; telephone: (202) 693-3013 (this is not a toll-free number); fax: 
(202) 693-3991; e-mail: [email protected].

SUPPLEMENTARY INFORMATION:

I. Background

    The accuracy and reliability of program reports submitted by States 
and grantees using Federal funds are fundamental elements of good 
public administration, and are necessary tools for maintaining and 
demonstrating system integrity. The President's Management Agenda to 
improve the management and performance of the Federal government has 
emphasized the importance of complete information for program 
monitoring and improving program results.
    States and grantees receiving funding under WIA Title I, Wagner-
Peyser Act, TAA, and SCSEP are required to maintain and report accurate 
program and financial information (WIA section 185 (29 U.S.C. 2935) and 
WIA Regulations 20 CFR 667.300(e)(2); Wagner-Peyser Act section 10 (29 
U.S.C. 49i), Older Americans Act section 503(f)(3) and (4) (42 U.S.C. 
3056a(f)(3) and (4)), and TAA Regulations 20 CFR 617.57). Further, all 
States and grantees receiving funding from ETA and the Veterans' 
Employment and Training Service are required to submit reports or 
participant records and attest to the accuracy of these reports and 
records.
    Recent performance audits conducted by the Department of Labor's 
Office of the Inspector General, however, found that the accuracy of 
reported performance outcomes cannot be assured due to insufficient 
local, State, and Federal oversight. To address this concern and meet 
the Agency's goal for accurate and reliable data, ETA committed to the 
development and implementation of a data validation process in order to 
ensure the accuracy of data collected and reported on program 
activities and outcomes.
    Data Validation. The data validation requirement for employment and 
training programs will strengthen the workforce system by ensuring that 
accurate and reliable information on program activities and outcomes is 
available. Data validation is intended to accomplish the following 
goals:
    [sbull] Ensure that critical performance data are accurate.
    [sbull] Detect and identify specific problems with a State's or 
grantee's reporting process, including software and data issues, to 
enable the State or grantee to correct the problems.
    [sbull] Help States and grantees analyze the causes of performance 
successes and failures by displaying participant data organized by 
performance outcomes. In addition, the process will allow States and 
grantees to select appropriate validation samples necessary to compute 
statistically significant error rates.
    Data validation consists of two parts:
    (1) Report validation evaluates the validity of aggregate reports 
submitted to ETA by checking the accuracy of the reporting software 
used to calculate the reports. Report validation is accomplished by 
processing an entire file of participant records into validation counts 
and comparing the validation counts to those reported by the State or 
grantee.
    (2) Data element validation assesses the accuracy of participant 
data records. Data element validation is performed by reviewing samples 
of participant records against source documentation to ensure 
compliance with Federal definitions.
    Data Validation Pilot Test. Two States participated in a pilot test 
of the validation process in the summer and fall of 2002. Grantees in 
the MSFW program, Native American Employment and Training program, and 
SCSEP will begin pilot tests by the end of CY 2003. The pilot States 
conducted validation for the WIA Title IB, Labor Exchange, and TAA 
programs. The States received preparatory training prior to beginning 
validation and technical assistance throughout the pilot from ETA's 
validation contractor. The pilot test indicated the following:
    [sbull] States and grantees will generally be able to conduct data 
validation with a reasonable but sustained level of effort.
    [sbull] The validation process allows States and grantees to 
identify and address reporting errors.
    [sbull] States and grantees do make reporting errors which need 
detecting and fixing.
    [sbull] The average staff requirements for a State to complete 
validation for the WIA Title IB, Labor Exchange, and TAA programs will 
be about 882 hours per year. The average annual time required by 
grantees operating MSFW programs, Native American Employment and 
Training programs, and SCSEP to complete validation is approximately 
102 hours. Start-up activities in the initial year of validation will 
require an additional 264 hours on average per State and 74 hours on 
average per grantee.

II. Desired Focus of Comments

    Currently, ETA is soliciting comments about the proposed new 
collection of information on the validity of data that States and 
grantees report to the Agency. ETA is seeking Office of Management and 
Budget (OMB) approval under PRA95 to establish a data validation 
requirement for the following employment and training programs: WIA 
Title IB, Labor Exchange, TAA, MSFW, Native American Employment and 
Training, and SCSEP. Data validation will increase the reporting burden 
by requiring States and grantees to submit reports on data validity to 
ETA.
    A copy of the proposed information collection request can be 
obtained by contacting the office listed above in the ADDRESSES section 
of this notice. The Department of Labor is particularly interested in 
comments which:
    [sbull] Evaluate whether the proposed collection of information is 
necessary for the proper performance of the functions of the Agency, 
especially whether the information will have practical utility;

[[Page 2632]]

    [sbull] Evaluate the accuracy of the Agency's estimate of the 
burden of the proposed collection of information, including the 
validity of the methodology and assumptions used;
    [sbull] Discuss how to enhance the quality, utility, and clarity of 
the information to be collected; and
    [sbull] Suggest how to minimize the burden of the collection of 
information on those who are to respond, including through the use of 
appropriate automated, electronic, mechanical, or other technological 
collection techniques or other forms of information technology (e.g., 
permitting electronic submissions of responses).

III. Current Actions

    The Department proposes the following plan for implementing and 
operating data validation:
    [sbull] In order to ensure the accuracy of reported information 
throughout the workforce investment system, States and grantees will be 
required to conduct data validation and submit validation output 
reports to ETA. States will initiate data validation for WIA Title IB, 
Labor Exchange, and TAA by the end of CY 2003, and grantees operating 
MSFW programs, Native American Employment and Training programs, and 
SCSEP will initiate validation during CY 2004.
    [sbull] Data validation will be required annually. States and 
grantees will be required to send validation output reports to ETA 
within 120 days after the submission of required annual reports or 
participant records to ETA. Report validation will be performed prior 
to the submission of reports. Data element validation will be completed 
within 120 days after required annual reports or participant records 
are due to ETA.
    [sbull] ETA has developed a set of validation tools discussed 
below--instructional handbooks, software, and user guides--that States 
and grantees can use to validate data. States and grantees may use an 
alternative methodology and tools as long as the methodology meets 
standards for sampling methods and confidence intervals. States or 
grantees that do not use the validation tools provided by ETA will be 
required to document that the alternative methodology is statistically 
valid.
    [sbull] In addition to performing validation, the ETA software can 
be used to generate the aggregate information required in reports 
submitted to ETA. States or grantees that use the software provided by 
ETA to generate this aggregate information will not be required to 
perform report validation.
    [sbull] ETA will establish acceptable levels for the accuracy of 
reports and data elements. These accuracy standards will be established 
in phases. The initial validation year will focus on detecting and 
resolving any issues with State and grantee data and reporting systems. 
Error rates collected in the second year will be analyzed, and, based 
on this information, standards for accuracy will be established prior 
to the third year of validation.
    [sbull] Once accuracy standards are established, States and 
grantees will be held accountable for meeting those standards and will 
be required to address any issues concerning data accuracy. States and 
grantees that fail to meet accuracy standards will receive technical 
assistance from ETA and will develop and implement a corrective action 
plan. Data that does not meet accuracy standards will not be acceptable 
for measuring performance, and may keep the State or grantee from being 
eligible for incentives that are awarded based on performance data. 
Significant or unresolved deviation from accuracy standards may be 
deemed a failure to report.
    Resources. The requirement to perform validation derives from 
States' and grantees' responsibility to provide accurate information on 
program activities and outcomes to ETA. States and grantees are 
expected to provide resources for conducting validation from their 
administrative funds. Validation of program performance is a basic 
responsibility of grantees, who are required to report on program 
performance, under Department of Labor regulations (29 CFR 95.51 and 
97.40). ETA has taken a number of steps to minimize the resources 
needed for data validation, including developing tools that States and 
grantees can use to perform validation. The estimates provided below, 
which are based on state pilot experiences, indicate that annual staff 
requirements for continuing operations of data validation will be on 
average 882 hours (or less than \1/2\ of a staff year) for a State and 
102 hours (or about \1/20\ of a staff year) for a grantee.
    Data Validation Tools. To reduce startup costs related to 
implementing data validation, ETA has developed standardized software, 
instructional handbooks, and user guides that States and grantees can 
use to perform data validation:
    [sbull] Software developed by ETA generates samples, worksheets, 
and reports on data accuracy. For report validation, the software will 
validate the accuracy of aggregate reports that are generated by the 
State's or grantee's reporting software and will produce an error rate 
for each reported count. For data element validation, the software 
generates a sample of the participant records and data elements for the 
state or grantee to validate. The software produces worksheets on which 
the validator records information after checking the source 
documentation in the sampled case files. The software calculates error 
rates for each data element, with confidence intervals of 3.5 percent 
for large States/grantees and 4 percent for small States/grantees.
    [sbull] Handbooks provide detailed information on the validation 
methodology, including sampling specifications and data element 
validation instructions for each data element to be validated.
    [sbull] User guides developed for each ETA validation software 
application guide States and grantees through the process of installing 
the application, building and loading a validation file, and completing 
report and data element validation.
    Data Recording and Reports. States and grantees will record the 
results of their validation on spreadsheet software prepared as an 
accompaniment to their handbooks. Initially, the spreadsheets can be 
transmitted by e-mail to ETA. Eventually, the results will be submitted 
in the same manner as other reports. The results will be stored in a 
dataset in the National Office in Washington, DC, and compiled in an 
annual validation accuracy report.
    Training and Technical Assistance. ETA provided validation training 
to States in regional sessions during the summer of 2003. Training for 
grantees of the MSFW and Native American Employment and Training 
programs will be held during winter 2003/04, and training will be 
provided for SCSEP grantees during spring 2004. States and grantees may 
obtain technical assistance on validation procedures and the use of the 
validation tools by contacting ETA's data validation contractor.
    Type of Review: New.
    Agency: Employment and Training Administration.
    Title: Data Validation Requirement for Employment and Training 
Programs.
    OMB Number: 1205-0NEW.
    Recordkeeping: States and grantees must maintain complete records 
of all validation activities for three years. The retention requirement 
will apply to records of all validation activities, including files, 
worksheets, reports, and source documentation.
    Affected Public: State, local, and tribal government entities and 
private non-profit organizations.
    Total Respondents: 317 (53 states will perform validation for the 
WIA Title IB,

[[Page 2633]]

Labor Exchange, and TAA programs annually. 264 grantees operating MSFW 
programs, Native American Employment and Training programs, and SCSEP 
will perform validation annually).
    Frequency: Complete data validation annually.
    Total Responses: 317 (53 responses from states annually and 264 
responses from grantees annually).
    Estimated Time per Response: 882 hours per year on average for a 
state to complete validation of the WIA Title IB, Labor Exchange, and 
TAA programs. 102 hours per year on average for a grantee operating a 
MSFW program, Native American Employment and Training program, or SCSEP 
to perform validation.
    Total Burden Hours: An estimated 46,732 hours per year will be 
required for all states to complete validation for the WIA Title IB, 
Labor Exchange, and TAA programs. An estimated 13,992 hours will be 
necessary by all states for startup activities in the initial year of 
validation. An estimated 26,830 hours per year will be required for all 
grantees operating MSFW programs, Native American Employment and 
Training programs, and SCSEP to perform validation. An estimated 19,552 
hours will be necessary by all grantees for startup activities in the 
initial year of validation.
    Total Burden Cost (startup): The start-up cost is estimated to be 
$454,740 for all states in the initial year of validation for the WIA 
Title IB, Labor Exchange, and TAA programs ($8,580 on average per 
state). The start-up cost is estimated to be $312,322 for all grantees 
in the initial year of validation for MSFW, Native American Employment 
and Training, and SCSEP ($1,183 on average per grantee).
    Total Burden Cost (operating): The cost is estimated to be 
$1,518,791 per year for all states to complete validation for the WIA 
Title IB, Labor Exchange, and TAA programs ($28,656 on average per 
state). The cost is estimated to be $495,767 per year for all grantees 
operating MSFW programs, Native American Employment and Training 
programs, and SCSEP to perform validation ($1,878 on average per 
grantee).

Summary of Burden

                 Calculation of Combined Annual Burden for WIA Title IB, Labor Exchange, and TAA
----------------------------------------------------------------------------------------------------------------
                                        No. of states     Annual hours     Rate in $/hr\1\        Cost        Cost
------------------------------------------------------------------------------------------------------------ ------
Large State.........................                18             1,332            $32.50           $43,297
Medium State........................                18               836             32.50            27,180
Small State.........................                17               453             32.50            14,718
All States..........................                53            46,732             32.50         1,518,791
Average per State...................  ................               882             32.50           28,656
----------------------------------------------------------------------------------------------------------------
\1\ Hourly rate is the estimated average hourly earnings for employees in State Unemployment Insurance (UI)
  agencies in FY 2003 (as used for FY 2003 UI budget formulation purposes).


                Calculation of Combined Startup Burden for WIA Title IB, Labor Exchange, and TAA
----------------------------------------------------------------------------------------------------------------
                                            No. of states         Hours       Rate in $/hr \1\        Cost
----------------------------------------------------------------------------------------------------------------
State...................................                53               264            $32.50            $8,580
All States..............................                53            13,992             32.50          454,740
----------------------------------------------------------------------------------------------------------------
\1\ Hourly rate is the estimated average hourly earnings for employees in State Unemployment Insurance (UI)
  agencies in FY 2003 (as used for FY 2003 UI budget formulation purposes).


              Calculation of Annual Burden for MSFW, Native American Employment and Training, SCSEP
----------------------------------------------------------------------------------------------------------------
                                           No. of grantees    Annual hours     Rate in $/hr\1\        Cost
----------------------------------------------------------------------------------------------------------------
MSFW Grantee............................                52               158      $10.75/32.50            $1,896
Native American Employment & Training                  144                53             10.75               569
 Grantee................................
SCSEP Grantee...........................                68               162       10.75/32.50             4,637
All Grantees............................               264            26,830       10.75/32.50           495,767
Average per Grantee.....................  ................               102       10.75/32.50            1,878
----------------------------------------------------------------------------------------------------------------
\1\ Hourly rates used to calcuate cost depends upon the type of organization receiving the grant. For State
  government grantees, the hourly rate is the estimated average hourly earnings for employees in State
  Unemployment Insurance (UI) agencies in FY 2003 (as used for FY 2003 UI budget formulation purposes). For
  private non-private grantees, the hourly rate is the average hourly earnings in the social assistance industry
  (May 2003, Current Employment Statistics Survey, U.S. Census Bureau).


             Calculation of Startup Burden for MSFW, Native American Employment and Training, SCSEP
----------------------------------------------------------------------------------------------------------------
                                           No. of grantees        Hours        Rate in $/hr\1\        Cost
----------------------------------------------------------------------------------------------------------------
MSFW Grantee............................                52                72      $10.75/32.50              $864
Native American & Training Grantee                     144                72             10.75               774
 Employment.............................
SCSEP Grantee...........................                68                80       10.75/32.50             2,293
All Grantees............................               264            19,552       10.75/32.50           312,322
Average per Grantee.....................  ................                74       10.75/32.50            1,183
----------------------------------------------------------------------------------------------------------------
\1\ Hourly rates used to calculate cost depends upon the type of organization receiving the grant. For State
  government grantees, the hourly rate is the estimated average hourly earnings for employees in State
  Unemployment Insurance (UI) agencies in FY 2003 (as used for FY 2003 UI budget formulation purposes). For
  private non-profit grantees, the hourly rate is the average hourly earnings in the social assistance industry
  (May 2003, Current Employment Statistics Survey, U.S. Census Bureau).


[[Page 2634]]

    Data validation is estimated to require an annual burden of 73,562 
hours and $2,015,000 for all six programs subject to the validation 
requirement. An additional 33,544 hours and $767,000 will be required 
for startup activities for all six programs in the initial year of 
validation.
    Comments submitted in response to this comment request will be 
summarized and/or included in the request for OMB approval of the 
information collection request; they will also become a matter of 
public record.

    Signed in Washington, DC, on January 9, 2004.
Emily Stover DeRocco,
Assistant Secretary for Employment and Training.
[FR Doc. 04-990 Filed 1-15-04; 8:45 am]
BILLING CODE 4510-30-P