[Federal Register Volume 63, Number 179 (Wednesday, September 16, 1998)]
[Notices]
[Pages 49557-49560]
From the Federal Register Online via the Government Publishing Office [www.gpo.gov]
[FR Doc No: 98-24849]


-----------------------------------------------------------------------

DEPARTMENT OF EDUCATION


National Assessment Governing Board

AGENCY: National Assessment Governing Board; Department of Education.

ACTION: Notice of information collection activity; request for comment.

-----------------------------------------------------------------------

SUMMARY: In compliance with the Paperwork Reduction Act (44 U.S.C. 3501 
et seq.), this notice announces a proposed information collection 
request (ICR) of the National Assessment Governing Board. The 
information collection is to conduct validity studies in conjunction 
with the pilot study of the proposed national tests in 4th grade

[[Page 49558]]

reading and 8th grade mathematics, in March 1999. Before submitting the 
ICR to the Office of Management and Budget (OMB), the Governing Board 
is soliciting comments on the information collection as described 
below.

DATES: Comments must be submitted on or before November 16, 1998.

ADDRESSES: Submit written comments identified by ``ICR: Voluntary 
National Test-Pilot Validity Studies'' by mail or in person addressed 
to Ray Fields, Assistant Director for Policy and Research, National 
Assessment Governing Board, Suite 825, 800 North Capitol Street, NW., 
Washington, DC 20002. Comments may be submitted electronically by 
sending electronic mail (e-mail) to [email protected]. Comments sent 
by e-mail must be submitted as an ASCII file avoiding the use of 
special characters and any form of encryption.
    All written comments will be available for public inspection at the 
address given above from 8 a.m. to 4:30 p.m., Monday through Friday, 
excluding legal holidays.

FOR FURTHER INFORMATION CONTACT:
Ray Fields, Assistant Director for Policy and Research, National 
Assessment Governing Board, Suite 825, 800 North Capitol Street, NW., 
Washington, DC 20002, Telephone: (202) 357-0395, e-mail: 
[email protected].

SUPPLEMENTARY INFORMATION: 

I. Information Collection Request

    The National Assessment Governing Board is seeking comments on the 
following Information Collection Request (ICR).
    Type of Review: New.
    Title: Validity Studies of the Voluntary National Tests in 4th 
Grade Reading and 8th Grade Mathematics.
    Affected Entities: Parties affected by this information collection 
are state, local, Tribal Government or non-public education agencies.
    Abstract: Pub. L. 105-78 vests exclusive authority to develop the 
voluntary national tests in the Governing Board and also prohibits the 
use of Fiscal Year 1998 funds for pilot testing, field testing, 
implementation, administration, or distribution of voluntary national 
tests. If Congress does not prohibit further development of the 
voluntary national tests after September 30, 1998, the Governing Board 
intends to begin pilot testing of items, (i.e. test questions) and 
conduct validity studies of test procedures in March 1999.
    Pub. L. 105-78 also requires the Governing Board to make four 
determinations about the voluntary national tests: (1) The extent to 
which test items selected for use on the tests are free from racial, 
cultural, or gender bias, (2) whether the test development process and 
test items adequately assess student reading and mathematics 
comprehension in the form most likely to yield accurate student 
achievement in reading and mathematics, (3) whether the test 
development process and test items take into account the needs of 
disadvantaged, limited English proficient, and disabled students, and 
(4) whether the test development process takes into account how 
parents, guardians, and students will appropriately be informed about 
testing content, purposes and uses.
    The purpose of the validity studies is to assess procedures for 
administering the proposed voluntary national tests in reading and 
mathematics. Since test administration can affect student performance, 
the validity studies determine if characteristics of test performance, 
such as non-standard conditions for students with special needs, affect 
student performance. Three studies are included in this information 
collection. Since the VNT is designed to be administered in two, 45-
minute sessions, the Effect of Break Length between Testing Sessions 
validity study will examine this impact on examinee test scores and 
make recommendations for future administrations of the test. The 
Effects of Calculator Type validity study will investigate how student 
familiarity with a particular calculator affects test performance. 
Design specifications for the 8th grade mathematics test call for the 
use of a calculator in one testing session. The National Assessment 
Governing Board undertakes this study to inform policy, for proposed 
future field and operational tests, about whether to issue standard 
calculators or permit students to take the test with their own 
calculator. The Effects of Extended Time and Small Group Administration 
Accommodations validity study investigates non-standard test 
administration procedures for the inclusion of students with 
disabilities and students with limited English proficiency in the 
voluntary national tests. This study considers the testing 
accommodations of extended time (up to two times the standard 
administration test length) and small group administration for students 
with disabilities, students with limited English proficiency (LEP) and 
comparison groups of non-disabled, non-LEP students. This study will 
measure the magnitude of the effect of test accommodations on student 
performance.
    Effects of Break Length between Testing Sessions. The 
specifications for the VNT call for the test to be administered in two-
45 minute sessions given on the same day but do not dictate the 
specific scheduling of these two sessions. As school personnel will 
administer the operational VNT, it is likely that there will be some 
variation in test administration procedures at participating schools. 
We anticipate that most schools will opt for two morning sessions with 
a short break between, but some schools may have logistical reasons for 
administering the test in one morning and one afternoon session. Data 
will be collected from test administrators on the test administration 
schedule and activities that occurred during testing sessions. This 
validity study would describe break lengths and activities, investigate 
their effects on test performance, and would allow AIR to advise NAGB 
on scheduling options for the VNT field-test and operational 
administration.
    The Effects of Calculator Type validity study will use released 
NAEP questions for which national statistics are available, rather than 
test questions for the VNT pilot test, Eighteen 8th grade students 
would be recruited to participate in a ``think-aloud'' procedure. 
Students would take one group of mathematics questions using either 
their own or a standard issue calculator, followed by another group of 
questions using the opposite calculator. While answering these 
questions, examinees would respond to interviewer prompts about their 
thinking processes. Students would also complete a brief questionnaire 
about their calculator use in schools. Data would be analyzed to 
determine the ways in which calculator use and familiarity with 
calculator features affects student performance.
    The Effects of Extended Time and Small Group Administration 
Accommodations validity study will investigate the impact of 
accommodations on the test performance of special populations; it will 
be conducted in two parts. In the first part, 900 students at each 
grade level will be added as an augmentation to the original pilot 
sample (300 students with disabilities who would take the VNT either 
unaccommodated or with accommodations that do not require altered test 
formats (e.g., large print, oral presentation), 300 students with 
limited English proficiency, and 300 non-disabled, non-LEP students) 
would either take the VNT under the standard time condition (two 45-
minute

[[Page 49559]]

sessions) or under an extended time accommodation (two sessions, each 
up to 90 minutes). Large schools and schools with large numbers of LEP 
students would be recruited to participate and would be assigned to one 
of the two conditions. Because of the small number of eligible students 
with disabilities in a given school, 110 schools (60 for 4th grade 
reading, 50 for 8th grade mathematics) will be recruited to participate 
in this study.
    For the second part, the accommodations of extended-time and small-
group administration would both be considered for a sample of 750 
students with disabilities eligible for small-group or extended time 
testing accommodations at each grade from schools participating in the 
pilot sample, both from students in classrooms that will be sampled 
from those schools for pilot study participation and from the students 
in the remaining classrooms in those schools. Thus, all students will 
be selected from schools selected to participate in the main pilot test 
and some of the students selected for participation in this study will 
also have been selected for inclusion in the main pilot test.
    Small group accommodations are often offered to students with 
disabilities but the manner of the accommodation varies. The small 
group accommodation could be a ``pull-out'' session in another 
classroom, or, due to space or staffing restrictions, the small group 
may be ``embedded'' in a larger setting (such as a library or 
cafeteria) where other activity is present. This study distinguishes 
between these two methods of providing small group accommodations for 
students with disabilities. A subsample of schools that are already 
participating in the main pilot VNT would be assigned to one of five 
conditions: (1) Standard time, small group ``pull-out'' administration, 
(2) extended time, small group ``pull-out'' administration (3) standard 
time, ``embedded'' small group administration, (4) extended time, 
``embedded'' small group administration, (5) standard time, standard 
(large) group administration. All eligible students with disabilities 
in the school would then take the VNT under the prescribed condition. A 
total of 750 students with disabilities would be recruited at each 
grade level, resulting in 150 students in each of the five conditions 
mentioned above. Due to the small number of eligible students with 
disabilities in a given school, it is estimated that 290 schools (4th 
grade) and 267 schools (8th grade) would be needed to complete the 
sample size. Parental consent will be sought for all students selected 
to participate under conditions different from those identified in 
their IEPs.
    All students in both parts of this study who take a test under the 
extended time would, at the end of the first 45 minutes of a testing 
session, be asked to switch to a different color pencil. Students in 
this condition could then answer remaining items or return to skipped 
items, using the different color pencil, until they have completed the 
test or until the end of the extended time period. All students (all 
conditions) would be asked to complete a short questionnaire about the 
length of the test.
    Data from this study would be analyzed to determine which groups 
benefit from the accommodation of extended time, which method of small 
group administration maximizes performance for students with 
disabilities, and how much accommodations affect test performance. 
Analyses would also be done to determine if the current test 
administration procedures provide sufficient time for students to take 
the VNT. Recommendations will be made for providing appropriate 
accommodations for students with disabilities and students with limited 
English proficiency on future field tests and operational tests of the 
VNT.
    In order to ensure adequate control and proper identification of 
the booklets of test items, and conduct necessary analyses of the data 
that results from the information collection, the following background 
information will be collected on the cover of the booklets of test 
questions: student name, date of birth, race/ethnicity, and sex (all to 
be supplied by the student), and special education status, limited 
English proficiency status, disadvantaged status, test administration 
accommodations, and primary language (collected by the test 
administrator under contract). Although students will write their name 
on each booklet for identification purposes during the administration 
of the pilot test, the students' names will be removed from the booklet 
shortly after the pilot test. Student names will not be included in the 
database for analysis and will not leave the school building where 
pilot testing is taking place. Instead, a unique numeric or 
alphanumeric identifier will be assigned to each booklet for tracking 
and analysis purposes. No third party notification or public disclosure 
burden is associated with this collection.

Burden Statement

    Effects of Break Length between Sessions: This study will not 
require any increase in burden for students above that required for the 
pilot study. School staff burden for this study is approximately 10 
minutes per school (672 for reading and 372 for math) or 174 hours 
total.
    Effects of calculator use study: The respondent burden for this 
study is 40.5 hours, or 2.25 hours for each of the 18 students 
participating in the Cognitive Laboratory study. This estimate is based 
on 90 minutes of cognitive think aloud and 45 minutes of test 
administration procedures by research staff.
    Testing accommodations validity study: The annual burden respondent 
estimate is based on 90 minutes of testing and 30 minutes of test 
administration activities (e.g., delivering instructions, handing out 
and collecting booklets, and providing background information as 
described above) per student, or two hours per student, in the standard 
time condition. Students who take the test in the extended time 
condition have up to three and a half hours each: 30 minutes of test 
administration activities, and up to 180 minutes of testing time, 
although this may be less if students finish early. Five hundred 
seventy-three of the students participating in this study will not 
require any increase in testing burden above that required for the 
pilot study, 382 will require an increase of 1.5 hours of testing 
burden, 327 will be new students with a total testing burden of 2 
hours, and 218 will be new students with a total testing burden of 3 
hours. In addition, there is an additional 6 minutes of burden per each 
of the 3,300 students to answer questions about the length of the test. 
Total student burden for the 3,300 students participating in the 4th 
grade reading test and 8th grade mathematics test is 7,270 hours. 
School staff burden for this study, for both reading and mathematics 
tests, is 225 hours. This includes questions about classroom practices 
to be asked of school staff by test administrators, 5 minutes each for 
the 2,700 students with disabilities and students with limited English 
proficiency included in the study. There is no school staff burden for 
the 600 students without disabilities or limited English proficiency.
    Participation in the pilot test and these validity studies is 
voluntary. State, local, and non-public education agencies are not 
mandated or required to participate.

[[Page 49560]]

Summary

    The total number of students involved in the validity studies, 
above that which has already been requested in the pilot VNT 
collection, is 3,318 with a total burden of 7310.5 hours. Total school 
staff burden for these validity studies is 399 hours.

II. Request for Comments

    The National Assessment Governing Board solicits comments to:
    (a) Evaluate whether the proposed collection of information is 
necessary for the proper performance of the functions of the Governing 
Board, including whether the information will have practical utility;
    (b) Evaluate the accuracy of the Governing Board's estimates of the 
burden of the proposed collection of information;
    (c) Enhance the quality, utility and clarity of the information to 
be collected;
    (d) Minimize the burden of the collection of the information on 
those who are to respond, including through the use of appropriate 
automated, mechanical or other technological collection techniques or 
other forms of information technology, e.g., permitting electronic 
submission of responses.

III. Public Record

    A record has been established for this action. A public version of 
this record, including printed, paper versions of electronic comments, 
is available for inspection from 8 a.m. to 4:30 p.m., Monday through 
Friday, excluding legal holidays. The public record is located in Suite 
825, 800 North Capitol Street, NW., Washington, DC 20002. Comments may 
be submitted electronically by sending electronic mail (e-mail) to 
[email protected]. Comments sent by email must be submitted as an 
ASCII file avoiding the use of special characters and any form of 
encryption.
    The official record for this action, as well as public version, as 
described above will be kept in paper form. Accordingly, the National 
Assessment Governing Board will transfer all comments received 
electronically into printer, paper form as they are received and will 
place the paper copies in the official record which will also include 
all comments submitted directly in writing. The official record is the 
paper record maintained at the National Assessment Governing Board, 
Suite 825, 800 North Capitol Street, NW, Washington DC 20002.

List of Subjects

    Pilot tests for the voluntary national tests in 4th grade reading 
and 8th grade mathematics, validity studies, and Information Collection 
Request.

    Dated: September 11, 1998.
Roy Truby,
Executive Director, National Assessment Governing Board.
[FR Doc. 98-24849 Filed 9-15-98; 8:45 am]
BILLING CODE 4000-01-M