[Federal Register Volume 65, Number 34 (Friday, February 18, 2000)]
[Notices]
[Pages 8348-8350]
From the Federal Register Online via the Government Publishing Office [www.gpo.gov]
[FR Doc No: 00-4016]


=======================================================================
-----------------------------------------------------------------------

DEPARTMENT OF EDUCATION


National Assessment Governing Board; Information Collection 
Request

AGENCY: National Assessment Governing Board; Department of Education.

ACTION: Notice of revised information collection request; Request for 
comment.

-----------------------------------------------------------------------

SUMMARY: The National Assessment Governing Board (NAGB) is revising the 
Notice of Proposed Information Collection Request (ICR) published on 
January 17, 2000 in two ways. First, this notice extends the time for 
public comment to the National Assessment Governing Board to March 17, 
2000. Per instructions in the January 17, 2000 notice, submit written 
comments identified by ``ICR: VNT Research and Validation Support 
Studies (Option Year 2)'' by mail or in person addressed to: Ray 
Fields, Assistant Director, National Assessment Governing Board, Suite 
825, 800 North Capitol Street, NW, Washington, DC 20002. Comments may 
also be submitted electronically by sending electronic mail (e-mail) to 
[email protected]. Electronic comments must be identified by the title 
of the ICR.
    Second, this notice is to inform the public that an emergency 
review of this Information Collection Request has been requested in 
accordance with the Paperwork Reduction Act of 1995 (the Act) (44 
U.S.C. Chapter 3507(j)). Approval by the Office of Management and 
Budget (OMB) has been requested by March 17, 2000. Emergency review is 
requested because of an unanticipated event outside the control of the 
National Assessment Governing Board. Interested persons are invited to 
submit comments to the address below on or before March 17, 2000 to 
inform the emergency review by OMB.

ADDRESSES: Written comments regarding the emergency review should be 
addressed to the Office of Information and Regulatory Affairs; 
Attention: Danny Werfel, Desk Officer: Department of Education; Office 
of Management and Budget; 725 17th Street, NW, Room 10235; New 
Executive Office Building; Washington, DC 20503 or should be 
electronically mailed to the internet address [email protected]. 
Submit written comments identified by ``ICR: VNT Research and 
Validation Support Studies (Option Year 2).'' The National Assessment 
Governing Board will forward to OMB any comments received from the 
public in response to the January 17, 2000 notice inviting requests for 
public comment on this ICR and in response to this notice, extending 
the public comment period to March 17, 2000.

SUPPLEMENTARY INFORMATION: Section 3506 of the Act (44 U.S.C. Chapter 
35) requires that the Director of OMB provide interested federal 
agencies and the public an early opportunity to comment on information 
collection requests. The Office of Management and Budget (OMB) may 
amend or waive the requirement for public consultation to the extent 
that public participation in the approval process would defeat the 
purpose of the information collection, violate State or federal law, or 
substantially interfere with any agency's ability to perform its 
statutory obligations. In compliance with the Paperwork Reduction Act 
(44 U.S.C. 3501 et seq.), this notice revises a proposed information 
collection request (ICR) of the National Assessment Governing Board 
(the Governing Board, or NAGB) published on January 17, 2000. The 
information collection is to conduct two research and validation 
support studies related to test development for the proposed Voluntary 
National Test (VNT) during Spring 2000.

[[Page 8349]]

    Copies of this ICR may be obtained from Ray Fields, Assistant 
Director, National Assessment Governing Board, Suite 825, 800 North 
Capitol Street, NW, Washington, DC 20002. Telephone: (202) 357-0395; e-
mail: Ray__Fields @ED.Gov.

I. Information Collection Request

    The National Assessment Governing Board is seeking comments on the 
following Information Collection Request (ICR).
    Title: Voluntary National Tests (VNT): Research and Validation 
Support Studies (Option Year 2).
    Affected Entities: Parties affected by this information collection 
are individuals and State, local, or Tribal SEAs or LEAs.
    Abstract: In order to comply with the mandates of Public Law 105-
78, the National Assessment Governing Board (NAGB) proposes to conduct 
two research and validation support studies. Congress vested exclusive 
authority in the Governing Board for test development for the proposed 
VNT. At the same time, Congress prohibited pilot testing and field 
testing of questions developed for the proposed VNT. No test question 
developed for the proposed VNT will be used in these research studies. 
Instead, test questions used for the National Assessment of Educational 
Progress (NAEP) will be employed. This is to ensure that the 
prohibition on pilot and field testing is not violated, while still 
providing for research needed to answer questions related to test 
development.
    The data collected will serve two purposes: (a) Provide information 
on the feasibility of a calibration linkage between the proposed 
Voluntary National Tests (VNT) and the National Assessment of 
Educational Progress (NAEP) (more specifically--between a test designed 
to give individual results and a survey designed to report group 
results); and (b) provide information needed to inform policy and 
practice related to test accommodations for students with limited 
English proficiency, specifically, to help guide the development of an 
8th grade mathematics test booklet in two languages (i.e., a ``dual 
language'' booklet in this case in English and Spanish).
    The two research studies will also assist NAGB in making three of 
the four determinations required by Congress: (1) The extent to which 
test items selected for use on the tests are free from racial, cultural 
or gender bias; (2) whether the test development process and test items 
adequately assess student reading and mathematics comprehension in the 
form most likely to yield accurate information regarding student 
achievement in reading and mathematics; and (3) whether the test 
development process and test items take into account the needs of 
disadvantaged, limited English proficient and disabled students.
    The first study is directed toward establishing the feasibility of 
a calibration linkage between a test form resembling an individual test 
and a survey of group results--the National Assessment. Research 
questions to be answered include the following: What are the effects on 
the measurement of student performance of an individually administered 
test that shares a framework with NAEP but which differs somewhat from 
NAEP in content coverage, administration, and unit of analysis? Is it 
possible to establish a strong link between the group-focused results 
of NAEP and such an individually administered test? What inferences can 
be supported by such a link?
    4800 students from Grade 4 and 4800 students from Grade 8 are 
expected to participate in this study. The 9600 students will be 
divided equally across three conditions.
    Students in the first condition will take an ``NAEP Special Form'' 
booklet, consisting of NAEP items constructed to be as parallel as 
possible to the proposed VNT forms. This parallelism would include 
content coverage, timing, and shape of the test information function 
(TIF), which has been proposed to be flatter than the TIF for NAEP. 
Because empirical information on each item is needed to construct a 
form with a specified TIF, the items would come from the previous NAEP 
administration in the respective subjects.
    Students in the second condition would take ``Extended NAEP'' 
booklets, which are based on blocks of items from the 2000 NAEP 
administration and would be constructed to be representative of the 
content and statistical specifications (TIF) of NAEP. The forms for 
Grade 8 mathematics would consist of six intact 15-minute blocks 
administered in two 45-minute sessions. The forms for Grade 4 reading 
would consist of four NAEP reading blocks, also administered in two 45-
minute sessions. (Because the reading blocks are timed at 25 minutes 
each, some items will have to be deleted to fit into the reduced 
testing time.) The administration of these forms would be under 
conditions proposed for the VNT. To avoid the circularity of linking 
the same items to themselves, the items used in the Extended-NAEP forms 
should be distinct from those used in the NAEP Special Forms.
    In the first two conditions of this proposed study, the two types 
of forms would be spiraled together and administered to equivalent 
samples of students. Because the NAEP Special Forms and the Extended-
NAEP forms would be administered under the same conditions, issues of 
administration, timing, and motivation become moot. If the content 
match between the NAEP Special Forms and the simulated VNT forms could 
be made sufficiently close, a linking study between the two types of 
forms would approximate a linkage study between actual VNT forms and 
Extended-NAEP. If a calibration were successful, the resulting linkage 
interpretations would be in terms of student performance on NAEP when 
NAEP is given under VNT conditions.
    Students in the third condition differ from the other two in that 
they would be taking the ``NAEP Special Form'' under motivated 
circumstances. It is quite plausible that the same student would 
perform at a higher level under a motivated situation such as the VNT, 
where individual scores are obtained than under a low motivation 
situation such as the NAEP. This differential effect of motivation 
could impact achievement level cut-points (among other things) in ways 
that cannot be assessed in the two conditions described above. 
Consequently, the third condition of this study involves paying 
students $1 for every item they answer correctly. This procedure is 
directly modeled after research conducted on motivational interventions 
for the NAEP. A comparison of item parameters and test characteristic 
curves for the NAEP Special Forms under motivated and unmotivated 
conditions would provide information on the differential impact of 
motivation and how to adjust results from any subsequent linking study 
between the VNT and NAEP.
    The second study involves a series of subtasks directed toward 
informing NAGB's inclusion and accommodation policies regarding LEP 
students. These tasks are:

Subtask A

    Writing an issues paper covering theory and research related to the 
development of a dual language test. This paper would inform procedures 
to be used in the translation of items into the second language (i.e., 
Spanish) (Subtask B).

Subtask B

    Using released and secure NAEP 8th grade mathematics items to 
construct

[[Page 8350]]

simulated VNT-M test booklets (dual language and English-only 
versions). The English language version of this booklet will be the 
same as the one for the ``NAEP Special Form'' described earlier.

Subtask C

    Evaluating the psychometric equivalence of the dual language and 
English-only booklets via traditional quantitative analyses. Six 
hundred bilingual and LEP students will be recruited and randomly 
assigned to complete either the dual language or English-only version 
of the test booklet. Quantitative analyses will be conducted to examine 
the psychometric equivalence of the two test versions (mean 
differences; differential item functioning; correlations).

Subtask D

    Conducting focus groups of students immediately after they take the 
VNT-M to document students' overall experience with the two types of 
booklets. Sixty students will be recruited to do these focus groups, in 
order to obtain their insights and general reactions to the booklets.

Subtask E

    Conducting cognitive laboratory studies to obtain in-depth 
information on the validity of the translation and about how students 
use the dual language test. An additional nine LEP and nine English-
speaking students will be asked to participate in this study, in order 
to explore the performance of both Anglo and Hispanic LEP students to 
identify solution pathways that students choose to use.
    Subtasks C through E will allow for a thorough investigation into 
the cognitive processes that bilingual and limited English proficient 
(LEP) students employ when using the dual language version of the VNT-
M. In addition, they will provide information about factors other than 
mathematical knowledge and problem-solving ability that may have an 
effect on their performance on the test.
    The five subtasks listed above will offer answers to the following 
research questions to examine the quality of the dual language test, 
taking into account several features of the items:
    Cognitive: Do students understand the native language version of 
the test questions as a vehicle for assessing mathematics? (Subtasks C, 
D, E)
    Content: Is the content of the native language version of the test 
questions the same as the English version? (Subtasks B, C, D, E)
    Format: What considerations should be given to how the test 
questions appear on the pages of the test booklet? (Subtasks A, B)
    Cultural: Is the native language version clear and acceptable to 
the various communities in the United States for whom this is the 
native language? (Subtasks A, B, C, D, E)
    Academic: Are the grammar and language structure used in the native 
language version correct? (Subtasks B, D, E)
    Scoring: What considerations need to be made for scoring dual 
language test booklets? (Subtask A)
    Psychometric Equivalence: Is there a psychometric equivalence 
between the dual language version and the English only versions of the 
test? (Subtask C)
    A total of 10,800 students is expected to participate in the two 
studies (4800 4th graders and 4800 8th graders in the calibration 
linkage feasibility study; 1,200 LEP and bilingual students taking the 
dual language or English-only math test (from which there will be 60 
focus group participants); and 18 cognitive laboratory participants). 
These students will be recruited from 300 schools. Students in the 
motivated condition of the calibration linkage study, focus group 
participants and cognitive laboratory participants will receive a token 
monetary incentive. Also under consideration is a modest monetary 
incentive for each participating school.
    Burden Statement: Assuming a 2 hour burden for each of the 10,800 
students expected to participate in the two studies, a total of 21,600 
hours is estimated. An additional 300 hours of school burden (one hour 
per participating school) is expected, reflecting the time it would 
take to collect student background data for our research purposes. 
Participation in this study is voluntary. State, local, and non-public 
education agencies will not be mandated or required to participate.

II. Request for Comments

    The National Assessment Governing Board is especially interested in 
public comments that will assist it:
    (a) Evaluate whether the proposed collection of information is 
necessary for the proper performance of the functions of the Governing 
Board, including whether the information will have practical utility;
    (b) Evaluate the accuracy of the Governing Board's estimates of the 
burden of the proposed collection of information;
    (c) Enhance the quality, utility and clarity of the information to 
be collected;
    (d) Minimize the burden of the collection of the information on 
those who are to respond, including through the use of appropriate 
automated, mechanical or other technological collection techniques or 
other forms of information technology, e.g., permitting electronic 
submission of responses.

    Dated: February 15, 2000.
Roy Truby,
Executive Director, National Assessment Governing Board.
[FR Doc. 00-4016 Filed 2-17-00; 8:45 am]
BILLING CODE 4000-01-U