[Federal Register Volume 61, Number 107 (Monday, June 3, 1996)]
[Proposed Rules]
[Pages 27990-27993]
From the Federal Register Online via the Government Publishing Office [www.gpo.gov]
[FR Doc No: 96-13801]




[[Page 27989]]


_______________________________________________________________________

Part V





Department of Education





_______________________________________________________________________



34 CFR Part 701



Standards for Conduct and Evaluation of Activities Carried Out by the 
Office of Educational Research and Improvement (OERI)--Designation of 
Exemplary and Promising Programs; Proposed Rule

  Federal Register / Vol. 61, No. 107 / Monday, June 3, 1996 / Proposed 
Rules  

[[Page 27990]]



DEPARTMENT OF EDUCATION

34 CFR Part 701

RIN 1850-AA52


Standards for Conduct and Evaluation of Activities Carried Out by 
the Office of Educational Research and Improvement (OERI)--Designation 
of Exemplary and Promising Programs

AGENCY: Department of Education.

ACTION: Notice of proposed rulemaking (NPRM).

-----------------------------------------------------------------------

SUMMARY: The Assistant Secretary is developing these standards pursuant 
to the Office of Educational Research and Improvement's authorizing 
legislation, the ``Educational Research, Development, Dissemination, 
and Improvement Act of 1994.'' The major purpose of these standards is 
to provide quality assurance that programs designated by the Department 
of Education as either exemplary or promising have met criteria that 
will allow educators, professional organizations, and others to use 
these programs with confidence.

DATES: Comments must be received on or before August 2, 1996.

ADDRESSES: All comments concerning these proposed regulations should be 
addressed to Eve M. Bither, U.S. Department of Education, 555 New 
Jersey Avenue, N.W., Room 500, Washington, D.C. 20208-5530. Comments 
may also be sent through the Internet to: (Eve__B[email protected]).
    Comments that concern information collection requirements should be 
sent to the Office of Management and Budget at the address listed in 
the Paperwork Reduction Act section of this preamble.

FOR FURTHER INFORMATION CONTACT: Sharon Bobbitt, U.S. Department of 
Education, 555 New Jersey Avenue, N.W., Room 508, Washington, D.C. 
20208-5643. Telephone: (202) 219-2126. Internet: 
(Sharon__B[email protected]).
    Individuals who use a telecommunications device for the deaf (TDD) 
may call the Federal Information Relay Service (FIRS) at 1-800-877-8339 
between 8 a.m. and 8 p.m., Eastern time, Monday through Friday.

SUPPLEMENTARY INFORMATION:

Background

    On March 31, 1994, President Clinton signed Pub. L. 103-227, which 
includes Title IX, the ``Educational Research, Development, 
Dissemination, and Improvement Act of 1994'' (the ``Act''). The Act 
restructured the Office of Educational Research and Improvement (OERI) 
and provided it with a broad mandate to conduct an array of research, 
development, dissemination, and improvement activities aimed at 
strengthening the education of all students.

Statutory Requirements

    The legislation directed the Assistant Secretary to develop, in 
consultation with the National Educational Research Policy and 
Priorities Board, such standards as may be necessary to govern the 
conduct and evaluation of all research, development, and dissemination 
activities carried out by the Office to ensure that those activities 
meet the highest standards of professional excellence. The legislation 
requires that the standards be developed in three phases.
    In the first phase, standards were promulgated to establish the 
peer review process and evaluation criteria to be used for the review 
of applications for grants and cooperative agreements and proposals for 
contracts. The final regulations setting out these standards were 
published in the Federal Register on September 14, 1995 (60 FR 47808). 
These proposed regulations address the second phase of development by 
establishing the criteria to be used in reviewing potentially exemplary 
and promising educational programs. The Assistant Secretary will 
publish at a later date additional proposed regulations for phase three 
of the standards, which are to govern evaluation of the performance of 
recipients of grants and contracts and cooperative agreements with 
OERI.
    The OERI legislation requires that expert panels be established to 
review educational programs and recommend to the Secretary those 
programs that should be designated as exemplary or promising and 
disseminated through the Department's National Education Dissemination 
System. The legislation further requires the Assistant Secretary to 
develop standards that describe the procedures the panels will use in 
reviewing the educational programs. Section 941(a)(3) of the 
legislation broadly defines ``educational program'' to include 
``educational polices, research findings, practices and products.'' 
Educational programs may range in size and complexity from an 
individual instructional program--such as an elementary school science 
program--to a comprehensive reform initiative involving multiple goals 
and participants. Programs at all levels of education--preschool, 
elementary, secondary, and postsecondary--are eligible for 
consideration.
    The Act also requires that the Assistant Secretary review the 
procedures utilized by the National Institutes of Health (NIH), the 
National Science Foundation (NSF), and other Federal departments or 
agencies engaged in research and development and actively solicit 
recommendations from research organizations and members of the general 
public.
    In developing the review and evaluation procedures for the proposed 
standards, OERI has reviewed and considered dissemination practices and 
procedures used for identifying promising and exemplary programs by 
various foundations, research organizations, associations, and Federal 
agencies including NIH, NSF, the Federal Emergency Management Agency, 
the Department of Health and Human Services, and the National Endowment 
for the Arts. OERI adapted these review and evaluation procedures as 
appropriate.

Proposed Standards

    The proposed standards have been developed by the Assistant 
Secretary in consultation with the Board. The standards proposed in 
this NPRM--
     Require that expert panels be established to review 
educational programs and recommend to the Secretary those programs that 
should be designated as exemplary or promising and disseminated through 
the Department's National Education Dissemination System; and
     Establish a process that panels will use to review and 
evaluate educational programs and determine which programs to recommend 
to the Secretary for designation as exemplary or promising.
    Educational programs may be submitted at any time for consideration 
for designation as exemplary or promising. In addition, the Assistant 
Secretary will periodically establish and announce in the Federal 
Register specific topic areas of high priority for which programs will 
be invited or sought out. The legislation also provides that the 
Secretary may identify educational programs for the panels to review.
    Educational program submissions may include, as evidence of the 
effectiveness of the program, a range of assessments, evaluative 
information from users, and other objective performance indicators that 
are appropriate to the program. The legislation ensures that a panel 
may not eliminate any program from consideration based on the lack of 
one type of supporting data such as test scores.

[[Page 27991]]

    A standing group of experts, which will include teachers and 
others, will be appointed by the Assistant Secretary as appropriate. 
From that group, the Assistant Secretary will select members who have 
relevant knowledge and experience in specific topic areas to form 
expert panels to review programs in accordance with the criteria in 
these proposed regulations.
    In determining whether an educational program should be recommended 
as exemplary or promising, the panel is required by the legislation to 
consider (a) whether, based on empirical data, the program is effective 
and should be designated as exemplary, or (b) whether there is 
sufficient evidence to demonstrate that the program shows promise for 
improving student achievement and should be designated as promising. 
These proposed regulations require a panel to evaluate whether a 
program has met all of the criteria of educational effectiveness set 
forth in Subpart C of these proposed regulations. A panel may determine 
that a program shows promise for improving student achievement and 
recommend that the program be designated as promising if the program 
has met all of the criteria with respect to one context, or with one 
population. A panel may determine that a program is effective and 
recommend that the program be designated as exemplary if the program 
has met all of the criteria with respect to multiple contexts, or with 
multiple populations.
    Use of these criteria for evaluating programs will ensure that 
programs disseminated by the Department are high-quality, research-
based programs that have provided evidence indicating they have 
improved teaching or learning or both. The Department's dissemination 
system is designed to make programs available to the public as quickly 
as possible. The system will enable the Department to respond to all 
forms of requests for information and assistance, and to support the 
applications of research and best practice. The system will use 
electronic networking and the capabilities of:

National Research Institutes;
Educational Resources Information Center (ERIC);
Regional Educational Laboratories;
Department-supported technical assistance providers;
National Library of Education; and
Other public and private nonprofit entities, including education 
associations and networks.

    Prior to the adoption of these standards, exemplary programs were 
validated by the Department's Program Effectiveness Panel (PEP) and 
disseminated through the National Diffusion Network (NDN). With the 
adoption of these standards, the Department will recognize and 
disseminate promising educational programs in addition to exemplary 
programs.

Regulatory Flexibility Act Certification

    The Secretary certifies that these proposed regulations would not 
have a significant economic impact on a substantial number of small 
entities.
    The small entities that would be affected by these proposed 
regulations are small local educational agencies (LEAs) and private 
schools receiving Federal funds under this program. However, the 
regulations would not have a significant economic impact on the small 
LEAs and private schools affected, because the regulations would not 
impose excessive regulatory burdens or require unnecessary Federal 
supervision. The regulations would impose minimal requirements to 
ensure the proper expenditure of program funds.

Paperwork Reduction Act of 1995

    Section 701.4 contains information collection requirements. As 
required by the Paperwork Reduction Act of 1995, the Department of 
Education will submit a copy of this section to the Office of 
Management and Budget (OMB) for its review. (44 U.S.C. 3504(h))
    These regulations affect the following types of entities eligible 
to submit a program for review: Any public or private agency, 
organization or institution, or individual.
    The public reporting burden is estimated to range from 2 to 6 hours 
for each program submitted for review. The actual burden will be 
determined by how much descriptive information about their program each 
entity wishes to provide.
    Organizations and individuals desiring to submit comments on the 
information collection requirements should direct them to the Office of 
Information and Regulatory Affairs, OMB, Room 10235, New Executive 
Office Building, Washington, D.C. 20503; Attention: Wendy Taylor.
    The Department considers comments by the public on this proposed 
collection of information in--
     Evaluating whether the proposed collection of information 
is necessary for the proper performance of the functions of the 
Department, including whether the information will have practical 
utility;
     Evaluating the accuracy of the Department's estimate of 
the burden of the proposed collection of information, including the 
validity of the methodology and assumptions used;
     Enhancing the quality, usefulness, and clarity of the 
information to be collected; and
     Minimizing the burden of the collection of information on 
those who are to respond, including through the use of appropriate 
automated, electronic, mechanical, or other technological collection 
techniques or other forms of information technology (e.g., permitting 
electronic submission of responses).
    OMB is required to make a decision concerning the collection of 
information contained in these proposed regulations between 30 and 60 
days after publication of this document in the Federal Register. 
Therefore, a comment to OMB is best assured of having its full effect 
if OMB receives it within 30 days of publication. This does not affect 
the deadline for the public to comment to the Department on the 
proposed regulations.

Invitation To Comment

    Interested persons are invited to submit comments and 
recommendations regarding these proposed regulations.
    All comments submitted in response to these proposed regulations 
will be available for public inspection, during and after the comment 
period, in Room 600, 555 New Jersey Avenue, NW., Washington, DC, 
between the hours of 8:30 a.m. and 4 p.m., Monday through Friday of 
each week except Federal holidays.

Assessment of Educational Impact

    The Secretary particularly requests comments on whether the 
proposed regulations in this document would require transmission of 
information that is being gathered by or is available from any other 
agency or authority of the United States.

List of Subjects in 34 CFR Part 701

    Education, Educational research, Reporting and recordkeeping 
requirements.

    Dated: May 22, 1996.

(Catalog of Federal Domestic Assistance Number does not apply)
Sharon P. Robinson,
Assistant Secretary for Educational Research and Improvement.

    The Secretary proposes to amend Chapter VII of Title 34 of the Code 
of Federal Regulations by adding a new Part 701 to read as follows:

[[Page 27992]]

PART 701--STANDARDS FOR CONDUCT AND EVALUATION OF ACTIVITIES 
CARRIED OUT BY THE OFFICE OF EDUCATIONAL RESEARCH AND IMPROVEMENT 
(OERI)--DESIGNATION OF EXEMPLARY AND PROMISING PROGRAMS

Subpart A--General

Sec.
701.1  What is the purpose of these standards?
701.2  What definitions apply?
701.3  What entity is eligible to submit a program for review?
701.4  What must an entity submit for review?

Subpart B--Selection of Panel Members

701.10  How are panels established?
701.11  Who may serve as a member of the standing group?
701.12  How is the membership of expert panels determined?

Subpart C--The Expert Panel Review Process

701.20  How does an expert panel evaluate programs?
701.21  What is the difference between an exemplary and a promising 
program?
701.22  What criteria are used to evaluate programs for exemplary or 
promising designation?

    Authority: 20 U.S.C. 6011(i)

Subpart A--General


Sec. 701.1   What is the purpose of these standards?

    (a) The standards in this part implement section 941(d) of the 
Educational Research, Development, Dissemination, and Improvement Act 
of 1994.
    (b) These standards are intended to provide quality assurance that 
programs designated by the Department of Education as either exemplary 
or promising have met criteria that will allow educators, professional 
organizations, and others to use these programs with confidence.

(Authority: 20 U.S.C. 6011(i)(2)(E))


Sec. 701.2   What definitions apply?

    Definition in the Educational Research, Development, Dissemination, 
and Improvement Act of 1994. The following term used in this part is 
defined in 20 U.S.C. 6041(a)(3):

Educational program

(Authority: 20 U.S.C. 6041(a)(3))


Sec. 701.3   What entity is eligible to submit a program for review?

    Any public or private agency, organization, or institution, or an 
individual, may submit an educational program for review.

(Authority: 20 U.S.C. 6011(i)(2)(E))


Sec. 701.4   What must an entity submit for review?

    (a) To have its educational program considered for designation as 
exemplary or promising, the eligible entity must submit to the 
Secretary a description of the program and a discussion of the 
program's educational effectiveness, responsive to the criteria in 
Subpart C, Sec. 701.22.
    (b) Information submitted must include, to the extent relevant to 
the particular program--
    (1) A program abstract of 250 words or less;
    (2) A description of the salient features of the program;
    (3) A description of the program's philosophy and history;
    (4) Site information, including demographics;
    (5) A description of evaluation results;
    (6) Funding and staffing information; and
    (7) Organization name, address, telephone and fax numbers, e-mail 
address (if available), and contact person.

(Authority: 20 U.S.C. 6011(i)(2)(E))

Subpart B--Selection of Panel Members


Sec. 701.10   How are panels established?

    (a) The Assistant Secretary, in consultation with the National 
Educational Research Policy and Priorities Board established under 20 
U.S.C. 6021, establishes a standing group of educational experts. The 
Assistant Secretary may expand the membership of the standing group as 
necessary.
    (b) The Assistant Secretary selects members from the standing 
group, based on their areas of expertise, to serve on expert panels in 
specific topic areas for the purpose of reviewing and evaluating 
educational programs and recommending, to the Secretary, those programs 
that should be designated as exemplary or promising.

(Authority: 20 U.S.C. 6011(i)(2)(E), 6041(d))


Sec. 701.11   Who may serve as a member of the standing group?

    An individual may serve as a member of the standing group for the 
purpose of reviewing and evaluating educational programs for exemplary 
or promising status if that individual possesses two or more of the 
following qualifications:
    (a) Demonstrated expertise and experience in one or more specific 
educational areas.
    (b) Demonstrated expertise and experience across a broad range of 
educational policies and practices.
    (c) Experience in evaluating educational programs.
    (d) Experience or expertise in developing educational products.
    (e) Current employment as a teacher, principal or other school-
based or community-based professional (such as a guidance counselor, 
school media specialist, or health professional).

(Authority: 20 U.S.C. 6011(i)(2)(E), 6041(d))


Sec. 701.12   How is the membership of expert panels determined?

    (a) For the review of each program, or group of programs, the 
Assistant Secretary establishes an expert panel comprised of 
individuals who are members of the standing group.
    (b) In establishing the membership of each expert panel, the 
Assistant Secretary--
    (1) Selects individuals who have in-depth knowledge of the subject 
area or content of the program or group of programs to be evaluated;
    (2) Selects at least one current teacher, principal, or other 
school-based or community-based professional;
    (3) Ensures that no more than one-third of the panel members are 
employees of the Federal Government; and
    (4) Ensures that each panel member does not have a conflict of 
interest, as determined in accordance with paragraph (c) of this 
section, with respect to any educational program the panel member is 
asked to review.
    (c) Panel members are considered employees of the Department for 
the purposes of conflicts of interest analysis and are subject to the 
provisions of 18 U.S.C. 208, 5 CFR 2635.502, and the Department's 
policies used to implement those provisions.

(Authority: 20 U.S.C. 6011(i)(2)(E), 6041(d))

Subpart C--The Expert Panel Review Process


Sec. 701.20   How does an expert panel evaluate programs?

    (a) Each panel member shall--
    (1) Independently review each program based on the criteria in 
Sec. 701.22;
    (2) Provide written comments based on an analysis of the strengths 
and weaknesses of the program according to the criteria;
    (3) Participate in site visits if appropriate; and
    (4) Participate in a meeting of the expert panel, if appropriate, 
to discuss the reviews.
    (b) A panel may not eliminate an educational program from 
consideration based solely on the fact that the program does not have 
one specific type of supporting data, such as test scores.

[[Page 27993]]

    (c) Each expert panel shall make a recommendation to the Secretary 
as to whether the program is exemplary, promising, or neither.

(Authority: 20 U.S.C.6011(i)(2)(E), 6041(d))


Sec. 701.21   What is the difference between an exemplary and a 
promising program?

    (a) A panel may recommend to the Secretary that a program be 
designated as promising if the panel determines that the program has 
met each of the criteria of educational effectiveness in Sec. 701.22 
with respect to one context or one population.
    (b) A panel may recommend to the Secretary that a program be 
designated as exemplary if the panel determines that the program has 
met each of the criteria of educational effectiveness in Sec. 701.22 
with respect to multiple contexts or multiple populations.

(Authority: 20 U.S.C.6011(i)(2)(E), 6041(d)(2))


Sec. 701.22   What criteria are used to evaluate programs for exemplary 
or promising designation?

    In determining whether an educational program (``program'' includes 
educational polices, research findings, practices and products) should 
be recommended as exemplary, promising, or neither, each expert panel 
shall consider the following criteria of educational effectiveness:
    (a) Evidence of success. The expert panel considers--
    (1) Whether, based on a range of assessments, information from 
users, or other indicators as appropriate, the program contributes to 
solving substantial or important problems in teaching or learning; and
    (2) The extent to which--
    (i) Program effects are beneficial to the populations for whom the 
program was designed; or
    (ii) The product performs as expected for the educational consumers 
it was said to benefit.
    (b) Quality of the program. The expert panel considers--
    (1) Whether the program has clear goals, is based on sound research 
and practice, and incorporates accurate and up-to-date content;
    (2) Whether the program represents a substantially improved 
alternative to existing options;
    (3) The extent to which the program promotes equity and is free of 
bias based on race, gender, age, culture, ethnic origin, disability, or 
limited English proficiency status;
    (4) Whether the program is based on high expectations for the 
success of all participants;
    (5) Whether the program is appropriate to the target audiences; and
    (6) The extent to which any materials associated with the program 
conform to accepted standards of technical quality.
    (c) Educational significance. The expert panel considers--
    (1) The extent to which the program has the potential to increase 
knowledge or understanding of educational problems, and issues, or 
effective strategies for teaching or learning; and
    (2) Whether the program is described clearly enough so that it can 
be adapted or adopted in new sites.
    (d) Usefulness to others. The expert panel considers--
    (1) Whether the cost of the program (including money, staff time, 
and other required resources) is reasonable in light of expected 
benefits and compared to other alternatives; and
    (2) Whether the program is available for use by others.

(Authority: 20 U.S.C. 6011(i)(2)(E), 6041(d)(2))

[FR Doc. 96-13801 Filed 5-31-96; 8:45 am]
BILLING CODE 4000-01-P