[Federal Register Volume 63, Number 207 (Tuesday, October 27, 1998)]
[Rules and Regulations]
[Pages 57570-57575]
From the Federal Register Online via the Government Publishing Office [www.gpo.gov]
[FR Doc No: 98-28729]



[[Page 57569]]

_______________________________________________________________________

Part VII





Department of Education





_______________________________________________________________________



34 CFR Part 702



Standards for Conduct and Evaluation of Activities Carried Out by the 
Office of Educational Research and Improvement (OERI)--Evaluation of 
the Performance of Recipients of Grants, Cooperative Agreements, and 
Contracts; Final Rule

  Federal Register / Vol. 63, No. 207 / Tuesday, October 27, 1998 / 
Rules and Regulations  

[[Page 57570]]



DEPARTMENT OF EDUCATION

34 CFR Part 702

RIN 1850-AA54


Standards for Conduct and Evaluation of Activities Carried Out by 
the Office of Educational Research and Improvement (OERI)--Evaluation 
of the Performance of Recipients of Grants, Cooperative Agreements, and 
Contracts

AGENCY: Office of Educational Research and Improvement, Department of 
Education.

ACTION: Final regulations.

-----------------------------------------------------------------------

SUMMARY: The Assistant Secretary establishes regulations pursuant to 
OERI's authorizing legislation, the Educational Research, Development, 
Dissemination, and Improvement Act of 1994. The major purpose of these 
standards is to ensure that the research, development, and 
dissemination activities carried out by the recipients of grants from 
and contracts and cooperative agreements with OERI meet the highest 
standards of professional excellence.

EFFECTIVE DATE: These regulations take effect November 27, 1998.

FOR FURTHER INFORMATION CONTACT: Sharon Bobbitt, U.S. Department of 
Education, 555 New Jersey Avenue, NW, Room 508C, Washington, D.C. 
Telephone: (202) 219-2126. Internet: (Sharon--B[email protected]). 
Individuals who use a telecommunications device for the deaf (TDD) may 
call the Federal Information Relay Service (FIRS) at 1-800-877-8339 
between 8 a.m. and 8 p.m., Eastern time, Monday through Friday.
    Individuals with disabilities may obtain this document in an 
alternate format (e.g., Braille, large print, audiotape, or computer 
diskette) on request to the contact person listed in the preceding 
paragraph.

SUPPLEMENTARY INFORMATION:

Background

    On March 31, 1994, President Clinton signed Pub. L. 103-227, which 
includes Title IX, the Educational Research, Development, 
Dissemination, and Improvement Act of 1994 (the Act). The Act 
restructured OERI and provided it with a broad mandate to conduct an 
array of research, development, dissemination, and improvement 
activities aimed at strengthening the education of all students.

Statutory Requirements

    The Act directed the Assistant Secretary to develop, in 
consultation with the National Educational Research Policy and 
Priorities Board (the Board), such standards as may be necessary to 
govern the conduct and evaluation of all research, development, and 
dissemination activities carried out by OERI to ensure that these 
activities meet the highest standards of professional excellence. The 
Board is responsible for reviewing and approving the standards. The 
legislation requires that the standards be developed in three phases.
    In the first phase, standards were created and promulgated to 
establish the peer review process and evaluation criteria to be used 
for the review of applications for grants and cooperative agreements 
and proposals for contracts. The final regulations setting out these 
standards were published on September 14, 1995 (60 FR 47808). In the 
second phase, standards were created and promulgated to establish the 
criteria to be used in reviewing potentially exemplary and promising 
educational programs. The final regulations setting out these standards 
were published on November 17, 1997 (62 FR 61427).
    In the third phase, which is the subject of these final 
regulations, the Act requires that OERI develop standards for 
evaluating and assessing the performance of all recipients of grants 
from and cooperative agreements and contracts with OERI. This 
evaluation must take place both during and at the conclusion of the 
performance of the grant, cooperative agreement, or contract, and must 
include the use of a system of peer review for the final assessment.
    In developing the standards, the Assistant Secretary was required 
to review the procedures utilized by the National Institutes of Health 
(NIH), the National Science Foundation (NSF), and other Federal 
departments or agencies engaged in research and development and to 
solicit recommendations from research organizations and members of the 
general public. OERI has reviewed the procedures used to evaluate the 
performance of recipients of grants, contracts, or cooperative 
agreements by several offices within NIH and NSF, the Office of Energy 
Research in the Department of Energy, the Food and Drug Administration, 
the National Institute of Standards and Technology, the National 
Aeronautics and Space Administration, and the University Research 
Initiative of the Department of Defense. Recommendations concerning 
these standards have been obtained from the American Educational 
Research Association, the Council for Educational Development and 
Research, and the Organization of Research Centers.

Standards

    The standards have been developed by the Assistant Secretary in 
consultation with the Board. These standards cover all grants, 
cooperative agreements, and contracts administered by OERI, ranging 
from the smallest purchase orders and commissioned papers to the 
largest research projects and research centers. The standards:
     Require at least one interim assessment as well as a final 
assessment of the performance of recipients of grants, cooperative 
agreements, and contracts.
     Establish procedures for selecting peer review panels to 
conduct the assessments.
     Establish procedures and criteria that the peer review 
panels use in conducting the assessments.
     Establish specific additional criteria that peer review 
panels use in conducting the assessments for National Research and 
Development Centers, Regional Educational Laboratories, Field-Initiated 
Studies, and ERIC Clearinghouses.
    In an effort to fulfill the law's intention of ensuring high-
quality research, development, and evaluation, OERI has developed 
standards in which interim and final assessments may be supplemented by 
a self-assessment by the recipient of a grant, cooperative, agreement, 
or contract. The Board and the Assistant Secretary believe that the 
collection and review of evidence on one's own performance is itself a 
useful tool for improvement.
    The Government Performance and Results Act requires the 
establishment of performance indicators for Department activities. 
Information collected pursuant to those indicators will be considered, 
as appropriate, in the evaluation of individual recipients.
    On February 24, 1998, the Assistant Secretary published a notice of 
proposed rulemaking (NPRM) for these standards in the Federal Register 
(63 FR 9393). These final regulations contain four major changes from 
the NPRM. These changes are fully explained in the ``Analysis of 
Comments and Changes'' elsewhere in this preamble. The major changes 
pertain to clarification of the purpose of the regulation, how OERI 
determines the number of interim assessments necessary, the role of 
Department of Education staff in the assessments, and the use of 
interim assessments as a source of information for the final 
assessment.

Analysis of Comments and Changes

    In response to the Secretary's invitation in the NPRM, four parties

[[Page 57571]]

submitted comments on the proposed regulations. In addition to the 
public comment, comments from the Board's Subcommittee on Standards are 
addressed as required by the legislation. The full Board approved the 
final regulations at a meeting on September 18, 1998. An analysis of 
the comments and of the changes in the regulations since publication of 
the NPRM follows.
    Major issues are grouped according to subject with appropriate 
sections of the regulations referenced in parentheses. Technical and 
other minor changes--and suggested changes the Secretary is not legally 
authorized to make under the applicable statutory authority--are not 
addressed.

Purpose (Sec. 702.1)

    Comments: Three commenters suggested that the purpose of the 
standards be clarified. One commenter suggested that the standards 
themselves cannot ensure the highest standards of professional 
excellence. Another commenter asked specifically whether the purpose 
for conducting assessments was to make decisions about future funding 
or to provide a system for monitoring and enhancing current and future 
projects.
    Discussion: The Secretary agrees that the purpose of the standards 
should be clarified to go beyond their stated statutory purpose, which 
is to ``ensure the highest standards of professional excellence,'' to 
include the objectives of continuously improving the quality of funded 
activities and of considering the results as one of the factors in 
determining continuation funding for multi-year awards.
    Changes: Section 702.1 has been modified to include a provision 
that the purpose of the standards is to provide feedback to help 
improve the quality of funded activities and to provide information for 
consideration as continuation funding decisions are made.

Additional Activities that May be Evaluated  (Sec. 702.3)

    Comment: One commenter thought that the statement that these 
standards could be applied to other activities funded by the Department 
was too broad and should be deleted.
    Discussion: The Secretary believes that this statement is 
necessarily broad to allow all Department programs to use these 
standards, when appropriate, to assess the performance of any of their 
funded activities without developing their own unique regulations. This 
statement is also consistent with the earlier standards which 
established the peer review process and evaluation criteria to be used 
for the review of applications for grants and cooperative agreements 
and proposals for contracts.
    Changes: None.

Number of Interim Assessments  (Sec. 702.4)

    Comments: Two commenters suggested changes to this provision. One 
commenter suggested since there may be more than one interim 
assessment, that it be clear in Sec. 702.4(d)(1). The OERI Board 
suggested that the requirements for a single interim assessment for 
total awards of $5,000,000 or less be modified to reflect total awards 
of $3,000,000 or less.
    Discussion: In response to the comments, the Secretary now believes 
that considerations such as difficulty in achieving project objectives 
rather than the dollar levels of awards should determine whether a 
particular project merits more than one interim assessment. Elimination 
of the dollar threshold clarifies the original intent of this section 
which is to require that all awards receive one interim assessment. 
More than one interim assessment will be performed only when a 
recipient is having difficulty achieving project objectives as 
determined by the initial interim assessment or through the monitoring 
efforts of Department of Education staff. The Assistant Secretary will 
make the determination of the number of interim assessments on a case-
by-case basis.
    Changes: Section 702.4(b) has been modified to delete the dollar 
threshold and to reflect that all awards will receive at least one 
interim assessment. A new paragraph 702.4(c) has been added to clarify 
that the Assistant Secretary will require more than one interim 
assessment when a recipient has been identified, either in the initial 
interim review or through monitoring efforts of Department of Education 
staff, as having difficulty in achieving project objectives. Former 
paragraph 702.4(c) has been redesignated as Sec. 702.4(d). Section 
702.4(d)(1) has been modified to define an interim assessment as ``any 
assessment'' conducted during a recipient's period of performance.

Definitions  (Sec. 702.5)

    Comment: One commenter suggested that the terms referred to in this 
section include the specific definitions and not references to the OERI 
statute and to the Education Department General Administrative 
Regulations.
    Discussion: The Secretary believes that providing the citations for 
specific terms rather than the definitions themselves keeps regulations 
short and concise while still cross referencing easily accessible 
resources for the definitions.
    Changes: None.

Characteristics of Peer Reviewers  (Sec. 702.10)

    Comment: One commenter suggested that paragraph 702.10(a) ``(4) 
knowledge of a broad range of education policies and practices;'' be 
deleted from the list of knowledge and expertise required of peer 
reviewers, because it is redundant with the other criteria and is very 
vague.
    Discussion: The Secretary believes that this criterion provides for 
a balance between specific program knowledge and a broader perspective 
of education policies and practices and is therefore not redundant with 
the other, more focused, characteristics required of peer reviewers.
    Changes: None.

Role of Department Staff  (Sec. 702.10)

    Comments: Two commenters expressed concern over the appropriate 
role of the OERI staff in the review process. One commenter urged the 
Department to use all outside reviewers. The other commenter 
acknowledged the knowledge and skills of the OERI staff but suggested 
that staff not serve as peer reviewers within the primary division of 
an agency in which they work and that each peer review panel be limited 
to one Department staff person. This commenter suggested that the staff 
focus on the important role of mentoring and designing competitions.
    Discussion: The Secretary agrees that the primary role of the OERI 
staff should be management of competitions including assessing the 
results of peer reviews and monitoring awards. The Secretary believes 
that the purpose of the peer review process should be to acquire the 
perspective of outside experts independent of OERI. The Secretary also 
believes that there may be exceptional circumstances where expertise 
resides in OERI or in the Department, or where outside reviewers are 
not required such as in the review of small purchase orders. The 
exceptions should be determined by the Assistant Secretary.
    Changes: Section 702.10(d) has been reworded to preclude OERI and 
other Department staff from serving as peer reviewers except in 
exceptional circumstances as determined by the Assistant Secretary.

Conflict of Interest  (Sec. 702.11)

    Comment: One commenter was concerned that while the conflict of 
interest requirements were ``legally correct'' they failed to address 
the problem occasioned by reviewers who

[[Page 57572]]

may have ideological or methodological view points that differ from 
those of the recipient to be evaluated, or who are affiliated with 
competing institutional organizations.
    Discussion: The commenter appears to be concerned that the proposed 
conflict of interest provision does not address the potential problem 
of bias on the part of a panel against a particular grantee on 
ideological or other grounds. The Secretary first believes that it is 
essential to retain the present language, which parallels the provision 
in the standards at 34 CFR 701.11(c), because it highlights the 
important issue of improper financial gain or the appearance of 
improper gain. However, the Secretary agrees that adding a requirement 
to the effect that panels selected by the Assistant Secretary reflect a 
broad range of perspectives could strengthen the regulation.
    Changes: A new paragraph ``(c)'' has been added to Sec. 702.13 
requiring the Assistant Secretary, to the greatest extent feasible, to 
select peer reviewers for each evaluation who represent a broad range 
of perspectives.

Sources of Information  (Secs. 702.22 and 702.23)

    Comment: One commenter suggested that the use of Government 
Performance and Results Act (GPRA) information should be encouraged 
rather than required for both interim and final assessments. The 
commenter is concerned that information currently being collected under 
GPRA to evaluate the effectiveness of a program or a system-level 
activity will not provide information relevant to the assessment of 
individual awards under that program or system-level activity and 
therefore should not be required.
    Discussion: The Secretary agrees that information obtained by GPRA-
related reports on the effectiveness of a program or system level 
activity, e.g., how effectively a program is meeting the overall 
objectives defined for it in its authorizing legislation, may not 
necessarily include information related to an individual award being 
reviewed under this regulation. However, the Secretary believes that 
information on the effectiveness of the particular program under which 
a recipient receives funding will help to provide a context for the 
review of an individual award and must be considered by the panel. 
Moreover, these regulations make it clear that the GPRA information is 
only one of a number of sources used in conducting the review.
    Changes: None.
    Comment: One commenter suggested that the findings and information 
from interim assessments would be an important source of information 
for the final assessments and should be included under Sec. 702.23(a).
    Discussion: The Secretary agrees that the results of interim 
assessments should be a source of information for final assessments.
    Change: Section 702.23(a) has been modified to add a new paragraph 
(Sec. 702.23(a)(5)) to require that the results of interim assessments 
be considered as a source of information for final assessments.

Evaluation Criteria  (Sec. 702.24)

    Comments: Two commenters suggested changes to this section. One 
commenter suggested that there be a single menu of criteria for the 
standards, because the proposed menu is too long. The second commenter 
suggested that since Field Initiated Studies are not likely to provide 
services, the word ``services'' be deleted from the criterion in 
Sec. 702.24(e)(4)(ii): ``* * * addresses issues of national 
significance through its products or services, or both.''
    Discussion: The Secretary believes the current menu approach 
provides a comprehensive strategy for assessing the performance of all 
activities, ranging from the smallest purchase order to the largest 
research investments. The categories in the regulation reflect the 
specific authorities in the OERI statute. In addition, the menu 
provides for other criteria for future research investments that do not 
fit within the statutory authorities yet also must be assessed. A 
single menu would, of necessity, be too generic to apply to the wide 
range of activities covered by these standards. The Secretary agrees 
that assessing ``services'' is not appropriate for Field Initiated 
Studies projects.
    Change: Section 702.24(e)(4)(ii) has been modified to delete the 
word, ``services.''

Paperwork Reduction Act of 1995

    Under the Paperwork Reduction Act of 1995, no persons are required 
to respond to a collection of information unless it displays a valid 
OMB control number. The valid OMB control number assigned to the 
collection of information in these final regulations is displayed at 
the end of the affected sections of the regulations.

Assessment of Educational Impact

    In the NPRM the Secretary requested comments on whether the 
proposed regulations would require transmission of information that is 
being gathered by or is available from any other agency or authority of 
the United States.
    Based on the response to the NPRM and on its own review, the 
Department has determined that the regulations in this document do not 
require transmission of information that is being gathered by or is 
available from any other agency or authority of the United States.

Electronic Access to This Document

    Anyone may view this document, as well as all other Department of 
Education documents published in the Federal Register, in text or 
portable document format (pdf) on the World Wide Web at either of the 
following sites:

http://ocfo.ed.gov/fedreg.htm
http://www.ed.gov/news.html

    To use the pdf you must have the Adobe Acrobat Reader Program with 
Search, which is available free at either of the previous sites. If you 
have questions about using the pdf, call the U.S. Government Printing 
Office at (202) 512-1530 or, toll free, at 1-888-293-6498.
    Anyone may also view these documents in text copy only on an 
electronic bulletin board of the Department. Telephone: (202) 219-1511 
or, toll free, 1-800-222-4922. The documents are located under Option 
G--Files/Announcements, Bulletins and Press Releases.

    Note: The official version of this document is the document 
published in the Federal Register.

List of Subjects in 34 CFR Part 702

    Education, Educational research, Reporting and recordkeeping 
requirements.

(Catalog of Federal Domestic Assistance Number does not apply.)

    Dated: October 22, 1998.
C. Kent McGuire,
Assistant Secretary for Educational Research and Improvement.
    The Secretary amends Chapter VII of Title 34 of the Code of Federal 
Regulations by adding a new Part 702 to read as follows:

[[Page 57573]]

PART 702--STANDARDS FOR CONDUCT AND EVALUATION OF ACTIVITIES 
CARRIED OUT BY THE OFFICE OF EDUCATIONAL RESEARCH AND IMPROVEMENT 
(OERI)--EVALUATION OF THE PERFORMANCE OF RECIPIENTS OF GRANTS, 
COOPERATIVE AGREEMENTS, AND CONTRACTS

Subpart A--General

Sec.
702.1  What is the purpose of these standards?
702.2  What activities must be evaluated by these standards?
702.3  What additional activities may be evaluated by these 
standards?
702.4  When is performance assessed under these standards?
702.5  What definitions apply?

Subpart B--Selection of Peer Review Panels

702.10  What are the characteristics of peer reviewers?
702.11  What constitutes a conflict of interest for grants and 
cooperative agreements?
702.12  What constitutes a conflict of interest for contracts?
702.13  How are peer reviewers selected for panels?

Subpart C--The Evaluation Process

702.21  How does a peer review panel evaluate the performance of a 
recipient?
702.22  What information does a peer review panel consider for an 
interim assessment?
702.23  What information does a peer review panel consider for a 
final assessment?
702.24  What evaluation criteria must be used for performance 
assessments?

    Authority: 20 U.S.C. 6011(i), unless otherwise noted.

Subpart A--General


Sec. 702.1   What is the purpose of these standards?

    (a) The standards in this part implement section 912(i) of the 
Educational Research, Development, Dissemination, and Improvement Act 
of 1994 (the Act).
    (b) These standards establish criteria and a peer review process to 
provide recipients of OERI grants, cooperative agreements and contract 
awards with assessments of their projects.
    (1) The purpose of the assessments is to provide feedback to 
recipients to improve the quality of funded activities and to provide 
information to OERI as it determines if a recipient of a multi-year 
award merits continuation funding.
    (2) The criteria and peer review process are intended to address 
the statutory requirement that the research, development, and 
dissemination activities carried out by the recipients of grants from 
and contracts and cooperative agreements with the Office of Educational 
Research and Improvement (OERI) meet the highest standards of 
professional excellence.

(Authority: 20 U.S.C. 6011(i)(2)(F))


Sec. 702.2   What activities must be evaluated by these standards?

    These standards apply to activities carried out by OERI using funds 
appropriated under section 912(m) of the Act including activities 
carried out by the following entities or programs:
    (a) The National Education Research Institutes.
    (b) The Office of Reform Assistance and Dissemination.
    (c) The Educational Resources Information Center.
    (d) The Regional Educational Laboratories.
    (e) The Teacher Research Dissemination Demonstration Program.
    (f) The Goals 2000 Community Partnerships Program.
    (g) The National Educational Research Policy and Priorities Board.

(Authority: 20 U.S.C. 6011(i)(1))


Sec. 702.3   What additional activities may be evaluated by these 
standards?

    The Secretary may apply these standards to other activities funded 
by the Department, as appropriate.

(Authority: 20 U.S. C 6011 (i)(1))


Sec. 702.4   When is performance assessed under these standards?

    (a) The Secretary will assess the performance of recipients of OERI 
grants, contracts, and cooperative agreements subject to these 
standards during and at the conclusion of their period of performance.
    (b) The Department requires at least one interim assessment by a 
peer review panel for all awards.
    (c) The Assistant Secretary will approve and require more than one 
interim assessment when an award is identified, either by the initial 
interim review or by Department of Education staff monitoring the 
award, as having difficulty in achieving project objectives.
    (d) A final assessment by a peer review panel is required for all 
awards.
    (e) As used in this part--
    (1) Interim assessment is any assessment conducted during a 
recipient's period of performance.
    (2) Final assessment is one conducted at the conclusion of a 
recipient's period of performance.

(Authority: 20 U.S.C. 6011(i)(2)(F))


Sec. 702.5  What definitions apply?

    (a) Definitions in the Educational Research, Development, 
Dissemination, and Improvement Act of 1994. The following terms used in 
this part are defined in 20 U.S.C. 6011(l)(1):

Development
Dissemination
Educational Research

    (b) Definitions in the Education Department General Administrative 
Regulations. The following terms used in this part are defined in 34 
CFR 77.1:

Application
Award
Department
Grant
Project
Secretary

    (c) Definitions in the Federal Acquisition Regulation. The 
following term used in this part is defined in 48 CFR Chapter 1: 
Contract Proposal.
(Authority: 20 U.S.C. 6011(i)(2)(F)

Subpart B--Selection of Peer Review Panels


Sec. 702.10  What are the characteristics of peer reviewers?

    (a) The Assistant Secretary selects each peer reviewer. Each peer 
reviewer must have the necessary knowledge and expertise in the area of 
the project being reviewed to evaluate the performance of a recipient. 
This experience may include--
    (1) Expert knowledge of subject matter in the area of the 
activities to be reviewed;
    (2) Expert knowledge of theory or methods or both in the area of 
the activities to be reviewed;
    (3) Practical experience in the area of the activities or type of 
institution or both to be reviewed;
    (4) Knowledge of a broad range of education policies and practices;
    (5) Experience in managing complex organizations; or
    (6) Expertise and experience in evaluation theory and practice.
    (b) Each peer reviewer must be free of conflict of interest, as 
determined in accordance with Sec. 702.11 or Sec. 702.12.
    (c) The Assistant Secretary may solicit nominations for peer 
reviewers from professional associations, nationally recognized 
experts, and other sources.
    (d) OERI and other Department staff who possess the qualifications 
in paragraphs (a) and (b) of this section may serve as peer reviewers 
only in exceptional circumstances as determined by the Assistant 
Secretary.

(Authority: 20 U.S.C. 6011(i)(2)(B))


Sec. 702.11  What constitutes a conflict of interest for grants and 
cooperative agreements?

    A peer reviewer assessing the performance of the recipient of a 
grant

[[Page 57574]]

from or cooperative agreement with OERI is considered an employee of 
the Department for the purposes of conflict of interest analysis. As an 
employee of the Department, the peer reviewer is subject to the 
provisions of 18 U.S.C. 208, 5 CFR 2635.502, and the Department's 
policies used to implement those provisions.

(Authority: 20 U.S.C. 6011(i)(2)(B))


Sec. 702.12  What constitutes a conflict of interest for contracts?

    A peer reviewer assessing the performance of the recipient of a 
contract with OERI is considered an employee of the Department in 
accordance with the Federal Acquisition Regulation (FAR), 48 CFR 3.104-
4(h)(2). As an employee of the Department, the peer reviewer is subject 
to the provisions of the FAR, 48 CFR Part 3, Improper Business 
Practices and Personal Conflict of Interest.

(Authority: 41 U.S.C. 423)


Sec. 702.13  How are peer reviewers selected for panels?

    (a) The Assistant Secretary assigns peer reviewers to panels that 
conduct the performance assessments.
    (b) The Assistant Secretary may establish panels by category of 
recipient, such as a panel to review the performance of all Regional 
Educational Laboratories. Each recipient is evaluated individually by 
reviewers who have been assigned to this type of panel.
    (c) In establishing panels, the Assistant Secretary, to the 
greatest extent feasible, selects peer reviewers for each evaluation 
who represent a broad range of perspectives.

(Authority: 20 U.S.C. 6011(i)(2)(B))

Subpart C--The Evaluation Process


Sec. 702.21  How does a peer review panel evaluate the performance of a 
recipient?

    (a) In each evaluation, a peer review panel--
    (1) Considers relevant information about the recipient's 
performance, as described in Secs. 702.22 and 702.23; and
    (2) Makes judgments about the recipient's performance, using the 
criteria in Sec. 702.24.
    (b) Each peer reviewer prepares a report based on the reviewer's 
assessment of the quality of the project according to the evaluation 
criteria.
    (c) After each peer reviewer has evaluated each project 
independently, the panel may be convened to discuss the strengths and 
weaknesses of the project. Each reviewer may then independently re-
evaluate each project with appropriate changes made to the written 
report.
    (d) The report of the interim assessment must include any 
recommendations the peer reviewer may have for improving the 
recipient's performance.
    (e) The report of the final assessment must contain each peer 
reviewer's evaluative summary of the recipient's performance, from the 
beginning of the contract, grant, or cooperative agreement to its 
conclusion.

(Authority: 20 U.S.C. 6011(i)(2)(F))


Sec. 702.22  What information does a peer review panel consider for an 
interim assessment?

    (a) Sources of information for the interim assessment must 
include--
    (1) The original request for proposals or grant announcement and 
the contract proposal or grant application;
    (2) Documentation of any changes in the work described in the 
contract, grant, or cooperative agreement, including reasons for the 
changes;
    (3) Any progress reports delivered to the Department or made 
available to the public by the recipient;
    (4) Examples of products delivered to the Department or made 
available to the public by the recipient;
    (5) Any relevant reports written by OERI staff, including reports 
of site visits by OERI staff;
    (6) Any performance evaluations conducted under the FAR or the 
Education Department General Administrative Regulations (34 CFR Part 
75).
    (7) Any relevant information provided by the recipient in response 
to Government Performance and Results Act (GPRA) (Pub. L. 103-62) 
requirements; and
    (8) Any reports from program evaluations commissioned by the 
Department.
    (b) Sources of information for the interim assessment may also 
include--
    (1) A self-assessment, prepared by the recipient, addressing the 
criteria in Sec. 702.24;
    (2) One or more site visits by the peer review panel;
    (3) One or more oral or written presentations to the panel by the 
recipient describing its performance; or
    (4) Other information about the recipient's performance.

(Approved by the Office of Management and Budget under control 
number 1850-0746)

(Authority: 20 U.S.C. 6011(i)(2)(F))


Sec. 702.23  What information does a peer review panel consider for a 
final assessment?

    (a) Sources of information for the final assessment must include--
    (1) The original request for proposals or application notice and 
the contract proposal or grant application, together with documentation 
of any changes in the work described in the proposal or application, 
including reasons for the changes;
    (2) If consistent with the recipient's contract, grant, or 
cooperative agreement with OERI, a written report or oral presentation 
or both by the recipient summarizing its activities and 
accomplishments;
    (3) Any relevant information provided by the recipient in response 
to Government Performance and Results Act (GPRA) (Pub. L. 103-62) 
requirements;
    (4) Any reports from program evaluations commissioned by the 
Department; and,
    (5) Any relevant information provided by the interim assessment.
    (b) The final assessment may also include other sources of 
information, such as one or more of those listed in Sec. 702.22.

(Approved by the Office of Management and Budget under control 
number 1850-0746)

(Authority: 20 U.S.C. 6011(i)(2)(F))


Sec. 702.24  What evaluation criteria must be used for performance 
assessments?

    (a) Peer reviewers (and those recipients who conduct self-
evaluations) shall use the criteria in paragraph (b) of this section to 
assess performance and, in case of interim assessments, to identify 
areas in which the performance of recipients may need improvement.
    (b) The following evaluation criteria are to guide the assessment 
process undertaken by peer reviewers. The peer reviewers determine the 
extent to which recipients meet these criteria:
    (1) Implementation and management. (i) Peer reviewers shall 
consider the degree to which the recipient has fully executed its 
program of work. In doing so, peer reviewers shall consider evidence on 
the extent to which the recipient completes the work described in the 
approved application or contract, including any approved modifications, 
in the time period proposed and in an efficient manner.
    (ii) In examining the degree of implementation, peer reviewers may 
also consider evidence on the extent to which--
    (A) The recipient implements and utilizes a quality assurance 
system for its products or services or both; and
    (B) The recipient conducts self-assessment or self-evaluation 
activities, including periodically seeking out independent critiques 
and evaluations of its work, and uses the results to improve 
performance.
    (2) Quality. (i) Peer reviewers shall consider the degree to which 
the

[[Page 57575]]

recipient's work approaches or attains professional excellence. In 
determining quality, peer reviewers shall consider evidence on the 
extent to which--
    (A) The recipient utilizes processes, methods, and techniques 
appropriate to achieve the goals and objectives for the program of work 
in the approved application; and
    (B) The recipient applies appropriate processes, methods, and 
techniques in a manner consistent with the highest standards of the 
profession.
    (ii) In determining quality, peer reviewers may also consider the 
extent to which the recipient conducts a coherent, sustained program of 
work informed by relevant research.
    (3) Utility. (i) In determining the utility of the recipient's 
products or services or both, peer reviewers shall consider evidence on 
the extent to which the recipient's work (including information, 
materials, processes, techniques, or activities) is effectively used by 
and is useful to its customers in appropriate settings.
    (ii) In determining utility, peer reviewers may also consider the 
extent to which the recipient has received national recognition; e.g., 
articles in refereed journals and presentations at professional 
conferences.
    (4) Outcomes and impact. (i) Peer reviewers shall consider the 
results of the recipient's work. In examining outcomes and impact, peer 
reviewers shall consider evidence on the extent to which--
    (A) The recipient meets the needs of its customers; and
    (B) The recipient's work contributes to the increased knowledge or 
understanding of educational problems, issues, or effective strategies.
    (ii) In examining outcomes and impact, peer reviewers may also 
consider the extent to which recipients address issues of national 
significance through its products or services or both.
    (c) For National Research and Development Centers, peer reviewers 
also shall consider evidence on the extent to which recipients meet the 
following criteria:
    (1) Quality. (i) The recipient uses a well-conceptualized framework 
and sound theoretical and methodological tools in conducting 
professionally rigorous studies; and
    (ii) The recipient conducts work of sufficient size, scope, and 
duration to produce sound guidance for improvement efforts and future 
research.
    (2) Utility. The recipient documents, reports, and disseminates its 
work in ways to facilitate the effective use of its work in 
appropriately targeted settings.
    (3) Outcomes and impact. (i) The recipient's work contributes to 
the development and advancement of theory in the field of study, 
including its priority area; and
    (ii) The recipient addresses issues of national significance 
through its products or services or both.
    (d) For the Regional Educational Laboratories, peer reviewers also 
shall consider evidence on the extent to which recipients meet the 
following criteria:
    (1) Quality. (i) The recipient utilizes a well-conceptualized 
framework and sound theoretical and methodological tools in conducting 
professionally rigorous studies;
    (ii) The recipient conducts work of sufficient size, scope, and 
duration to produce sound guidance for improvement efforts; and
    (iii) The recipient's products are well tested and based on sound 
research.
    (2) Utility. The recipient documents, reports, and disseminates its 
work in ways to facilitate its effective use in appropriately targeted 
settings, particularly in school improvement efforts of States and 
localities.
    (3) Outcomes and impact. (i) The recipient assists States and 
localities to implement comprehensive school improvement strategies 
through the provision of research-based information (including well-
tested models and strategies), materials and assistance; and
    (ii) The recipient's work results in widespread access to 
information regarding research and best practices, particularly within 
its region.
    (e) For Field-Initiated Studies, peer reviewers also shall consider 
evidence on the extent to which recipients meet the following criteria:
    (1) Implementation and management. The recipient's work responds to 
the goals, objectives and mission of the National Institute from which 
it is funded.
    (2) Quality. The recipient utilizes a well-conceptualized framework 
and sound theoretical and methodological tools in conducting 
professionally rigorous studies.
    (3) Utility. The recipient documents, reports, and disseminates its 
work in ways to facilitate its effective use in appropriately targeted 
settings.
    (4) Outcomes and impact. (i) The recipient's work contributes to 
the development and advancement of theory and knowledge in the field of 
study; and
    (ii) The recipient addresses issues of national significance 
through its products.
    (f) For the ERIC Clearinghouses, peer reviewers also shall consider 
evidence on the extent to which recipients meet the following criteria:
    (1) Quality. The recipient applies an integrated approach to 
acquiring and disseminating significant and high-quality educational 
literature and materials to maintain and enhance the ERIC database.
    (2) Utility. The recipient contributes to the development of the 
ERIC database as a source of literature and materials that reflects 
trends and issues within its scope.
    (3) Outcomes and impact. (i) The recipient meets the informational 
and educational needs of its customers through dissemination and 
outreach approaches and the development of an array of print and non-
print materials; and
    (ii) The recipient provides national leadership on the use of 
current computer, networking, and information technology.

(Approved by the Office of Management and Budget under control 
number 1850-0746)

(Authority: 20 U.S.C. 6011(i)(2)(F))

[FR Doc. 98-28729 Filed 10-26-98; 8:45 am]
BILLING CODE 4000-01-U