[Federal Register Volume 63, Number 36 (Tuesday, February 24, 1998)]
[Proposed Rules]
[Pages 9392-9397]
From the Federal Register Online via the Government Publishing Office [www.gpo.gov]
[FR Doc No: 98-4690]



[[Page 9391]]

_______________________________________________________________________

Part V





Department of Education





_______________________________________________________________________



34 CFR Part 702



Standards for Conduct and Evaluation of Activities Carried Out by the 
Office of Educational Research and Improvement (OERI); Evaluation of 
the Performance of Recipients of Grants, Cooperative Agreements, and 
Contracts; Proposed Rule

  Federal Register / Vol. 63, No. 36 / Tuesday, February 24, 1998 / 
Proposed Rules  

[[Page 9392]]



DEPARTMENT OF EDUCATION

34 CFR Part 702

RIN 1850-AA54


Standards for Conduct and Evaluation of Activities Carried Out by 
the Office of Educational Research and Improvement (OERI); Evaluation 
of the Performance of Recipients of Grants, Cooperative Agreements, and 
Contracts

AGENCY: Department of Education.

ACTION: Notice of proposed rulemaking.

-----------------------------------------------------------------------

SUMMARY: The Assistant Secretary proposes to establish regulations 
pursuant to OERI's authorizing legislation, the Educational Research, 
Development, Dissemination, and Improvement Act of 1994. The major 
purpose of these standards is to ensure that the research, development, 
and dissemination activities carried out by the recipients of grants 
from and contracts and cooperative agreements with OERI meet the 
highest standards of professional excellence.

DATES: Comments must be received by the Department on or before April 
27, 1998.

ADDRESSES: All comments concerning these proposed regulations should be 
addressed to Sharon Bobbitt, U.S. Department of Education, 555 New 
Jersey Avenue, NW., room 508c, Washington, DC 20202-5651. Comments may 
also be sent through the Internet to: [email protected]
    You must include the term Phase III in the subject line of your 
electronic message.
    Comments that concern information collection requirements must be 
sent to the Office of Management and Budget at the address listed in 
the Paperwork Reduction Act section of this preamble. A copy of those 
comments may also be sent to the Department representative named in 
this section.

FOR FURTHER INFORMATION CONTACT: Sharon Bobbitt. Telephone: (202) 219-
2126. Internet: (Sharon__B[email protected]). Individuals who use a 
telecommunications device for the deaf (TDD) may call the Federal 
Information Relay Service (FIRS) at 1-800-877-8339 between 8 a.m. and 8 
p.m., Eastern time, Monday through Friday.
    Individuals with disabilities may obtain this document in an 
alternate format (e.g., Braille, large print, audiotape, or computer 
diskette) on request to the contact person listed in the preceding 
paragraph.

SUPPLEMENTARY INFORMATION:

Invitation to Comment

    Interested persons are invited to submit comments and 
recommendations regarding these proposed regulations. To ensure that 
public comments have maximum effect in developing the final 
regulations, the Department urges commenters to identify clearly the 
specific section or sections of the proposed regulations that each 
comment addresses and to arrange comments in the same order as the 
proposed regulations.
    The Secretary particularly requests comments on the role of 
Department of Education staff in the implementation of the Standards. 
For example, should Department staff serve as reviewers on peer review 
panels under these regulations? See proposed Sec. 702.10(d) of these 
regulations in this regard. Should there be a maximum number or maximum 
percentage of Department staff on peer review panels? Should the 
participation of Department staff vary by size of the grant, contract, 
or cooperative agreement? What other issues about the role of 
Department staff in the peer review process should the Secretary 
consider?
    All comments submitted in response to these proposed regulations 
will be available for public inspection, during and after the comment 
period, in Room 600, 555 New Jersey Avenue, N.W., Washington, D.C., 
between the hours of 8:30 a.m. and 4:00 p.m., Eastern time, Monday 
through Friday of each week except Federal holidays.
    On request the Department supplies an appropriate aid, such as a 
reader or print magnifier, to an individual with a disability who needs 
assistance to review the comments or other documents in the public 
rulemaking docket for these proposed regulations. An individual with a 
disability who wants to schedule an appointment for this type of aid 
may call (202) 205-8113 or (202) 260-9895. An individual who uses a TDD 
may call the Federal Information Relay Service at 1-800-877-8339, 
between 8 a.m. and 8 p.m., Eastern time, Monday through Friday.
    To assist the Department in complying with the specific 
requirements of Executive Order 12866 and its overall requirement of 
reducing regulatory burden, the Secretary invites comments on whether 
there may be further opportunities to reduce any regulatory burdens 
found in these proposed regulations.

Background

    On March 31, 1994, President Clinton signed Pub. L. 103-227, which 
includes Title IX, the Educational Research, Development, 
Dissemination, and Improvement Act of 1994 (the Act). The Act 
restructured OERI and provided it with a broad mandate to conduct an 
array of research, development, dissemination, and improvement 
activities aimed at strengthening the education of all students.

Statutory Requirements

    The Act directed the Assistant Secretary to develop, in 
consultation with the National Educational Research Policy and 
Priorities Board (the Board), such standards as may be necessary to 
govern the conduct and evaluation of all research, development, and 
dissemination activities carried out by OERI to ensure that these 
activities meet the highest standards of professional excellence. The 
Board is responsible for reviewing and approving the standards. The 
legislation requires that the standards be developed in three phases.
    In the first phase, standards were created and promulgated to 
establish the peer review process and evaluation criteria to be used 
for the review of applications for grants and cooperative agreements 
and proposals for contracts. The final regulations setting out these 
standards were published on September 14, 1995 (60 FR 47808). In the 
second phase, standards were created and promulgated to establish the 
criteria to be used in reviewing potentially exemplary and promising 
educational programs. The final regulations setting out these standards 
were published on November 17, 1997 (62 FR 61427).
    In the third phase, which is the subject of this notice of proposed 
rulemaking (NPRM), the Act requires that OERI develop standards for 
evaluating and assessing the performance of all recipients of grants 
from and cooperative agreements and contracts with OERI. This 
evaluation must take place both during and at the conclusion of the 
performance of the grant, cooperative agreement, or contract, and must 
include the use of a system of peer review for the final assessment.
    In developing the standards, the Assistant Secretary was required 
to review the procedures utilized by the National Institutes of Health 
(NIH), the National Science Foundation (NSF), and other Federal 
departments or agencies engaged in research and development and to 
solicit recommendations from research organizations and members of the 
general public. OERI has reviewed the procedures used to evaluate the 
performance of recipients of grants, contracts, or cooperative 
agreements by several offices within NIH and NSF, the Office of Energy 
Research in the

[[Page 9393]]

Department of Energy, the Food and Drug Administration, the National 
Institute of Standards and Technology, the National Aeronautics and 
Space Administration, and the University Research Initiative of the 
Department of Defense. Recommendations concerning these standards have 
been obtained from the American Educational Research Association, the 
Council for Educational Development and Research, and the Organization 
of Research Centers. Public comment is invited in response to this 
NPRM.

Standards

    The standards have been developed by the Assistant Secretary in 
consultation with the Board. The standards in this NPRM would:
     Require interim and final assessments of the performance 
of recipients of grants, cooperative agreements, and contracts.
     Establish procedures for selecting peer review panels to 
conduct these assessments.
     Establish procedures and criteria that the peer review 
panels use in conducting these assessments.
     Establish specific additional criteria that peer review 
panels use in conducting these assessments for National Research and 
Development Centers, Regional Educational Laboratories, Field-Initiated 
Studies, and ERIC Clearinghouses.
    In an effort to fulfill the law's intention of ensuring high-
quality research, development, and evaluation, OERI has developed 
standards in which interim and final assessments may be supplemented by 
a self-assessment by the recipient of a grant, cooperative, agreement, 
or contract. The Board and the Assistant Secretary believe that the 
collection and review of evidence on one's own performance is itself a 
useful tool for improvement.
    These standards cover all grants, cooperative agreements, and 
contracts administered by OERI, ranging from the smallest purchase 
orders and commissioned papers to the largest research projects and 
research centers. The Department will require a single interim 
assessment by a peer review panel for total awards of $5,000,000 or 
less. At least one interim review by peer review panel will be required 
for larger awards. A final assessment by a peer review panel will be 
required for all awards.
    The Government Performance and Results Act requires the 
establishment of performance indicators for Department activities. 
Information collected pursuant to those indicators will be considered, 
as appropriate, in the evaluation of individual recipients.

Executive Order 12866

    1. Potential Costs and Benefits
    These proposed regulations have been reviewed in accordance with 
Executive Order 12866. Under the terms of the order the Secretary has 
assessed the potential costs and benefits of this regulatory action.
    The potential costs associated with the proposed regulations are 
those resulting from statutory requirements and those determined by the 
Secretary as necessary for administering this program effectively and 
efficiently. Burdens specifically associated with information 
collection requirements are identified and explained elsewhere in this 
preamble under the heading Paperwork Reduction Act of 1995.
    In assessing the potential costs and benefits--both quantitative 
and qualitative--of these proposed regulations, the Secretary has 
determined that the benefits of the proposed regulations justify the 
costs.
    The Secretary has also determined that this regulatory action does 
not unduly interfere with State, local, and tribal governments in the 
exercise of their governmental functions.
    To assist the Department in complying with the specific 
requirements of Executive Order 12866, the Secretary invites comment on 
whether there may be further opportunities to reduce any potential 
costs or increase potential benefits resulting from these proposed 
regulations without impeding the effective and efficient administration 
of the program.
Summary of Potential Costs and Benefits
    The potential costs of the proposed regulations are discussed in 
this preamble under the Paperwork Reduction Act of 1995. The benefit of 
these standards is to ensure that the research, development, and 
dissemination activities carried out by the recipients of grants from 
and contracts and cooperative agreements with OERI meet the highest 
standards of professional excellence.

2. Clarity of the Regulations

    Executive Order 12866 requires each agency to write regulations 
that are easy to understand.
    The Secretary invites comments on how to make these proposed 
regulations easier to understand, including answers to questions such 
as the following: (1) Are the requirements in the proposed regulations 
clearly stated? (2) Do the proposed regulations contain technical terms 
or other wording that interferes with their clarity? (3) Does the 
format of the proposed regulations (grouping and order of sections, use 
of headings, paragraphing, etc.) aid or reduce their clarity? Would the 
proposed regulations be easier to understand if they were divided into 
more (but shorter) sections? (A ``section'' is preceded by the symbol 
``Sec. '' and a numbered heading; for example, Sec. 702.2 What 
activities must be evaluated by these standards?) (4) Is the 
description of the proposed regulations in the ``Supplementary 
Information'' section of this preamble helpful in understanding the 
proposed regulations? How could this description be more helpful in 
making the proposed regulations easier to understand? (5) What else 
could the Department do to make the proposed regulations easier to 
understand?
    A copy of any comments that concern how the Department could make 
these proposed regulations easier to understand should be sent to 
Stanley M. Cohen, Regulations Quality Officer, U.S. Department of 
Education, 600 Independence Avenue, SW. (Room 5121, FB-10), Washington, 
D.C. 20202-2241.

Regulatory Flexibility Act Certification

    The Secretary certifies that these proposed regulations would not 
have a significant economic impact on a substantial number of small 
entities.
    The small entities that would be affected by these proposed 
regulations are small local educational agencies (LEAs) and private 
schools receiving Federal funds under this program. However, the 
regulations would not have a significant economic impact on the small 
LEAs and private schools affected because the proposed regulations 
would not impose excessive regulatory burdens or require unnecessary 
Federal supervision. The proposed regulations would impose minimal 
requirements to ensure the proper expenditure of program funds.

Paperwork Reduction Act of 1995

    Sections 702.22 and 702.23 contain information collection 
requirements. As required by the Paperwork Reduction Act of 1995 (44 
U.S.C. 3504(h)), the Department of Education has submitted a copy of 
these sections to the Office of Management and Budget (OMB) for its 
review.
    Collection of Information: Standards for Evaluation of the 
Performance of Recipients of OERI Grants, Cooperative Agreements, and 
Contracts.
    These regulations affect the following types of entities eligible 
to enter into

[[Page 9394]]

grants, cooperative agreements, or contracts: any public or private 
agency, organization or institution, or individual.
    The public reporting burden is estimated to range from 8 to 120 
hours for each interim or final assessment. The actual burden will be 
determined by how much descriptive information each recipient wishes to 
provide.
    Organizations and individuals desiring to submit comments on the 
information collection requirements should direct them to the Office of 
Information and Regulatory Affairs, OMB, room 10235, New Executive 
Office Building, Washington, DC 20503; Attention: Desk Officer for U.S. 
Department of Education.
    The Department considers comments by the public on this proposed 
collection of information in--
     Evaluating whether the proposed collection of information 
is necessary for the proper performance of the functions of the 
Department, including whether the information will have practical use;
     Evaluating the accuracy of the Department's estimate of 
the burden of the proposed collection of information, including the 
validity of the methodology and assumptions used;
     Enhancing the quality, usefulness, and clarity of the 
information to be collected; and
     Minimizing the burden of the collection of information on 
those who are to respond, including through the use of appropriate 
automated, electronic, mechanical, or other technological collection 
techniques or other forms of information technology; e.g., permitting 
electronic submission of responses.
    OMB is required to make a decision concerning the collection of 
information contained in these proposed regulations between 30 and 60 
days after publication of this document in the Federal Register. 
Therefore, a comment to OMB is best assured of having its full effect 
if OMB receives it within 30 days of publication. This does not affect 
the deadline for the public to comment to the Department on the 
proposed regulations.

Assessment of Educational Impact

    The Secretary particularly requests comments on whether the 
proposed regulations in this document would require transmission of 
information that is being gathered by or is available from any other 
agency or authority of the United States.

Electronic Access to This Document

    Anyone may view this document, as well as all other Department of 
Education documents published in the Federal Register, in text or 
portable document format (pdf) on the World Wide Web at either of the 
following sites:

http://gcs.ed.gov/fedreg.htm
http://www.ed.gov/news.html

    To use the pdf you must have the Adobe Acrobat Reader Program with 
Search, which is available free at either of the previous sites. If you 
have questions about using the pdf, call the U.S. Government Printing 
Office toll free at 1-888-293-6498.
    Anyone may also view these documents in text copy only on an 
electronic bulletin board of the Department. Telephone: (202) 219-1511 
or, toll free, 1-800-222-4922. The documents are located under Option 
G-- Files/Announcements, Bulletins and Press Releases.

    Note: The official version of this document is the document 
published in the Federal Register.

List of Subjects in 34 CFR Part 702

    Education, Educational research, Reporting and recordkeeping 
requirements.

    Dated: December 23, 1997.
Ricky Takai,
Acting Assistant Secretary for Educational Research and Improvement.

(Catalog of Federal Domestic Assistance Number does not apply)

    The Secretary proposes to amend Chapter VII of Title 34 of the 
Code of Federal Regulations by adding a new part 702 to read as 
follows:

PART 702--STANDARDS FOR CONDUCT AND EVALUATION OF ACTIVITIES CARRIED 
OUT BY THE OFFICE OF EDUCATIONAL RESEARCH AND IMPROVEMENT (OERI)--
EVALUATION OF THE PERFORMANCE OF RECIPIENTS OF GRANTS, COOPERATIVE 
AGREEMENTS, AND CONTRACTS

Subpart A--General

Sec.
702.1  What is the purpose of these standards?
702.2  What activities must be evaluated by these standards?
702.3  What additional activities may be evaluated by these 
standards?
702.4  When is performance assessed under these standards?
702.5  What definitions apply?

Subpart B--Selection of Peer Review Panels

702.10  What are the characteristics of peer reviewers?
702.11  What constitutes a conflict of interest for grants and 
cooperative agreements?
702.12  What constitutes a conflict of interest for contracts?
702.13  How are peer reviewers selected for panels?

Subpart C--The Evaluation Process

702.21  How does a peer review panel evaluate the performance of a 
recipient?
702.22  What information does a peer review panel consider for an 
interim assessment?
702.23  What information does a peer review panel consider for a 
final assessment?
702.24  What evaluation criteria are used for performance 
assessments?

    Authority: 20 U.S.C. 6011(i), unless otherwise noted.

Subpart A--General


Sec. 702.1  What is the purpose of these standards?

    (a) The standards in this part implement section 912(i) of the 
Educational Research, Development, Dissemination, and Improvement Act 
of 1994 (the Act).
    (b) These standards are intended to ensure that the research, 
development, and dissemination activities carried out by the recipients 
of grants from and contracts and cooperative agreements with the Office 
of Educational Research and Improvement (OERI) meet the highest 
standards of professional excellence.

(Authority: 20 U.S.C. 6011(i)(2)(F))


Sec. 702.2  What activities must be evaluated by these standards?

    These standards apply to activities carried out by OERI using funds 
appropriated under section 912(m) of the Act including activities 
carried out by the following entities or programs:
    (a) The National Education Research Institutes.
    (b) The Office of Reform Assistance and Dissemination.
    (c) The Educational Resources Information Center.
    (d) The Regional Educational Laboratories.
    (e) The Teacher Research Dissemination Demonstration Program.
    (f) The Goals 2000 Community Partnerships Program.
    (g) The National Educational Research Policy and Priorities Board.

(Authority: 20 U.S.C. 6011(i)(1))

[[Page 9395]]

Sec. 702.3  What additional activities may be evaluated by these 
standards?

    (a) The Secretary may apply these standards to other activities 
funded by the Department.

(Authority: 20 U.S.C. 6011(I)(1))


Sec. 702.4  When is performance assessed under these standards?

    (a) The Secretary will assess the performance of recipients of OERI 
grants, contracts, and cooperative agreements subject to these 
standards during and at the conclusion of their period of performance.
    (b) The Department requires a single interim assessment by a peer 
review panel for total awards of $5,000,000 or less. At least one 
interim review by peer review panel is required for larger awards.
    (c) A final assessment by a peer review panel is required for all 
awards.
    (d) As used in this part--
    (1) Interim assessment is one conducted during a recipient's period 
of performance.
    (2) Final assessment is one conducted at the conclusion of a 
recipient's period of performance.

(Authority: 20 U.S.C. 6011(I)(2)(F))


Sec. 702.5  What definitions apply?

    (a) Definitions in the Educational Research, Development, 
Dissemination, and Improvement Act of 1994.
    The following terms used in this part are defined in 20 U.S.C. 
6011(1):

Development
Dissemination
Educational research

    (b) Definitions in the Education Department General Administrative 
Regulations. The following terms used in this part are defined in 34 
CFR 77.1:

Application
Award
Department
Grant
Project
Secretary

    (c) Definitions in the Federal Acquisition Regulation. The 
following term used in this part is defined in 48 CFR Chapter 1:

Contract Proposal

(Authority: 20 U.S.C. 6011(I)(2)(F))

Subpart B--Selection of Peer Review Panels


Sec. 702.10  What are the characteristics of peer reviewers?

    (a) The Assistant Secretary selects each peer reviewer. Each peer 
reviewer must have the necessary knowledge and expertise in the area of 
the project being reviewed to evaluate the performance of a recipient. 
This experience may include--
    (1) Expert knowledge of subject matter in the area of the 
activities to be reviewed;
    (2) Expert knowledge of theory or methods or both in the area of 
the activities to be reviewed;
    (3) Practical experience in the area of the activities or type of 
institution or both to be reviewed;
    (4) Knowledge of a broad range of education policies and practices;
    (5) Experience in managing complex organizations; or
    (6) Expertise and experience in evaluation theory and practice.
    (b) Each peer reviewer must be free of conflict of interest, as 
determined in accordance with Sec. 702.11 or 702.12.
    (c) The Assistant Secretary may solicit nominations for peer 
reviewers from professional associations, nationally recognized 
experts, and other sources.
    (d) OERI and other Department staff who possess the qualifications 
in paragraphs (a) and (b) of this section may serve as peer reviewers.

(Authority: 20 U.S.C. 6011(I)(2)(B))


Sec. 702.11  What constitutes a conflict of interest for grants and 
cooperative agreements?

    A peer reviewer assessing the performance of the recipient of a 
grant from or cooperative agreement with OERI is considered an employee 
of the Department for the purposes of conflict of interest analysis. As 
an employee of the Department, the peer reviewer is subject to the 
provisions of 18 U.S.C. 208, 5 CFR 2635.502, and the Department's 
policies used to implement those provisions.

(Authority: 20 U.S.C. 6011(I)(2)(B))


Sec. 702.12  What constitutes a conflict of interest for contracts?

    A peer reviewer assessing the performance of the recipient of a 
contract with OERI is considered an employee of the Department in 
accordance with the Federal Acquisition Regulation (FAR), 48 CFR 3.104-
4(h)(2). As an employee of the Department, the peer reviewer is subject 
to the provisions of the FAR, 48 CFR Part 3, Improper Business 
Practices and Personal Conflict of Interest.

(Authority: 41 U.S.C. 423)


Sec. 702.13  How are peer reviewers selected for panels?

    (a) The Assistant Secretary assigns peer reviewers to panels that 
conduct the performance assessments.
    (b) The Assistant Secretary may establish panels by category of 
recipient, such as a panel to review the performance of all Regional 
Educational Laboratories. Each recipient is evaluated individually by 
reviewers who have been assigned to this type of panel.

(Authority: 20 U.S.C. 6011(I)(2)(B))

Subpart C--The Evaluation Process


Sec. 702.21  How does a peer review panel evaluate the performance of a 
recipient?

    (a) In each evaluation, a peer review panel--
    (1) Considers relevant information about the recipient's 
performance, as described in Secs. 702.22 and 702.23; and
    (2) Makes judgments about the recipient's performance, using the 
criteria in Sec. 702.24.
    (b) Each peer reviewer prepares a report based on the reviewer's 
assessment of the quality of the project according to the evaluation 
criteria.
    (c) After each peer reviewer has evaluated each project 
independently, the panel may be convened to discuss the strengths and 
weaknesses of the project. Each reviewer may then independently re-
evaluate each project with appropriate changes made to the written 
report.
    (d) The report of the interim assessment must include any 
recommendations the peer reviewer may have for improving the 
recipient's performance.
    (e) The report of the final assessment must contain each peer 
reviewer's evaluative summary of the recipient's performance, from the 
beginning of the contract, grant, or cooperative agreement to its 
conclusion.
(Authority: 20 U.S.C. 6011(I)(2)(F))


Sec. 702.22  What information does a peer review panel consider for an 
interim assessment?

    (a) Sources of information for the interim assessment must 
include--
    (1) The original request for proposals or grant announcement and 
the contract proposal or grant application;
    (2) Documentation of any changes in the work described in the 
contract, grant, or cooperative agreement, including reasons for the 
changes;
    (3) Any progress reports delivered to the Department or made 
available to the public by the recipient;
    (4) Examples of products delivered to the Department or made 
available to the public by the recipient;
    (5) Any relevant reports written by OERI staff, including reports 
of site visits by OERI staff;
    (6) Any performance evaluations conducted under the FAR or the 
Education Department General Administrative Regulations (34 CFR part 
75).

[[Page 9396]]

    (7) Any relevant information provided by the recipient in response 
to Government Performance and Results Act (GPRA) (Pub. L. 103-62) 
requirements; and
    (8) Any reports from program evaluations commissioned by the 
Department.
    (b) Sources of information for the interim assessment may also 
include--
    (1) A self-assessment, prepared by the recipient, addressing the 
criteria in Sec. 702.24;
    (2) One or more site visits by the peer review panel;
    (3) One or more oral or written presentations to the panel by the 
recipient describing its performance; or
    (4) Other information about the recipient's performance.

(Authority: 20 U.S.C. 6011(I)(2)(F))


Sec. 702.23  What information does a peer review panel consider for a 
final assessment?

    (a) Sources of information for the final assessment must include--
    (1) The original request for proposals or application notice and 
the contract proposal or grant application, together with documentation 
of any changes in the work described in the proposal or application, 
including reasons for the changes;
    (2) If consistent with the recipient's contract, grant, or 
cooperative agreement with OERI, a written report or oral presentation 
or both by the recipient summarizing its activities and 
accomplishments;
    (3) Any relevant information provided by the recipient in response 
to Government Performance and Results Act (GPRA) (Pub. L. 103-62) 
requirements; and
    (4) Any reports from program evaluations commissioned by the 
Department.
    (b) The final assessment may also include other sources of 
information, such as one or more of those listed in Sec. 702.22.

(Authority: 20 U.S.C. 6011(I)(2)(F))


Sec. 702.24  What evaluation criteria must be used for performance 
assessments?

    (a) Peer reviewers (and those recipients who conduct self-
evaluations) shall use the criteria in paragraph (b) of this section to 
assess performance and, in case of interim assessments, to identify 
areas in which the performance of recipients may need improvement.
    (b) The following evaluation criteria are to guide the assessment 
process undertaken by peer reviewers. The peer reviewers determine the 
extent to which recipients meet these criteria:
    (1) Implementation and management. (i) Peer reviewers shall 
consider the degree to which the recipient has fully executed its 
program of work. In doing so, peer reviewers shall consider evidence on 
the extent to which the recipient completes the work described in the 
approved application or contract, including any approved modifications, 
in the time period proposed and in an efficient manner.
    (ii) In examining the degree of implementation, peer reviewers may 
also consider evidence on the extent to which--
    (A) The recipient implements and utilizes a quality assurance 
system for its products or services or both; and
    (B) The recipient conducts self-assessment or self-evaluation 
activities, including periodically seeking out independent critiques 
and evaluations of its work, and uses the results to improve 
performance.
    (2) Quality. (i) Peer reviewers shall consider the degree to which 
the recipient's work approaches or attains professional excellence. In 
determining quality, peer reviewers shall consider evidence on the 
extent to which--
    (A) The recipient utilizes processes, methods, and techniques 
appropriate to achieve the goals and objectives for the program of work 
in the approved application; and
    (B) The recipient applies appropriate processes, methods, and 
techniques in a manner consistent with the highest standards of the 
profession.
    (ii) In determining quality, peer reviewers may also consider the 
extent to which the recipient conducts a coherent, sustained program of 
work informed by relevant research.
    (3) Utility. (i) In determining the utility of the recipient's 
products or services or both, peer reviewers shall consider evidence on 
the extent to which the recipient's work (including information, 
materials, processes, techniques, or activities) is effectively used by 
and is useful to its customers in appropriate settings.
    (ii) In determining utility, peer reviewers may also consider the 
extent to which the recipient has received national recognition; e.g., 
articles in refereed journals and presentations at professional 
conferences.
    (4) Outcomes and impact. (i) Peer reviewers shall consider the 
results of the recipient's work. In examining outcomes and impact, peer 
reviewers shall consider evidence on the extent to which--
    (A) The recipient meets the needs of its customers; and
    (B) The recipient's work contributes to the increased knowledge or 
understanding of educational problems, issues, or effective strategies.
    (ii) In examining outcomes and impact, peer reviewers may also 
consider the extent to which recipients address issues of national 
significance through its products or services or both.
    (c) For National Research and Development Centers, peer reviewers 
also shall consider evidence on the extent to which recipients meet the 
following criteria:
    (1) Quality. (i) The recipient uses a well-conceptualized framework 
and sound theoretical and methodological tools in conducting 
professionally rigorous studies; and
    (ii) The recipient conducts work of sufficient size, scope, and 
duration to produce sound guidance for improvement efforts and future 
research.
    (2) Utility. The recipient documents, reports, and disseminates its 
work in ways to facilitate the effective use of its work in 
appropriately targeted settings.
    (3) Outcomes and impact. (i) The recipient's work contributes to 
the development and advancement of theory in the field of study, 
including its priority area; and
    (ii) The recipient addresses issues of national significance 
through its products or services or both.
    (d) For the Regional Educational Laboratories, peer reviewers also 
shall consider evidence on the extent to which recipients meet the 
following criteria:
    (1) Quality. (i) The recipient utilizes a well-conceptualized 
framework and sound theoretical and methodological tools in conducting 
professionally rigorous studies;
    (ii) The recipient conducts work of sufficient size, scope, and 
duration to produce sound guidance for improvement efforts; and
    (iii) The recipient's products are well-tested and based on sound 
research.
    (2) Utility. The recipient documents, reports, and disseminates its 
work in ways to facilitate its effective use in appropriately targeted 
settings, particularly in school improvement efforts of States and 
localities.
    (3) Outcomes and impact. (i) The recipient assists States and 
localities to implement comprehensive school improvement strategies 
through the provision of research-based information (including well-
tested models and strategies), materials and assistance; and
    (ii) The recipient's work results in widespread access to 
information regarding research and best practices, particularly within 
its region.
    (e) For Field-Initiated Studies, peer reviewers also shall consider 
evidence on the extent to which recipients meet the following criteria:

[[Page 9397]]

    (1) Implementation and management. The recipient's work responds to 
the goals, objectives and mission of the National Institute from which 
it is funded.
    (2) Quality. The recipient utilizes a well-conceptualized framework 
and sound theoretical and methodological tools in conducting 
professionally rigorous studies.
    (3) Utility. The recipient documents, reports, and disseminates its 
work in ways to facilitate its effective use in appropriately targeted 
settings.
    (4) Outcomes and impact. (i) The recipient's work contributes to 
the development and advancement of theory and knowledge in the field of 
study; and
    (ii) The recipient addresses issues of national significance 
through its products or services or both.
    (f) For the ERIC Clearinghouses, peer reviewers also shall consider 
evidence on the extent to which recipients meet the following criteria:
    (1) Quality. The recipient applies an integrated approach to 
acquiring and disseminating significant and high-quality educational 
literature and materials to maintain and enhance the ERIC database.
    (2) Utility. The recipient contributes to the development of the 
ERIC database as a source of literature and materials that reflects 
trends and issues within its scope.
    (3) Outcomes and impact. (i) The recipient meets the informational 
and educational needs of its customers through dissemination and 
outreach approaches and the development of an array of print and non-
print materials; and
    (ii) The recipient provides national leadership on the use of 
current computer, networking, and information technology.

(Authority: 20 U.S.C. 6011(I)(2)(F))

[FR Doc. 98-4690 Filed 2-23-98; 8:45 am]
BILLING CODE 4000-01-P