[Federal Register Volume 80, Number 12 (Tuesday, January 20, 2015)]
[Rules and Regulations]
[Pages 2606-2609]
From the Federal Register Online via the Government Publishing Office [www.gpo.gov]
[FR Doc No: 2015-00463]


=======================================================================
-----------------------------------------------------------------------

DEPARTMENT OF EDUCATION

34 CFR Parts 75 and 77

RIN 1855-AA10
[Docket ID ED-2014-OII-0116]


Direct Grant Programs and Definitions That Apply to Department 
Regulations

AGENCY: Department of Education.

ACTION: Final regulations.

-----------------------------------------------------------------------

SUMMARY: On August 13, 2013, the Department of Education (the 
Department) published final regulations in the Federal Register to 
amend the Education Department General Administrative Regulations 
(EDGAR). On October 22, 2014, the Department published a notice of 
proposed rulemaking to make additional amendments to EDGAR. In this 
document, the Department amends EDGAR to add a definition of ``What 
Works Clearinghouse Evidence Standards'' (WWC Evidence Standards) in 
our regulations to standardize references to this term. In addition, 
the Department amends the definition of ``large sample'' in our 
regulation. We also make technical edits to our regulations to improve 
the consistency and clarity of the regulations. Finally, we redesignate 
a section of our regulations and include in that redesignated section 
an additional provision that allows the Secretary to give special 
consideration to projects supported by evidence of promise.

DATES: These regulations are effective February 19, 2015.

FOR FURTHER INFORMATION CONTACT: Allison Moss, U.S. Department of 
Education, 400 Maryland Avenue SW., Room 4W319, Washington, DC 20202. 
Telephone: (202) 205-7726 or by email: [email protected].
    If you use a telecommunications device for the deaf (TDD) or text 
telephone (TTY), call the Federal Relay Service (FRS), toll free, at 1-
800-877-8339.

SUPPLEMENTARY INFORMATION: In this document, we amend EDGAR to 
standardize a term and improve the consistency and clarity of our 
regulations.
    Summary of the Major Provisions of This Regulatory Action: The 
Department revises EDGAR to include a definition for ``What Works 
Clearinghouse Evidence Standards,'' provide the Secretary the 
flexibility to establish a separate competition for or award 
competitive preference to discretionary grant applications supported by 
evidence of promise, and revise certain definitions to improve clarity.

References to the WWC Handbook

    The Department adds a definition of ``What Works Clearinghouse 
Evidence Standards'' to 34 CFR part 77. This definition incorporates 
the most recent version of the What Works Clearinghouse (WWC) 
Procedures and Standards Handbook (WWC Handbook), Version 3.0, which 
was made public in March 2014. Instead of continuing to separately cite 
the WWC Handbook in various provisions of parts 75 and 77, we add, to 
part 77, a single definition of the WWC Evidence Standards that 
incorporates the current version of the WWC Handbook, and then include 
that defined term, as applicable, throughout parts 75 and 77.
    The WWC Handbook, first published in 2008, documents the systematic 
review process and the standards by which the WWC reviews studies. 
Version 3.0 of the WWC Handbook significantly expands the examples used 
to illustrate how the WWC Evidence Standards are applied in various 
contexts. Although previous versions of the WWC Handbook focused on 
only one WWC product--the intervention report--Version 3.0 includes 
information on several additional WWC products, including practice 
guides, single-study reviews, and quick reviews.
    By adding a definition of ``WWC Evidence Standards'' and updating 
the applicable references throughout 34 CFR parts 75 and 77 to 
incorporate the most recent version of the WWC Handbook, the Department 
will provide more effective guidance to applicants and grantees as they 
design and implement rigorous evaluations of their projects.

Special Consideration for Discretionary Grant Applications 
Demonstrating ``Evidence of Promise''

    These final regulations amend Sec.  75.266 and redesignate it as 
Sec.  75.226. Previously, this section provided that the Secretary may 
give special consideration, through establishing a separate competition 
or awarding competitive preference, to discretionary grant applications 
supported by strong evidence of effectiveness or moderate evidence of 
effectiveness. In our experience using evidence in discretionary grant 
competitions, we have learned it is beneficial to also include in 34 
CFR 75.266 (which we have redesignated as 34 CFR 75.226) a provision 
for giving special consideration to applications supported by evidence 
of promise, which is a less rigorous standard, because evidence of 
effectiveness in the education field continues to develop. By including 
evidence of promise in newly redesignated 34 CFR 75.226, we allow more 
flexibility to discretionary grant programs oriented towards supporting 
evidence-based projects.

Definition of ``Evidence of Promise''

    We amend the definition of ``evidence of promise'' to replace the 
reference to ``quasi-experimental study'' with

[[Page 2607]]

``quasi-experimental design study,'' to clarify that the term used in 
the definition of ``evidence of promise'' is ``quasi-experimental 
design study,'' which is defined later in 34 CFR 77.1(c). We also 
change the paragraph designations in this definition for internal 
consistency. We make these changes in order to align the definition of 
``evidence of promise'' so it is consistent with the other defined 
terms to which it makes reference and to ensure that applicants and 
grantees receive consistent and clear information when referencing that 
definition.

Definition of ``Large Sample''

    The Department modifies the definition of ``large sample'' in 34 
CFR part 77 to remove the requirement that analysis units be randomly 
assigned to treatment or control groups. In implementing our 
discretionary grant programs, we discovered a discrepancy between the 
existing definition, specifically its references to random assignment 
of students, teachers, classrooms, schools, or other single analysis 
units to treatment or control groups, and the definition of ``moderate 
evidence of effectiveness'' in 34 CFR 77.1. Under the definition of 
``moderate evidence of effectiveness,'' a quasi-experimental design 
study (as defined in 34 CFR 77.1) that includes a large sample could 
meet the standard, but many such studies do not randomly assign units 
of analysis to treatment or control groups. We revise the definition of 
``large sample'' to eliminate the random assignment of analysis units 
into treatment or control groups as a mandatory element. Therefore, for 
instance, a quasi-experimental design study with a sample of 350 or 
more students (or other single analysis units), or 50 or more groups 
(such as classrooms or schools) that contains 10 or more students, 
could meet the definition of ``moderate evidence of effectiveness'' in 
34 CFR 77.1.
    There are no differences between the NPRM and these final 
regulations.
    Public Comment: In response to our invitation in the NPRM, we did 
not receive any comments on the proposed regulations.

Executive Orders 12866 and 13563 Regulatory Impact Analysis

    Under Executive Order 12866, the Secretary must determine whether 
this regulatory action is ``significant'' and, therefore, subject to 
the requirements of the Executive order and subject to review by the 
Office of Management and Budget (OMB). Section 3(f) of Executive Order 
12866 defines a ``significant regulatory action'' as an action likely 
to result in a rule that may--
    (1) Have an annual effect on the economy of $100 million or more, 
or adversely affect a sector of the economy, productivity, competition, 
jobs, the environment, public health or safety, or State, local, or 
tribal governments or communities in a material way (also referred to 
as an ``economically significant'' rule);
    (2) Create serious inconsistency or otherwise interfere with an 
action taken or planned by another agency;
    (3) Materially alter the budgetary impacts of entitlement grants, 
user fees, or loan programs or the rights and obligations of recipients 
thereof; or
    (4) Raise novel legal or policy issues arising out of legal 
mandates, the President's priorities, or the principles stated in the 
Executive order.
    This final regulatory action is not a significant regulatory action 
subject to review by OMB under section 3(f) of Executive Order 12866.
    We have also reviewed these regulations under Executive Order 
13563, which supplements and explicitly reaffirms the principles, 
structures, and definitions governing regulatory review established in 
Executive Order 12866. To the extent permitted by law, Executive Order 
13563 requires that an agency--
    (1) Propose or adopt regulations only on a reasoned determination 
that their benefits justify their costs (recognizing that some benefits 
and costs are difficult to quantify);
    (2) Tailor its regulations to impose the least burden on society, 
consistent with obtaining regulatory objectives and taking into 
account--among other things and to the extent practicable--the costs of 
cumulative regulations;
    (3) In choosing among alternative regulatory approaches, select 
those approaches that maximize net benefits (including potential 
economic, environmental, public health and safety, and other 
advantages; distributive impacts; and equity);
    (4) To the extent feasible, specify performance objectives, rather 
than the behavior or manner of compliance a regulated entity must 
adopt; and
    (5) Identify and assess available alternatives to direct 
regulation, including economic incentives--such as user fees or 
marketable permits--to encourage the desired behavior, or provide 
information that enables the public to make choices.
    Executive Order 13563 also requires an agency ``to use the best 
available techniques to quantify anticipated present and future 
benefits and costs as accurately as possible.'' The Office of 
Information and Regulatory Affairs of OMB has emphasized that these 
techniques may include ``identifying changing future compliance costs 
that might result from technological innovation or anticipated 
behavioral changes.''
    We are issuing these final regulations only on a reasoned 
determination that their benefits justify their costs. In choosing 
among alternative regulatory approaches, we selected those approaches 
that maximize net benefits. Based on the analysis that follows, the 
Department believes that these final regulations are consistent with 
the principles in Executive Order 13563.
    We also have determined that this regulatory action does not unduly 
interfere with State, local, or tribal governments in the exercise of 
their governmental functions.
    In accordance with both Executive orders, the Department has 
assessed the potential costs and benefits, both quantitative and 
qualitative, of this regulatory action. The potential costs associated 
with this regulatory action are those resulting from statutory 
requirements and those we have determined as necessary for 
administering the Department's programs and activities.

Paperwork Reduction Act of 1995

    These regulations do not contain any information collection 
requirements.

Intergovernmental Review

    These regulations affect direct grant programs of the Department, 
some of which are subject to Executive Order 12372 and the regulations 
in 34 CFR part 79. One of the objectives of the Executive order is to 
foster an intergovernmental partnership and a strengthened federalism. 
The Executive order relies on processes developed by State and local 
governments for coordination and review of proposed Federal financial 
assistance.
    This document provides early notification of our specific plans and 
actions for these programs.

Assessment of Educational Impact

    In the NPRM we requested comments on whether the proposed 
regulations would require transmission of information that any other 
agency or authority of the United States gathers or makes available.
    Based on the response to the NPRM and on our review, we have 
determined that these final regulations do not require transmission of 
information that any other agency or authority of the United States 
gathers or makes available.

[[Page 2608]]

    Accessible Format: Individuals with disabilities can obtain this 
document in an accessible format (e.g., braille, large print, 
audiotape, or compact disc) on request to the person listed under FOR 
FURTHER INFORMATION CONTACT.
    Electronic Access to This Document: The official version of this 
document is the document published in the Federal Register. Free 
Internet access to the official edition of the Federal Register and the 
Code of Federal Regulations is available via the Federal Digital System 
at: www.gpo.gov/fdsys. At this site you can view this document, as well 
as all other documents of this Department published in the Federal 
Register, in text or Adobe Portable Document Format (PDF). To use PDF 
you must have Adobe Acrobat Reader, which is available free at the 
site.
    You may also access documents of the Department published in the 
Federal Register by using the article search feature at: 
www.federalregister.gov. Specifically, through the advanced search 
feature at this site, you can limit your search to documents published 
by the Department. (Catalog of Federal Domestic Assistance Number does 
not apply.)

List of Subjects

34 CFR Part 75

    Accounting, Copyright, Education, Grant programs--education, 
Inventions and patents, Private schools, Reporting and recordkeeping 
requirements.

34 CFR Part 77

    Education, Grant programs--education.

    Dated: January 9, 2015.
Arne Duncan,
Secretary of Education.

    For the reasons discussed in the preamble, the Secretary amends 
parts 75 and 77 of title 34 of the Code of Federal Regulations as 
follows:

PART 75--DIRECT GRANT PROGRAMS

0
1. The authority citation for part 75 continues to read as follows:

    Authority: 20 U.S.C. 1221e-3 and 3474, unless otherwise noted.


0
2. Section 75.210 is amended by revising paragraphs (h)(2)(viii) and 
(ix), and removing footnotes 1 and 2, to read as follows.


Sec.  75.210  General selection criteria.

* * * * *
    (h) * * *
    (2) * * *
    (viii) The extent to which the methods of evaluation will, if well-
implemented, produce evidence about the project's effectiveness that 
would meet the What Works Clearinghouse Evidence Standards without 
reservations.
    (ix) The extent to which the methods of evaluation will, if well-
implemented, produce evidence about the project's effectiveness that 
would meet the What Works Clearinghouse Evidence Standards with 
reservations.
* * * * *

0
3. Section 75.266 is redesignated as Sec.  75.226 and newly 
redesignated Sec.  75.226 is revised to read as follows:


75.226   What procedures does the Secretary use if the Secretary 
decides to give special consideration to applications supported by 
strong evidence of effectiveness, moderate evidence of effectiveness, 
or evidence of promise?

    (a) As used in this section, ``strong evidence of effectiveness'' 
is defined in 34 CFR 77.1(c);
    (b) As used in this section, ``moderate evidence of effectiveness'' 
is defined in 34 CFR 77.1(c);
    (c) As used in this section, ``evidence of promise'' is defined in 
34 CFR 77.1(c); and
    (d) If the Secretary determines that special consideration of 
applications supported by strong evidence of effectiveness, moderate 
evidence of effectiveness, or evidence of promise is appropriate, the 
Secretary may establish a separate competition under the procedures in 
34 CFR 75.105(c)(3), or provide competitive preference under the 
procedures in 34 CFR 75.105(c)(2), for applications supported by:
    (1) Evidence of effectiveness that meets the conditions set out in 
paragraph (a) of the definition of ``strong evidence of effectiveness'' 
in 34 CFR 77.1(c);
    (2) Evidence of effectiveness that meets the conditions set out in 
either paragraph (a) or (b) of the definition of ``strong evidence of 
effectiveness'' in 34 CFR 77.1(c);
    (3) Evidence of effectiveness that meets the conditions set out in 
the definition of ``moderate evidence of effectiveness;'' or
    (4) Evidence of effectiveness that meets the conditions set out in 
the definition of ``evidence of promise.''

    Authority: 20 U.S.C. 1221e-3 and 3474.

PART 77--DEFINITIONS THAT APPLY TO DEPARTMENT REGULATIONS

0
4. The authority citation for part 77 continues to read as follows:

    Authority: 20 U.S.C. 1221e-3 and 3474, unless otherwise noted.


0
5. In Sec.  77.1 paragraph (c) is amended by:
0
A. Revising the definitions of Evidence of promise, Large sample, 
Moderate evidence of effectiveness, Quasi-experimental design study, 
Randomized controlled trial, and Strong evidence of effectiveness.
0
B. Adding, in alphabetical order, the definition of What Works 
Clearinghouse Evidence Standards.
0
C. Removing footnotes 1 through 8.
    The revisions and addition read as follows:


Sec.  77.1  Definitions that apply to all Department programs.

* * * * *
    (c) * * *
* * * * *
    Evidence of promise means there is empirical evidence to support 
the theoretical linkage(s) between at least one critical component and 
at least one relevant outcome presented in the logic model for the 
proposed process, product, strategy, or practice. Specifically, 
evidence of promise means the conditions in both paragraphs (i) and 
(ii) of this definition are met:
    (i) There is at least one study that is a--
    (A) Correlational study with statistical controls for selection 
bias;
    (B) Quasi-experimental design study that meets the What Works 
Clearinghouse Evidence Standards with reservations; or
    (C) Randomized controlled trial that meets the What Works 
Clearinghouse Evidence Standards with or without reservations.
    (ii) The study referenced in paragraph (i) of this definition found 
a statistically significant or substantively important (defined as a 
difference of 0.25 standard deviations or larger) favorable association 
between at least one critical component and one relevant outcome 
presented in the logic model for the proposed process, product, 
strategy, or practice.
* * * * *
    Large sample means an analytic sample of 350 or more students (or 
other single analysis units), or 50 or more groups (such as classrooms 
or schools) that contain 10 or more students (or other single analysis 
units).
* * * * *
    Moderate evidence of effectiveness means one of the following 
conditions is met:
    (i) There is at least one study of the effectiveness of the 
process, product, strategy, or practice being proposed that meets the 
What Works Clearinghouse Evidence Standards without reservations, found 
a statistically significant favorable impact on a

[[Page 2609]]

relevant outcome (with no statistically significant and overriding 
unfavorable impacts on that outcome for relevant populations in the 
study or in other studies of the intervention reviewed by and reported 
on by the What Works Clearinghouse), and includes a sample that 
overlaps with the populations or settings proposed to receive the 
process, product, strategy, or practice.
    (ii) There is at least one study of the effectiveness of the 
process, product, strategy, or practice being proposed that meets the 
What Works Clearinghouse Evidence Standards with reservations, found a 
statistically significant favorable impact on a relevant outcome (with 
no statistically significant and overriding unfavorable impacts on that 
outcome for relevant populations in the study or in other studies of 
the intervention reviewed by and reported on by the What Works 
Clearinghouse), includes a sample that overlaps with the populations or 
settings proposed to receive the process, product, strategy, or 
practice, and includes a large sample and a multi-site sample.

    Note:  Multiple studies can cumulatively meet the large and 
multi-site sample requirements as long as each study meets the other 
requirements in this paragraph.

)* * * * *
    Quasi-experimental design study means a study using a design that 
attempts to approximate an experimental design by identifying a 
comparison group that is similar to the treatment group in important 
respects. These studies, depending on design and implementation, can 
meet What Works Clearinghouse Evidence Standards with reservations (but 
not What Works Clearinghouse Evidence Standards without reservations).
* * * * *
    Randomized controlled trial means a study that employs random 
assignment of, for example, students, teachers, classrooms, schools, or 
districts to receive the intervention being evaluated (the treatment 
group) or not to receive the intervention (the control group). The 
estimated effectiveness of the intervention is the difference between 
the average outcomes for the treatment group and for the control group. 
These studies, depending on design and implementation, can meet What 
Works Clearinghouse Evidence Standards without reservations.
* * * * *
    Strong evidence of effectiveness means one of the following 
conditions is met:
    (i) There is at least one study of the effectiveness of the 
process, product, strategy, or practice being proposed that meets the 
What Works Clearinghouse Evidence Standards without reservations, found 
a statistically significant favorable impact on a relevant outcome 
(with no statistically significant and overriding unfavorable impacts 
on that outcome for relevant populations in the study or in other 
studies of the intervention reviewed by and reported on by the What 
Works Clearinghouse), includes a sample that overlaps with the 
populations and settings proposed to receive the process, product, 
strategy, or practice, and includes a large sample and a multi-site 
sample.

    Note: Multiple studies can cumulatively meet the large and 
multi-site sample requirements as long as each study meets the other 
requirements in this paragraph.

    )(ii) There are at least two studies of the effectiveness of the 
process, product, strategy, or practice being proposed, each of which: 
Meets the What Works Clearinghouse Evidence Standards with 
reservations, found a statistically significant favorable impact on a 
relevant outcome (with no statistically significant and overriding 
unfavorable impacts on that outcome for relevant populations in the 
studies or in other studies of the intervention reviewed by and 
reported on by the What Works Clearinghouse), includes a sample that 
overlaps with the populations and settings proposed to receive the 
process, product, strategy, or practice, and includes a large sample 
and a multi-site sample.
* * * * *
    What Works Clearinghouse Evidence Standards means the standards set 
forth in the What Works Clearinghouse Procedures and Standards Handbook 
(Version 3.0, March 2014), which can be found at the following link: 
http://ies.ed.gov/ncee/wwc/DocumentSum.aspx?sid=19.
* * * * *
[FR Doc. 2015-00463 Filed 1-16-15; 8:45 am]
BILLING CODE 4000-01-P