[House Hearing, 113 Congress]
[From the U.S. Government Publishing Office]
EDUCATION RESEARCH: EXPLORING
OPPORTUNITIES TO STRENGTHEN THE
INSTITUTE OF EDUCATION SCIENCES
=======================================================================
HEARING
before the
COMMITTEE ON EDUCATION
AND THE WORKFORCE
U.S. HOUSE OF REPRESENTATIVES
ONE HUNDRED THIRTEENTH CONGRESS
FIRST SESSION
__________
HEARING HELD IN WASHINGTON, DC, SEPTEMBER 10, 2013
__________
Serial No. 113-30
__________
Printed for the use of the Committee on Education and the Workforce
[GRAPHIC(S) NOT AVAILABLE IN TIFF FORMAT]
Available via the World Wide Web:
www.gpo.gov/fdsys/browse/
committee.action?chamber=house&committee=education
or
Committee address: http://edworkforce.house.gov
_____
U.S. GOVERNMENT PRINTING OFFICE
82-662 PDF WASHINGTON : 2013
-----------------------------------------------------------------------
For sale by the Superintendent of Documents, U.S. Government Printing
Office Internet: bookstore.gpo.gov Phone: toll free (866) 512-1800; DC
area (202) 512-1800 Fax: (202) 512-2104 Mail: Stop IDCC, Washington, DC
20402-0001
COMMITTEE ON EDUCATION AND THE WORKFORCE
JOHN KLINE, Minnesota, Chairman
Thomas E. Petri, Wisconsin George Miller, California,
Howard P. ``Buck'' McKeon, Senior Democratic Member
California Robert E. Andrews, New Jersey
Joe Wilson, South Carolina Robert C. ``Bobby'' Scott,
Virginia Foxx, North Carolina Virginia
Tom Price, Georgia Ruben Hinojosa, Texas
Kenny Marchant, Texas Carolyn McCarthy, New York
Duncan Hunter, California John F. Tierney, Massachusetts
David P. Roe, Tennessee Rush Holt, New Jersey
Glenn Thompson, Pennsylvania Susan A. Davis, California
Tim Walberg, Michigan Raul M. Grijalva, Arizona
Matt Salmon, Arizona Timothy H. Bishop, New York
Brett Guthrie, Kentucky David Loebsack, Iowa
Scott DesJarlais, Tennessee Joe Courtney, Connecticut
Todd Rokita, Indiana Marcia L. Fudge, Ohio
Larry Bucshon, Indiana Jared Polis, Colorado
Trey Gowdy, South Carolina Gregorio Kilili Camacho Sablan,
Lou Barletta, Pennsylvania Northern Mariana Islands
Martha Roby, Alabama John A. Yarmuth, Kentucky
Joseph J. Heck, Nevada Frederica S. Wilson, Florida
Susan W. Brooks, Indiana Suzanne Bonamici, Oregon
Richard Hudson, North Carolina
Luke Messer, Indiana
Juliane Sullivan, Staff Director
Jody Calemine, Minority Staff Director
C O N T E N T S
----------
Page
Hearing held on September 10, 2013............................... 1
Statement of Members:
Kline, Hon. John, Chairman, Committee on Education and the
Workforce.................................................. 1
Prepared statement of.................................... 3
Miller, Hon. George, senior Democratic member, Committee on
Education and the Workforce................................ 3
Prepared statement of.................................... 5
Statement of Witnesses:
Christie, Kathy, vice president, knowledge/information
management & dissemination, Education Commission of the
States..................................................... 24
Prepared statement of.................................... 26
Kemple James J., executive director, Research Alliance for
New York City Schools, New York University................. 18
Prepared statement of.................................... 20
Long, Dr. Bridget Terry, Ph.D., Xander professor of education
and economics, academic dean, Harvard Graduate School of
Education; chair, National Board for Education Sciences.... 8
Prepared statement of.................................... 13
Scott, George A., Director for Education, Workforce, and
Income Security Issues, U.S. Government Accountability
Office..................................................... 6
Prepared statement of, Internet address to............... 8
Additional Submissions:
Dr. Long:
Document, ``Recommendations for the Reauthorization of
the Education Sciences Reform Act (ESRA)''............. 10
Document, ``NBES Markup, June 2012, Education Sciences
Reform, Public Law 107-279,'' Internet address to...... 13
EDUCATION RESEARCH: EXPLORING
OPPORTUNITIES TO STRENGTHEN THE
INSTITUTE OF EDUCATION SCIENCES
----------
Tuesday, September 10, 2013
U.S. House of Representatives
Committee on Education and the Workforce
Washington, DC
----------
The committee met, pursuant to call, at 10:01 a.m., in room
2175, Rayburn House Office Building, Hon. John Kline [chairman
of the committee] presiding.
Present: Representatives Kline, Petri, Foxx, Walberg,
Salmon, Rokita, Miller, Scott, Hinojosa, Tierney, Holt,
Grijalva, Polis, and Bonamici.
Staff present: Katherine Bathgate, Deputy Press Secretary;
James Bergeron, Director of Education and Human Services
Policy; Heather Couri, Deputy Director of Education and Human
Services Policy; Lindsay Fryer, Professional Staff Member;
Rosemary Lahasky, Professional Staff Member; Nancy Locke, Chief
Clerk; Krisann Pearce, General Counsel; Dan Shorts, Legislative
Assistant; Nicole Sizemore, Deputy Press Secretary; Alex
Sollberger, Communications Director; Alissa Strawcutter, Deputy
Clerk; Brad Thomas, Senior Education Policy Advisor; Tylease
Alli, Minority Clerk/Intern and Fellow Coordinator; Jeremy
Ayers, Minority Education Policy Advisor; Kelly Broughan,
Minority Education Policy Associate; Jamie Fasteau, Minority
Director of Education Policy; Scott Groginsky, Minority
Education Policy Advisor; Eunice Ikene, Minority Staff
Assistant; Brian Levin, Minority Deputy Press Secretary/New
Media Coordinator; and Megan O'Reilly, Minority General
Counsel.
Chairman Kline. A quorum being present, the committee will
come to order.
Good morning. Welcome to our hearing to discuss
opportunities to improve the Institute of Education Sciences.
I would like to extend a special welcome to our witnesses,
whose testimony will provide valuable insight into ways we can
better ensure parents, teachers, school leaders, and
policymakers have access to the most relevant education
research.
Established by the Education Sciences Reform Act in 2002,
the Institute of Education Sciences is responsible for
gathering information on education progress, conducting
research on educational practices in the nation's schools, and
examining the quality of federal education programs and
initiatives. The information collected and disseminated by the
institute helps schools identify and implement successful
education initiatives.
Additionally, the data allows taxpayers and congressional
leaders to keep tabs on the federal investment in education,
which is especially important in these times of fiscal
restraint. The Education Sciences Reform Act has been due for
reauthorization since 2008 and traditionally moves right after
the Elementary and Secondary Education Act. In July the House
approved the committee's legislation to rewrite ESEA, known as
the Student Success Act.
The Education Sciences Reform Act presents another
opportunity to help provide teachers and parents the tools
necessary to raise the bar in our schools and I look forward to
working with my colleagues on both sides of the aisle to
develop smart policies that will improve the law.
To lay groundwork for the reauthorization, last year
Ranking Member George Miller and I asked the Government
Accountability Office to conduct a study on the effectiveness
of the institute's research. Though the final report has yet to
be released, we have received a few preliminary findings that
highlight areas for improvement.
For example, GAO confirms the institute has greatly
improved the quality of education research over the last decade
but notes there is often a significant delay in disseminating
key data and findings to education officials. As a result, the
research is not always immediately relayed to parents and
school leaders, reducing its usefulness and relevancy.
GAO also found the institute does not always properly
evaluate the efficacy of its own programs and research arms,
which could lead to unnecessary costs, confusion, and
redundancies. Currently, the institute operates 10 regional
labs and 12 research and development centers to conduct
research, provide technical assistance, and distribute data.
Meanwhile, the Department of Education operates five content
centers and 16 comprehensive centers that serve some of the
same purposes.
As we develop policies to strengthen the institute, we
should consider streamlining the federal research structure to
reduce duplication, enhance accountability, and make it easier
for states and school districts to access important
information. We must also ensure the Institute of Education
Sciences has the flexibility necessary to modernize its
research methods and keep up with new developments in education
delivery and practice. Finally, we must acknowledge that the
value of the institute's research depends on its political
autonomy and take the necessary steps to protect the
organization's independence.
We are fortunate to have with us several witnesses who can
help us better understand what is and is not working with the
Institute of Education Sciences, including a representative
from GAO who can provide more information on the aforementioned
study. Their testimony will inform our efforts to reauthorize
the Education Sciences Reform Act and help us craft policies
that will improve the quality and usefulness of education
research.
With that, I now yield to my distinguished colleague,
George Miller, the senior Democratic member of the committee,
for his opening remarks.
[The statement of Chairman Kline follows:]
Prepared Statement of Hon. John Kline, Chairman,
Committee on Education and the Workforce
Established by the Education Sciences Reform Act in 2002, the
Institute of Education Sciences is responsible for gathering
information on education progress, conducting research on educational
practices in the nation's schools, and examining the quality of federal
education programs and initiatives.
The information collected and disseminated by the Institute helps
schools identify and implement successful education initiatives.
Additionally, the data allows taxpayers and congressional leaders to
keep tabs on the federal investment in education, which is especially
important in these times of fiscal restraint.
The Education Sciences Reform Act has been due for reauthorization
since 2008 and traditionally moves right after the Elementary and
Secondary Education Act. In July the House approved the committee's
legislation to rewrite ESEA, known as the Student Success Act. The
Education Sciences Reform Act presents another opportunity to help
provide teachers and parents the tools necessary to raise the bar in
our schools, and I look forward to working with my colleagues on both
sides of the aisle to develop smart policies that will improve the law.
To lay groundwork for the reauthorization, last year Ranking Member
George Miller and I asked the Government Accountability Office to
conduct a study on the effectiveness of the Institute's research.
Though the final report has yet to be released, we have received a few
preliminary findings that highlight areas for improvement.
For example, GAO confirms the Institute has greatly improved the
quality of education research over the last decade, but notes there is
often a significant delay in disseminating key data and findings to
education officials. As a result, the research is not always
immediately relayed to parents and school leaders, reducing its
usefulness and relevancy.
GAO also found the Institute does not always properly evaluate the
efficacy of its own programs and research arms, which could lead to
unnecessary costs, confusion, and redundancies. Currently the Institute
operates 10 regional labs and 12 research and development centers to
conduct research, provide technical assistance, and distribute data.
Meanwhile, the Department of Education operates five content centers
and 16 comprehensive centers that serve some of the same purposes.
As we develop policies to strengthen the Institute, we should
consider streamlining the federal research structure to reduce
duplication, enhance accountability, and make it easier for states and
school districts to access important information. We must also ensure
the Institute of Education Sciences has the flexibility necessary to
modernize its research methods and keep up with new developments in
education delivery and practice. Finally, we must acknowledge that the
value of the Institute's research depends on its political autonomy,
and take the necessary steps to protect the organization's
independence.
We are fortunate to have with us several witnesses who can help us
better understand what is and is not working within the Institute of
Education Sciences, including a representative from GAO who can provide
more information on the aforementioned study. Their testimony will
inform our efforts to reauthorize the Education Sciences Reform Act and
help us craft policies that will improve the quality and usefulness of
education research.
______
Mr. Miller. Thank you very much, Mr. Chairman, for holding
this important hearing.
As we begin the new school year our nation's schools stand
on the threshold of major transitions. Schools are implementing
new college and career ready standards, they are piloting new
assessments aligned to those standards, and teachers and
principals are being evaluated in new ways to measure and
improve their effectiveness. Districts and states are
implementing new accountability standards that focus intense
efforts on turning around their most struggling schools.
These are seismic changes. To ensure that children benefit,
we need to support robust research to identify what is working
and what can be improved.
When Congress passed the Education Sciences Reform Act in
2002 we did so to complement the bipartisan effort of No Child
Left Behind. We wanted to strengthen the quality and the rigor
of education research and we wanted to take advantage of the
new, rich data that would emerge from the NCLB requirements.
Eleven years later, thanks to NCLB and ESRA, we have a
wealth of data that can be used to measure what is working for
students, make corrections where things are not working, and
create ways to ensure continuous improvement in the future.
That is the task I hope this committee will take as we review
ESRA.
In 2002 the quality of education research was lacking. Much
of it was driven by politics rather than science. As a
response, we created the Institute of Education Sciences. Its
mission was to conduct scientifically rigorous research outside
the influence of politics or trend of the moment.
Since then, IES has crafted high standards for the research
it funds. It trains and supports researchers across the country
and the studies are peer reviewed. A decade later, education
research is more rigorous and sound.
However, research is not effective if it only answers
abstract questions or only published in professional journals.
Research must be relevant as well as rigorous. It must be
widely shared with those who work with students in order to
make a difference.
I am pleased that IES has taken steps in that direction. In
2012 IES overhauled the nation's 10 regional research labs. It
did so by connecting them with research alliances and
policymakers and practitioners. The alliances work with the
labs to identify pressing education problems in schools and
then the labs develop and test strategies for solving them.
Take New England, for example. The Northeastern regional
lab created a research alliance on early childhood. That
alliance gathered the region's early childhood stakeholders to
create a research agenda that focuses on standards,
assessments, and practices to improve early childhood
education. The resulting research will now be more useful to
practitioners that have a ready-made network for disseminating
it.
As we examine ESRA and the role of IES I hope that we will
keep this need for both rigor and relevance in mind. I hope
that we will keep in mind that we in Congress must be good
federal partners in this effort. That means we must provide
stable and sufficient resources to IES to do its job.
The IES budget in 2012 fell just short of $600 million.
That is less than 1 percent of our overall federal education
budget. Other fields invest far more in research and
development.
We should take a serious look at this. More money is not
always the answer, but sufficient money is.
Also, we must not forget that sequestration cuts this year
are limiting research right now, and another round of cuts
beginning in January could have a crippling impact in
destabilizing ongoing and vital research. I hope that my
colleagues will join me in seeking ways to invest in education
research in a smart way that avoids waste but also is sound and
avoids harmful austerity.
I thank all of the witnesses for appearing today and I am
pleased that there is such an interest and leadership in
addressing the quality of education research in America, and I
look forward to all of your testimony.
Thank you so much.
[The statement of Mr. Miller follows:]
Prepared Statement of Hon. George Miller, Senior Democratic Member,
Committee on Education and the Workforce
Chairman Kline, thank you for holding this important hearing. As we
begin a new school year, our nation's schools stand on the threshold of
major transitions.
Schools are implementing new college and career ready standards.
And they are piloting new assessments aligned to those standards.
Teachers and principals are being evaluated in new ways to measure and
improve their effectiveness. Districts and states are implementing new
accountability systems that focus intense efforts on turning around
their most struggling schools.
These are seismic changes. To ensure that children benefit, we need
to support robust research to identify what is working and what can be
improved.
When Congress passed the Education Sciences Reform Act in 2002, we
did so to complement the bipartisan effort of No Child Left Behind. We
wanted to strengthen the quality and rigor of education research. And
we wanted to take advantage of the new, rich data that would emerge
from NCLB requirements.
Eleven years later, thanks to NCLB and ESRA, we have a wealth of
data that can be used to measure what is working for students, make
corrections where things are not working, and create ways to ensure
continuous improvement in the future. That is the task I hope this
Committee will take up as we review ESRA.
In 2002, the quality of education research was lacking. Much of it
was driven by politics rather than science. As a response, we created
the Institute of Education Sciences. Its mission was to conduct
scientifically rigorous research, outside the influence of politics or
the trend of the moment.
Since then, IES has crafted high standards for the research it
funds. It trains and supports researchers across the country. Its
studies are peer reviewed. A decade later education research is more
rigorous and sound.
However, research is not effective if it only answers abstract
questions or is published in a professional journal. Research must be
relevant as well as rigorous. And it must be widely shared with those
who work with students, in order to make a difference.
I am pleased that IES has taken some steps in this direction. In
2012 IES overhauled the nation's 10 regional research labs. It did so
by connecting them with research alliances of policymakers and
practitioners. The alliances work with the labs to identify pressing
education problems in schools and then the labs develop and test
strategies for solving them.
Take New England, for example. The Northeastern regional lab
created a research alliance on early childhood. That alliance gathered
the region's early childhood stakeholders to create a research agenda
that focuses on standards, assessments, and practices to improve early
education. The resulting research will now be more useful to
practitioners and have a ready-made network for disseminating it.
As we examine ESRA and the role of IES, I hope we will keep this
need for both rigor and relevance in mind. I hope we also keep in mind
that we in Congress must be good federal partners in this effort. That
means we must provide stable and sufficient resources to IES to do its
job.
The IES budget for FY12 fell just short of $600 million. That is
less than 1 percent of our overall federal education budget. Other
fields invest far more in research and development. We should take a
serious look at this. More money is not always the answer, but
sufficient money is.
Also, we must not forget that sequestration cuts this year are
limiting research right now. And another round of cuts beginning in
January could have a crippling impact and destabilize ongoing and vital
research. I hope my colleagues will join me in seeking ways to invest
in education research in a smart way that avoids waste; but also in a
sound way that avoids harmful austerity.
I thank all the witnesses for appearing today. I am pleased that
there is such interest and leadership in addressing the quality of
education research in America.
I look forward to your testimony. I yield back.
______
Chairman Kline. Thank the gentleman.
Pursuant to committee rule 7(c), all committee members will
be permitted to submit written statements to be included in the
permanent hearing record, and without objection the hearing
record will remain open for 14 days to allow statements,
questions for the record, and other extraneous material
referenced during the hearing to be submitted in the official
hearing record.
It is now my pleasure to introduce our very distinguished
panel of witnesses.
First, Mr. George A. Scott is the director for education,
workforce, and income security with GAO. He has previously
testified before both the House and Senate on the agency's work
surrounding K-12 education and student financial aid programs.
Dr. Bridget Terry Long is the Xander professor of education
and economics and academic dean at the Harvard Graduate School
of Education. She is also chair of the National Board for
Education Sciences.
Dr. James Kemple is executive director of the Research
Alliance for New York City Schools at New York University.
And Ms. Kathy Christie is the vice president for knowledge
and information management and dissemination at the Education
Commission of the States.
Welcome, all of you. Before I recognize each of you to
provide your testimony let me remind you of our very nifty
lighting system here.
You will each have 5 minutes to present your testimony.
When you begin the light will turn green--no surprises here.
When you have a minute left the light will turn yellow, and
when your time is expired the light will turn red.
And I am reluctant to gavel a witness down but I would ask
you to try to wrap up at that point so that each of you has a
chance to provide your testimony and we have a chance to engage
in a discussion.
I will now like to recognize Mr. Scott for 5 minutes.
Sir, you are recognized.
STATEMENT OF GEORGE A. SCOTT, DIRECTOR FOR EDUCATION,
WORKFORCE, AND INCOME SECURITY ISSUES, U.S. GOVERNMENT
ACCOUNTABILITY OFFICE
Mr. Scott. Thank you.
Chairman Kline, Ranking Member Miller, and members of the
committee, thank you for inviting me here today to discuss the
preliminary results of our review of the Department of
Education's Institute of Education Sciences, IES.
The Education Sciences Reform Act outlines a broad mission
for IES, including to expand the knowledge and understanding of
education and to provide this information to a wide range of
stakeholders. My testimony will focus on the extent to which
IES supports high quality research, disseminates relevant
products to the education field, and coordinates within
education and with other federal agencies.
In summary, we found that since its creation IES has
substantially improved education research. In 2007 the Office
of Management and Budget concluded that IES had transformed the
quality and rigor of research within the Department of
Education. It had also increased the demand for scientifically
based evidence of effectiveness in the education field as a
whole.
Many stakeholders said that IES's research standards
improved the quality of and had a positive influence on
education research.
While IES has improved the quality of education research,
its research is sometimes of limited usefulness to policymakers
and practitioners. Some stakeholders told us that the
evaluations supported by IES may not be completed soon enough
to inform important policy decisions. For example, officials
from one organization said that IES's evaluation of the Race to
the Top and school improvement grant programs will not be
released in time to give the states opportunity to implement
lessons learned from these studies before funding for these
programs end.
To address concerns about the relevance of its research IES
is soliciting feedback from stakeholders. For example, IES
recently convened a group of state and local education
officials to discuss the strengths and weaknesses of the
regional education labs and the What Works Clearinghouse.
Despite these efforts, IES does not have a systematic
process for incorporating feedback from stakeholders into its
research agenda. We consider having such a process to be a key
element of promoting a sound federal research program.
Further, IES cannot demonstrate the impact of its efforts
to improve the quality and relevance of its research in some
areas. IES's performance measures no longer capture the full
range of IES research and priorities. In some cases these
measures are no longer relevant to managing the agency's
operations.
Additionally, IES does not publicly report on the
performance of the regional education labs, which constitute
one of the agency's largest investments. As we have previously
reported, without appropriate performance measures agencies may
be at risk for failing to achieve their stated goals.
IES's research and technical assistance group have taken
various steps to provide relevant research. They have engaged
policymakers and practitioners in planning research and
technical assistance activities. All three groups also use a
range of methods to disseminate their products.
Despite these efforts, stakeholders have raised concerns.
For example, some stakeholders said that they do not find
research by the regional education labs to be as relevant or as
timely as other sources of information. Stakeholders also noted
that further efforts are needed to better market the research
from these groups so that they reach intended audiences.
Finally, IES takes a number of steps to coordinate with
other federal agencies to increase the use of research evidence
in guiding funding decisions. IES also coordinates within
Education to facilitate collaboration among various research
programs.
However, despite these efforts, the Department of Education
faces challenges in funding and prioritizing evaluations.
According to Education officials, efforts to prioritize
evaluation projects are hindered in part because of statutory
requirements.
According to these officials, Education lacks the authority
to combine evaluation funds from programs across the department
and then use them to evaluate any other program. As a result,
some evaluations may not occur and high priority evaluations
may be delayed.
In conclusion, IES has made significant contributions to
strengthening the rigor of education research. However, it
could build on these efforts by continuing to improve its
ability to provide timely and relevant information.
In addition, with a systematic process for incorporating
stakeholder needs and more comprehensive and up-to-date
performance measures IES would position itself to more fully
achieve its mission. Also, the ability to prioritize
evaluations is critical to helping the Congress make informed
decisions about programs. As we complete our work we will
consider any recommendations needed to address these issues.
Mr. Chairman, this concludes my remarks. Thank you.
[The statement of Mr. Scott may be accessed at the
following Internet address:]
http://www.gao.gov/assets/660/657669.pdf
------
Chairman Kline. Thank you very much.
Dr. Long, you are recognized.
STATEMENT OF DR. BRIDGET TERRY LONG, XANDER PROFESSOR OF
EDUCATION AND ECONOMICS, ACADEMIC DEAN, HARVARD GRADUATE SCHOOL
OF EDUCATION, CHAIR, NATIONAL BOARD FOR EDUCATION SCIENCES,
INSTITUTE OF EDUCATION SCIENCES
Ms. Long. Thank you, and good morning.
Chairman Kline and members of the committee, thank you for
the opportunity to appear before you today. As noted, I am also
chair of the National Board for Education Sciences.
The board is independent of IES and we are tasked with
advising the IES director and reviewing and evaluating its
work. In this way, we provide a critical but also constructive
perspective on the activities of IES.
My testimony reflects the discussions and recommendations
by the board as well as my observations as an experienced
researcher.
As we work to raise student achievement, foster productive
learning environments, and bolster the social contributions of
our schools and universities, the knowledge, inventions, and
partnerships created through education research are essential.
It is through research that we determine the best ways to
produce the needed gains and help to make tough decisions about
how to use our limited funds.
During the short history of IES it has filled an essential
role in providing and encouraging the necessary conditions for
high quality research in education. I focused my comments on
three main contributions.
First, IES has taken the role of creating a series of
important public goods. By ``public goods'' I mean things that
benefit us all, but many of these goods would not have been
produced without government intervention.
As noted by Chairman Kline, IES provides the foundations of
factual information and research with the collection of clear,
consistent, high quality data. Additionally, IES serves as a
repository and distribution center of objective research.
This is vitally important because the education space is
filled with many organizations, companies, and individuals who
have varying objectives, agendas, and degrees of expertise.
Therefore, it can be difficult to discern between the many
studies, reports, and assertions of what is fact versus what is
fiction, and IES stands as the best authority of rigorous
research free from influence.
Second, IES has led the way in efforts to reevaluate and
redefine the standard of what is considered the best evidence.
Before the creation of IES many lamented that education
research was failing to answer important educational questions
in convincing ways. One example of what IES has done is push
for randomized control trials, which is a gold standard in
research and often used in the field of medicine.
By providing support and encouraging researchers to develop
ways of conducting RCTs while still being sensitive to the
needs of students, education research has progressed in
fundamental ways with new, important evidence on the effects of
key programs and interventions. Moreover, IES continues to
engage the field in conversations about rigor, as demonstrated
by technical work groups tasked with ensuring that evaluations
provide unbiased assessments.
Third, IES has influenced the kind of research that is
done. While there are many organizations that conduct education
research, most focus on only a handful of topics and are only
able to do projects of limited size. But education is all
encompassing, from the wide array of the types of students,
environments, needs, and goals, and there is much work to be
done.
With a national platform, IES has the unique ability to
leverage researcher and practitioner expertise by signaling and
providing incentives to conduct studies on issues of importance
for the country. This includes large-scale projects that would
not otherwise be conducted but shed an considerable light on
important issues.
Another contribution has been to emphasize the importance
of partnerships with researchers and schools, districts, and
educational agencies. By working closely with the field,
researchers are much more likely to produce research that is
relevant and useful in practice.
Finally, it is important to note that IES has been
instrumental in attracting talent to the study of education
with training, tools, and resources to support high quality
research.
While the accomplishments of IES are numerous, the board
and IES are committed to continuous improvement. As noted
earlier, IES has filled a gap for the nation by providing
clear, objective information. However, more could be done to
communicate and disseminate this information.
This is a challenging feat. Education has an incredibly
large range of stakeholders and multiple audiences to address,
including policymakers, practitioners--from teachers, to
superintendents, to state agencies--researchers, and students
and their families. Each group needs different kinds of
information in different forms.
There are many examples of success and promise. For
example, the practice guides distill a wealth of research into
clear steps for teachers to take to improve the learning of
students.
Additionally, there have been many efforts to improve the
work of IES. They include revisions to the Web site,
establishing a grant competition to create a research and
development center on knowledge utilization. To ensure reliance
and usability IES has also revised and renegotiated the
contracts for the RELs and just last month convened a product
feedback and development meeting with stakeholders from across
the country.
I have also entered into the record a full list of the
board's recommendations regarding ESRA.
[The information follows:]
THE NATIONAL BOARD OF EDUCATION SCIENCES (NBES)
Advisory Board to the Director of the Institute of Education Sciences
(IES),
U.S. Department of Education Dr. Bridget Terry Long, Chair
Recommendations for the Reauthorization of the
Education Sciences Reform Act (ESRA)
At the June 20, 2012 NBES meeting, Board members discussed
recommendations to revise ESRA. These recommendations build from
previous suggestions made by the NBES in May 2008 with several
additional changes and revisions. The recommendations fall into three
categories:
Definitional changes
Substantive changes in the Institute or Board's
functioning or powers
Administrative or ``housekeeping'' changes to the bill
definitional changes
1. Definitions related to ``Scientific Research,'' etc.
On pages 4-5, we recommend changes related to definitions of
``scientific research.'' Congress has moved towards defining principles
of scientific research rather than defining scientifically-based
research. As noted on page 5, the NBES agrees with the Department of
Education's position that any definition of ``scientific research,''
etc. in ESRA should be consistent with the definitions used in other
bills such as the Elementary and Secondary Education Act.
2. Changes in IES's mission
On page 7, the NBES recommends modifying the initial lines of IES's
mission to read:
The mission of the Institute is to provide national leadership in
expanding reliable evidence on which to ground education practice and
policy and to encourage its use by parents, educators, students,
researchers, policymakers, and the general public * * *
The major differences between our recommendation and the original
are to replace ``expanding fundamental knowledge'' with ``expanding
reliable evidence'' and to add the words ``encourage its use''. The new
definition, which focuses on providing evidence and encouraging its
use, is much closer to what IES does and is more objective than the
existing definition's focus on fundamental knowledge and understanding.
Note: This is not marked on the attached draft.
substantive changes related to powers, functions, and terms of office
1. Delegation of ``Other Activities''
Changes on page 9 would leave it at the discretion of the IES
Director to accept additional assignments from the Secretary of
Education if they were consistent with IES's mission and priorities.
The existing language gives the Secretary the power to simply assign
such activities to IES.
2. Provisions related to the IES Director
Page 9 removes language which is no longer relevant related to the
appointment of the first IES Director. In addition, language is added
making it possible for a Director to be nominated for a second term,
and for a sitting Director to serve up to an additional year if his/her
successor has not been appointed.
Additional new language on page 9-10 makes the IES Director
eligible for ``critical pay'' under the Federal Workforce Flexibility
Act of 2004. The explanatory note in the mark-up states, ``Many people
who might be qualified to be Director are unwilling to do so over a 6
year term at the rate of pay [specified in existing law]. This addition
provides pay flexibility in recruiting a director, and would be subject
to the recommendation of the Board.'' Page 13 adds the language related
to the Board's ability to make recommendations in this area.
New language on page 10 specifies that the IES Director reports
directly to the Secretary of Education. The explanatory note in the
mark-up states, ``A direct reporting line to the Secretary is important
to maintaining the status and independent functioning of IES within the
Department.''
3. Requirement that IES Director submit a biennial plan of activities
to the Board for advice
Page 10 adds new language to the Director's duties requiring him/
her to submit a biennial plan of activities to the Board every two
years. New language on page 12 adds reviewing and advising the Director
on the plan of activities to the Board's duties. (Note that the Board's
approval of the plan is not required.)
4. NBES: Organizations that advise the President on Board members
Existing language in ESRA requires the President to solicit advice
regarding individuals to serve on the Board from the National Academy
of Sciences, the National Science Board, and the National Science
Advisor. New language on page 13 would add the Board itself, the
American Educational Research Association, the Society for Research on
Educational Effectiveness, and the National Academy of Education to the
list of groups that the President must solicit advice from.
5. NBES: Board terms
Page 14 includes a number of changes aimed at fixing some of the
difficulties that the Board has consistently encountered since its
founding, including numerous vacancies and attenuated terms.
Page 14 adds language specifying that a member's 4-year term
commences from the date of their appointment. Current practice has been
to appoint members to 4-year slots, whose beginning and end dates are
calculated based on three cohorts tied to the original legislation.
That is: there are five Board slots tied to an initial term expiration
date of 11/28/08; five Board slots tied to an initial term expiration
date of 11/28/06; and five Board slots tied to an initial term
expiration date of 11/28/07. If a Board member was nominated or
confirmed to a slot that was close to its expiration date, their
effective term dating from their actual appointment might be as short
as a year. Page 14 also strikes language which is no longer relevant
related to the initial appointment of Board members to staggered terms.
Page 14 also adds a new provision (similar to the provision for the
IES Director) allowing Board members to serve up to an additional year
after their term has expired if their successor has not been appointed.
The net effect of the changes listed on page 14 will be to greatly
reduce the number of unfilled vacancies on the Board and to make it
much more likely that at all times the Board will have close to its
full complement of 15 members that ESRA stipulates.
6. NBES: Executive Director
New language on page 15 of the mark-up provides greater detail
regarding the Executive Director position. Board members favored
revising ESRA to give the Board hiring and evaluation authority over
the NBES Executive Director.
7. NBES: Charitable contributions
New language on page 16 would allow the Board to accept charitable
donations to further the mission of the Board. This would allow the
Board to provide coffee during advisory board meetings.
8. NBES: Standing committee structure
Pages 16 strikes language related to NBES's standing committee
structure. The existing language specifies a standing Board committee
corresponding to each national education center (e.g., NCES, NCER). The
Board has never functioned this way in practice. This is due, in part,
to the fact that at times Board membership has dwindled to as few as 6
members.
In place of the struck language, the mark-up adds new permissive
language on page 16 that allows the Board to establish standing
committees related to the Board's responsibilities.
9. Commissioner's pay
Similar to the new language related to the IES Director's pay, page
18 adds language allowing Commissioners to be eligible for critical pay
under the provisions of Federal Workforce Flexibility Act of 2004. The
rationale is similar to that for the IES Director: to enhance
recruitment flexibility for Commissioners.
10. The appointment process for the NCES Commissioner
Page 18 has existing language that has the NCES Commissioner be
appointed by the President and confirmed by the Senate. The Board made
no recommendation regarding changing to the appointment process for the
NCES Commissioner, because opinion was evenly divided among the
members. Some felt strongly that the current requirement that the NCES
Commissioner be appointed by the President and confirmed by the Senate
reflects a hard-won acknowledgment that education statistics deserve
national-level prestige. Others instead felt that the procedure for
appointing the NCES Commissioner should be the same as that for the
other IES Center commissioners to support smooth and efficient
functioning of IES.
11. National Research and Development Centers
Pages 21-22 strike existing language related to National Research
and Development Centers and replaces it with a section titled,
``Priorities for Long-Term Research Activities.'' The changes remove
language requiring the funding of at least eight National R&D Centers
as well as requirements related to the topics assigned to the centers.
The replacement language does not address the number of centers funded
and allows the NCER Commissioner to choose topics consistent with IES's
priorities. The feeling is that the NCER commissioner and the director,
with the counsel of the Board, should be able to determine the best
funding mechanisms and funding levels for advancing IES's long-term
research priorities rather than having Congress earmark particular
centers and levels of funding.
12. Removal of privacy protection for individual school information
Page 41 strikes language giving privacy protection to individually
identifiable information with respect to individual schools. The
explanatory note in the mark-up states that, ``Schools do not receive
privacy protection elsewhere in federal statute or regulations. Many
IES reports from NCES require that schools be identified, e.g., the
Common Core of Data. The prohibition on revealing school identity means
that useful information must be omitted from evaluation reports. There
is no compelling reason to maintain this protection for schools.''
13. Adjustment to the circumstances under which the Director or Board
members may be removed
Pages 45-46 add language that allows the President to remove the
Director and any Board member for cause, although the President must
inform the Board of the cause for which the appointee is being removed.
The original language did not include the words ``for cause,'' nor was
Board notification required. The original language also included the
Commissioner for Education Statistics in these provisions. However, the
mark-up's proposed change to make the Commissioner of Education
Statistics a Director-appointed position, like the other Commissioners,
means that the Commissioner should be struck from the provisions of
this section.
14. Expansion of authorization related to data bases to be included in
the statewide longitudinal data systems
Page 52 adds language specifying the Higher Education Act and IDEA
with regard to the development of statewide longitudinal data systems.
The existing language only specifies the Elementary and Secondary
Education Act. The change expands the authority so that a broader range
of educational records can be incorporated into the supported data
systems.
15. NAEP reports
Pages 55 and 57 add language related to authority over the content
and release of National Assessment of Educational Progress (NAEP)
reports. The mark-up's explanatory note states, ``There are ongoing
disputes between NCES and the National Assessment Governing Board
(NAGB) on the content and formatting of NAEP reports * * * These are
IES/NCES reports published under the authority of the Director and
Commissioner. The NCES commissioner needs to retain responsibility for
the content of the reports.'' And, ``NAGB has taken the position that
the NCES commissioner's role at the release event is entirely at the
discretion of NAGB. Because the findings being released are from an
NCES statistical report, the commissioner or his delegate should be
responsible for presenting the findings.''
administrative/housekeeping changes
1. Delegation of authority
Page 8 contain new language aimed at delineating the delegation of
authority between the IES Director and the Secretary of Education.
2. Removal of language related to the role of the National Assessment
Governing Board (NAGB)
Four provisions related to NAGB are struck on pages 8-9. The
rationale given was that a preceding paragraph clearly articulated
NAGB's role and so the subsequent four provisions were redundant.
3. Other changes in the Director's duties
Page 10 changes language related to peer review from ``establish''
to ``maintain,'' reflecting the fact that peer review procedures have
already been established.
Another change on page 11 specifies that the Director will
coordinate with the Secretary of Education to insure that IES's
findings are used by all of ED's technical assistance providers, and
not just the 15 comprehensive assistance centers.
4. Review of publications, not ``products''
Page 11 strikes the term ``products'' from the section pertaining
to the Director's review of evidence-based claims in ED publications.
The rationale is that IES cannot review products, only publications
that make scientific claims.
5. Requirement that IES priorities be proposed every 6 years
New language on page 11 requires the IES Director to submit
priorities for the Institute to the Board for approval at least every 6
years. This would put into statute what has occurred in practice.
6. Peer review standards and NCER
Page 19 strikes language requiring the National Center for
Education Research (NCER) to maintain peer review standards. This would
conform to IES's actual practice, which is to have the Scientific
Review Office maintain IES's standards related to peer review.
7. Replacing `Commissioner' with `Center'
Pages 26 and 29 replace `Commissioner' with `Center' for the sake
of consistency.
8. Removal of outdated language related to NCEE
Page 32 strikes language regarding the award of specific contracts
which is outdated and not relevant for reauthorization. Similarly,
pages 34-35 strikes additional outdated language that is not relevant
to reauthorization.
______
[An additional submission, the document, ``NBES Markup,
June 2012, Education Sciences Reform, Public Law 107-279,'' may
be accessed at the following Internet address:]
http://ies.ed.gov/director/board/pdf/NBESmarkupESRA.pdf
------
Ms. Long. So in summary, IES has made many contributions,
though there is still work to be done, and we look forward to
this discussion.
Thank you.
[The statement of Dr. Long follows:]
Prepared Statement of Dr. Bridget Terry Long, Ph.D., Xander Professor
of Education and Economics Academic Dean, Harvard Graduate School of
Education; Chair, National Board for Education Sciences
Mr. Chairman and members of the committee, thank you for the
opportunity to appear before you today.
My name is Dr. Bridget Terry Long, and I am the Academic Dean and
Xander Professor of Education and Economics at the Harvard Graduate
School of Education. Beyond my expertise as a researcher and faculty
member, I am also the Chair of the National Board for Education
Sciences, the advisory board of the Institute of Education Sciences
(IES). The National Board for Education Sciences is independent of IES,
and we are tasked with advising the Director and reviewing and
evaluating the work of IES. In this way, we provide a critical but also
constructive perspective on the activities of IES.
My testimony reflects discussions and recommendations made by the
Board as well as my observations as an experienced educational
researcher who has interacted with IES on many levels. My comments
today aim to provide an objective assessment of the role of IES, its
contributions, and areas for improvement.
The Role of IES
In our current environment, educational research has become even
more important as the penalties of poor achievement and lack of
opportunity have never been greater. As we work to raise student
achievement, foster productive learning environments, and bolster the
social contributions of our schools and universities, the knowledge,
inventions, and partnerships created through educational research are
essential--it is through research that we determine the best ways to
produce the needed gains and help to make tough decisions about how to
use our limited funds. Before we can debate what policies we should
implement, we first need a clear understanding of the facts and to have
an accurate sense of the real costs and benefits of any policy or
program. In essence, research is the foundation for improving
education.
During the short history of IES, it has filled an essential role in
providing and encouraging the necessary conditions for high-quality
education research. While its impact is evident in many ways, I focus
my comments on three main contributions. First, IES has taken the role
of creating a series of public goods that no one else would or could do
without concerns about possible bias. Second, it has led the way in
efforts to reevaluate and redefine the standard of what is considered
good evidence. Third, IES has influenced the kind of educational
research that is done by making possible large-scale studies, pushing
researchers to work closely with practitioners to ensure relevance and
usability, and holding an unwavering focus on serving the national
good.
(1) Creating Necessary Public Goods
As a federal entity, IES has taken leadership to provide several
key public goods needed to support a strong educational system and
research. By public goods, I mean things that benefit us all, but many
of these goods would not otherwise be produced without government
intervention. For example, IES provides the foundations of factual
information and research with the collection of clear, consistent,
high-quality data through the National Center for
Education Statistics (NCES).\1\ It is through the efforts of IES,
which conducts its work free from political influence, that we are able
to understand trends in our student populations, schools and
universities, and an array of inputs and outcomes that span early
childhood to adult education. These data also make possible a wealth of
research conducted on every aspect of education.
Additionally, IES serves as a repository and distribution center of
research, both studies funded by IES and those that are not. The
dissemination and communication of objective information is a critical
one for the nation. The education space is filled with many
organizations, companies, and individuals who have varying objectives,
agendas, and degrees of expertise. Therefore, it can be difficult to
sort between the many studies, reports, and assertions to determine
what is fact versus what is fiction. Moreover, the research community
often lacks the training and incentives to translate complex research
for a lay audience. In such a crowded space, IES stands as the best
authority of rigorous research free from influence. It has helped to
clarify what is known about issues related to large educational
debates. Moreover, it has been helpful in discerning between
conflicting and confusing reports on important issues. It has used its
convening power to bring together researchers from various backgrounds
to discuss the issues and coordinate research.\2\ It has also conducted
evaluations of federal initiatives.\3\
(2) Setting the Standards of ``Good Evidence''
Before the creation of IES, many lamented that educational research
was failing to answer important questions in convincing ways. The
varying quality of research and lack of attention to certain issues led
some to dismiss the educational research base as inadequate. IES has
changed this dramatically by leading a critical assessment of past
research and initiating a number of debates about what are appropriate
methods and standards of rigor for the different approaches to
educational research.
One concrete example of this has been the push for randomized
controlled trials (RCTs), which are considered the gold-standard of
research and often used in the field of medicine. Prior to IES's
leadership, RCTs were rarely conducted in education and not valued
among many researchers. However, by pushing the field, providing
support, and engaging researchers to develop ways of conducting such
analyses while still being sensitive to needs of students and
practitioners, educational research has progressed in fundamental ways
with new important evidence on the effects of key programs and
interventions. For example, in my own work with several colleagues,
which was partially funded by IES, we demonstrated that providing low-
and moderate-income families with streamlined personal assistance to
complete the federal college financial aid application had large
effects on college attendance and persistence. Because we used a
randomized controlled trial design, we were able to establish
convincingly that our intervention was not only the cause of the
educational gains; importantly, the program was also inexpensive.\4\
IES continues to engage the field in conversations about rigor in
educational research. This is demonstrated by technical working groups
that are establishing standards for specific research methodologies and
helping to ensure that evaluations provide unbiased and causally-valid
assessments.\5\ It is also worth noting that IES has developed a
rigorous peer review process for evaluating grant proposals.
(3) Encouraging Relevant, Rigorous Research for the
National Good
IES has used its resources and convening power to focus the field
on research that is both rigorous and focused on shedding light on the
major problems facing the country. By setting priorities and crafting
calls for research proposals (i.e., Requests for Proposals or RFPs),
IES has sent signals to the field about important topics that need
answers, rigorous standards that must be upheld, and the importance of
conducting research in partnership with practitioners.
Additionally, it has made possible research studies that would not
have otherwise been conducted.
While there are private foundations and other organizations that
support educational research, most focus on only a handful of topics
and fund projects of limited size. But education is all encompassing,
from the wide array of types of students, environments, needs, and
goals, and there is much work to be done. With a national platform, IES
has the unique ability to leverage researcher and practitioner
expertise by signaling and providing incentives to conduct studies on
issues of importance for the country. One way it has done this is by
designing research competitions that focus on the major issues and
areas of education. Along with this has come IES's emphasis on the
importance of external validity in research, meaning that it has called
for researchers to be accountable to external audiences on how the
findings for one set of schools might be applicable to another set of
schools.
IES has also been able to support large-scale projects that could
not be easily funded by others. To learn more certain issues, studies
must be large in scale and compare the experiences of districts across
states or large populations of students. Without support from IES, this
type of work would often not be possible, and the knowledge base that
is being built as a result of this work has been valuable in improving
student outcomes. Taken together, IES has both insured research on a
breadth of topics while also making possible large-scale studies that
have been incredibly beneficial to our understanding of how to help
students.
Another way IES has influenced the research community is by
highlighting the importance of partnerships between researchers and
schools, districts, or state educational agencies. Because the delivery
of education is the result of many actors, research can often be
improved by being designed and conducted while working with
practitioners. Additionally, by working closely with the field,
researchers are much more likely to produce research that is relevant
and useful in practice. However, such work can be difficult to manage
and implement.
IES has pushed and supported such connections to the benefit of the
research being conducted.\6\
Finally, it is important to note that IES has been instrumental in
attracting talent to the study of education. With the signals it sends
about important issues in education and the support it gives for
research, IES has helped to attract a growing number of researchers
with the tools and resources to support high-quality research and
partner with the field. IES is helping to produce the next generation
of scholars and innovators who will help to solve important problems in
education.
The Strengths, Challenges, and Continuous Improvement of IES
The accomplishments of IES are numerous, and the researchers and
innovators supported by IES funding will continue to have positive
impacts on the lives of students as well as many other parts of our
society. Nevertheless, in light of the Board and IES's commitment to
continuous improvement, it is clear more can and needs to be done. In
this spirit, the Board has worked to advise, review, and advance the
activities of IES. The Board has matured to be an important place of
feedback and expertise, and my comments here reflect continuing
discussions between the Board and IES staff about how to address
challenges facing the organization.
As I noted earlier, the dissemination and communication role of IES
is an important one.
IES has filled a gap for the nation by providing clear, objective
information and making it available to the public. While IES is a
strong producer and supporter of information of value, it is still
building capacity and expertise on how to disseminate that information,
including methods that use the latest technology and outreach methods.
This is a challenging feat. Unlike many other fields, education has
large range of stakeholders and multiple audiences to address,
including policymakers; practitioners from teachers to superintendents
to state agencies; researchers; and students and their families. Each
group needs different kinds of information in different forms.
The Board and IES staff believe strongly in the dissemination role
of IES, and we have held a number of discussions on how to improve
efforts. There are many examples of success and promise. For example,
the Practice Guides distill a wealth of research into clear steps
teachers can take to improve the learning of their students.\7\ The
What Works Clearinghouse was created with the idea of helping the
public understand research results and whether they were completed
using rigorous methods. The dissemination of recent data reports and
grant competitions include webinars and video media.\8\
However, more could be done in terms of reaching out to the many
audiences of educational data and research, and there are many efforts
underway at IES to address this challenge. They include:
Revisions to the website to make it easier to find
important research and facts. For instance, a new contract was awarded
this year to manage and enhance the What Works Clearinghouse.\9\
Additionally, as part of the RFP for the Education Resources
Information Center (ERIC), the contractor is expected to redesign the
IES website to improve search capabilities and provide basic
orientations to key topics and references for relatively inexperienced
users.\10\
IES added new requirements to research grant competitions
for researchers to develop dissemination plans for their studies.
Moreover, NCSER released a report on how to make research more
understandable, and it was presented to its grant recipients.\11\
Establishing a grant competition to create a Research and
Development Center on Knowledge Utilization. This Center will explore
questions of how education researchers can make their work more
relevant and useful to practitioners located in state and local
education agencies and in individual schools. This work is meant to
address concerns that often there is only limited adoption of evidence-
based practices.\12\
Related to the issue of dissemination is the relevance and
usability of the research produced and funded by IES. This has been a
major focus of IES, and there are many instances of the Institute
meeting this goal. As noted above, the growing attention to the
importance of partnerships has broadened the number of studies done in
concert with schools and districts, and this approach increases the
likelihood that the results will be relevant and useful for
practitioners. Still, this has been an area of constant reevaluation,
and there have been many activities recently to improve this function
of IES. For example:
Revising and renegotiating the contracts for the Regional
Education Labs (RELs). For example, earlier this year, IES released
revised criteria for REL proposals and products. The criteria focus on
issues related to the technical rigor of products (e.g., data quality,
analysis methods), the relevance of the work (i.e., whether it provides
evidence that can inform a practitioner's action or decision), and the
readability of the products (i.e., whether the information is clear for
its intended audiences). NCEE has also been working to build the
capacity of the REL program by conducting webinars to help the RELs
meet increasing standards in writing, collaboration, and
measurement.\13\
Just recently, on August 12, 2013, IES convened a Product
Feedback and Development Meeting with stakeholders from across the
country to get suggestions about how to improve the usability and
relevance of the products and services of the WWC and RELs.\14\
As an independent body tasked with providing constructive feedback
to IES, the Board has been pleased with the fact that our feedback and
that of others on these issues has been incorporated into the work of
the Institute, and we believe these activities will help to strengthen
IES's impact.
Another challenge facing IES is balancing the need to work in many
areas with the reality of having limited resources. Because it is
important to understand so many facets of education and the populations
it impacts, it can be difficult to prioritize some areas over others or
to decide not to fund research in some areas at all. Touch choices
sometimes have to be made. For instance, this year, IES will not hold
research competitions in special education.\15\ However, IES is not
taking a haphazard approach to this dilemma. Recent discussions between
the Board and IES staff have concerned if and how the Institute might
decide to prioritize funding decisions. Moreover, IES is attempting to
understand and improve the impact of the overall portfolio of research
supported with IES funding. Together, we have been examining the
research portfolios of NCER and NCSER to understand how IES might
better target its research funding.
Revising ESRA: Recommendations from the NBES
At the June 20, 2012 NBES meeting, Board members discussed specific
recommendations to revise ESRA. These recommendations build from
previous suggestions made by the Board in May 2008 with several
additional changes and revisions. Most notably, we suggest:
Establishing a requirement that the IES Director submit a
biennial plan of activities to the Board for advice. Currently, the IES
Director is only required to submit his or her priorities to the Board
every six years. Although the Board has many informal opportunities to
provide feedback to the Director based on the strong working
relationship between the current Board and current Director, the
expectation of more frequent formal feedback should be documented.
Changing the term of a Board member to commence from the
date of confirmation so that members have a full four years of service
Automatically extending by one year the terms of Board
members whose successors have not yet been appointed; this would help
to ensure that the Board always has a sufficient number of members to
be effective
Giving the Board hiring and evaluation authority over the
NBES Executive Director to ensure this role is independent of IES given
the assessment duties of the Board
Allowing for flexibility in the pay of the IES Director
and Commissioners by making these positions eligible for ``critical
pay'' under the Federal Workforce Flexibility Act of 2004
Removing privacy protection for individual schools in data
reports, a protection that does not exist in any other federal statue
or regulation. The current prohibition on revealing school identity
means that useful information must be omitted from evaluation reports.
A full list of our recommendations and a marked-up copy of ESRA
have been entered into the official record.
Conclusions
In summary, to have an informed populace and clarity on how best to
educate our children and ourselves, there must be a robust foundation
of high-quality data, rigorous, objective research and strong
communication of evidence on what works and what does not. It is clear
that IES has made substantial contributions to our understanding of how
to improve education and is engaged in activities to address the
challenges it faces. There is more work to be done, and as noted by our
recommendations, the Board believes some changes to ESRA would improve
the functioning of IES and the Board for the continued benefit of the
country.
endnotes
\1\ Before the creation of IES, the Office of Educational Research
and Improvement (OERI) led efforts to collect educational data. IES has
built upon these surveys in its current activities.
\2\ For example, the National Center for Education Research (NCER)
serves as a hub to facilitate collaboration among a diverse,
interdisciplinary group of researchers who are a part of the Reading
for Understanding Initiative (RfU). NCER is funding six research teams
to advance theories and develop interventions to improve reading
comprehension from pre-K through grade 12. Five of the teams are
testing interventions to improve reading comprehension through a
variety of curricula, supplemental materials, and professional
development opportunities.
\3\ For instance, in September 2012, NCEE released the State and
District Receipt of Recovery Act Funds: A Report from Charting the
Progress of Education Reform--An Evaluation of the Recovery Act's Role,
which documents how funding was spent and includes the characteristics
of funded schools and districts, amounts, etc. It is part of a larger
study of major Federal funding efforts and reflects an NCEE effort to
get interim reports out to the public more quickly.
\4\ Bettinger, Eric P., Bridget Terry Long, Philip Oreopoulos, and
Lisa Sanbonmatsu. (2012) ``The Role of Application Assistance and
Information in College Decisions: Results from the H&R Block FAFSA
Experiment,'' Quarterly Journal of Economics, vol. 127, no. 3, pp.
1205-1242.
\5\ NCEE formed the technical methods group to work on issues and
strategies that assure evaluations of education interventions provide
unbiased and causally valid assessments. The technical methods working
group aims to advance and provide guidance for those specialists who
are embarking on evaluations in education. More information is
available here: http://ies.ed.gov/ncee/tech--methods/
\6\ For instance, in 2012, IES created the Research-Practitioner
Partnerships in Education Research program, which supports partnering
around issues and problems of practice identified by the state and
local education agencies. It is administered by the National Center for
Education Research (NCER) as part of the research grant program.
\7\ One example of a Practice Guide is Teaching Elementary School
Students To Be Effective Writers, which was released by the What Works
Clearinghouse. It offers a framework and examples, and is part of
NCEE's interest in providing practice guides that are narrowly focused
and useful to classroom teachers.
\8\ For example, to explain the new NAEP Technology and Engineering
Literacy (TEL) assessment, the National Center for Education Statistics
(NCES), created a video. The video describes what the assessment
covers, gives examples, and makes clear that the goal of the assessment
is to learn whether students have the skills needed to address the
challenges of our evolving society. Additionally, an online tutorial
allows users to get a sense of the test. More information is available
here: http://nces.ed.gov/nationsreportcard/tel/
\9\ Report from the National Center for Education Evaluation and
Regional Assistance (NCEE) to the NBES concerning activities from March
to May 2013.
\10\ Report from the National Center for Education Evaluation and
Regional Assistance (NCEE) to the NBES concerning activities from
October 2012 to February 2013.
\11\ The report entitled, Translating the Statistical
Representation of the Effects of Education Interventions into More
Readily Interpretable Forms, can be found here: http://ies.ed.gov/
ncser/pubs/20133000/
\12\ More information is available here: http://ies.ed.gov/funding/
ncer--rfas/randd.asp
\13\ Report from the National Center for Education Evaluation and
Regional Assistance (NCEE) to the NBES concerning activities from March
to May 2013.
\14\ More information is available here: http://ies.ed.gov/ncee/
wwc/event.aspx?sid=28
\15\ The National Center for Special Education Research (NCSER)
will not hold research or research training competitions for FY 2014.
Researchers interested in the study of children, youth, and adults with
disabilities may be eligible for funding under the NCER competitions.
______
Chairman Kline. Thank you very much.
Dr. Kemple, you are recognized.
STATEMENT OF DR. JAMES KEMPLE, EXECUTIVE DIRECTOR, RESEARCH
ALLIANCE FOR NEW YORK CITY SCHOOLS, NEW YORK UNIVERSITY
Mr. Kemple. Chairman Kline and members of the committee,
thank you for the opportunity to appear before you today.
Continuing the theme, from my perspective as an educator
and researcher for more than 30 years, the Education Sciences
Reform Act and the Institute of Education Sciences have
produced incredibly important changes in the quality, quantity,
and use of education research.
Until ESRA and IES, education research was allowed to
function at a standard that would never pass muster in public
health, employment and training, welfare policy, let alone
agriculture and medicine. The dearth of good evidence in
education and the inability to effectively communicate lessons
from the little evidence that did exist left us with a legacy
of repeatedly reinventing the wheel and chasing fads rather
than building a reliable and useful track record of what
worked, what did not work, for whom, and under what conditions.
While I believe we are much better off than we were 12
years ago prior to IES, I think we are still burdened with that
legacy from more than two generations of ineffective research.
From a pure numbers perspective, IES has commissioned and
released findings from 90 studies that now meet the widely
agreed upon gold standard for research--the randomized
controlled trial. To my knowledge, that is 89 more such studies
than both of IES's predecessors combined had produced, and
there are many more in the pipeline.
However, I believe IES's influence extends well beyond the
scientific research studies and the activities it has supported
over the last 12 years. The principals embedded in ESRA and in
IES's work have also changed the way that federal, state, and
local policymakers evaluate and use education research.
In New York City, where I lead a partnership with the
city's schools, the schools' chancellor, most senior staff in
the New York City Department of Education, and, yes, even the
mayor now ask pointed questions about whether the research that
they are presented with meets the appropriate scientific
standards. And I am also encouraged by the fact that when they
do see high quality evidence they are much more inclined to use
the resulting evidence in their decision-making even when that
evidence suggests that programs are ineffective and should
probably be discontinued.
I think it is also worth noting that IES has supported
training programs to develop a new generation of researchers
and research organizations that are equipped to meet those
higher standards of evidence.
So in short, from my perspective the transformations that
have occurred under IES have moved education research much
closer to the caliber of research conducted for decades within
the U.S. Departments of Education and Labor and the NIH.
Now, while applauding IES's many important accomplishments,
I would also describe it as a work in progress. There are
several areas where I think the institute can be improved.
First, in some cases I believe IES has promoted scientific
rigor at the expense of policy and practical relevance. Second,
IES has not invested enough in building partnerships and lines
of communications between researchers and policymakers and
practitioners, and those are just the people that should inform
and benefit from the work that we produce. And then third, I
think IES can be smarter in the strategies it uses to make its
work accessible.
My sense is that the original framework established by ESRA
already provides for advancement in each of these areas, and I
think IES is headed in the right direction. The key challenges,
I believe, lie in helping the current leadership of IES
continue to make strides in each of the directions I just
outlined.
So I want to suggest four core principles that I believe
should guide the further strengthening of IES, and I will call
these the four Ps: P1, preserve scientific rigor; P2,
prioritize relevance for policymakers and practitioners; P3,
promote wider use of high quality evidence; and P4, prepare for
the future.
Under P1, I think it is specifically important for IES to
continue to place a premium on funding research that
establishes strong, causal connections between specific
education policies and practices and the outcomes that we care
most about for our students: achievement in literacy,
mathematics, and the sciences, social development, and
preparation for college and careers.
P2: In prioritizing rigorous research that is more relevant
to policymakers and practitioners I think we have two
challenges here. One is strengthening the relationships and
collaboration between researchers, policymakers, and
practitioners; and the second is ensuring that our studies
address critical questions about how and why education
practices and policies work or do not work, not just the thumbs
up did they work or not. I think in some cases IES's pursuit of
rigor has sometimes narrowed the scope of research to the
thumbs up, thumbs down, did it work or not.
And in promoting wider use and application or research, I
think it is important that IES treat dissemination as a
continuous process, not just an event that occurs at the end of
a study, and that they make smarter uses of technology to
promote and disseminate its work.
And then finally, P4: In preparing a next generation of
education researchers who are committed to scientific rigor, I
believe IES should continue its support for the pre-and post-
doctoral training programs that are ensuring that our best and
brightest are going into the field of education research.
Let me conclude by saying this is one of the most important
issues--education, evidence-building--that the federal
government can play in supporting education throughout the
United States and I urge the committee to take up the
reauthorization of ESRA. Thank you.
[The statement of Mr. Kemple follows:]
Prepared Statement of James J. Kemple Executive Director,
Research Alliance for New York City Schools at New York University
Mr. Chairman and Members of the Committee: My name is Jim Kemple. I
serve as the founding Executive Director of the Research Alliance for
New York City Schools at New York University. Our organization is a
nonpartisan research center that conducts rigorous studies on topics
that matter to the policymakers, educators and other stakeholders who
work with New York City's public schools. We strive to advance equity
and excellence in education by providing evidence about policies and
practices that promote student success. Prior to my current position, I
worked for 18 years at MDRC, overseeing scientific evaluations of
education, welfare-to-work, and employment and training initiatives
across the country. Before that, I served as director for the Higher
Achievement Program here in DC, and I taught high school math.
Thank you for the opportunity to testify before you today. I can
think of few more important roles for the federal government to play in
education than its support for building and communicating rigorous
evidence about what works to improve teaching and learning across the
country. The current economic and fiscal environment makes it more
important than ever to use scientific evidence to inform difficult
decisions about how to allocate scarce resources, and to invest in
building more and better evidence about the efforts we make to
strengthen our schools, particularly efforts that flow from the federal
government.
From my perspective as an educator and researcher for more than 30
years, the Education Sciences Reform Act (ESRA) and the Institute of
Education Sciences (IES) have produced incredibly important changes.
Until ESRA and the creation of IES, education research was allowed to
function at standards that would never pass muster with public health,
employment and training, or welfare policy, let alone with medicine or
agriculture. As a nation, we have paid a heavy price. The paucity of
good evidence in education, and the inability to effectively
communicate lessons from the little scientific evidence that did exist,
left us with a legacy of reinventing the wheel and chasing fads rather
than building a reliable and useful track record of what worked, what
did not work, for whom and under what circumstances. While I believe we
are much better off now than we were 12 years ago, we are still saddled
by that legacy.
Since its inception, IES has funded and released findings from 90
studies that meet the widely agreed-upon ``gold standard'' for
research, the randomized controlled trial. That's 89 more such studies
than all of IES's predecessors combined. However, I believe IES's
influence extends well beyond the specific research studies and
activities is has supported. It has changed the way federal, state, and
local policymakers evaluate and use education research.
In New York City, where I lead a research partnership with the city
schools, the Mayor, the Schools Chancellor, and most senior staff in
the Department of Education now ask pointed questions about whether the
research that is presented to them meets scientific standards. When
they see high-quality research, they are much more inclined to use the
resulting evidence in their decision-making. For example, New York City
discontinued its use of financial incentives for school performance in
the face of solid evidence that these incentives did not improve
student achievement. By the same token, the City has reinforced its
commitment to creating and sustaining small schools of choice, citing
scientific evidence that these schools are significantly improving
graduation and college readiness rates, particularly among some of the
city's most vulnerable students.
IES has also helped develop a new generation of researchers and
research organizations that are equipped to meet those high standards
of evidence. More than 25 universities are now attracting the nation's
best and brightest to training programs in rigorous education science.
While these young people come from multiple disciplines, they are
committed to conducting high-quality education research that will be
useful to policymakers and practitioners.
In my view, these transformations have moved education research
much closer to the caliber of research conducted for decades through
the U.S. Departments of Labor and Health and Human Services and the
NIH.
While applauding these important accomplishments, I would also
describe IES as a work in progress. There are several areas where the
Institute could be improved. First, in some cases, I believe IES has
promoted scientific rigor at the expense of policy and practical
relevance.
Second, IES has under-invested in building partnerships and lines
of communication between researchers and the policymakers and educators
who should inform and benefit from its work. Third, I think IES can do
more to make its work accessible. My sense is that the original
framework established by ESRA allows for advancements in these areas.
The key challenge lies in helping the current leadership of IES
continue to make strides in each of these directions.
I would like to organize my remarks around four core principles
that I believe should guide the further strengthening of IES. I'll call
these the four Ps: Preserve scientific rigor; Prioritize relevance for
policy and practice; Promote greater use of high quality research; and
Prepare for the future. Each of these principles should be seen as
reinforcing and complimenting the others.
P1: Preserve the commitment to scientific rigor. Specifically, IES
should continue to place a premium on funding research that establishes
strong causal connections between specific education policies and
practices and student outcomes we care about: most notably, achievement
in literacy, math, and the sciences; social development; and college
and career readiness.
Prior to ESRA and IES, the federal investment in education research
generated reasonably good evidence about the nature of the problems we
face in our schools, but yielded weak and unsubstantiated claims about
how various approaches may or may not have solved those problems and
improved teaching and learning. Even after only 12 years of work under
IES, the education research community is finding that many of those
claims, both positive and negative, turned out to be plain wrong.
For example, over the past 12 years the federal government
allocated hundreds of millions of dollars for academic enhancements to
after-school programs, innovative teacher professional development
programs, cutting-edge adolescent literacy programs, and computer-based
tutoring tools. Many of these investments were guided by compelling
theory but, due to the legacy of low quality research in education, the
evidence base for their actual effectiveness was weak. Fortunately,
Congress and the U.S. Department of Education had the foresight to make
sure that these new investments were accompanied by rigorous
evaluations under IES to learn about their impact on teaching and
learning. Unfortunately, it turned out that most these initiatives, on
average, had little or no impact.
In a more encouraging example, many federal, state and local
policymakers are currently working to expand pre-kindergarten programs.
Due in part to the growing commitment to scientific rigor in education
research, these policymakers no longer have to rely on one single study
from the 1960s involving less than 120 toddlers who were exposed to an
incredibly expensive set of services and supports before they entered
school. Evidence from a growing number of credible studies is showing
the benefits of affordable early intervention.
Federal support for these kinds of rigorous impact studies is
crucial to developing a more effective educational system.
P2: Prioritize rigorous education research that is more relevant to
policymakers and practitioners. This challenge has two parts: 1)
supporting partnerships and collaboration between researchers,
policymakers and practitioners, and 2) ensuring that studies address
questions about how and why education practices and policies work or do
not work.
Prioritizing Partnerships: My organization in New York City, and
similar groups in more than a dozen other cities around the country,
are beginning to demonstrate the value of partnerships that include
researchers, policymakers, administrators, and educators. By working
together to set research priorities, interpret results and put findings
to use, we are accelerating the pace at which research can inform
policy and practice.
For instance, we have worked with the New York City Department of
Education to enhance the largest school survey in the nation, which
collects vital information from students, parents and teachers; we have
produced individual reports for schools involved in our studies, to
help them improve in real time; and we have examined the effects of the
City's high school choice process on low-achieving students, producing
insights that have been useful to both the district and local community
groups that are helping students navigate the system. This is a far cry
from the typical end product of research, which generally targets
academic colleagues and so often sits on our shelves collecting dust.
From my perspective, IES is making strides toward promoting this kind
of collaboration and should continue to do so.
Similarly, the U.S. Department of Education in general, and IES in
particular, should continue to encourage links between federal
programmatic funding and IES-directed studies that build solid evidence
about the impact of these investments. While IES's independence should
remain paramount, I believe a great deal of its struggle to be relevant
can be traced to the limited role that other offices in the Department
of Education (as well as the Departments of Health and Human Services
and Labor) have had in prioritizing the research questions it pursues
and to the limited role that IES has played in getting buy-in from
those offices.
Until recently, IES has had even less interaction with State and
Local Education Authorities, who in the end are the ultimate consumers
of the evidence IES produces. This is beginning to change, with new
initiatives like IES's Research-Practice Partnership program and the
requirement that the Regional Education Laboratories (RELs) conduct
their work through what are called ``research alliances,'' which are
formal advisory groups comprised of state and local education
policymakers and practitioners. I would encourage the continuation and
expansion of these efforts.
Prioritizing How and Why Questions: In my view, IES's pursuit of
scientific rigor has sometimes narrowed the scope of research to focus
exclusively on the question ``did it work or not?'' This obscures the
kind of information we desperately need to make education better.
Specifically, I would encourage IES to expand its pursuit of
questions about how, why, for whom and under what circumstances things
work or do not work. These questions should be essential to rigorous
evaluations of program effectiveness and, in my opinion, will be
especially valuable when we find something that did not work.
For example, I helped conduct a study of what are called High
School Career Academies, a promising school reform initiative that is
supposed to prevent students from dropping out and help them enter the
workforce after high school. On average, we found that the programs had
no effect on dropout rates (although it had large positive effects on
workforce outcomes). However, when we dug a little deeper and looked at
students who were at the highest risk of dropping out and were enrolled
in the most dysfunctional schools, we found quite large dropout
reductions.
This was in spite of the fact that the programs in these schools
were not very strong; they were just much better than anything else
available to those students. We were able to attribute these effects to
the program's emphasis on personal relationships and high expectations,
particularly in the context of an otherwise chaotic environment. We
would not have learned this without expanding the prevue of our
research beyond the thumbs up or thumbs down question of ``did it work
or not.'' As a result, that study is listed in the What Works
Clearinghouse--IES's definitive resource on scientifically validated
research in education--as one of only seven with evidence of positive
effects on keeping students in high school.
Questions about how and why policies and practices work or do not
work are important both in the context of new initiatives and when
proven practices are being adapted to new circumstances. In particular,
IES should continue its recent investments in what is called
``continuous improvement research''--a process by which data collection
and analysis are integrated into program development and
implementation. While still in its infancy, this seems like a promising
method for using rigorous research to help schools become more
effective over time.
P3: Promote wider use and application of education research. Again,
I believe this is a dual challenge: 1) treating dissemination as a
continuous process rather than a single event at the end of a study,
and 2) making smarter use of technology to organize and provide access
to high-quality evidence.
Promoting Dissemination as a Continuous Process: In my experience,
this also ties back to the importance of building relationships between
researchers and the audiences we are trying to reach. For example, the
Career Academies study I mentioned earlier was a 15-year evaluation
(yes, 15 years). This work, involving literally hundreds of
contributors and collaborators, has had a profound influence on career
and technical education. While the study found that the Career
Academies produced sustained positive effects for on long term
workforce outcomes for young people, the story of the study's influence
began before we collected a single piece of data. My colleagues and I
at MDRC started this project by asking both leaders in the field and
teachers and administrators in schools what they thought would be worth
learning about innovative approaches to the school-to-work transition.
We continued this dialog at each step in the study providing a wide
range of audiences with early and long-term findings and asking for
their guidance about how our work could be more useful. As a result,
key stakeholders bought into the research process from its inception;
they were able to confront the results, even though not all of them
were positive; and, most importantly, they continue to this day to work
diligently to reform and strengthen their programs to be better aligned
with what we found made a difference.
Prioritizing Smarter Use of Technology: IES has led the effort to
bring dissemination of high-quality education research into the late
20th Century (although probably not the 21st Century) through its
creation of the What Works Clearinghouse--a compendium of studies that
have been screened for scientific merit and catalogued by topic. It has
also supported related resources like the Better Evidence Encyclopedia
and the Society for Research on Educational
Effectiveness. More recently, IES has issued a call for the
establishment of a Center on Knowledge Utilization, whose mission will
be to study how educators and policymakers use research. I believe
these are investments worth sustaining and increasing, particularly if
they continue their development of research-based practice guides in
addition to their mandate to serve as arbiters of what constitutes
scientifically valid evidence.
However, to advance the use of rigorous research, I think IES will
need to make smarter use of technology to make this work more
accessible and user-friendly. I do not think this is a matter of
keeping better track of how many reports get published or how many
website visits they receive. This is beyond my area of expertise, but I
am struck by the ease with which I can find pretty useful and generally
reliable information about restaurant and movie reviews, and ratings of
cars and appliances. I am hard pressed to believe that those of us who
care about making high-quality research more widely available do not
have something to learn from these efforts.
P4: Prepare a next generation of education researchers who are
committed to scientific rigor, to relevance for policy and practice,
and to applying what they learn in the field. This may be the most
important legacy of ESRA and IES.
Hundreds of talented young people have completed or are enrolled in
training programs supported by IES that place a special emphasis on
teaching about scientific research methods. I have had the privilege of
working directly with nearly a dozen of these young scholars, including
six who are now students at NYU's Steinhardt School of Culture,
Education and Human Development. I have been impressed by their
competence, certainly, but mostly by their passion for making a
difference in schools. None of these folks will see themselves as being
successful if their primary accomplishments are to accumulate a long
list of articles in prestigious journals and receive tenure at a
prominent university. While many of them are certainly destined for
these accomplishments, I am convinced that they will make their biggest
impact in producing high-quality work that education policymakers and
practitioners value and use.
From that perspective, I believe IES's support for pre- and post-
doctoral training programs and for its methodological and professional
development activities are critical investments. I particularly applaud
IES's recent call for universities to form partnerships with schools
and school districts as key components of their doctoral training
programs and to ensure the graduate students spend time working closely
with administrators and teachers to learn about their needs and
interests.
In closing, I think we must recognize that despite the great leap
forward precipitated by ESRA and IES, the reality is that most states
and school districts still use rigorous research in policy and
administrative decisions much too infrequently. This is, in part, about
the role that ideology and politics play (both constructively and
disruptively), but it is also because the policy and practitioner
communities not been very involved in the production of evidence and
setting of research priorities. There is still very limited evidence on
issues that matter to them; the evidence that does exist is often hard
to understand and apply; and there is little incentive for them to
produce or use rigorous evidence. The recommendations I am offering
here would go a long way toward addressing these issues and would help
make schools and school districts more active partners in education
research.
Of course, there are more than 14,000 jurisdictions that make
policy and administer K-12 education in the US. The role of the federal
government is limited at best (with only 7 percent of education
expenditures covered by federal funding and limited capacity to manage
implementation). It seems imperative that a not-insignificant portion
of this limited federal investment be accompanied by two requirements--
similar to those we've seen in the Investing in Innovation Fund: 1)
that SEAs and LEAs use federal resources to support initiatives that
have credible evidence of their effectiveness, and 2) when such
evidence is lacking, that they be willing to participate in rigorous
research that will help fill this gap. Together with the four Ps I have
proposed, over time, this approach could help our nation build a firmer
foundation of evidence and ultimately produce better outcomes for our
students and teachers.
______
Mr. Rokita [presiding]. Thank you, Doctor.
Ms. Christie, you are recognized for 5 minutes.
STATEMENT OF KATHY CHRISTIE, VICE PRESIDENT, KNOWLEDGE/
INFORMATION MANAGEMENT AND DISSEMINATION, EDUCATION COMMISSION
OF THE STATES
Ms. Christie. The Education Commission of the States, or
ECS, was created by states, for states nearly 50 years ago. We
help states learn what other states are doing, what new ideas
are emerging, and what the research says.
We provide unbiased information. We don't advocate for
certain education policies. And we don't pick sides.
We are the only state-focused national organization that
works across all levels and sectors of education, from pre-K to
post-secondary, and across branches of government.
The strength of the evidence underpinning policy levers and
initiatives is critical to the success of the policy process,
but policymakers seldom know what the evidence says on every
issue. Much depends on the quality of their staff and whether
they know where to go to find an evidence base. And yet, they
have to make decisions every day, whether they can answer these
questions or not.
That is why having timely, succinct, and understandable
research available is so important and why organizations like
ECS play such a vital role.
My role at ECS lets me sit in a national crow's nest,
watching the horizon for the education problems states are
struggling with and for what they are doing to solve these
problems. But it is difficult to make sure that every education
committee chair, governor's advisor, state superintendent,
governing board member, and higher education entity knows about
our resources. We very much understand the difficulties of
getting good research into the hands of those who can do
something about it.
The Institute for Education Sciences has some entities
available to attempt to address some of these needs, including
regional education laboratories, commonly known as RELs,
comprehensive centers, and content centers. In the past the
production from RELs and centers seemed uneven. Resources
seemed to take a long time to come to fruition, and by the time
they did sometimes the window of opportunity to inform
decisions had passed, and decision-makers were not always at
the table to set the agenda.
Today the What Works Clearinghouse is building a strong
base of easily accessible program reviews and pointing toward
interventions that work and that don't work. Summaries are now
less academic and easier to follow. Practice guides provide
good direction. Conclusions are presented in a more
straightforward manner. Readers can actually easily access
areas by topic.
Conclusions, I am sorry. IES might consider how to more
clearly distinguish between findings regarding whether studies
meet standards of evidence and whether programs actually impact
learning. Overall, however, the site has improved greatly. We
link to the clearinghouse by issue area, which is an efficient
way to immediately capture updates for our constituents.
In addition, content centers on issues of importance to
states, such as turnarounds and state capacity, can be spot on
for meeting state needs. But IES could work to ensure that
vetting and review for activities and outputs does not inhibit
the development of timely, relevant, digestible research and
assistance.
The Best Evidence Encyclopedia, which has funding ties to
IES, is another excellent resource. And my understanding of the
new breed of RELs is that they are working to establish
research alliances across those states with state leaders at
the table.
To put these entities in perspective, though, I would like
to highlight what is good about one of my favorite sources, the
National Bureau for Economic Research. Nearly every week I get
a message highlighting several of their new studies.
These studies are relevant to the problems I see states
struggling with, such as compulsory attendance. They are
timely; they are prolific. IES could look to this model for
improving relevance and timeliness.
Like most academic studies, these are so academic that
people are not going to read them, and for the most part they
are not openly accessible. So ECS is working to translate
studies like this and capture the key findings and
recommendations in our research studies database. We organize
them by frequently asked questions such as, ``Preschool: How
prepared do teachers need to be?''
Since 2008 we have entered 193. We are very thankful for
the GE Foundation for supporting this work.
The reason for this effort is clear: When busy people ask,
``What does the research say?'' any response to that question
needs to be timely, relevant, digestible, and trusted.
Here are the four final points I would like to make.
One: Research matters not only to those implementers in the
field--the superintendents, principals, and teachers--but to
those who are committed to improving the system of education.
Two: The gold standard matters, but while optimal, it is
not always possible. The real world will continue to demand
that policies be crafted based on hypotheses that are
relatively well supported by evidence or whether early evidence
is simply promising. IES could do a better job of ensuring that
topics fit with what matters to states and that its research
helps answer not only which programs work but also which
policies or state investments hold promise and which elements
of those policies matter most so that state-level elected
officials might act on them.
Three: IES needs state leaders to perceive it as an
unbiased, honest broker, so increasing the independence of IES
could be key.
And four: IES should consider a coordinated effort to
transparently evaluate and hold itself accountable on a set of
performance measures that are important to states.
Thank you for letting me be here.
[The statement of Ms. Christie follows:]
Prepared Statement of Kathy Christie, Vice President, Knowledge/
Information Management & Dissemination, Education Commission of the
States
Mr. Chairman and Members of the Committee: My name is Kathy
Christie. I am Vice President, Knowledge/Information Management &
Dissemination for the Education Commission of the States (ECS). The
Education Commission of the States (ECS) was created by states, for
states, almost 50 years ago.
Since 1965, ECS has worked with state policymakers to improve
America's public P-20 education system. We provide unbiased
knowledgeable advice to state leaders and help them to learn from one
another's experiences. What makes ECS unique in the crowded education
policy arena is that we work with policymakers, researchers, thought
leaders and practitioners across all levels and sectors of education--
from pre-K to postsecondary--and across branches of government. We are
the only state-focused national organization that brings together
governors, state legislators, chief state school officers, higher
education officials, and business leaders to advance policies that
improve our educational system. To accomplish this work, we undertake
the following kinds of activities:
We help states learn what other states are doing, identify
best practices, what new ideas in education are emerging, and what the
research says.
We provide unbiased information. We don't advocate for
certain education policies and we don't pick sides.
Our website is one of the best in the country to find
information on hundreds of education issues. You can check out our
website or call us directly; either way, we will provide the
information that's needed.
Most people are not aware that it was ECS--the only nationwide
interstate compact for education--which was responsible for the
creation of NAEP, the National Assessment of Educational Progress. It
is this kind of state collaboration and state insight that makes ECS
unique. The message I bring here today is about the power of state-
level leaders to make a difference in the educational opportunities
available to children.
State Leaders Influence System-Level Reforms
Policymakers at every level in states--chief state school officers,
governing board members, legislators, and governors--all have a role to
play in developing and implementing education initiatives. Very often
these various policy actors have different opinions about how laws
should be shaped. Consequently, evidence related to the hypotheses that
underpin state-level policies and evidence on the relative
effectiveness of state-level initiatives are critical to the success of
the policy process.
They hope, and we do too, that the decisions they make will drive
improvements in what can be a large, bureaucratic system. A constituent
of ours once described the education system as a big pillow. You punch
it--and make a big dent, but gradually that dent disappears and the
pillow puffs out to its original shape. You punch it again, making
another dent. But after a bit the pillow again puffs back out to its
original shape. We all understand how tough it is to make real change.
State-level policymakers seldom know what the evidence says on
every issue. Much depends on the quality of their staff and whether
they know where to go to find an evidence base. Even when they have
capable staff with sufficient time, it can be a challenge to gather and
prepare evidence in a way that allows policymakers to quickly
understand whether it is sufficiently robust to support the decision
they will make.
When considering a hypothesis behind a bill, legislators routinely
ask: What is the level of evidence supporting this proposal? Is it
minimal, with a higher risk to success? Is there some evidence
available, but it fails to highlight the most critical factors that
need to be in the policy? Or is there extensive preliminary research
and piloting, with interventions that have been aligned at all levels
and across agencies--a sufficiently robust knowledge base on which to
guide large-scale decisions?
And yet state legislators have to make decisions every day--whether
they can answer these questions or not. That's why having timely,
succinct and understandable research available is so important and why
organizations like ECS play a vital role in state-level education
policy.
National Perspective
My role at the Education Commission of the States lets me sit in a
national crows' nest, watching the horizon--across state boundaries--
for the education problems states are struggling with and for what they
are doing to solve those problems. ECS scans news clippings every day
and pushes the most relevant out via email--every day. We track the
policies that state legislators are enacting, and we add them every
week to the most extensive, freely-available database of its kind in
the country. Every day we are culling from the professional and
academic literature and pushing the best back out via our web site. But
it is difficult to make sure that every chair of an education
committee, every governor's education policy advisor, every state
superintendent, every state board member and every higher education
agency head knows about those resources. We very much understand the
difficulties of getting good research into the hands of those who can
do something about it. And these challenges don't even begin to touch
the difficulty of reaching every legislator and agency head and
governing board member across the country.
What I have learned in over 20 years with ECS is that we reinvent
the wheel time and time again. Policymakers don't pay enough attention
to history. We might read the research and go, ``oh, yeah, that's
right'' but then we too often jump at some new shiny, glittery answer
or lobby for a new research study rather than taking time to unearth
the root cause of a problem or step back to analyze the existing
research--the research that while there, might not be broadly available
or is so incomprehensible, we don't know what to make of it.
The Institute of Education Sciences has some entities available to
attempt to address some of these needs, including Regional Education
Laboratories (commonly known as RELs), comprehensive centers, and
content centers. Past RELS seemed uneven in production of resources,
particularly those that might remain relevant and useful long after
individual instances of technical assistance or convenings. Resource
development or projects seemed to take a long time to come to fruition,
and by the time they did, sometimes the window of opportunity to inform
decisions had passed. And decision-makers were not always at the table
to set the agenda.
Today the What Works Clearinghouse (WWC) is building a strong base
of accessible program reviews and pointing toward interventions that
work--and that don't work. WWC has improved over time. Summaries are
now less academic and easier to follow. Practice guides provide good
direction for practitioners. Conclusions are presented in a more
straightforward manner. Readers can easily access areas related to
specific topics. IES might consider how to more clearly distinguish
between 1) findings regarding whether studies meet standards of
evidence and 2) evaluations of actual program effects on learning.
Overall, however, the site has improved greatly. ECS is able to link to
WWC issue areas via relevant topic areas (e.g., literacy) on
www.ecs.org, so as studies are added to WWC, it is not necessary for us
to add each new review to our site. This is efficient and immediately
captures updates for our constituents.
Content centers that focus on topics that matter to states--
turnarounds and state capacity, for example--can be spot on for meeting
state needs. IES might review processes to ensure that vetting and
review processes for activities and outputs of these new centers and
for the new RELs does not inhibit the development of timely, relevant,
digestible research and assistance.
The Best Evidence Encyclopedia is another excellent resource: a
free web site created by the Johns Hopkins University School of
Education's Center for Data-Driven Reform in Education (CDDRE) under
funding from the Institute of Education Sciences. And my understanding
of the new breed of Regional Education Labs (RELS) is that they are
working to determine what states need and helping to establish research
alliances across those states--with state leaders at the table. But
like many education policy organizations, including ECS, RELs may
struggle to raise awareness of available resources and how they can
support their states and school districts.
From my crows' nest, I seek out and cull research reports from the
comprehensive centers and the content centers--although some are more
prolific than others.
To put this comment in perspective, I'd like to highlight what's
good about one of my favorite entities--the National Bureau for
Economic Research. Every week I get an email summarizing the education
studies they have completed. Every week. Not four per year. Not one per
month. Every week. They're called Working Papers, so one's immediate
assumption is that they are food for thought--not to be considered
definitive, but findings to think about. Why are they so compelling?
They are relevant to the problems I see states struggling with--like
compulsory attendance, for instance. Does the age at which kids start
school matter? Does the cut-off age (5 by September 1, for example) for
attendance matter? Does the upper compulsory attendance age make a
difference over the course of a lifetime? NBER studies are relevant and
timely. They look at the types of issues that governors and state
legislatures can influence via policy. They are prolific producers of
what most would agree is quality research.
These studies are dense. Their titles are often just abysmal and if
they were movies, no one would buy tickets (e.g. Interpreting the
Evidence on Life Cycle Skill Formation), even though they have much to
offer. They are so academic that most state policymakers--or their
excellent staff--are not going to read them. The same goes for studies
coming out of AERA. And for the most part, the studies are not openly
accessible. But they have something to say to you. And to state
legislators. And to governors. So ECS is working to translate studies
like this and capture the key findings, recommendations, and
implications for policy in its Research Studies Database. We organize
them by frequently asked questions such as ``Preschool: How prepared do
teachers need to be?'' or ``High school curriculum: How important is
rigor?'' Since 2008, we have entered key findings and policy
implications from 193 studies into The ECS Research Studies Database.
We are very thankful to the GE Foundation for supporting this work.
The database could be easier to use. It could be ``prettier.'' We
have created standards for inclusion, and an important standard is that
studies need to have appeared in peer-reviewed journals. But we do make
exceptions.
For example, I personally mined Crossing the Finish Line--a 2009
book by William Bowen, Matthew Chingos and Michael McPherson--names
probably familiar to many in this room--that looked at what impacts
student persistence and success across postsecondary institutions.
Compelling, compelling statements built from statistical analyses are
buried throughout, but they were most certainly read by more academics
and higher education leaders than by policymakers. The authors'
analysis supports, for example, the clearly articulated statement that
``both parental education and family income are strongly associated
with graduation rates even after controlling for related differences in
student characteristics, particularly academic preparation. However,
family income, not parental education, is primarily responsible for the
overall relationship between SES and time-to-degree.''
If another credible, vetted resource emerges to counter such
findings, ECS will not hesitate to include it in the database as well.
As exceptions to the peer-review rule, we include NBER studies,
many of which eventually are published in peer-reviewed journals--but
in the interim, they reflect the food for thought that state leaders
need.
Do enough state leaders know about this resource? No. Have we been
successful in marketing it? Probably not. We always include new studies
in our weekly e-newsletter, e-Connection, and they always get the most
hits; regrettably, we have not been particularly successful in
marketing its availability.
The reason we acted, though, is clear. When a consistent element of
questions is ``what does the research say?'' the response needs to be
timely, relevant, digestible, and trusted.
Here are the four final points I would like to make:
1. Research matters not only to those implementers in the field--
the superintendents, principals and teachers--but to those who are
committed to improving the system of education.
2. The gold standard matters. But while optimal, it is not always
possible. The real world will continue to demand that policies be
crafted based on hypotheses that are ``relatively well'' supported by
evidence or where the early evidence is ``promising.'' IES could do a
better job of ensuring 1) that topics fit with what matters in states;
2) that its research helps answer not only ``which programs work'' but
also which policies or state investments hold promise--and which
elements of those policies matter most so that elected officials might
act on them.
3. IES needs state leaders to perceive it as an unbiased, honest
broker, so increasing the independence of IES could be key.
4. In that regard, IES might want to consider a coordinated effort
to transparently evaluate and hold itself accountable on a set of
performance measures that are important to states.
______
Mr. Rokita. Thank you, Ms. Christie.
Thank the witnesses for their testimony.
I now would like to recognize Chairman Walden, Walberg for
5 minutes.
Mr. Walberg. Thank you, Mr. Chairman. Put a W in front of
it and we are always at the end, so we all look the same. Thank
you.
Appreciate the panel being here today, and in the process
of doing a lot of reauthorization of laws that are in place,
agencies that are in place, the question continues to come to
my mind of why.
Ms. Christie, should reauthorization of Education Science
Reform Act take place?
Ms. Christie. I think IES is an incredible resource. Like
all of us, it can get a lot better.
Mr. Walberg. Mr. Scott, I would ask you the same question.
From your perspective with GAO, should reauthorization of the
Education Science Reform Act--of course that is very pertinent
to IES--take place?
Mr. Scott. Well, I think as our work demonstrates, IES has
played a significant role in improving the quality and the
rigor of education research, and given the number of efforts
that are currently underway to reform the U.S.'s educational
system, it is critically important that we continue to have
some vehicle such as IES to ensure that the various programs
that are underway can be properly evaluated.
Mr. Walberg. Ms. Christie, is it the responsibility of the
federal government to do this research, or could it be done
equally as well--maybe more efficiently--at the private sector
level and, in fact, at the states in competition for
educational quality, promoting research entities that would
definitely be looking at states and their responsibility for
education?
Ms. Christie. We love to highlight states that in every
bill put in an evaluation component to--so that they are
modeling continuous improvement to their constituents, their
schools, their districts. Many times it is the cost component
of that that is difficult, and there are other times when if
there are common problems across the states--and there are;
they are nearly all common even though we all would like to
think we are absolutely unique--but when you have common
problems that is the power of ECS. You do things as a
collaborative. You do things that keep the scale within reach.
And I think the federal level has always had a role in
keeping the spotlight on equity and ensuring all kids have
opportunities, and I think that there are certain things that
are probably fitting for both to do.
Mr. Walberg. I guess my concern comes back that also on the
federal level bureaucracies develop that take $600 million of
resources that certainly could be effective at a private sector
entity, looking for best practices with those unique students
we have. And yet, as you I think accurately state, with their
uniqueness there is a great deal of sameness as well, that
parents, local school boards, school districts,
superintendents, teachers want to achieve in the outcome of
students.
And I guess I am not hearing, as of yet, a strength in the
answer that this could not be done in somewhat a market-based
approach at the local level--of states specifically--of doing
the research that is necessary with best practices that are out
there and a clear understanding that we need to achieve those
and how do we do that.
So respond to that, Ms. Christie, if you would, please?
Ms. Christie. The big component here is trust. So if you
have a lot of private sector folks doing research, which even
right now, I mean, when IES contracts with REL providers, those
are basically private contractors.
There needs to be a sense that what is coming out is
unbiased and that it can be trusted. And I worry that if that
gets outside of an independently verified group that you could
lose that trust, and I think that would be a huge gap, then, in
the research.
Mr. Walberg. Well, Mr. Chairman, I would yield back my time
after saying that if we are talking about trust in context with
the federal government and independence in context with the
federal government we have a major hurdle to get over.
I yield back.
Mr. Rokita. I thank the gentleman.
Ranking Member Miller is recognized for 5 minutes.
Mr. Miller. Thank you very much.
Sounds like there is a considerable amount of trust here at
this table in this process, with some suggestions for
improvement--but not eradicating the concept that IES has been
put here.
Dr. Kemple, you--I think the panel seems to agree that we
are doing pretty well on the rigor side here. The questions is
about how do you get that dissemination to take place that is
actually useful at the state level or even at the district
school level.
And that has always been a problem for me. I would say
before we got into this process, what we had was a landscape
littered with 5-year pilot projects that just died and nobody
ever asked what happened to them or did it work or not, and
then somebody came along over and over and over again, very
often with the same pilot project just in a different state,
different district around here.
So how are we doing on this dissemination in terms of
following through on it--not just getting it there, but then
what is done to help people if the state decides they want to
go through it? I mean, we are sort of going through this with
Common Core. Our governor just put up $1 billion to help the
districts with the implementation. I mean, this is a big hurdle
in terms of taking new findings of rigorous research and
suggesting to people, ``This may be a better way to do
something than you have been doing for 15 years the other
way.''
How are we doing on that hurdle?
Mr. Kemple. I think this is a two-fold problem. I think it
starts with thinking about dissemination and promoting the use
of research as a process rather than as an event. Too often we
place much too much emphasis on waiting until the study is done
until we actually think about what we have been able learn from
that.
I think, A, it is important to think about dissemination as
it is starting before even one iota of data is collected to
make sure we are asking the right questions; I think it is
important that we incorporate the policymakers and the
expertise and the practitioners who are part of the formation
of the research projects or the evidence-building process in a
continuous way throughout a project so that we are feeding the
information that we are learning in process into a decision-
making process and into a learning process.
And I think thirdly, it is important to make sure that as
we package results, that as we put results together, we are
translating those into practice guides so that we are being led
by the best evidence possible in a way that is actionable and
that makes the practitioner community--or puts the practitioner
community in a position to learn from both the work in progress
and the work at its conclusion.
Mr. Miller. Dr. Long, this is your field. How is it going?
Ms. Long. I would say that IES is doing a wonderful job in
terms of production and leveraging the field to produce the
research. I think the dissemination part of it is a work in
progress.
And actually, as the board and IES have talked about it
much more in terms of communication, which is a two-way street
between IES, the work that is produced, and what is known with
the field and what the field needs in order to do better work.
Part of what is involved with that is translation--taking
very rigorous research, which education has gone forward
tremendously in having causal results, but translating that for
the layperson, for the teacher who is in the classroom so they
know how to use it, and then learning how to disseminate it.
IES needs to do more----
Mr. Miller. Ms. Christie, is that your job? At that stage
is that your job, the handoff here to the states?
Ms. Christie. We hand off everything we can get our hands
on to the states, yes----
Mr. Miller. No, but I am just asking, you know, you get
this new research, you get a model for dissemination or to the
importance of this research, and I just want to--so where do
the states pick this up and make a decision?
Ms. Christie. Yes, we----
Mr. Miller. Not all research is welcome, you know, I find
from time to time in the education establishment because it
suggests substantial change--significant change. So how do the
governors pave the way when you have rigorous research that
suggests you have got to change directions? I mean, we are now,
what, 2 years into the Common Core with governors modifying and
rethinking the model back and forth to where we are today.
Ms. Christie. Well, if I could make one suggestion, so much
of what is in the What Works Clearinghouse, for example, is
about the programs, not necessarily about those big policy
issues that a governor needs to take up.
But I think research is welcome, it just needs to be
crafted in a way that folks like me--I mean, I am just a
translator; I am not an academic--that I can understand and
then I can put into practice.
Mr. Miller. Well, Mr. Kemple, how do we get the feedback
loop back to the researchers and the disseminators at IES based
upon what you have learned in modification? Because if you look
at teacher sites, teachers are always modifying somebody else's
lesson plans, somebody else's approach, and it is rather an
interesting process. How do you get what you are doing in New
York back to the IES and others about how this is going?----
Mr. Kemple. Well, I think just in the way that a teacher
collaborative, as you mentioned, works, as someone is learning
from--taking what one teacher has done and they are building
from another, those folks are all at the table together talking
both about what their goals and objectives are, what kinds of
problems they are facing in the classroom, and what kinds of
different solutions they have each come up with on their own.
Again, I think this is a process of making sure that all of
those folks are somehow at the table as we discuss the nature
of the problems that we face, the solutions we have each come
up with, and then cycling that back into an effort at
continuous improvement.
Mr. Miller. Thank you.
Mr. Rokita. I thank the ranking member.
Mr. Tierney is recognized for 5 minutes.
Mr. Tierney. Thank you.
Thank the witnesses for their testimony. This, I think,
follows along with Mr. Miller's questions a little bit, and it
may--I think it is a fair question to ask. Can any of you tell
me about a program or practice that was actually affected by
the research--one that was either scaled up or eliminated or
changed in some way?
Mr. Kemple. Sure. I was privileged to lead a study of what
are called high school career academies, which are high school
reform initiatives aimed at dropout prevention and helping
young people make healthy transitions into the workforce and to
college. This is a 15-year study--yes, 15 years. So if we had
waited 15 years for this to translate into some change in
practice I think we would probably have lost any momentum or
chance of making a difference.
But this was an initiative that found little or no impact
on college readiness, although many of the young people in the
study went on to college, but found massive effects on their
capacity to find good jobs, keep good jobs, and climb a career
ladder in the 8 years following high school graduation. We were
able to form a coalition of stakeholders in school districts,
at the state level, and among expert organizations across the
country that were part of helping us form that research project
and part of the process that we used to disseminate what we
were learning over the course of the 15-year study, so that by
the time we got toward the end of that project many of those
groups had already begun to synthesize the work into the
creation of organizations that were aimed at supporting the
standards for implementing--for creating and sustaining these
career academy programs and for a continuous improvement
process to work on some of the weaker aspects, such as the
academic curriculum and the college access programs.
Mr. Tierney. So 15 years--that is before the 2002----
Mr. Kemple. That is right.
Mr. Tierney [continuing]. Authorization of this act. So was
that done under this act or something else?
Mr. Kemple. That was actually done independently through
private funding.
Mr. Tierney. Okay. Well, my question to you is can you give
me an example of something that occurred under this act that
either changed in some way a program or a practice----
Ms. Christie. Dr. Long?
Ms. Long. Yes. I can speak about some of my own work that
was supported by IES.
Mr. Tierney. Please.
Ms. Long. Working with two of my colleagues, Eric Bettinger
and Phil Oreopoulos, we worked with tax preparers to help
families fill out their student financial aid forms, the FAFSA.
And so after completing their taxes we only needed 5 additional
minutes to pre-populate the FAFSA using software and then
asking them a few additional questions as well as giving them
information about college prices and what they needed to do in
order to access those institutions. We found huge results in
this randomized controlled design of getting many more students
into college by just offering 5 minutes of assistance.
This report was first released in 2009, right around the
debates about FAFSA simplification, was it worthwhile, and we
have subsequently gotten an additional grant from IES to
continue to see how we can expand these kinds of services and
community tax sites around the country.
Mr. Tierney. And did the FAFSA reforms in any way play off
of those findings, and did they----
Ms. Long. We certainly fed that information into the
debates and received many calls from both states as well as
many members of Congress around the federal government to
understand that, yes, information is a huge barrier. This was a
randomized controlled trial, simplifying things and giving
assistance, and it turned out it was very cost effective.
Mr. Tierney. Dr. Scott, or Mr. Scott, do you find enough of
those examples in your work, in your review of this agency to
continue to warrant its continuation?
Mr. Scott. I think, as we noted in my statement, IES is
uniquely positioned to act in this area. I think it is
important, though, that as we also pointed out, that they take
some steps to improve. For example, in the areas of continuing
to get feedback from practitioners and policymakers, continuing
to hold various aspects of their operations accountable, such
as the RELs, and then continuing to measure and report out on
their activities. So I think IES is uniquely positioned to
contribute in this area but there also needs to be some
improvements to their operations and accountability.
Mr. Tierney. Thank you.
Do practitioners--do actual teachers and superintendents
and principals--have a way of connecting with IES and looking
at these reviews periodically and interacting? Is there some
setup to encourage that?
Ms. Long. There are several efforts. There is information
that is available online. There are constant efforts to
translate and make this information available. For example, the
practice guides have been cited, which really break down the
research to what teachers can do in their classrooms.
But this is a work in progress. Much more needs to happen,
and IES is taking steps to get more feedback from the field
about what they need and how they need to use that information.
Mr. Tierney. And are you working with colleges that teach
people to become teachers as to how they might access a tool
like this and make it part of their overall practice as they go
out and teach?
Ms. Long. That is one of the audiences, but much more could
be done with them.
Mr. Tierney. Thank you.
Thank you, Mr. Chairman.
Mr. Rokita. I thank the gentleman.
Ms. Bonamici is recognized for 5 minutes.
Ms. Bonamici. Thank you very much, Mr. Chairman.
And thank you to the witnesses for being here today for
this discussion. I, especially as a member of this committee,
truly believe that education is an investment in our country's
future and so it is important for us to get the policy right
here.
And, Ms. Christie, I want to say that as a former state
legislator, thank you. Frequently relied on the Education
Commission of the States and your work there.
I especially appreciate your pointing out in your written
testimony that you have learned over the years that we tend to
reinvent the wheel, and also pointing out that it is important
to take the time to really unearth the root cause of a problem,
and I hope that this hearing today will help with both of those
areas--help us not reinvent the wheel, but also not neglect the
importance of looking at the root cause of the problem. And I
have often spoken about the importance of, for example, looking
at poverty and homelessness and how that affects students in
school.
So I wanted to talk a little bit about what many of you
have mentioned both in your written and oral testimony, and
that is the relevance of research. So I am from Oregon, and we
have the Oregon Leadership Network Research Alliance, which is
under our Education Northwest, and they have really been
working to bring together the practitioners on the ground with
the policymakers, understanding that the research needs to be
accessible to both of these groups, and I agree.
So I would like to talk a little bit about what relevance
means.
And, Dr. Kemple, you said that many states and school
districts don't use rigorous evidence because they have not
been involved in producing or guiding what research is done. So
you also mentioned that it is important to prioritize
educational research that is relevant to policymakers and
practitioners.
So can you all talk a little bit about what does relevant
mean and who is determining relevancy? So are the practitioners
and the policymakers all saying that the same things are
relevant? And how do their views differ?
And then who ultimately makes that decision? If we are
saying we want relevant research, who is going to make that
decision? Is that going to be us in--as policymakers, is it
going to be the practitioners, or is it going to be the
researchers, and why?
Mr. Kemple. So briefly, I think relevance means two things.
One, I think it means, whose questions are we asking as
researchers or how do we--whose questions are we prioritizing
as researchers? I think that is a two-way conversation. I think
it has to include people who are the ultimate consumers of
their research. If it is a question for Congress about whether
they would like to make sure that an investment in early
childhood education, adolescent literacy, afterschool
programming is paying off in terms of improving--achieving the
goals that it was set out to, either improving teaching or
improving learning among young people, we ought to know that
those are priorities for what gets learned and that
conversation needs to happen between the people who are making
the tough decisions about how to allocate scarce resources for
the benefit of children.
Secondly, though, I think it means that we have to place a
higher priority on the questions about why and how and under
what conditions are investments in education improvement make a
difference or don't make a difference. In my view, it is just
as important to find out something doesn't work as it is to
find out that it doesn't work--or that it does work, and--but
going beyond the thumbs up, thumbs down to say, ``Why did this
work? What were the causal mechanisms? What was the appropriate
context?'' Those are the things that will matter most and
translate well enough in Oregon as well as New York State or
Arizona or Texas.
Ms. Bonamici. Thank you.
And from the other witnesses, do you want to give me some
input on how those determinations are made about what research
is done? How is it determined that it is relevant? And how can
we better improve those--that communication between the
practitioners and the policymakers and the researchers?
Ms. Long. I think the importance of partnerships cannot be
underscored enough. And this is a movement that IES is going
towards, that there has to be a feedback mechanism where you
are doing research in partnership with the field, with the
teachers, the schools, the universities, so that as you learn
as you go along and it is a continual improvement type of
model. Because what is relevant can also change very quickly.
At the same time, you need a federal organization that is
going to be objective and look at the national good and say,
``This is what is important for the country. This is the
information that we all need to know.'' And some of the myths
that we have had before, we need to realize those don't work
and other things do, while also feeding that information to
people who are making decisions--the policymakers--to
understand, given our limited resources, we have to make
decisions, we have to prioritize. So that is also very
important and relevant.
Ms. Bonamici. Thank you.
Ms. Christie?
Ms. Christie. Sure. I would like to push on a little bit
different way that might not have been talked about in the
past.
There are groups like ECS and the National Conference of
State Legislatures, National Governors Association, Council of
Chief State School Officers, the state boards organization,
that if someone reached out to all of us we could weigh in on
what we hear from the states. I mean, I watch every day e-clips
across the country, we have a policy database that even someone
at ECS could put together a team and sit down and actually look
at what are the policies that are being enacted and what are
states struggling with? Because they are like the tip of the
iceberg for what people need to know.
Ms. Bonamici. Thank you.
I see my time is expired. Thank you, Mr. Chairman.
Mr. Rokita. Thank the gentlelady.
Ms. Foxx is recognized for 5 minutes.
Ms. Foxx. Thank you, Mr. Chairman.
And I want to thank our panel for being here today.
And I want to say, I want to associate myself with the
train of questioning that Mr. Miller was going at before, I
believe, and other folks, in terms of the way we are looking at
this issue, and particularly in the way of application. That
old saying about deja vu all over again, I have been in this
business for a long time and this panel--we could be in a time
warp here, actually, because I believe I first heard this kind
of debate and this kind of discussion back in the 1960s and
maybe 1970s and 1980s when I was on the school board, when I
did my master's degree.
We have been dealing with this issue forever in terms of
how do we get the research--how do we get the knowledge that we
have applied appropriately in the places where it can do the
most good? And it seems as though we haven't figured that out
yet.
I mean, people have been decrying the fact that folks in
education just ignore the research and the results.
So what I would like to ask each one of you is do you have
some examples of where you have seen the research that we--of
programs that work well, methods that work well--where has it
been applied well, appropriately, and how do we replicate those
situations without it costing a lot of money?
Anybody?
Mr. Kemple. I guess one example I would point to for now
is--and highlight the fact that I think one thing that really
does differentiate us now from the 1960s, 1970s, 1980s, and
even 1990s is that what was not on the table then was whether
we had evidence we could really believe. I think by and large
we were making big investments in research that really didn't
provide us with any causal link between the programs and
policies that were being paid for and the outcomes that we
cared about for the young people in the schools.
I think one--what we are starting to see--and among those
things were teacher preparation. I think what we see is a long
litany of research that suggests that it doesn't matter how
much--where people were trained, what their test scores were
coming out of graduate schools of education, and whether, in
fact, they had a master's degree. What mattered most was their
experience in the classroom and the way that they were
mastering their subject matter material.
We now have some pretty solid evidence that some
alternative routes to certification or entry into teaching are
turning out to be as if not more effective than traditional
routes through schools of education. I think we are starting to
see a proliferation of strategies to draw in talented people
from across the country to opt for teaching as opposed to other
highly prestigious occupations, and some of that is growing out
of the research that suggests that initiatives like Teach for
America or the New Teacher Project are producing positive gains
for young people in classrooms.
Ms. Long. To build from that, even today there is the
release of a report looking at alternative certification
programs that is being released that was partly funded by IES,
and it has completely changed the way people discuss what is a
high quality teacher. Many of the old models that we have been
using for years of just get a master's degree have been called
into question and because of the data that IES collects we know
that the way that teachers are distributed across schools
varies incredibly. Where there are high-need schools with
students that are suffering, we have to figure out policies of
how to get teachers there.
I think the other thing IES has definitely changed is the
way that people are trained to do education research and who is
attracted to do education research. You now see in graduate
schools of education people who used to be former teachers, who
worked in community-based organizations, who now, combined with
all of these rigorous tools, are doing research in a very
different way than they did in the past.
The other thing that is very different now is technology,
and this is something that IES is grappling with: How can you
use technology to communicate and get feedback from the field?
And that is everything from building a Web site, which is
certainly not enough, to podcasts and other kinds of uses of
technology, and we have seen several examples--everything from
how they are communicating with teachers to parents to trying
to distill some of this information so that they can use it.
Ms. Christie. Can I weigh in?
The early literacy and the early learning research I think
has had a dramatic effect. The problem is I graduated with a
teaching degree in 1970, and at that time I knew that the first
three years of a child's life were the most important ones, and
that is why later in my career I decided to stay home with my
kids for 12 years.
So you are absolutely right about the recycle. And ECS had
a huge early learning initiative in the 1970s, so we have
known. The trick is, exactly, why do we keep moving on to the
next easier thing to do than getting back to what we need to
do, how do we push that back out, how do we make sure we are
actually making progress? And those are the really tough
questions, I believe.
Mr. Rokita. I thank the gentlelady.
Ms. Foxx. Can I indulge for 10 seconds?
Anybody who comes in this room who is in the education
business and uses the word training will get my lecture. We
train animals and we educate people. So those of you who are in
the business of education, I would ask you to not talk about
training but talk about education.
Mr. Rokita. I thank the gentlelady.
Mr. Scott is recognized for 5 minutes.
Mr. Scott of Virginia. Thank you, Mr. Chairman.
And I have some of the same questions that the ranking
member had about what you actually do with the information. For
example, I notice that the What Works Clearinghouse studied
many dropout prevention programs. Exactly what do we know about
dropping out and what can be done about it on a local level?
Ms. Christie. Can I jump in?
Mr. Scott of Virginia. Sure.
Ms. Christie. I think the data that is some of what IES is
funding--for example, with early warning indicators and that
sort of thing--is a tremendous leap forward, and I think states
are paying a great deal of attention to that, trying to figure
out how to even get those early warning indicators down into
the very early grades to identify kids very early on.
The interventions that are in the What Works Clearinghouse
on trying to prevent kids from dropping out of school, I think
there is the kind of information there that people can get
their arms around.
I have to tell you that at the state level we moved offices
several years ago and we had box after box after box of state
dropout initiatives. There was not one piece of evidence tied
or--in any of those boxes. Nothing had been evaluated.
So it is just so critically important, and it doesn't
always get done.
Ms. Long. I think the goals of the What Works Clearinghouse
are very admirable. Education is a very crowded space and there
are many opinions and there are many organizations that are
releasing different reports.
And because much of the research is very complex, the
average person oftentimes cannot understand it and so IES has
stepped in as a federal role to say, ``We will be an objective,
unbiased source. We will help to translate what all this
complex work means.''
And that is where the What Works Clearinghouse steps in,
looking at studies to say, ``People are making these grand
statements. Can you actually believe these results?''
To walk away and say, ``This program does or does not
work.'' Knowing that it does not work is also very important.
But the What Works Clearinghouse, I would say, has many
opportunities for improvement in how it interacts with the
field, from the way it does its studies to making sure that
people in the field know that it exists. There have been recent
changes even to its Web site so that when someone goes it is
organized in terms of frequently asked questions--I want to
know about how to help my preschooler--and starting to organize
and revamp the site in that way.
So it is definitely an area of continuous improvement, but
the goal of trying to give an objective, clear, easy to
understand resource is absolutely something that we need.
Mr. Scott of Virginia. Once you know what works how do you
get--do you have a training mechanism so teachers can actually
get trained in what works or is it just sit there, they find
out what works but don't know how to do it?
Mr. Kemple. I think the mechanisms for translating the
evidence into practice are--in many ways are still wanting. I
think from a policymaking perspective and from the perspective
of trying to decide how to allocate scarce resources, I think
one critical lever may very well be ensuring that if a state or
a district wants to use federal resources or a district wants
to use state resources, or if a school has access to some
flexible resources they should be obligated to demonstrate that
the way that they would like to use those resources has
appropriate and rigorous evidence behind its effectiveness, and
that they have a way of being able to link with the people who
either have produced that evidence or started to translate it
into guidelines--that they can use to implement their
programs----
Mr. Scott of Virginia. Well, but how does--how does a
teacher get trained in what works? I mean, he or she is sitting
in the classroom, you read this research works, how do you
learn how to do it?
Ms. Long. I think that is where the practice guides come
in, because that is taking of research and putting it into the
pieces of the day-to-day job of a teacher, now what do you do
with this information? How do you actually apply it in your
classroom? And so that is another translation function that IES
has taken upon itself.
Mr. Scott of Virginia. And is it working?
Ms. Christie?
Ms. Christie. If I could just make one suggestion that I
don't hear talked about at all, and that is if every time there
was a new piece that was put into the What Works Clearinghouse
or a new conclusion drawn, if that could be pushed out to
school boards across this country, it is school boards, then,
that help decide how professional development is delivered, how
teachers collaborate within the districts, what they should be
looking at.
And IES already has the contact information for every
district. You wouldn't have to have all the boards. The
districts then could relay that to the----
Mr. Scott of Virginia. My time is almost expired. I wanted
to get in another question about whether or not there is
research on nonverbal communications, which can be very
important in how children react to education.
Mr. Kemple. Not something I am very familiar with.
Mr. Scott of Virginia. You are not?
Mr. Kemple. Not, no.
Ms. Christie. I am not sure what you are asking.
Mr. Scott of Virginia. Nonverbal communications and teacher
interacting with the student, conveying caring or not caring
and the children reacting to that. Is there any research on
that?
Ms. Christie. I am not familiar with the research on that
but I am familiar with the tools that IES has to quickly find
out who else knows or done any research on that.
Mr. Scott of Virginia. Thank you, Mr. Chairman.
Mr. Rokita. I thank the gentleman.
Mr. Holt is recognized for 5 minutes.
Mr. Holt. Thank you.
I thank the witnesses.
This is encouraging. I am hearing a--what I had hoped to
hear about IES.
I will begin by saying we badly need rigorous research. We
have been hampered for, well, decades--maybe forever--because
every policymaker, every school board member, policymaker on
the state or federal level was a student and, therefore, is an
expert on education. And so we end up with the same old, with
frequent overlays of short-lived fads. And so I hope that we
can make the most of this rigorous research.
The proposal came from Mr. Walberg on the other side that
we devolve this to the states or turn it over to the private
sector. Let me ask two specific questions: Are most states
doing rigorous, relevant research? Does anybody----
Ms. Long. My sense is the answer is no because they don't
have the capacity to do it.
Mr. Holt. Okay. Is the private sector doing sufficient
studies of rigorous, unbiased research that teachers or
policymakers can trust?
Mr. Kemple. On their own initiative I would say very few,
if any.
Mr. Holt. So the conclusion I draw from that is we need
this. So let me ask the next question: Do we need more of this?
Are we close to saturation in producing the kinds of rigorous
studies that we need to wisely spend many tens of billions of
dollars every year in actually--well, in education?
Ms. Long. No. Not at all.
Monitoring the applications for research proposals over
time to IES, the quality of them has increased dramatically
over time. And unfortunately, many of them are not funded, and
during the last year there were a number of very highly rated
research proposals through the peer review process that
unfortunately were not able to be funded.
So there are many great----
Mr. Holt. More than were funded? Were a minority of the
well-reviewed proposals funded?
Ms. Long. That is correct. That is correct.
Mr. Holt. Okay. Does the IES review the--have there been
studies within the IES of the application of former studies?
Does the IES review the application over the years or the
follow up that occurs on those studies?
Ms. Long. There has been a lot of attention. It has looked
back and it is hard to find out what impact those have had, and
so this year there is actually a competition to establish a
center of knowledge utilization just to address that issue as
well as requiring funded researchers to have dissemination
plans and communication with the field so that their research
results are used by those practitioners.
Mr. Holt. Changing the subject, one of the things we are
asked to consider reauthorizing would be the state longitudinal
data systems. Are any of the witnesses expert in the data
systems? My specific question is, are we learning from the good
data systems things that are useful in the classroom?
Mr. Kemple. I think we are just beginning that process. In
New York City we have formed an explicit research alliance that
works closely with the New York City public school system and
have been able to acquire its entire administrative records
database, along the same lines as what the state longitudinal
data systems are supposed to be doing across the country.
It has turned out to be an incredibly valuable tool both
for research and for policy and practice. I think to the degree
that we can continue to help states both create these systems
and then have them link up with the capacity to make use of
those for research purposes, I think we will quickly see
mechanisms that will allow that work to penetrate into schools
and classrooms.
Mr. Holt. And then I guess this is specifically for Dr.
Kemple: How can research be embedded in the programs that are
authorized and funded? How good a job are we doing at it? How
could we make that happen?
Mr. Kemple. Again, I think this is something that has a
long way to go, I think in two ways--one, particularly when
there are hard decisions about how to make use of resources
that can be used for innovation. I think any use of resources
for innovation, be they from the federal, state, or local
level, should be accompanied either by strong evidence that
this innovation has the capacity to change teaching and
learning in schools for the better; or B, if there is lacking
evidence, that there is a requirement that the participants or
the recipients of the funding be willing to participate in a
rigorous research study to establish whether or not those
resources are paying off in better teaching and learning.
Mr. Holt. Thank you.
Mr. Rokita. I thank the gentleman.
Mr. Hinojosa, you are recognized for 5 minutes.
Mr. Hinojosa. Thank you, Mr. Chairman. I thank you all for
inviting this wonderful panel to talk to us about improving
teaching and learning in classrooms across our country.
I want to say that I am a strong supporter of the regional
educational laboratories and the comprehensive center programs.
My first question is going to go to Dr. Long about the new
emphasis at the Institute of Education Sciences, you call IES,
on knowledge utilization, which I think is just a fancy way of
saying we want to ensure that educators know about the research
on what works.
Therefore, can you tell us about the work at IES Sciences
in this area with early childhood development?
Ms. Long. With early childhood development specifically?
Unfortunately, I can't speak specifically to that area. I can
say, generally the issue has been how do we translate our
research, research that might be on early childhood education--
while also trying to understand from the practitioners' view,
what are the questions that they have, what are their pressing
needs. You have to understand how they do their work so how
they might actually change practice with the research results,
so that it can't be just one-directional from IES. What IES
needs to do and is trying to do is build a two-way street so we
have communication from the people who work with our young
children of what they need, what are the pressing questions,
and how they might--how they do their work and then how the
research might be able to inform that.
Mr. Hinojosa. I am going to ask another question and
probably come back to you, Dr. Long.
Ms. Christie, you spoke about your learning the importance
of education from cradle to age 3 years of age, and you used
your children that you stayed home and educated them at home.
Why do you think that state and federal elected officials do
not consider a high priority to invest in early education
programs?
And the reason I ask you that question is that years ago
Buck McKeon was chairman of our Subcommittee on Higher Ed and
took a codel to China. We were trying to find out why it was
that they would usually beat the United States on competition--
international scholastic competition. And one of the people who
answered our question was an older gentleman--a professor--and
he said it was a very simple answer.
The formula he said was early reading plus writing equals
success in school, and that they started reading to the
children the moment they were born. So they read to them in the
cradle and then they already knew how to read by the time they
were 3 years old and pecking on a computer at age 4 to tell us
what they read. It was that simple.
So you tell me, was it research that you read that made you
believe that you wanted to stay home and spend the time you did
with your children from the time they were born on?
Ms. Christie. When I was in my preparation program for
being educated as a teacher, yes, that was part of the program.
I do believe state policymakers are very compelled to address
those early years. Sometimes it is simply a fiscal issue, and I
think we are seeing the investments go up.
We track state of the state addresses by all the governors
and I can tell you, it was a prominent part of a number of
governors this year and that is not unusual. We do see that. So
they are very--they are not uninvolved in it. They are very,
very interested and most of the time it is a fiscal issue.
Mr. Hinojosa. Dr. Long, do you have research that shows
that it is working, where they invest in the regions of the
country in the cradle to age 3 and 4 on early childhood
development?
Ms. Long. There is certainly a research basis for those
early investments have long-term impacts. There have even been
studies that looked at children in preschool and then followed
them years and years later to see that they were doing better
in high school and better in college. So there is that research
basis.
And I would say in a short amount of time, as we have
started to collect this information, fund additional
information, and put it in a central location so we can figure
out how all the different pieces of the puzzle fit together, we
are starting to come out with some strong conclusions about
what does work, although there is so much more that we don't
understand and so many things we have found that don't work,
and so we need alternatives.
Mr. Hinojosa. Thank you, Mr. Chairman. I yield back.
Mr. Rokita. I thank the gentleman.
I just have a couple questions.
Again, I want to thank the witnesses. I appreciate all your
testimony. Let me start with Mr. Scott.
Your testimony stated, I believe, that the performance
measures that IES uses do not reflect current programs, so can
you provide some specific examples of this?
Mr. Scott. Mr. Chairman, I do think it is important to note
that when you look across any number of the current performance
measures IES has in place, they are actually doing quite well
in many areas. A couple areas that we have identified where we
believe there needs to be more transparency and accountability
includes the RELs as well as some of the new research grant
programs that IES has not yet developed performance measures
for those activities.
Mr. Rokita. And what can we be doing better to--or IES be
doing better to accurately evaluate its programs? Is that the
same thing?
Mr. Scott. I think first of all, involve policymakers and
practitioners in the research agenda setting; develop relevant,
timely, up-to-date performance measures; have the information
necessary to evaluate those activities; and then hold those
activities accountable.
Mr. Rokita. What was the second to the last one?
Mr. Scott. Develop the measures, involve stakeholders in
the research agenda setting----
Mr. Rokita. What would a couple examples of performance
measures be in your mind?
Mr. Scott. You know, for example, identifying feedback
opportunities to involve stakeholders, measuring how you are
doing in relation to what the stakeholders' needs are. They do
have a number of measures related to the What Works
Clearinghouse, in terms of the level--the number of
interventions that have been supported by their activities.
And so I do think, you know, they have made progress in
this area, but particularly as it relates to the regional
educational labs and some of the new grant programs, we do
believe it is critically important that you establish
performance measures in those areas--particularly the RELs.
That is a significant investment on the part of IES, and so to
not have public reporting and not have public accountability
around that activity we believe is a key area for improvement.
Mr. Rokita. And again, what would--in your mind, what would
that public reporting look like?
Mr. Scott. As again, you know, having key indicators,
performance measures, in terms what the expectations are for
the RELs, but then having feedback----
Mr. Rokita. What would that be? I am trying to get you to
be specific and quantitative.
Mr. Scott. Quantitative. Could be, once again, meeting the
needs--having a feedback loop in terms of meeting the needs of
the stakeholders, having clear expectations in terms of how you
expect the RELs to be engaged with the research alliance, and
then hold them accountable for that cooperation.
Mr. Rokita. Okay. I am going to leave you alone.
[Laughter.]
Mr. Scott. I don't want to get too prescriptive here. I
mean, one of the things we----
Mr. Rokita. Okay. Fair enough.
Mr. Scott [continuing]. Do believe is having both
quantitative and qualitative measures in place.
Mr. Rokita. There we go. You got----
Mr. Scott. Customer service satisfaction. That is the word
I was looking for when I was talking about getting feedback. I
mean, the National Center for Education Statistics does a
really good job of that, and when we talked to the stakeholders
that is one of the things they pointed out is that a lot of the
projects and the products produced by NCES are really useful,
and there are clearly opportunities for IES to do that across
more of its activities is to get that direct customer
satisfaction survey information and use it directly.
Mr. Rokita. Okay.
Dr. Long, your testimony--I conclude IES has a broad
mission to provide useful information to many different
audiences. In times of limited resources, though, and this was
touched on a little bit earlier, in your opinion, who can
benefit most from IES products? Remember, we are broke.
Ms. Long. Yes. There are many audiences. I think
policymakers absolutely need the information that is coming
from IES to make better decisions what to do with limited
resources, what works, and for what cost.
I think researchers absolutely sending signals to them
about how they should spend their time. I don't think IES needs
to do everything; it needs to leverage the field. And by using
the signals, the incentives, its convening power to take the
researchers out in the field, the organizations, whether they
be states or school boards, to get that information out to
them.
I think it also does have a translation responsibility to
the field to make clear what do you do with this research
information, but then again, working in concert with other
organizations and associations to get the information out
there.
Mr. Rokita. And for what audience? For policymakers, you
say, and for who else?
Ms. Long. I said for policymakers, I said education
researchers should help direct where they are putting their
efforts. I said in terms of the field--so teachers, principals,
superintendents--the translation function of IES is very
important. But again, to get that out I think it is working in
concert with organizations and associations that are in the
field--not for them to necessarily do everything else, but they
have the unique position of being completely objective, having
convening power like no one else does, and so they can send
signals, incentives, bring people together in a very different
way than any other organization.
Mr. Rokita. Okay. Fair enough.
And then in the 30 seconds I have left, if the two
remaining witnesses would like to respond to either of my
questions, you are welcome to.
Dr. Kemple, in about 10 seconds?
Mr. Kemple. Yes. I think the Congress and the U.S.
Department of Education I think would be the primary
beneficiaries of IES's work, both on the quality and the
quantity of work that gets produced. And in terms of answering
questions about what--not just about effective practices but
also about ineffective practices, I think it is the
responsibility to the public to be able to----
Mr. Rokita. Thank you.
Mr. Kemple [continuing]. Invest in learning about whether--
--
Mr. Rokita. Appreciate that.
Ms. Christie?
Ms. Christie. I would like to suggest that folks look at
the government performance accountability system in Washington,
go to their site, look at the kinds of metrics they report on
on their site. It is very impressive, and I wish more folks,
both state and federal, would do the similar thing.
Mr. Rokita. Okay. Thank you.
Again, I thank the witnesses and now I yield to the Ranking
Member Miller for his closing remarks.
Mr. Miller. Thank you very much, and I want to thank the
chairman of the subcommittee, and the Chairman, for holding
this hearing. I think it is very important.
You know, I think that the testimony is pretty clear that
we have developed a much better system than we had before, and
the question is, how do you hold on to that and how do you
maintain the integrity of that system and sort of grow the
confidence of the practitioners, if you will, from governors on
down all the way to the classroom.
And, Mr. Scott, how do you get into a situation, as you
point out on page six here, on this question of the peer review
process starting to stretch itself out and growing from 117
days in 2011 to 175 days in 2012? I mean, this can be the death
of an organization.
Mr. Scott. And I think that is----
Mr. Miller. You know, we are in a dynamic system where, at
our level in our districts people think about this in 9-month
segments, and what is new has got to show up on a fairly
regular basis, I mean, and coincide with the needs of the
users, which are, dictated by the school year if nothing else.
Mr. Scott. Well, I think that issue sort of points to two
things that we talk about in our statement, one being the
importance of sort of balancing the rigorous research with
ensuring it is timely, and useful, and relevant. And the second
issue is having IES use--IES gathers a lot of information about
a lot of things going on under its--or in its purview, but we
have found that at times there are gaps in how they use that
information to oversee the research, and the issue of the peer
review is a perfect example of that.
It is not clear to us that they were really aware of this
until we started asking some questions about what was going on
with the timing here, and now they are paying attention to it.
Our point to IES would be there are other processes that you
have in place that you should be constantly monitoring to
ensure things don't spin out of control or start to grow and
expand in a way that negatively impacts the end users of the
information and the research.
Mr. Miller. So you think this can, in fact, be corrected,
in terms of getting the peer review condensed--I mean, if
people have signed on to participate in that process, maybe
they have signed on to do 1,000 of these, I don't know, but at
some point you have got to know what your contractors--what the
capacity is to participate. You may want them for their name,
for their specialties and their expertise, but if they don't
allow the time so they can fully participate you have got to
move on and find, I think, somebody else at some point.
Mr. Scott. It is important for IES to have the right
information to monitor the performance and then to take the
necessary corrective actions to ensure that that peer review
process doesn't continue to grow and negatively impact its
ability to provide the research.
Mr. Miller. Can you just put in,--quickly, your concerns
about the regional education labs? You come across that in your
testimony----
Mr. Scott. If you look historically at some of the
challenges around the regional education labs, we continue to
be concerned that there are not clear performance measures for
the RELs, that it is not publicly available. IES is collecting
some information. IES also has an ongoing evaluation of the
previous cohort of the RELs that is due to be completed at some
point here in the near future.
We do think it is important, though, for accountability
reasons that there be some public accounting for those RELs,
that they have clear performance measures and indicators, and
then that IES take the necessary action to hold them
accountable for their performance.
Mr. Miller. I mean, this is--at least as it has been
presented--this is supposed to be some high performance
operation, and the question that you are raising to some extent
is whether or not that, in fact, you are getting a high
performance operation in some of the regions----
Mr. Scott. Well, we have heard comments from certain
stakeholders that certain RELS are more productive than others,
the relevance of the research that certain RELs produce is more
relevant than others. I think the question we have for IES is,
you know, at what point are we going to have more public
accounting for the performance of the RELs? That is where you
start--public accounting for their performance. And then you
can make the necessary decisions after then how to move
forward.
Mr. Miller. Thank you.
Thank you, Mr. Chairman.
Mr. Rokita. I thank the ranking member.
Let me again and finally thank the witnesses for your
testimony. It was very enlightening. Very much appreciate it.
I think what we learned is that the quality of education
research has greatly improved in the last decade, and for that
we are thankful. And we are thankful in particular for your
leadership in the field.
However, like most things in life, there are places that we
can look to strengthen the evaluation and performance
requirements for IES programs and ensure that the research
coming out of it is rigorous, relevant, and useful to education
practitioners. And I think you saw that in the questioning by
several members this morning. I think that ought to be our next
goal and a continuation of the work that we have done.
So I look forward to working with my colleagues to
reauthorize the Education Sciences Reform Act in a positive way
that recognizes the fiscal condition we are in, how to leverage
not only the resources of IES but its partners at the federal
and state level, and certainly the leadership of each of the
witnesses here today.
And with that, seeing no further business before this
committee, this hearing is adjourned.
[Whereupon, at 11:27 a.m., the committee was adjourned.]