[House Hearing, 111 Congress]
[From the U.S. Government Publishing Office]
HOW DATA CAN BE USED TO INFORM EDUCATIONAL OUTCOMES
=======================================================================
HEARING
before the
COMMITTEE ON
EDUCATION AND LABOR
U.S. House of Representatives
ONE HUNDRED ELEVENTH CONGRESS
SECOND SESSION
__________
HEARING HELD IN WASHINGTON, DC, APRIL 14, 2010
__________
Serial No. 111-54
__________
Printed for the use of the Committee on Education and Labor
Available on the Internet:
http://www.gpoaccess.gov/congress/house/education/index.html
U.S. GOVERNMENT PRINTING OFFICE
55-849 WASHINGTON : 2010
-----------------------------------------------------------------------
For Sale by the Superintendent of Documents, U.S. Government Printing Office
Internet: bookstore.gpo.gov Phone: toll free (866) 512-1800; (202) 512�091800
Fax: (202) 512�092104 Mail: Stop IDCC, Washington, DC 20402�090001
COMMITTEE ON EDUCATION AND LABOR
GEORGE MILLER, California, Chairman
Dale E. Kildee, Michigan, Vice John Kline, Minnesota,
Chairman Senior Republican Member
Donald M. Payne, New Jersey Thomas E. Petri, Wisconsin
Robert E. Andrews, New Jersey Howard P. ``Buck'' McKeon,
Robert C. ``Bobby'' Scott, Virginia California
Lynn C. Woolsey, California Peter Hoekstra, Michigan
Ruben Hinojosa, Texas Michael N. Castle, Delaware
Carolyn McCarthy, New York Mark E. Souder, Indiana
John F. Tierney, Massachusetts Vernon J. Ehlers, Michigan
Dennis J. Kucinich, Ohio Judy Biggert, Illinois
David Wu, Oregon Todd Russell Platts, Pennsylvania
Rush D. Holt, New Jersey Joe Wilson, South Carolina
Susan A. Davis, California Cathy McMorris Rodgers, Washington
Raul M. Grijalva, Arizona Tom Price, Georgia
Timothy H. Bishop, New York Rob Bishop, Utah
Joe Sestak, Pennsylvania Brett Guthrie, Kentucky
David Loebsack, Iowa Bill Cassidy, Louisiana
Mazie Hirono, Hawaii Tom McClintock, California
Jason Altmire, Pennsylvania Duncan Hunter, California
Phil Hare, Illinois David P. Roe, Tennessee
Yvette D. Clarke, New York Glenn Thompson, Pennsylvania
Joe Courtney, Connecticut
Carol Shea-Porter, New Hampshire
Marcia L. Fudge, Ohio
Jared Polis, Colorado
Paul Tonko, New York
Pedro R. Pierluisi, Puerto Rico
Gregorio Kilili Camacho Sablan,
Northern Mariana Islands
Dina Titus, Nevada
Judy Chu, California
Mark Zuckerman, Staff Director
Barrett Karr, Minority Staff Director
C O N T E N T S
----------
Page
Hearing held on April 14, 2010................................... 1
Statement of Members:
Kline, Hon. John, Senior Republican Member, Committee on
Education and Labor........................................ 4
Prepared statement of.................................... 6
Study, ``Children's Educational Records and Privacy,''
dated October 28, 2009, Internet address to............ 5
Miller, Hon. George, Chairman, Committee on Education and
Labor...................................................... 1
Prepared statement of.................................... 3
Questions for the record submitted to Mr. Wenning, on
behalf of Mr. Kucinich................................. 74
Statement of Witnesses:
Hartley, Katie, teacher, value added data specialist, Miami
East Local Schools, Miami County, OH....................... 28
Prepared statement of.................................... 31
Kitchens, Joe, superintendent, Western Heights School
District, Oklahoma City, OK................................ 24
Prepared statement of.................................... 26
Reidenberg, Joel R., professor of law and founding academic
director, Center on Law and Information Policy, Fordham
University School of Law................................... 32
Prepared statement of.................................... 36
Wenning, Richard J., associate commissioner, Colorado
Department of Education.................................... 9
Prepared statement of.................................... 11
Responses to Mr. Kucinich's questions for the record..... 75
HOW DATA CAN BE USED TO
INFORM EDUCATIONAL OUTCOMES
----------
Wednesday, April 14, 2010
U.S. House of Representatives
Committee on Education and Labor
Washington, DC
----------
The committee met, pursuant to call, at 10:01 a.m., in room
2175, Rayburn House Office Building, Hon. George Miller
[chairman of the committee] presiding.
Present: Representatives Miller, Payne, Scott, Woolsey,
Hinojosa, McCarthy, Tierney, Kucinich, Wu, Holt, Davis, Hirono,
Altmire, Hare, Shea-Porter, Polis, Sablan, Titus, Chu, Kline,
Petri, Biggert, McMorris Rodgers, Guthrie, Cassidy, Roe, and
Thompson.
Staff present: Andra Belknap, Press Assistant; Calla Brown,
Staff Assistant, Education; Jody Calemine, General Counsel;
Jamie Fasteau, Senior Education Policy Advisor; Denise Forte,
Director of Education Policy; David Hartzler, Systems
Administrator; Fred Jones, Staff Assistant, Education; Kara
Marchione, Education Policy Advisor; Sadie Marshall, Chief
Clerk; Bryce McKibbon, Staff Assistant; Charmaine Mercer,
Senior Education Policy Advisor; Alex Nock, Deputy Staff
Director; Lillian Pace, Policy Advisor, Subcommittee on Early
Childhood, Elementary and Secondary Education; Rachel Racusen,
Communications Director; Meredith Regine, Junior Legislative
Associate, Labor; Alexandria Ruiz, Staff Assistant; Melissa
Salmanowitz, Press Secretary; Mark Zuckerman, Staff Director;
Stephanie Arras, Legislative Assistant; James Bergeron, Deputy
Director of Education and Human Services Policy; Kirk Boyle,
General Counsel; Casey Buboltz, Coalitions and Member Services
Coordinator; Barrett Karr, Staff Director; Alexa Marrero,
Communications Director; Brian Newell, Press Secretary; Susan
Ross, Director of Education and Human Resources Policy; Mandy
Schaumburg, Education Policy Counsel; and Linda Stevens, Chief
Clerk/Assistant to the General Counsel.
Chairman Miller [presiding]. A quorum being present, the
committee will come to order.
Good morning. Welcome to our witnesses and our members
joining us here. Today we will explore how effective data
systems can help improve educational outcomes. This is a part
of a series of hearings this committee is holding as we work in
a bipartisan way to reauthorize the Elementary and Secondary
Education Act.
My colleagues have demonstrated their dedication to this
bipartisan reauthorization process and all bring valuable
expertise to the table.
Mr. Holt and Mrs. McCarthy particularly have been leaders
in the data arena for several years. Notably, together they
have previously introduced legislation to improve data use in
schools across the nation.
Data is absolutely critical to education reform. Just like
a successful company relies on sales reports to measure their
success, schools need data to make informed and educated
decisions about what is working and what isn't.
In many schools and districts, data is not used in the most
meaningful way to make decisions, or even at all. It is
unacceptable that education is only--the major enterprise in
this country, on the whole, that doesn't use data to make
decisions.
Teachers, parents, and school administrators and states
need access to real-time information to know exactly how
students are faring in school. We took a big step forward to
address this need in the American Recovery and Reinvestment Act
when we required states to comply with the four assurances in
order to be eligible for the historic investment in education.
These assurances helped move the ball a little farther down
the field for schools that are asking states to adopt college-
and career-ready standards tied to better assessments, to turn
around the lowest performing schools, and to ensure teacher
talent that is distributed fairly and to establish data systems
that use the data to improve schools.
We asked for these commitments from states, especially on
the data front, for two reasons. One, we can no longer accept
an education system that is willing to settle for less than the
best of all of our students.
The millions of students in classrooms today are our future
innovators and engineers. If we are going to regain our footing
as a global competitor in the world, we need to demand the best
of our students, our teachers and our schools.
And two, we need an effective longitudinal data system with
focus on safety and privacy for our students that works to help
schools succeed.
Schools need student-level information in order to better
educate every child, both for their own benefit and for our
future as a nation. In Western Heights school district in
Oklahoma, for example, school officials use data systems to
help determine which students are--were the lowest performing.
They realized that their mobile students, those who moved
from school to school, were achieving at the lowest levels, and
dropping out at the highest. After implementing the data
system, the dropout rate in the district fell by 11 points in 2
years.
If districts implemented early warning indicator systems in
middle schools, they could identify the students most likely to
drop out of high school and reach those students before they
get off track.
If a principal uses data to help identify teachers'
strengths in the classroom, the principal could work to
replicate those achievements on a school-wide level.
If researchers were able to investigate state-level data,
they could share the practices that are working best to help
students succeed.
Without data, schools are operating in the dark. Simply
put, data systems work. That is why there has been a tremendous
focus on data, in the next iteration of the Elementary and
Secondary Education Act, the new law can be a real catalyst for
positive change in our schools.
Since we announced we were working to rewrite ESEA, we have
heard from thousands of stakeholders. Their input has been
incredibly helpful. We all agree that the status quo is failing
our children and won't lead our children to the future.
It is time we put the needs of our students and teachers at
the top of our priorities. We can't let our students suffer the
failures of a system that doesn't support them. We have an
obligation to the children of this country to get it right the
first time. That is why the data is so absolutely critical.
It is time to give teachers the tools they need to make
data-based, informed decisions in the classroom. Critics of the
use of data are operating under an antiquated school of
thought. We have to take our schools to the future.
When data is properly presented and where people are given
skills to use it and know the purpose behind it, data can be a
most valuable tool to school success.
I want to thank in advance our witnesses for being here
today and for their testimony that they will give in a moment.
At this time I would like to recognize Congressman Kline,
the senior Republican on the committee.
[The statement of Mr. Miller follows:]
Prepared Statement of Hon. George Miller, Chairman, Committee on
Education and Labor
Good morning.
Today we'll explore how effective data systems can help improve
education outcomes.
This is a part of a series of hearings this committee is holding as
we work in a bipartisan way to reauthorize the Elementary and Secondary
Education Act.
My colleagues have demonstrated their dedication to this bipartisan
reauthorization process and all bring valuable expertise to the table.
Mr. Holt and Mrs. McCarthy in particular have been leaders in the
data arena for several years. Notably, together they have previously
introduced legislation to improve data use in schools across the
nation.
Data is absolutely critical to education reform.
Just like any complex organization relies on multiple indicators to
measure their success, schools need data to make informed and educated
decisions about what is working and what isn't.
But in many schools and districts, data is not used in the most
meaningful way to make decisions, or even at all.
It is unacceptable that education is the only major enterprise in
this country that, on the whole, doesn't use data as to make decisions.
Teachers, parents, school administrators and states need access to
real time data to know exactly how students are faring in school.
We took a big step forward to address this need in the American
Recovery and Reinvestment Act when we required states to comply with
four assurances in order to be eligible for the historic investments in
education.
These assurances helped move the ball a little farther down the
field for schools by asking states to adopt college and career ready
standards tied to better assessments, turn around the lowest perform
schools, ensure teacher talent is distributed fairly and establish data
systems to use data to improve schools.
We asked for these commitments from states, especially on the data
front, for two reasons.
One, we can no longer accept an education system that is willing to
settle for less than the best for our students.
The millions of students in classrooms today are the future
innovators and engineers.
If we are going to regain our footing as a global competitor in the
world, we have to demand the best for our schools, our teachers and our
schools.
And two, we know an effective longitudinal data system with a focus
on the safety and privacy of our students works to help schools
succeed.
Schools need student level information in order to better educate
every child--both for their own benefit and for our future as a nation.
In the Western Heights school district in Oklahoma, for example,
school officials used a data system to help determine which students
were the lowest performing.
They realized their mobile students, those who moved from school to
school, were achieving at the lowest levels and dropping out at the
highest.
After implementing a data system, the dropout rate in the district
fell by 11 points in two years.
If districts implement early warning indicator systems in middle
schools, they could identify the students most likely to drop out of
high school and reach those students before they get off track.
If a principal uses data to help identify teachers' strengths in
the classroom, the principal could work to replicate their achievements
on a school wide level.
If researchers were able to investigate state-level data, they
could share the practices that are working best to help students
succeed.
Without data, schools are operating in the dark. Simply put, data
systems work.
That's why there has to be a tremendous focus on data in the next
iteration of the Elementary and Secondary Education Act, so the new law
can be a real catalyst for positive change in our schools.
Since we announced we were working to rewrite ESEA, we've heard
from thousands of stakeholders. Their input has been incredibly
helpful.
We all agree that the status quo is failing our children and won't
lead our children to the future.
It's time we put the needs of our students and teachers at the top
of our priorities.
We can't let our students suffer the failures of a system that
doesn't support them.
We have an obligation to the children of this country to get it
right the first time.
This is why data is so absolutely critical.
It's time we give teachers the tools they need to make data-based,
informed decisions in the classrooms.
Critics of the use of data are operating under an antiquated school
of thought. We have to take our schools to the future.
When data is properly presented and when people are given skills to
use it and know the purpose behind it, data can be the most valuable
tool for school success.
I'd like to thank our witnesses for being here today. I look
forward to hearing your testimony.
______
Mr. Kline. Thank you, Mr. Chairman.
Good morning to the witnesses and to all present. We are
here this morning to examine how data can be used to inform
educational outcomes. To be sure, educational data systems play
an integral role in efforts to create more sophisticated
academic performance measures. In other words, data help us
understand how our students and their teachers are performing.
Yet no conversation about educational data systems would be
complete without a discussion of student privacy. Technological
advances and research opportunities have created a thirst for
individualized student data like never before. Our commitment
to privacy and data protection must intensify at the same pace.
Unfortunately, the research indicates not nearly enough is
being done to safeguard our students' records. We will hear
this morning from Professor Joel Reidenberg of the Center on
Law and Information Policy at Fordham Law School--welcome,
Professor--who has been at the forefront in examining the
privacy implications of longitudinal data systems.
These massive state-controlled databases collect personally
identifiable information about schoolchildren, information
designed to be interoperable among a variety of data systems,
leaving open the possibility that this data could be mined for
uses far beyond its intended purposes.
And, Mr. Chairman, I request unanimous consent to include
Professor Reidenberg's report from October 2009.
Chairman Miller. Without objection, it will be made part of
the record of the hearing.
[The study, ``Children's Educational Records and Privacy,''
dated October 28, 2009, may be accessed at the following
Internet address:]
http://law.fordham.edu/assets/CLIP/CLIP--Report--Childrens--Privacy--
Final.pdf
______
Mr. Kline. Thank you.
Professor Reidenberg will discuss his findings in detail,
but there are two areas of concern I would like to highlight.
First, the Fordham study found privacy protections lacking in
most states. In some cases, states are not even complying with
the federal educational rights and privacy act.
Second, the study highlighted the risk that individual
state data systems could be sewn together to create a de facto
national database, a massive federal collection of individuals
student information that could include not just academic
histories but sensitive personal data, including Social
Security numbers, demographic and financial characteristics,
discipline records and health or behavioral information.
The study describes it this way ``Common data standards by
definition facilitate the combination of multiple data sets
into one national data warehouse of K-12 children, which in
turn could be combined with data from post-secondary data
systems to create an unprecedented national database of
personal information.''
The prospect of these data systems being used for more than
academic tracking in grade school is hardly far-fetched. In
fact, the American Recovery and Reinvestment Act, the stimulus
bill, which we now know contained a host of provisions having
nothing to do with job creation, included an additional $250
million for the existing state longitudinal data systems.
According to the U.S. Department of Education, the long-
term goal of the program is to enable all states to create
comprehensive P-20 systems which will track students from
almost literally the cradle to their careers.
The emphasis on interoperability makes clear these systems
are intended to link personal and academic information from
elementary and secondary school to workforce data systems that
attract--that track adults later in life. These vast
collections of information could significantly undermine
individual privacy, particularly if they are compromised
through ineffective security measures.
In this era of technology and vast Web-based information
archives, data that becomes public can never again truly be
kept private. The potential privacy cost of these data systems,
particularly if they do not maintain proper safeguards, cannot
be ignored.
Yet we must also consider the monetary costs associated
with significant new data collection requirements. States and
local school districts take on significant financial and
personnel burdens to comply with data collection requirements.
At a time when local schools are seeking less red tape and
fewer federal requirements, we must carefully weigh the
potential benefits with these costs. The stimulus significantly
expanded the scope of federal involvement in student data
collection, the consequences of which are only just beginning
to emerge.
I remain deeply concerned about student privacy both under
current programs and in light of proposed expansions in data
collection and use through reauthorization of the Elementary
and Secondary Education Act.
I shared a number of these concerns in a letter to
Secretary Duncan in February of this year, and I am eager to
continue a dialogue about how individual privacy protection
will be maintained and strengthened.
As I said at the outset, data systems are an important
component of our efforts to measure and improve student
academic achievement and teacher quality. Yet as technology
advances, we must ensure the data collected is narrow in scope
and tightly controlled with its use carefully monitored.
The more data collected, the greater the risk of exposure,
which is why every effort must be made to bring privacy laws
into the 21st century to protect the student information.
Thank you, Mr. Chairman. I yield back.
[The statement of Mr. Kline follows:]
Prepared Statement of Hon. John Kline, Senior Republican Member,
Committee on Education and Labor
Thank you Chairman Miller. We're here this morning to examine how
data can be used to inform educational outcomes. To be sure,
educational data systems play an integral role in efforts to create
more sophisticated academic performance measures. In other words, data
help us understand how our students--and their teachers--are
performing.
Yet no conversation about educational data systems would be
complete without a discussion of student privacy. Technological
advances and research opportunities have created a thirst for
individualized student data like never before. Our commitment to
privacy and data protection must intensify at the same pace.
Unfortunately, the research indicates not nearly enough is being
done to safeguard our students' records. We'll hear this morning from
Professor Joel Reidenberg (RIDE-en-berg) of the Center on Law and
Information Policy at Fordham Law School, who has been at the forefront
in examining the privacy implications of longitudinal data systems.
These massive, state-controlled databases collect personally
identifiable information about school children--information designed to
be interoperable among a variety of data systems, leaving open the
possibility that this data could be mined for uses far beyond its
intended purposes.
Mr. Chairman, I request unanimous consent to include Professor
Reidenberg's October 2009 report--entitled ``Children's Educational
Records and Privacy: A Study of Elementary and Secondary School
Reporting Systems''--in the printed hearing record.
Professor Reidenberg will discuss his findings in detail, but there
are two areas of concern I'd like to highlight. First, the Fordham
study found privacy protections lacking in most states--in some cases,
states are not even complying with the Federal Educational Rights and
Privacy Act.
Second, the study highlighted the risk that individual state data
systems could be sewn together to create a de facto national database--
a massive federal collection of individual student information that
could include not just academic histories but sensitive personal data
including social security numbers, demographic and financial
characteristics, discipline records, and health or behavioral
information. The study describes it this way: ``Common data standards,
by definition, facilitate the combination of multiple data sets into
one national data warehouse of K-12 children, which in turn could be
combined with data from post-secondary data systems to create an
unprecedented national database of personal information.''
The prospect of these data systems being used for more than
academic tracking in grade school is hardly far-fetched. In fact, the
American Recovery and Reinvestment Act--the so-called stimulus bill,
which we now know contained a host of provisions having nothing to do
with job creation--included an additional $250 million for the existing
state longitudinal data systems.
According to the U.S. Department of Education, the long-term goal
of the program is to enable all states to create comprehensive P-20
systems, which will track students from almost literally the cradle to
their careers. The emphasis on ``interoperability'' makes clear these
systems are intended to link personal and academic information from
elementary and secondary school to workforce data systems that track
adults later in life.
These vast collections of information could significantly undermine
individual privacy, particularly if they are compromised through
ineffective security measures. In this era of technology and vast web-
based information archives, data that become public can never again
truly be kept private.
The potential privacy cost of these data systems--particularly if
they do not maintain proper safeguards--cannot be ignored. Yet we must
also consider the monetary costs associated with significant new data
collection requirements.
States and local school districts take on significant financial and
personnel burdens to comply with data collection requirements. At a
time when local schools are seeking less red tape and fewer federal
requirements, we must carefully weigh the potential benefits with these
costs.
The stimulus significantly expanded the scope of federal
involvement in student data collection, the consequences of which are
only just beginning to emerge. I remain deeply concerned about student
privacy, both under current programs and in light of proposed
expansions in data collection and use through reauthorization of the
Elementary and Secondary Education Act. I shared a number of these
concerns in a letter to Secretary Duncan in February of this year, and
I am eager to continue a dialogue about how individual privacy
protections will be maintained and strengthened.
As I said at the outset, data systems are an important component of
our efforts to measure and improve student academic achievement and
teacher quality. Yet as technology advances, we must ensure the data
collected is narrow in scope and tightly controlled, with its use
carefully monitored.
The more data collected, the greater the risk of exposure--which is
why every effort must be made to bring privacy laws into the 21st
century to protect student information.
Thank you, and I yield back.
______
Chairman Miller. I thank the gentleman, and I would like
now to introduce the panel of witnesses for the hearing.
But without objection, I would first yield to the gentleman
from Colorado, Mr. Polis, to briefly introduce our first
witness, Richard Wenning.
Mr. Polis. Thank you, Mr. Chairman.
It is really my honor to introduce Rich Wenning, who I got
to know during my time on the state board of education and
being involved with educational reform in Colorado.
Rich Wenning is currently the associate commissioner at the
Colorado Department of Education, where he leads the Colorado
Department of Education's public policy development and the
design and implementation of Colorado's great educational
accountability system and growth model, which we are going to
be hearing about. He is the architect of the Colorado Growth
Model.
Before he joined Colorado Department of Education, Mr.
Wenning was vice president for quality and accountability at
the Colorado League of Charter Schools, where I had the
opportunity to work with him in that capacity as well.
Mr. Wenning served as an executive on loan to the
superintendent of Denver public schools, where he focused on
strengthening the district's performance management practices.
Before Mr. Wenning moved to Colorado from Washington, D.C.,
he was president of the Education Performance Network, an
affiliate of the New American Schools, not to be confused with
the New America School, which is the charter school that I had
founded and run prior to getting here, where he led a
consulting practice focused on educational accountability
systems and new school development.
Mr. Wenning also served as a senior policy advisor to the
CEO of the D.C. public schools during the school district's
takeover by the congressionally appointed D.C. Control Board.
While at D.C. public schools, he headed the Office of
Intergovernmental Affairs and Educational Accountability.
Prior to joining D.C. public schools, Mr. Wenning served as
a clerk for the Senate Appropriations Subcommittee on the
District of Columbia and as a staff member on the Senate
Appropriations Subcommittee on Labor, Education and Health and
Human Services.
Mr. Wenning began his career at the Government
Accountability Office where he led research on accountability
and equity issues as well as market-based education reform
strategies.
And it is my honor to introduce Mr. Wenning to our
Education and Labor Committee.
Yield back.
Chairman Miller. Thank you very much.
Our next witness will be Mr. Kitchens, a superintendent of
public schools for the last 15 years. Joe Kitchens is the
national leader in the use of data systems to improve school
achievement.
Mr. Kitchens was instrumental in developing longitudinal
data that enables teachers to make immediate and effective
decisions in the classroom. In 2008 the Data Quality Campaign
recognized Mr. Kitchens as the district data leader of the
year.
Katie Hartley is currently--teaches junior high math at
Miami East Junior High in West Central Ohio. I am just trying
to get my geography down here--in the analysis and the use of
value-added data to make informed decisions concerning
curriculum, instruction and academic programming.
Joel Reidenberg is a professor of law and founding academic
director of the Center of Law and Information Policy at Fordham
Law School. He is an expert on information technology law and
policy. Professor Reidenberg examines information privacy and
Internet regulation.
Professor Reidenberg has served as an advisor on data
privacy, including special assistant to the attorney general
for the state of Washington.
Welcome to all of you. Before we begin, let me explain the
lighting system. When you begin a green light will go on. You
will have 5 minutes to make your presentation. In fact, we are
going to extend to you a couple of minutes because I know some
of you are also demonstrating the use of this information and
you have got to pass a computer back and forth.
And then when the red light comes on, you can see that--in
a coherent fashion, if you can bring your testimony to a close,
we would appreciate it.
So, Mr. Wenning, we are going to begin with you. Welcome.
STATEMENT OF RICHARD WENNING, ASSOCIATE COMMISSIONER, COLORADO
DEPARTMENT OF EDUCATION
Mr. Wenning. I think we are on now. There we go.
So, Mr. Chairman, members of the committee, thank you for
inviting my testimony today on behalf of the Colorado
Department of Education.
I would like to provide my remarks in the context of
Colorado's effort to create an aligned state and federal
accountability system that maximized the use of longitudinal
data to support state and local performance management
purposes.
Educational accountability systems include three basic
components: rewards, sanctions and public reporting. Colorado's
approach to educational accountability attempts to balance
these components to promote local ownership of high-quality
performance information.
Local ownership drives insight and action by users--
students, parents, educators, administrators, business leaders,
all members of the public. The key is fostering a common
understanding among these stakeholders.
Colorado believes that the results we expect must start
with the end in mind, and that is our statutory bright line
principle of 100 percent of students becoming college-and
career-ready by the time they graduate.
This universal goal clarifies our public responsibility and
the focus of our accountability and performance management
systems, and we are very pleased to see this principle in the
president's blueprint for ESEA reauthorization.
Growth models like the Colorado Growth Model make it
possible to establish ambitious growth expectations for every
student, based on what they need to be on track, and roll this
up for state and federal accountability purposes.
The clarity of the goal of readiness by exit, particularly
in the context of common high standards, supports an essential,
powerful and ongoing conversation between every student and his
or her teacher and parents about how much growth the student is
making and whether it is good enough.
Most important is a conversation about how each student,
teacher and parent must work together to ensure that goals and
standards are met. And I refer to the capacity to
constructively engage in this fundamental conversation, using
information effectively to make adjustments and achieve goals,
as Performance Management Capacity.
This is the essential role of state longitudinal data
systems. The key function: help parents constructively engage
with educators and become knowledgeable choosers of schools.
The availability of outstanding instructional improvement
and social collaboration technologies and incentives for using
them, particularly through initiatives focusing on educator
effectiveness, represent vital tools for bringing about
breakthrough improvements in performance.
Thanks to advantageous timing--major advances in technology
coinciding with Race to the Top--the nation is in a position to
provide students and teachers the tools they need to achieve
the results we expect. And we believe that we are primed to
bring about breakthrough educator collaboration about
performance and practice.
Underpinning this collaboration in Colorado is a new and
public conversation about performance fostered by SchoolView
and the Colorado Growth Model. SchoolView and the Colorado
Growth Model are state-owned tools that run on open-source
software, and we are happy to share them with other states at
no cost.
We are pleased that several other states have already
adopted our growth model, promoting cross-state collaboration
and comparisons--Massachusetts, Indiana, Arizona, and I just
learned that Wisconsin will be adopting the growth model as
well that we have developed.
What I am going to do is just show you a quick
demonstration of what the public has access to, and you have
got screen shots of the password-protected version in my
testimony. And of course, that is secure data that only
educators have access to, but that does allow an educator to
get down to an individual student and have a conversation with
mom and dad about how a child's progress is doing.
The next key step is merging that with instructional
resources so every teacher and student can be engaged in
information about how each child is progressing.
I am going to quickly bring up two districts. First I want
to orient you to the basic four quadrant diagram that we always
use. We look at growth on the horizontal axis and achievement
on the vertical axis, so that we can understand schools'
performance in a simple manner.
I am going to pick Adams 14 School District, and then I am
going to go ahead and pick Pueblo School District, and we are
going to contrast two middle schools. Let me find Pueblo here.
And you have got the screen shots of this as well. I am
going to go and hit--choose only middle schools. And this is a
nice tool just to help benchmark performance.
The horizontal line in the middle reflects the average
percent proficient or advanced in the state of Colorado. The
vertical line is at 50th percentile growth. That is a year's
growth in a year's time.
I am going to highlight two schools. As we can see, we have
just highlighted Kearney Middle School with an enrollment of
470, a little bit below average in achievement, 44 percent
proficient or advanced. Median growth percentile of 74--that
means students at Kearney, the typical child here, makes as
much progress or more than 74 percent of kids in Colorado with
the same starting point.
Here, in a school that is a little bit above average in
achievement, Corwin International Magnet School, the median
growth percentile is 28, meaning the typical student in Corwin
is only growing as well as 28 percent of the kids in Colorado
with the same starting point.
Now we can go ahead in here--and this, of course, is all
anonymous data at this point. We are going to disaggregate by
other groups. And we can see that students eligible for free or
reduced-price lunch are growing at a 29th growth percentile,
meaning they are only doing as well as 29 percent of the kids
in the state with the same starting point.
We go up a level and we can take a look at this school,
which, again, would have lower achievement but much higher
growth, and take a look at students in the other group category
here, and we can see that for low-income students at Kearney,
their growth percentile is 74.5, meaning they have got very
high growth, making very high progress, even though in our
current system of looking at AYP both of these schools would
look the same, but we can see that there are dramatically
different growth rates among them.
So you know, this kind of disclosure fosters a much more
informed understanding of school and student performance, one
that all of our stakeholders are becoming familiar with and
interestingly, our educator associations are strong advocates
of, because of the ability for teachers to understand what
performance is like in different schools.
Federal policy can either support or hinder the
understanding, ownership and effective use of performance
information at the individual, local and state level through
the metrics required and rewards and sanctions established.
As we reauthorize ESEA, it is critical that we get the
federal, state and local roles right and give states sufficient
latitude to build the performance capacity--the performance
management capacity of stakeholders to achieve the breakthrough
results that we need.
Incremental changes in this relationship in access to data
won't even come close to the unprecedented productivity
expectations we seek for public education in the United States
as we aim to getting all students ready for college and career
success.
Thank you, Mr. Chairman, members of the committee, and I
will be, of course, happy to respond to any questions that you
may have.
[The statement of Mr. Wenning follows:]
Prepared Statement of Richard J. Wenning, Associate Commissioner,
Colorado Department of Education
Thank you for inviting my testimony on behalf of the Colorado
Department of Education at today's hearing. I'd like to provide my
remarks in the context of Colorado's effort to create an aligned state
and federal accountability system focused on all students reaching
college and career readiness by high school graduation.
How is Colorado refining its use of student performance data to
improve accountability for student growth, better inform school
improvement efforts, and more clearly communicate with the public?
Educational accountability systems include three basic components:
rewards, sanctions and public reporting. Colorado's approach to
educational accountability attempts to balance these components to
promote local ownership of high-quality performance information. We
believe this local ownership drives insight and action by users:
students, parents, educators, administrators, policymakers, business
leaders, and the public-at-large.
Colorado believes that the results we expect must start with the
end in mind: namely our statutory bright-line principle of all students
becoming college- and career-ready by high school graduation. This
universal goal clarifies our public responsibility and the focus of our
accountability and performance management systems: we must maximize
individual student academic growth toward the destination of college
and career readiness. We were very pleased to see this principle
reflected in the President's Blueprint for ESEA Reauthorization.
However, the Blueprint's intended use of the 2020 date for school
vs. state accountability is unclear. Colorado feels strongly that an
arbitrary date certain is not helpful for states to calibrate their
school accountability systems. This is because a very credible date
exists for every student, namely their graduation date. Growth models
make it possible to establish ambitious growth expectations for every
student, based on what they need to be on track and also allow a roll
up for state and federal accountability purposes. This concept is
discussed further below.
The clarity of the goal of readiness by exit, particularly in the
context of common high standards, supports an essential, powerful and
ongoing conversation between every student and his or her teachers and
parents about how much growth the student is making, whether it is good
enough to catch up to proficiency (if the student is not proficient),
keep up at proficiency (if the student is already proficient), or to
move up to advanced levels of achievement. Most important is a
conversation about how each student, teacher and parent must work
together to ensure that the student meets goals and standards. I refer
to the capacity to constructively engage in this fundamental
conversation, using information effectively to make adjustments and
achieve goals, as Performance Management Capacity. Plain and consistent
language (like catch up and keep up) promotes meaningful conversations
and illustrates the importance of focusing on the user of information
when designing accountability systems.
The availability of outstanding instructional improvement and
social collaboration technologies and incentives for using them
(particularly through initiatives focusing on educator effectiveness)
represent vital tools and opportunities for break-through performance
improvements. Thanks to advantageous timing--major advances in
technology coinciding with Race to the Top--the nation is in a position
to provide students and educators the tools they need and deserve to
achieve the outcomes we expect. We are primed to promote break-through
educator collaboration about performance and practice. This is the
essential role of state longitudinal data systems.
Underpinning this collaboration in Colorado is a new and broad
public conversation about performance fostered by SchoolView and the
Colorado Growth Model (see figures below). SchoolView is a state-owned
tool that we are happy to share with other states. The Colorado Growth
Model uses an open-source methodology run on open-source software. We
are making the display tools available at no cost to other states
through a memorandum of understanding, including commitment to the
Creative Commons intellectual property agreement we use.
The Colorado Growth Model was approved by the U.S. Department of
Education for use in its growth model pilot. It uses a common measure
to describe how much growth each student makes and how much growth is
needed to reach state standards. In doing so, it provides a complete
history of individual test scores for all students. The model depicts
growth in a user-friendly and interactive display that provides clear
information about student progress toward reaching state proficiency
levels within a specific period of time.
The Colorado Growth Model supports a common understanding of how
individual students and groups of students progress from year to year
toward state performance standards based on where each student begins.
The model focuses attention on measuring and maximizing student
progress over time and reveals where, and among which students, the
strongest growth is happening--and where it is not. It recognizes that
the most effective schools are those that produce the highest sustained
rates of student academic growth over time. Those schools may or may
not be schools with the highest test scores every year.
The Colorado Growth Model applies the common measure of Individual
Student Growth Percentiles to school, district and state performance in
a normative and criterion-referenced manner. The growth model provides
a growth percentile ranging from 1 to 99 for every student--also
described as ``Low,'' ``Typical'' or ``High''--and provides the
percentile needed for a student to reach Partially Proficient,
Proficient and Advanced levels within one, two, or three years.
The model provides Median Growth Percentiles that are useful for
benchmarking purposes and analysis of gaps in growth rates among groups
of students. The overall State Median Growth Percentile for every grade
is 50, so it is useful to look for differences from the 50th percentile
when benchmarking the growth of the typical student.
The model also provides information on the adequacy of growth to
reach and maintain state-defined performance levels--we refer to these
as Catch Up and Keep Up. On Track to Catch Up identifies students
scoring Unsatisfactory or Partially Proficient in the prior year who
achieved enough growth to reach Proficient within three years or by
10th grade. On Track to Keep Up identifies students already scoring
Proficient or Advanced who achieved enough growth to stay at least
Proficient over three years or until 10th grade.
The Colorado Growth Model fills an important gap in the current
accountability system required by NCLB. To close the achievement gaps
that plague our education system, we must eliminate gaps in how
children are growing academically and ensure that our neediest students
grow faster--more than a year's growth in a year's time--so that they
catch up. The following graphics show the percentage of students
achieving enough growth to catch up or keep up in Colorado.
Because AYP today is focused on each school's percentage of
students who score ``at proficiency'' each year, it creates an overly
anxious short-term focus on students ``on the cusp'' of proficiency--
the ones who should be easiest to push over the hump and therefore give
schools a better rating.
Instead, we should encourage teachers to focus on maximizing every
child's progress toward ambitious standards--and developing every child
to his or her full potential--while encouraging schools to focus on
long-term effectiveness. The federal accountability system should
measure whether that is happening. As we measure the performance of
schools and districts, we must provide individual student data that
educators need in order to focus on improving student learning. Every
educator and parent should know in plain language how much growth a
child has achieved and how much growth each child needs to reach state
standards.
Consistent with these design principles, the Colorado Department of
Education used SchoolView to deploy a set of interactive Web-based
display tools to provide Colorado Growth Model information about
district, school and student performance to parents, educators and the
public. (See images at end of document.) These display tools enable and
promote new, well-informed conversations about learning among
educators, students and parents while providing unprecedented public
transparency in support of accountability, which allows us to disclose
more, use fewer punitive labels, drive strong stakeholder buy-in, and
foster sustained public pressure for reform.
Colorado is very interested in collaborating with other states to
create a common data visualization platform to drive broad public
understanding about educational effectiveness and cross-state
performance benchmarking. We are pleased that Arizona and Indiana have
elected to work with us on this effort. In addition, Massachusetts has
adopted our growth model for its use. Several other states are expected
to adopt it as well.
How can federal policy best promote improved student achievement?
Federal policy can promote dramatically improved student outcomes
by ensuring a coherent accountability system focused squarely on
building the performance management capacity of stakeholders. For this
to happen, the federal role in local school management decisions must
be redefined in a manner that recognizes and respects the essential
role that states, local educational agencies, schools and individual
educators must play if sustained high is to become the norm. Federal
policy can either support or hinder the understanding, ownership, and
effective use of performance information at the individual, local and
state levels through the metrics required and rewards and sanctions
established.
State education agencies (SEAs) play a critical role, and SEAs
should be re-purposed to support school effectiveness. This will
require federal support. SEAs must become reliable providers and
brokers of high-quality support and service to schools and districts.
They must focus on sustaining continuous improvement in schools and
districts while also ensuring that they meet compliance obligations. To
achieve this aim, SEAs will need to invest in research and development,
program evaluation, and diagnostic school and district reviews focused
on improvement efforts. This may require reallocation of resources.
SEAs will also need to develop coherent knowledge management strategies
to sustain their capacity levels.
Flexibility is also necessary. Expanding allowable uses of funds
would allow SEAs to invest in capacity-building strategies to deliver
ambitious, desired results. ESEA reauthorization should extend far
greater leeway in the use of federal funds at the state and local
levels, but only to those SEAs that adopt high-quality accountability
systems based on internationally benchmarked standards for college and
career readiness. Incorporating these expectations into the
reauthorization of ESEA will go far in ensuring students are truly
prepared for college or rewarding careers.
Provide Flexibility in Identifying Low-Performing Schools for
Intervention
In reauthorizing ESEA, Congress should be cautious in prescribing
the details of how to identify the bottom five percent of schools based
on achievement and growth. Some flexibility is needed so that states
can calibrate accountability systems to meet the performance
improvement needs of their particular schools and districts. The
essential condition is that states must have a credible approach and
rationale and be publicly transparent in how they do this. For states
without an approved accountability system designed to identify the
bottom five percent, ESEA could contain a default approach.
For example, there are more chronically low-performing schools in
Colorado than we can effectively intervene in with federal School
Improvement Grant [1003(g)] resources. (See figures below.) As we
prioritize schools for intervention, we would like to consider
persistence and severity of need and whether the intervention fits the
problem and can have a scalable impact. Also, to help ensure success,
we need to engage communities to understand and support the change.
Uncertainty about who is on the ``federal list'' vs. the ``state list''
has been unhelpful and has set back our efforts to take on our lowest-
performing schools.
To illustrate, consider two hypothetical low-performing schools.
One is a high-poverty, chronically underperforming high school with
1,000 students and the other is a high-poverty, 50-student alternative
education school with 20 continuously enrolled students from one year
to the next. The alternative school focuses on students who have been
incarcerated or have drug treatment needs and helps transition kids
back to regular high school or helps students earn GEDs. Many of these
very students have experienced failure and disengagement at the
comprehensive high school. Both schools are persistently low-
performing, but the large high school is a few schools higher in the
rankings and thus doesn't make it on the ``Tier 2'' list. However, its
poor performance is a direct cause of the need for the alternative
school, now targeted for turnaround.
Colorado would like discretion to determine which school to serve--
to attack root causes rather than symptoms. The large high school is a
good fit for turnaround. The alternative school is not. Forcing a
leadership change at the alternative school could have a negative
impact on student engagement and the school is doing about as well as
other alternative schools. Without a doubt, we need to take on
improvements in our alternative schools. However, state ownership and
discretion are critical when we determine where to invest scarce
resources in order to increase the supply of high-performing schools,
to reach the largest number of students and maximize positive impact.
Conclusions on which schools constitute the bottom five percent
depend on the particular analytical lens one uses to identify schools
for intervention. Consider the following graphics. The first graphic
shows the lowest-performing five percent of schools in Colorado based
on standardized growth and achievement data (growth weighted 2:1) over
a combined three-year period across reading and math. The second and
third graphics show the same schools highlighted by subject area. The
axes reflect combined three-year student median growth rates and
percentages proficient or advanced. While the first graphic suggests a
tight cluster of low-performing schools, the other graphics show the
variability of performance by subject area. The point here is that
there is not just one way to identify the lowest five percent.
Performance profiles vary by elementary, middle and high school levels.
Some schools perform better in one subject or the other. ESEA should
leave room for state discretion in making these determinations.
______
Chairman Miller. Thank you very much.
Mr. Kitchens? You need your microphone.
STATEMENT OF JOE KITCHENS, ED.D, SUPERINTENDENT OF SCHOOLS,
WESTERN HEIGHTS SCHOOLS
Mr. Kitchens. Yes. Okay. Got it?
Chairman Miller. Is the green light on? Yes, I think you
are on.
Mr. Kitchens. Yes.
Chairman Miller. Thank you.
Mr. Kitchens. Thank you very much.
It is indeed an honor to be here today, and I have my
trusted assistant, Dr. Lisa McLaughlin, with me today, and she
drives the boat here, so to speak.
I wanted to start with a picture. You know, a picture
sometimes can tell a thousand words, and so we have a picture
that we would like to show the committee, and this is a picture
of our graduation last year.
And this is the way it has been for the last 5 years at
Western Heights schools in Oklahoma City. Every year in a
message of accountability we go to our people and we say,
``This is a cohort of students that we enrolled in the ninth
grade, and this is how many of that cohort dropped out, and
this is how many of that cohort graduated, and this is how many
of that cohort that we totally impacted and served.''
And the message is simple, I think, in America right now.
This is not the same country as when I was growing up in the
1960s. I entered the ninth grade. I knew who I would graduate
with. I understood that. Now it is not quite that way.
In this particular diagram, every year we lose--or every 4
years we lose 45 percent of our students, Mr. Chairman. We have
to do something about this. We have to retool America's schools
to deal with this issue of mobility.
And so if you come to our school district this year, I
promise you this will be the way it will be, and we will tell
in the most serious way of accountability how things are going
with our public school.
If we can go forward, at Western Heights, we, too, operate
and give our public an understanding of what is going on with
our school system. We use an enterprise model. We want to show
the performance of every student and every teacher's classroom,
and we practice what we call managed access for privacy
control.
That is, the teachers used records of her students and her
students only. Parent use the records of her family, their
family, only. Principal views records of students in the site
only. And it took us a while to get to that point, but that is
where we are.
So we are going to go forward, and we are going to go
directly into our network right now, live, and we will show you
a student's record that we have permission to show, and--if we
didn't lose our login. We are going to have to log in right
quick and catch the student record. Timed out on us.
And as I said, we maintain total control, managed access
control, of the record. If the status of a family changes, Mr.
Chairman, we have automated controls that shut down access to
the family record. And that way, we are protecting the privacy
of individuals.
And we need to come on down to the student record, Mr.
Anderson. Okay. And now you see the record. And this is real-
time access of data. So we are going to look at Sean's
enrollment, and it is not just real time. It is historical
data.
So let's look at current enrollment. Let's look at all
enrollment. That is the current enrollment status, and he has
been enrolled with us for 6 or 7 years, so we have historical
data. One of the critical things about data in America is to--
in the schools is to have access to historical data.
Let's look at schedule data. This is not something that you
will normally see. And this is a current schedule. This is live
data of Sean's current schedule, but in historical terms, let's
look at Sean's schedule for the last 3 years.
So for the past 3 years, we have been able to pull up
historical schedules. If you wanted an electronic transcript,
this is the way it would have to work. That is year seven.
Let's get year eight. So, year nine data. Okay. Go back up. Go
get year eight data. That is year nine. Okay. So you can see
that we are moving back and forth in schedules over time.
So let's go to attendance information. So we are gathering
all of this information on the child, and we are going to get
daily attendance right now, today. And if you could, let's just
get it for the year, full year. And there it is. And if you
want historical data, we can go back in time and pull it for 3
or 4 years, every year. Okay.
Let's offer the grades, the current grades. And as I said,
the mother signed a release and understands how this is being
used. And there are the grades. And let's look at the
assessment information. Okay. Let's look at results on
assessments. And let's look at the ACT plan. And here is the
data on the plan.
Let's go back. Let's look at a state test. The EOIs--these
are state tests. And you are actually seeing how the record can
pull up. Let's get it. And Algebra I assessment--we can go
back. We can hit another assessment.
And every assessment that the district gives is now
available to the teacher, and the teacher of record only. And
this allows us to move in and out, okay.
Now, at ``other''--and this is what I call a cross
boundary--we are actually pulling data from the child nutrition
system into the system. And so we have what we call cross-
boundary transformation of data, twelve disparate data systems
working simultaneously in managed access. The only reason
anybody here is able to see this today is because this parent
has signed off to let that happen. We can shut that down, okay.
And let's go in at that point. Now, I would say to the
people here in D.C., to our government, it is time to deal with
mobility. You know, we have to do this on behalf of our
children.
This is not the same society as we had 25, 30, 40 years ago
in the sense of we have people on the move. And we need to move
data with children so we can make informed decisions about
their educational lives.
Thank you.
[The statement of Mr. Kitchens follows:]
Prepared Statement of Joe Kitchens, Superintendent, Western Heights
School District, Oklahoma City, OK
We live in a world where rapid advances in technology are
commonplace, and leveraging technology to improve productivity is
expected. With the passage of The American Recovery and Reinvestment
Act (ARRA) of 2009, there now exists a ``once-in-a-lifetime''
opportunity to realize dramatic leaps in educational improvement to
prepare our children for the future. According to the Data Quality
Campaign (DQC, 2010), ``the education sector is on the cusp of becoming
an information-based enterprise.'' It follows that the development of
enterprise-based data systems are essential for the nation's
educational progress.
A true enterprise-based system always has, as its focus, the
``product'' to be produced. In an education-based enterprise
environment, the ``product'' is student success. An effective
education--based enterprise system provides for the creation, storage
and use of data from multiple disparate sources. Diverse data
collection, combined with the application of effective rules for data
management, means that enterprise-based educational systems hold great
promise for impacting the school improvement process in a positive
manner. Educators at all levels, from local classrooms to district
offices to state and federal education agencies, must recognize that
true school improvement--the type that is lasting and meaningful--will
occur only when school systems and agencies are simultaneously
supported via interdependent, classroom-driven longitudinal data
systems that provide near real-time, appropriately aggregated/
disaggregated data to students, teachers, parents and other
stakeholders, including state and federal agencies.
The evolution of effective enterprise-based education systems will
determine whether districts and states will actually be able to create
huge improvements in success that these times demand. School
improvement must become dynamic, where success is emulated and failure
is eliminated. Such effective classroom-based, enterprise-oriented
longitudinal data systems can be empowered through the use of emerging
technologies (with protection of private data via managed access), so
that stakeholders at all levels may better understand the real-time
impact of success and failure in our nation's classrooms.
Enterprise-based Longitudinal Data Systems
So, where do we begin? The classroom, of course! There are many
educational issues to consider:
While enterprise systems should be designed to support
``any time, any place'' learning, where does the majority of student
learning occur at this time? Answer: in the classroom.
Where do teachers and students most commonly interact in
support of learning activities? Answer: in the classroom.
At a minimum, where should educators strive to develop an
immediate impact on student learning? Answer: It all begins in the
classroom.
Who can most effectively impact student learning? Answer:
the teacher.
Who among us can best influence students to achieve their
potential? Answer: teachers, peers, parents, and mentors--those
typically engaged in student support activities.
All education initiatives should be challenged as to what value-add
they will bring to the nation's students. It only makes sense that real
and effective investment in the national education system must be
initiated and measured in terms of individual student growth. Effective
learning is personal, sometimes complex and always best supported by
quality data analysis that informs instruction on a continuous, near
real-time basis. It makes ``BIG'' sense that statewide longitudinal
data systems (SLDS) and their continuous management be inexorably and
effectively linked to America's classrooms. Real school improvement in
America is contingent on the simultaneous development of seamless,
enterprise-based longitudinal data systems at classroom, site, district
and state levels across the nation that is reflected back to the
enterprise system product--in this case, student success.
The United States Department of Education (USDOE) has supported the
creation and deployment of SLDS/enterprise-based initiatives in almost
all of the states. These efforts need to be integrated and become a
very critical aspect of educational improvement activities in all of
America's schools. It is crucial that Americans have confidence that
public education programs are in fact improving. When there are
problems in schools, the public must know that those problems will be
successfully addressed. This presents the case for enterprise-based,
multi-level school management systems within a state's existing
infrastructure. In principle, real school improvement activities must
originate at the individual student level. Growth modeling of
individual student success over time is absolutely the most valuable
tool that local administrators can provide to students, their parents
and teachers. If the development and deployment of SLDS architecture
continues from a ``top-down'' perspective without effective evidence of
coordinated linkage of student data over time, then how can these
efforts ever establish a definitive value-add for instruction?
The Impact
For the future of education, the importance of developing
enterprise-based SLDS solutions is immense. It is the only way to
address the issue of high student mobility that currently exists and
will continue to increase. Our cohort-driven statistical analyses
indicate that the nation may be missing the opportunity to effectively
and appropriately educate a large segment of our country's student
population (i.e., the mobile students). Some of our findings indicate
that mobile students fail academically and drop out of school at twice
the rate as non-mobile students. Enterprise-based systems which can
support the distribution of near real-time, high-value data that
informs instruction are absolutely essential in addressing the mobility
problems of America's students. Our data indicates that, over a four
year period, more than 50% of our secondary students are mobile. In
some districts across the country, the numbers may be much higher.
There is no solace to be gained--rather, great danger exists--when
districts or states report that non-mobile students are succeeding
academically while the plight of mobile children is ignored.
The investment of millions of dollars in longitudinal data analysis
should assist the USDOE and state educational agencies (SEAs) to become
more accountable to the American public. However, there are other
compelling reasons to use enterprise-based longitudinal data systems,
such as establishing near real-time instructional need, and assisting
in the delivery of timely instructional supports at the classroom level
while creating and distributing student growth model analyses that
validate instructional efforts.
Suggested Actions
The Council of Chief State School Officers (CCSSO) and most states
have worked to create their own versions of learning standards.
Attempts to update learning standards, whether at the federal or state
levels, must continue as the scope of knowledge grows. As long as
learning standards are modified and assessments are revised, there will
always be a need to ``bridge the gap'' between the ``old'' and the
``new'' standards. We cannot afford to rebuild our education system
every time learning standards change. There is a critical need in
education to establish a common language that simultaneously and
definitively describes the scope (what we teach) and methodology (how
we teach) of past, current, and proposed instructional efforts at every
level. In successful, enterprise-based solutions within corporate
environments, the establishment of a ``common vocabulary'' is
recognized and highly valued. We must proactively establish flexible
and definitive descriptors of what we will teach our students and then
map this common vocabulary to all valued state and national standards
of instruction. This process of ``setting standards for standards''
could greatly improve the flexibility, efficiency and effectiveness of
America's school systems, especially for mobile students. Such an
effort in the basic core of curriculum needs to be, at a minimum, a
PreK-16 effort to support the transition of students at all educational
levels.
In most successful companies within the corporate world, when a new
vocabulary is introduced, it requires the development and adoption of
new ``business processes'' that will provide new capacities to create,
store, and use data more productively. These new business processes
also require a review of data transmission at every level of functional
operations. Since there currently is a heightened interest at the
federal and state levels to collect academic performance data in the
aggregate, and since there is an associated need for school districts/
sites to develop academic performance measures at the student level,
there should be a concerted effort to study and develop new ``rules''
for enterprise-based management of educational data.
In summary, it must be noted that the deployment of effective
enterprise-based, longitudinal data systems is not widely evident in
America's schools. Efforts to improve the transparency of the nation's
school systems are dependent on the establishment of enterprise-based
longitudinal data systems. Furthermore, other issues such as quality
control, performance-based pay, and professional development are
dependent on the establishment of enterprise-based longitudinal data
systems at every level of education, including the classroom and
student levels.
REFERENCE
Data Quality Campaign (DQC), 2010. 2009-10 Progress Report on State
Data Systems and Use. Washington, DC:
www.DataQualityCampaign.org
______
Chairman Miller. Thank you.
Ms. Hartley, welcome. I think we are going to--are we
passing the computer down, Ms.--watch out there, now. We are
going to have water and coffee and computers all over the
place.
STATEMENT OF KATIE HARTLEY, M.ED., JUNIOR HIGH MATH TEACHER AND
VALUE ADDED SPECIALIST, MIAMI EAST LOCAL SCHOOLS
Ms. Hartley. Good morning.
Chairman Miller. Good morning.
Ms. Hartley. Thank you for the opportunity to speak. My
name is Katie Hartley and I teach--I currently teach middle
school math at Miami East Local School in West Central Ohio. I
am also the district's value-added specialist, and I am also a
regional value-added specialist for the western region of Ohio.
A nonprofit organization called Battelle for Kids brought
this thing called value-added data to Ohio in 2002, and the
superintendent we had at the time had the foresight to get us
involved in the program. I was selected to be trained as a
value-added specialist.
And so as both--I am here today to speak to you both as a
teacher who has used value-added data to inform decisions in my
classroom and also as a value-added specialist who has worked
with other groups of teachers to improve their practices.
Value added data, at the very basic level, is a way to
measure how much students grow in a year's time. It is the data
analysis that takes a student's test history and test history
of students like that child and they use all of this
information to make a prediction for how a student should score
on an assessment at the end of a school year.
And then we compare that prediction with how the child
actually does, and the difference between how the child should
score and how they actually score, then, is attributed to
teacher-and school-level decisions. So in a way, it is a means
to measure the effectiveness of a teacher and of a school.
At Miami East, we have used these data, like I said, for
the last 8 years. We have used some with different groups of
teachers. And I am very proud to say that for the last 2 years
we have achieved the ``excellent with distinction'' rating from
the state of Ohio, which is the highest level schools and
districts can receive.
You receive that rank by not only showing high achievement
scores, graduation rates, attendance rates with our students,
but also by showing positive growth scores, high value-added
scores, for our students.
I would like to go ahead and show you live some reports
that we are able to use in Ohio. You can see from here that the
EVOS report access has two different logins. There is the
public-level access, which I am going to show you, and then
there is an educator login.
That is a role-based access. In other words, district
leaders have access to district data. School leaders have
access to school data. And teachers have access to teacher-
level data. At this time in Ohio, parents do not have access to
their individual child's value-added data on the state system.
That is up to districts to decide how that is disseminated to
parents.
So we can scroll through every district in Ohio. And we can
automatically see a report here, and this is for Miami schools'
reading value-added scores. It is a very basic evaluation of
growth scores for students in our school district.
The analysis starts in grade four, and the intuitive nature
of the green, yellow, red--green obviously means that students
in those grade levels in reading made more than a year's
growth. They had high value-added scores. Yellow would mean
that they were close to making a year's worth of growth. And
then the red would be areas where students did not make a
year's worth of growth in that subject.
And this is also historical. We can look at data from 2007,
2008 and 2009. We can look at not just how did our students
perform last year but how have they over time performed in this
subject at this grade level. And then there is a 3-year average
here.
I can go back up here to the top, and I can choose, instead
of reading, math. In Ohio, under the Ohio system, we only do
value-added measures at the state level for reading and math.
Districts do have the option to be enrolled in a project called
Project SCORE, which Miami is in, that gives additional value-
added data for science and social studies. It also gives scores
for third graders, which the state does not give, and then it
also--we have a high school pilot.
But we can see this is now math. We were looking at reading
before. Now we are looking at some math value-added scores for
Miami East, again by grade level and by year. So we can see
that over time our fourth grade math students are doing a--
making tremendous gains.
In fifth grade math, we have gone from a green to a yellow
to a red, so as a fifth grade teacher or as a principal of that
building, you know, we need to think about what are the--what
has been happening in fifth grade math that has led to these
changes over time.
Sixth grade math, we are green. Then we drop down to yellow
but jump back up to green. So we made some adjustments there.
And so you can see from this, we can, as a school and as
teachers, look at how our students have grown and make
decisions about how we are teaching, what we are teaching, and
what we can do differently to impact that.
I unfortunately don't have a visual for this, because it
contains student-level data, but I would like to give an
example--I taught fifth grade math. This was probably about 6
years ago. And one of the pieces of information that teachers
receive is a disaggregated report--in other words, it tells us
how we grew our top achieving kids and our bottom achieving
kids and all kids in between.
And what I saw in one of the reports that I got for
students in my class was the fact that my high achieving
students had very high growth scores and my low achieving
students had very low growth scores, and that is obviously a
big red flag.
So as a teacher I had to examine what I was teaching, how I
was teaching it, how I was assessing it, how I was addressing
the needs of those lower achieving students, made some
modifications, did some different things with assessment,
instruction, brought in some volunteers, did some small group
work, did some after school work with those students, and was
able to use value-added scores from the following school year
to measure whether or not those changes had been effective.
Luckily, they were, and our low achieving students were
able to make those gains that we wanted them to make and, in
fact, across the board our middle and high achieving students
also made positive growth gains based on that.
And then one last thing I would like to show, which I think
is important, is the ability that we have to look at students
in particular teachers' rooms. These are from last school year.
These are sixth grade math reports. There were three different
teachers in our district that taught sixth grade math.
And just being able to look at the different strengths that
teachers have--this is a report for Teacher A, and we can see
that the--these are broken apart into low achieving students,
middle achieving students and high achieving students. The
green bar there in the middle would represent students at that
level making a year's worth of growth, making the--making it
where they are predicted to make it.
And we can see that Teacher A is helping her low and high
achieving students to make a year's growth, but luckily is
taking her middle achieving students and taking them even
further. Those children in the middle are scoring higher than
they are predicted to score based on their test history and
students like them in the past have scored.
Teacher B has a different look. Teacher B is making
positive growth with her lowest achieving students. The low
achieving students in Teacher B's class were making more than a
year's growth in a year's time.
Middle achieving students were making it where they were
predicted to. And high achieving students were making it just a
little bit lower than they were predicted to, and this is an
important thing to examine.
We often in schools--when we are measured on whether or not
children pass a state test, then that tends to be the focus.
And we sometimes forget those students at the high end who we
know are very, very--they are gifted. They are very bright.
They are going to pass the state test with little to no
intervention from the school.
We still need, as a school, to look at how we have grown
those children, have we met their needs. And so if we look at
Teacher A and Teacher B, they obviously have very different
strengths.
And this is a very important piece of information that then
needs to be shared between these two teachers and the
principal--you know, how is Teacher A working with students
that is helping those middle achieving students make the gains
that they are making, and how is Teacher B doing things that is
helping those low achieving students make the gains that they
are making.
In other words, not all teachers have the same strengths,
and if we can leverage the differences and the strengths that
teachers have and use that in a forum together to discuss how
we are teaching, that is the real power of using value-added
data.
Thank you.
[The statement of Ms. Hartley follows:]
Prepared Statement of Katie Hartley, Teacher, Value Added Data
Specialist, Miami East Local Schools, Miami County, OH
Hello Chairman Miller, Ranking member Kline and Members of the
Committee: Good morning, my name is Katie Hartley and I am a teacher
and value added data specialist for Miami East Local Schools in Miami
County, Ohio. I'm here today to talk to you about how I have used value
added and achievement data in my classroom and with other groups of
teachers to make decisions about curriculum and instruction.
Battelle for Kids, a nonprofit organization, brought value added
data analysis to schools in Ohio in 2002, and Miami East was one of the
first school districts in the state to begin to use this kind of
information. Value added data models use a student's individual test
history, along with historical data of other students to predict each
student's performance. Each student's actual performance is then
compared to their predicted performance to find a value added score.
The difference between a student's predicted performance and actual
performance (positive or negative) is attributed to the school and/or
teacher. This value added measurement allows schools and teachers to
evaluate the effectiveness of current enacted curriculum and
instructional practices.
Over the past eight years I have used these value added scores from
students in my classes to evaluate the strengths and weaknesses of my
teaching, made changes accordingly, and made judgments about these
changes with value added scores from subsequent tests. For example,
when low achieving students in my fifth grade math class received lower
value added scores than high achieving students in the same class, I
had to examine what skills I was teaching in that class, how I was
teaching those skills, and how I was measuring students' understanding
and mastery of the skills. I had to decide what I was doing in my
classroom that was allowing high achieving students to score even
higher than predicted, but was keeping my low achieving students from
scoring where they were predicted. I decided to keep the curriculum the
same since I was teaching all the skills and knowledge that the Ohio
Department of Education put forth for fifth graders in math, but
decided to change some of my instructional and evaluation techniques. I
incorporated more cooperative learning opportunities for students to
work together, more hands on activities for students, more games that
practiced essential skills, and also arranged for many low achieving
students to have additional help with their math work either from a
volunteer or myself. When the value added scores came out the following
year, students at all achievement levels (high, middle and low-
achieving students) had much higher value added scores than the year
before. Without the value added scores for students in my classes, I
would not have known I needed to make these changes, nor would I have
had a means to measure the effectiveness of the changes I made in my
teaching. Without a longitudinal data system with the ability to link
student scores over time, this information would not have been
available. In other words, I would not be as effective a teacher
without these data, and without the support of my local and state
agencies. Dr. Todd Rappold, my district superintendent, and Dr. Deborah
Delisle, state superintendent, both believe strongly in the use of data
to inform educational decisions, and in giving educators the tools they
need to do this effectively and successfully.
I have also worked with all teachers at Miami East Schools on the
use of value added and achievement data to make decisions, and plan for
instruction for each school year. Our ability to look at student level
data both for achievement and value added scores has allowed us to make
many improvements in teaching and learning in our schools. Miami East
has received the top rating the state of Ohio gives school districts,
`Excellent with Distinction' two years in a row. This rating is
reserved for school districts that not only have high achievement
scores, high graduation and attendance rates, but also have at least
two consecutive years of positive value added scores. The staff at
Miami East has demonstrated a dedication to using data to improve
instruction, and our students have benefited from this work. The
quality of the education students at Miami East receive is directly
correlated to their access to longitudinal student level data,
professional development time and resources around the use of value
added data to inform instruction, and the leadership and support of the
state superintendent, the district superintendent, and the district
value added specialist. A quality education for Miami East students is
made possible by quality student level data.
______
Chairman Miller. Mr. Reidenberg? Am I pronouncing your name
correctly?
STATEMENT OF JOEL R. REIDENBERG, J.D., PROFESSOR OF LAW AND
DIRECTOR OF CENTER ON LAW AND INFORMATION POLICY, FORDHAM
UNIVERSITY SCHOOL OF LAW
Mr. Reidenberg. Reidenberg.
Chairman Miller. Reidenberg. My apologies.
Mr. Reidenberg. Good morning, Mr. Chairman. Thank you very
much to you and to the distinguished members of the committee
for inviting me to testify this morning.
It is really an honor and a privilege to be able to address
the important privacy issues associated with databases of
children's educational records.
My testimony this morning is going to draw on the Fordham
study that the ranking member introduced a few minutes ago that
I co-directed along with my colleague Jamela Debelak, who is
here with me, from Fordham.
I am testifying today, though, as an academic expert and I
am not representing the views of any organization. What I would
like to do in this oral part is to summarize the written
statement I have provided to the committee for the record.
My research in this area on K through 12 educational record
databases began in October of 2006. At the time I was serving
as an elected member of the Board of Education in Millburn, New
Jersey--Millburn Township in New Jersey. So I was very
sensitive to how we measured the performance of our schools,
the performance of our teachers.
But as a board member, I heard a speech by the commissioner
of education at the time extolling the roll-out of the New
Jersey SMART data warehouse--that is what New Jersey calls it.
And this was a database that was to contain very detailed
identifiable information on our district's children that we, as
a school district, were going to be required to report to the
state.
And in listening to the commissioner speak, I was struck
that there seemed to be almost no thought given to the privacy
considerations. There seemed to be no thought given to whether
the data was necessary for an educational evaluation purpose,
whether there were access or use restrictions on the data that
the state was collecting from us. There were no data retention
policies associated with the data that the state was
collecting.
As a board member, I felt as though I was in a position of
knowing that my district was about to violate FERPA in sending
this information to the state.
I was troubled that the data warehouse was established
without public transparency. It was a surprise to all of us.
And then as an academic, I started to look into the program and
saw that it was part of a national trend driven by No Child
Left Behind, recently reinforced by the stimulus bill.
And I went back to Fordham and set out with a research team
to try to learn what was existing across the country. And let
me stress that our study and I do not challenge the importance
and the legitimacy of data collection and the use of data to
inform the educational decisions that we just heard about, and
to make assessments of performance.
But rather, what I seek to do is highlight the critical
need for policy makers to address publicly and to incorporate
privacy rules in the planning and development of these systems.
I would like to make three points from the Fordham study.
The first is that states are warehousing children's sensitive
personal information at the state level. Our study found that
most states have established state-wide databases of children's
information.
Typically, it was in identifiable form at the state level,
because very few of the states have firewalls that would
effectively separate the children's identity from the state
officials who would have access to or be maintaining the
databases.
Approximately one-third of the states are using Social
Security numbers as the identifier for children at the state-
level database. For a disturbing number of states--and I can
cite states like Alabama, Arizona, Maryland, Nevada, Oklahoma--
key information on the data warehousing programs, such as the
types of data being collected, were simply not publicly
available.
Our team of researchers--we had eight graduate students
looking at this, trying to find it--weren't able to find the
information. It means that state governments are conducting
major data processing operations essentially in secret from
parents and from the public at large.
We found that sensitive data is collected, certainly, for
NCLB reporting obligations, things like test scores, race,
disability status. But we also found that other data was
commonly collected that didn't appear to be for NCLB reporting
purposes and didn't appear on its face to be associated with
core educational assessment purposes.
So for example, 22 percent of the states were collecting at
the state level, in--often in identifiable form, whether
particular students were pregnant when they were in school.
Forty-six percent of the states were collecting mental health
illness information, whether students had been jailed.
Louisiana requires the state--the school districts to
report to the state level by Social Security number whether
students use foul language in class.
Data seems to be collected for other goals that--like
delivery of social services. So for example, there are states
that collect the birth weight of a teenage mother's baby. So it
is important for social services purposes, but the question
that we pose is is it necessary for that to be part of an
educational record in identifiable form at the state level.
We found that the United States Department of Education was
promoting interoperable standards. Interoperable standards are
important and valuable for the efficiency and the efficacy of
the data collection, but it also means that creating a national
database of schoolchildren becomes a turnkey operation,
particularly if little attention is paid to privacy in the
construction of the databases.
The second point that I would like to raise is the Fordham
study documented that basic privacy protections were lacking,
and rules need to be implemented to assure children's data is
adequately protected.
The lack of transparency for the data warehouses was deeply
troubling, and our research team had significant difficulty and
was unable to find publicly available information on what the
data being collected by the states was.
That means for parents, there are secret surveillance
systems of their children. For the public, it means that state
governments aren't accountable because the public doesn't know
what they are doing.
We found most states did not have detailed access and use
restrictions on the data held by the state. Most states did not
require database users to enter into confidentiality
agreements. Most states did not have data retention policies.
And it is very significant, because that means when states
collect information on discipline, children's interaction with
the juvenile justice system in particular, the juvenile justice
system seals those records, often expunges those records when
the child reaches 18. The state educational databases do not.
There is no data retention requirement. There is no
expungement. There is no seal. We found this to be both
surprising and troubling.
Most states are using identifiable children's information
at the state level. Do they really need to know the identity of
particular children? Anonymous information proves to be very
difficult, to actually make the data anonymous.
We heard the example from Colorado. The data is anonymous,
yes, but deleting names, creating new I.D.s, isn't sufficient.
Computer science techniques today make it very easy to re-
identify data. If you can look at clusters and cross matching
and cross referencing clusters, it becomes very simple to re-
identify from purportedly anonymous information. That is a very
significant finding that we saw in the data.
And lastly, we found that just the sheer scope of data
collection reflects that states do not seem to be worrying
about the very basic privacy principle of data minimization--in
other words, that data collections not just be fishing
expeditions because we think the data at a point in the future
will be useful.
The third point that I would like to raise from the Fordham
study is that strong security is necessary to minimize the
risks of data invasions, scandals and meltdowns. We are talking
about centralized warehouses of children's personal
information. That is a target.
Data security measures won't address some of the essential
policy decisions for privacy, like use restrictions, like data
minimization or retention periods. But what they do do--they
play a critical role in implementing the protection to prevent
unauthorized access, to prevent unauthorized use, and to
prevent disclosures.
It is inevitable that children's information will be
compromised from these central databases. Just look at the
financial services sector and how, in the banking industry, we
have seen the number of data leaks.
In the education sphere, from state databases we already
had the experience of Nashville, Tennessee. About a year ago,
all of the educational information on public schoolchildren in
the city of Nashville and 6,000 parents were disclosed on the
Internet, freely available on the Internet, because it was not
properly secured.
Data loss will occur. A hundred thousand students and
teachers in Greenville, North Carolina had their information
lost when a laptop was stolen. Data spying and voyeurs and
predators will go after the information. So we have to be very
careful how we secure it.
Importantly, states should avoid storing identifiable
information. That is the best--one of the best protections.
State-of-the-art encryption is necessary. Access controls, use
restrictions, need to be implemented.
And like the Internal Revenue Service, audit logs that
indicate when problems are there, misuse is there, intrusions
have occurred ought to be kept.
Let me conclude by recommending three steps that Congress
can take to protect children. As a condition of continued
federal funding of state warehouses of children's information,
I think Congress should first require that states articulate
through statute or regulation the justification for the
collection of each element of identifiable information. This
assures that legitimate uses are transparent and sufficiently
compelling to warrant the privacy tradeoffs.
Second, require that states define specific data retention
periods that are clearly linked to the specific purpose for
which the data is originally collected. This minimizes the risk
of data spills, protects against mission creep.
And lastly, that states be required to adopt an oversight
mechanism for the collection and use of children's educational
data. We have seen this in the Department of Homeland Security.
Congress required DHS to have a chief privacy officer. Congress
has required the Department of Justice to have a chief privacy
officer. This model provides for transparency to the public and
oversight for compliance with privacy requirements.
Thank you very much.
[The statement of Mr. Reidenberg follows:]
Prepared Statement of Joel R. Reidenberg, Professor of Law and Founding
Academic Director, Center on Law and Information Policy, Fordham
University School of Law
Good morning Mr. Chairman, Ranking Member, and distinguished
members of the Committee. I would like to thank you for the invitation
to testify today and to commend you for recognizing the importance of
privacy protections in the development of databases of children's
educational records.
My name is Joel Reidenberg. I am a Professor of Law and the
Academic Director of the Center on Law and Information Policy
(``CLIP'') at the Fordham University School of Law. As an academic, I
have written and lectured extensively on data privacy law and policy.
Of relevance to today's hearing, I directed with Jamela Debelak, CLIP's
Executive Director, the CLIP report ``Children's Educational Records
and Privacy: A Study of Elementary and Secondary School State Reporting
Systems'' (Oct. 28, 2009), http://law.fordham.edu/childrensprivacy. I
am a former chair of the Association of American Law School's Section
on Defamation and Privacy and have served as an expert adviser on data
privacy issues for the Federal Trade Commission, the European
Commission and during the 103rd and 104th Congresses for the Office of
Technology Assessment. I have also served as a Special Assistant
Attorney General for the State of Washington in connection with privacy
litigation. In appearing today, I am testifying as an academic expert
and my views should not be attributed to any organization with which I
am affiliated.
My testimony today draws on the Fordham study and I would like to
make three points directly from it:
1. States are warehousing sensitive information about identifiable
children.
2. The Fordham CLIP study documents that privacy protections are
lacking and rules need to be developed and implemented to assure that
children's educational records are adequately protected.
3. As part of basic privacy standards, strong data security is
necessary to minimize the risks of data invasions, scandals and melt-
downs from centralized databases of children's personal information.
My research focus on the treatment of K-12 educational records
began in October 2006. As an elected member of the Millburn Township
Board of Education in New Jeresey, I heard a speech by the state
commissioner of education extolling the roll-out of the NJ SMART data
warehouse later that fall. The NJ SMART program required our district
to provide detailed, sensitive information about our school children on
an identifiable basis to the state's central database. None of the
commissioner's plans indicated any effort to focus data collection on
truly necessary information, nor did they reflect any limitation on the
purposes for use of the data once collected, nor did the plans appear
to have any means for parents to check the accuracy of state-held
information, and nor did the plans have any limitations on the length
of storage. The only recognition that privacy might be affected by NJ
SMART was an architecture that included data security mechanisms. As a
Board member, I was disturbed that the state had given our district a
mandate that would invade our children's privacy for ill-defined
purposes in a way that appeared to put the district in clear violation
of the Family Educational Rights and Privacy Act (``FERPA''). I was
equally troubled that this database was established without public
transparency and debate on the policy ramifications for children's
privacy. Our Board and others we asked had not even heard about the
program.
In delving further into the New Jersey program, it became apparent
that New Jersey was part of a national trend to create state data
warehouses of children's educational records driven by No Child Left
Behind and more recently expanded by the American Recovery and
Reinvestment Tax Act of 2009. The national trend similarly had emerged
without public debate regarding privacy. As a result, we launched the
Fordham CLIP study to determine what existed across the country at the
state level, to assess whether states were protecting the privacy of
the children's information in these databases and to make best
practices and legislative reform recommendations as appropriate.
At the outset, I would like to stress that our study and I do not
challenge the importance and legitimacy of data collection and use to
better inform educational outcomes. Rather, I seek to highlight the
critical need for policy makers to incorporate privacy rules in the
planning and implementation of these systems so that the important and
legitimate goals of educational accountability do not undermine privacy
and so that the important and legitimate privacy concerns do not pose
unnecessary obstacles to educational accountability.
1. States are warehousing children's sensitive personal information
The Fordham study found that most states have established state-
wide databases of children's educational records. The information held
at the state level is typically identified or identifiable to
individual children because the databases use unique identifiers for
each child and very few states use systems that establish a firewall to
keep the identity of individual students known only at the local level.
One-third of the states track students through their social security
numbers. In other words, most states are developing systems that
centralize at the state level each individual child's information
rather than transferring data aggregated by cohorts to the state level.
For a disturbing number of states such as Alabama, Arizona,
Maryland, Nevada and Oklahoma, key information on the data warehouse
programs including the types of data that were being collected and used
were not publicly available. This means that state governments are
conducting major data processing operations involving children's
sensitive information essentially in secret from parents.
In states where information was publicly available on the data
warehouse programs, the Fordham study found that states were collecting
children's personal information to comply with NCLB reporting
obligations such as test scores, race, ethnicity, gender, and
disability status. However, the states were also collecting sensitive
information well beyond NCLB reporting requirements. The following
table gives some examples of the sensitive data collected by states.
Many additional data elements included in the state databases do
not appear to be collected for NCLB reporting purpose nor for core
educational assessment purposes. Louisiana schools, for example, must
report to the state the social security number of each child who is
disciplined for the use of foul language in school.
Data warehouses appear to gather data for other goals like the
delivery of social services. For example, Florida uses social security
numbers to collect information about its K-12 children and collects the
birth weight of a teenage mother's baby. While the birth weight of a
teenage mother's baby can be valuable information to anticipate social
service needs, the decision to include this information as part of an
educational record at the state level permanently linked to the
teenager and the baby raises many privacy risks that need to be
justified and balanced against the actual benefits for the mother and
child. The following table illustrates some of these types of data
found in the state data warehouses.
In developing data warehouses, the U.S. Department of Education has
encouraged the use of interoperable data standards. Organizations, such
as the Data Quality Campaign and the Standards Interoperability
Framework Association, have significantly advanced the development of
common data protocols. These common protocols are valuable to improve
the efficiency of data collection and use. But, the use of
interoperable data standards across state lines also means that the
creation of a national database of children becomes a turn-key
operation. Until the recent efforts of the Data Quality Campaign, basic
privacy protections were not included as key components of the work on
common data standards.
2. The lack of privacy protection
The Fordham study showed that the state data warehouses of
children's information typically lacked basic privacy protections and,
often, were not in compliance with FERPA.
As a starting point, the states' lack of transparency for these
databases is deeply troubling. Our research team had significant
difficulty and was unable to find publicly available information on the
data collected by many states. As far as parents are concerned, this
means that state governments have created secret surveillance systems
for their children. The non-transparent nature of these systems also
means that state government can avoid public accountability for its
treatment of children's personal information.
The technical architectures generally did not adequately seek to
de-identify children's information at the state level. To the extent
that outcome assessment can effectively be accomplished by examining
cohorts at the state level, rather than individual children, there is
no need for the state educational agency to have individual student
records. The use of truly anonymous information would avoid privacy
issues. However, we did not systematically see careful attention to
architectures that established identity firewalls. Professors Krish
Muralidhar and Rathindra Sarathy have demonstrated that re-
identification of specific children from purportedly anonymous student
information is already a problem in the context of public reporting on
school performance.\1\
---------------------------------------------------------------------------
\1\ Krish Muralidhar & Rathindra Sarathy, ``Privacy Violations in
Accountability Data Released to the Public by State Educational
Agencies,'' paper presented to the Federal Committee on Statistical
Methodology Research Conference, Washington DC, November 2-4, 2009
available at: http://gatton.uky.edu/faculty/muralidhar/
EdPrivacyViolation.pdf (last visited Apr. 9, 2010).
---------------------------------------------------------------------------
Data minimization, a basic privacy principle that collections of
personal information should not be conducted as general fishing
expeditions, is absent as a guiding policy for the state warehouses.
The scope of sensitive children's information that is collected by
states appears to be excessive with respect to the context and core
educational purposes of the databases.
The state data warehouses generally did not have clear legal
limitations on the purpose for which data could be accessed and used.
Without purpose limitations, states, such as New Jersey, are in facial
violation of FERPA. FERPA only permits local schools to report data to
state agencies in identifiable format for ``audit and evaluation''
purposes. The lack of purpose limitations strongly suggests that states
will begin a mission creep and use children's educational data for a
multiplicity of purposes unrelated to assuring the educational
performance of the state's schools. Most states also did not explicitly
require state officials to agree to confidentiality before accessing
student information.
The states by and large ignore data retention policies. The lack of
storage limits means that a child's third grade peccadillo and youthful
indiscretions will indeed become a ``permanent record'' since states
store detailed disciplinary and social information, including in some
instances if a child was the victim of bullying. The lack of storage
limitations is a facial violation of FERPA as FERPA requires that data
transferred to state authorities for audit and evaluation purposes not
be retained longer than necessary to accomplish those permissible
purposes. The lack of durational limits also undermines other important
public policies. For example, the detailed disciplinary information
collected on identified students, including involvement and convictions
under the juvenile justice system will be held indefinitely as part of
the ``educational records'' database. While the juvenile records are
typically sealed and may be expunged when a minor reaches adulthood,
the state's educational database without a data retention policy does
not provide any such protection.
Many states outsource the data processing services for their data
warehouses. While security and confidentiality provisions can be found
in some of these contracts, the clauses are typically very circumspect
with respect to the vendor's obligations. Vendor contracts are
generally silent with respect to uses and retention of data by the
vendor.
The Fordham CLIP study identified key privacy protections that need
to be implemented for children's educational record databases:
States should implement a technical architecture to
prevent access to identifiable information beyond the school officials
who need to know
States that outsource data processing should have
comprehensive agreements that explicitly address privacy
States should limit data collection to necessary
information for articulated, defined purposes
States should have specific data retention policies and
procedures
States should explicitly provide for limited access and
use of the children's data
States should provide public notice of state data
processing of children's information
3. Strong data security is necessary to minimize the risks of data
invasions, scandals and melt-downs from centralized databases
of children's personal information
In addition to basic privacy protections, data security is critical
when information relating to identifiable children is centralized at
the state level. Data security measures do not address the essential
policy decisions for privacy protections like data minimization,
purpose limitations, and defined storage periods. But, data security
measures play a critical role in the implementation of privacy
protections specifically with respect to the prevention of unauthorized
access, use and disclosure of personal information.
The centralization of children's information at the state level
increases the risks and scope of loss from security incidents. The
centralization means that data security breaches will be on a larger
scale than if data were held solely at the local level. For example,
according to the Congressional Research Service up to 1.4 million
residents of Colorado had their names, social security numbers and
birth dates compromised when a database from the state department of
human services was stolen from a private contractor in Texas.\2\
---------------------------------------------------------------------------
\2\ CRS Report for Congress, Data Security Breaches: Context and
Incident Summary, p. 62 (May 7, 2007) available at: http://www.fas.org/
sgp/crs/misc/RL33199.pdf
---------------------------------------------------------------------------
It is inevitable that security of the children's information will
be compromised. The experiences in the financial services sector that
have been revealed by data security breach notification laws reflect
the magnitude of this risk. Despite the deployment of significant
resources and the economic incentive for banks to avoid liability, the
number of compromised credit cards in the United States is staggering.
The Heartland Payment Systems breach alone in 2009 involved more than
100 million credit and debit card transactions. State departments of
education have neither the resources nor the same high level of
incentive to protect children's information to the degree that the
financial services sector does.
The substantial security risks to children's educational records in
data warehouses can be illustrated by a few examples:
Data spills occur when school or state officials fail to
assure adequate access controls and encryption for student records
Hackers gain access to data when it is insufficiently
protected
Data loss and theft compromise educational records when
they are insufficiently protected
Data spys and voyeurs who are internal employees with
access privileges abuse their access to personal information for
personal gain
Strong data security for children's educational records is, thus,
essential. Four critical features for a strong security system are:
States should avoid the storage of identifiable
information whenever possible.
States should use state-of-the art encryption to protect
children's data
States should have robust access control and use
authorization policies in place
States should, like the IRS, maintain audit logs that
track system use to detect intrusions and police internal misuse
Conclusion
The Fordham CLIP Study recommends several measures that I believe
Congress should consider as a condition of continued federal funding of
state data warehouses of children's information:
1. Require that states articulate through statute or regulation the
justification for the collection of each element of identifiable
information. This assures that the legitimate uses are transparent and
sufficiently compelling to warrant the privacy trade-offs.
2. Require that states define specific data retention limitations
that are clearly linked to the specific purposes for which the data is
originally collected. This reduces the risks of data spills, protects
against mission creep, and
3. Require that states adopt an oversight mechanism for the
collection and use of children's educational data. A Chief Privacy
Officer in the state departments of education would, like the CPOs in
the federal Department of Homeland Security and Department of Justice,
provide transparency to the public and oversight for compliance with
privacy requirements.
______
Chairman Miller. Thank you.
And thank you to all of you for your testimonies.
Ms. Hartley, I am going to start with you, and then if
maybe Mr. Kitchens and Mr. Wenning can respond to the question,
but I think it--you showed us Teacher A and B, and the--as we
look forward to a more collaborative workplace and school site,
and hopefully between best practices between schools and what a
district's goals are, when we look at that information on how a
particular teacher was doing in math or reading with low
achieving, high achieving, incomes, students--however you mix
them--the next logical step, it would seem to me, is not only
to be sharing that information and the ability of teachers to
assess how they are doing, but also what other teachers are
doing, and then hopefully having a principal, an academic
principal, that is prepared to see how he--how you can then
share that information.
Is that, in fact, being done? Because again, it is an
interesting graph, but if it is then not utilized--and what is
the ability and the time constraints of others on utilizing
that data to the benefit of the--of those teachers and,
clearly, of the students, if you could transfer those talents
across the students that they are responsible for?
Ms. Hartley. That is a great question. It is being done. It
is being done in my school district. Each grade level and
department, grades kindergarten through 12, are--meet regularly
to write what we call action plans on a yearly basis, and those
action plans are based on data.
Action plans, while they are written as a grade level or as
a department, are largely based on value-added scores that we
have seen.
So those are real graphs that Teacher A and Teacher B
really looked at in the fall of this school year and really had
some conversations about their instructional practices, their
assessment techniques, and in some ways the grouping of
students, how we place students in specific classes, how we--
you know, how do we group students.
And some decisions were made based on those graphs, and we
are hoping that the result of that, then, will be that the
strengths that Teacher A brought to the table and the strengths
that Teacher B brought to the table will become both of their
strengths.
Chairman Miller. If I can just add to that, and then I am
going to--I am going to go to Mr. Kitchens and Mr. Wenning, the
question also, then--how is that tool not just the
collaboration between those teachers but in terms of further
professional development--how is that information used?
Yes.
Ms. Hartley. Okay.
Chairman Miller. Just quickly you, and then----
Ms. Hartley. Okay. As a building or a district leader, you
would definitely want to know that information. And as the
instructional leader of a building, the principal's
responsibility is to ensure the best education possible for
every child in that building.
While it is not necessarily being done, I would like to see
individual professional development efforts be made to teachers
based on some of the information that we have had there.
For example, if, you know, Teacher A was not necessarily
making the growth--their low and high achieving students were
making an average year's worth of growth, which is great, so
Teacher A should really be focusing on professional development
that will target some of their higher achieving students.
And I know for a fact that Teacher A is actually working
with a gifted specialist this year on what she can do in her
classroom to engage those students and bring out the growth
measures that they would like to see from those students.
Chairman Miller. Thank you.
Mr. Kitchens?
Mr. Kitchens. Sure. One of the things that we like to do
with our network is create professional learning communities in
our district. And that is to say how would we unite all of the
fifth grade teachers of the district so that they could share
knowledge of how well their children were performing as a group
district-wide, or how well they were performing in a site.
So in every school site of our district, we have organized
our teachers into professional learning communities. And we
have instructional leaders in the schools.
And I am very proud to tell you, you know, we pay our
teachers extra duty to observe as leaders in the professional
learning community, and--much like we pay our coaches for
athletic extra duties, and it is very important to us to
establish dialogue, because collaboration is the key.
And using the data and knowing and understanding where our
students need assistance is our number one priority in
establishing that professional learning community within the
school district and within the school sites to focus on the
areas that we----
Chairman Miller. But does the data allow you to segment
those learning communities so, again----
Mr. Kitchens. Yes.
Chairman Miller [continuing]. If the teacher is doing well
in reading--assuming they have multiple responsibilities----
Mr. Kitchens. That is right.
Chairman Miller [continuing]. If they are doing well in
reading, you can segment them because of the data to work with
a group of math----
Mr. Kitchens. Yes.
Chairman Miller [continuing]. Instructors.
Mr. Kitchens. Yes, absolutely, and we see that as a key
and--to foster that communication and action plan. We ask each
school and site to foster or develop an action plan related to
the data, and that action plan is the business of the PLC.
Chairman Miller. Thank you.
Mr. Wenning?
Mr. Wenning. Thank you, Mr. Chairman. And the question cuts
to the heart of the purpose of this data, and that is to
connect student results to specific instructional resources for
that student, for those educators, and professional development
for them.
In Colorado we have a very large state, very expansive with
a lot of rural districts, and one of the keys is allowing that
kind of collaboration among educators about results and
instructional resources to happen state-wide so that our
educators connected--that are in our rural areas have access to
our educators that are in our urban areas, and that the
instructional resources are actually shared across the state
because of this information we have at the student level that
is available only to educators with a right to it.
But then at a broader level, just connecting strengths or
weaknesses in students more broadly to specific strategies, to
professional development paths, and allowing collaboration
through our new tools across the state is a real key attribute
of this longitudinal data.
Chairman Miller. Thank you.
Mr. Kline?
Mr. Kline. Thank you, Mr. Chairman.
And again, thanks to the witnesses. I apologize for being
absent for a bit. It is the crazy way we do business here,
where we scoot back and forth between different committee
hearings.
I will confess that I captured some of the testimony via
screen. It is another magical piece of technology and very
useful.
I had a meeting with about 20 school superintendents in my
district just during the Easter break, talking about No Child
Left Behind, and Elementary and Secondary Act reauthorization,
and Race to the Top, and blueprints, and all of those things, a
wonderful, wonderful round-table discussion.
And one of the things that the superintendents did express
was it would be great if we had data and a common system so
that we could share this information, and we could track
student progress, teacher performance and all of those things
that we have been talking about.
I say that just to emphasize that I am really keenly aware
of the value and the importance of this kind of data. But as I
said in my opening statement, I am very, very concerned about
the potential abuse of this data. So I am going to go to
Professor Reidenberg, if I could.
I did listen to your testimony, and you were citing some
pretty scary things, frankly, and your example of the justice
system seals a minor's record and it becomes invisible to
everybody, but potentially in a student education database that
misconduct as a student would be available forever--so I am, if
anything, even more alarmed after seeing the results of the
Fordham study and your testimony.
I was struck by some of the information that seems to be
captured in these databases. For example, the birth weight of a
student's baby, the student's birth order, information that
doesn't seem to have anything to do with a student's progress.
Do you have any--did your study reveal or do you have any
opinion on why this information is collected and what it could
be used for?
Mr. Reidenberg. We found the information by going through
the--what are known as the data dictionaries. These are the
coding books that the local schools will use to report data to
the states. So we will see different codes for various--
describing data elements.
And that is where it comes up. We didn't find any
statements explaining why that--those particular data points
were collected. I have had some conversations I can answer
separate from the study. Birth weight of a teenage mother's
baby is important for a variety of social services, health
services, for the baby and often the mother that, in some
states, they believe that is important to be provided as part
of the educational package so that that mother can succeed in
school.
I think what that reflects, though, is it is a mission
creep. It is using the educational record database for lots of
uses beyond the straight educational tracking system that one
would usually associate it with. It also tends to be a surprise
that they are collecting that.
New Jersey, my state, collects information--asks schools to
report who the students' health insurance carrier is. They want
to know certain medical test results as part of the state
database on an identifiable basis.
Mr. Kline. Mr. Wenning, is Colorado collecting that sort of
information? Is that in the database, the student's child's
birth weight, and order of birth, and financial status, income
of the parents? Is that in your database?
Mr. Wenning. Thank you, Mr. Chairman. No, that is not in
our educational data warehouse.
Mr. Kline. Anywhere.
Mr. Wenning. Not in education. I don't know----
Mr. Kline. No, no, I am talking about in----
Mr. Wenning. But in our educational data warehouse, no, we
don't have information on birth weight and, really, everything
we collect has been specified in statute or rule, and it is--
the concerns raised are important.
But no, the most important thing is we know what the data
is going to be used for, and there is actually a use case for
it. There is a major shift going on between state agencies
moving towards compliance entities to entities that are trying
to provide service and support to the field. And I think we are
catching up with that at this point.
Some of this data may very well have been collected for old
purposes that are no longer relevant. But no, not in Colorado.
We don't collect information on birth weight and put that in
our state education data warehouse.
Mr. Kline. I am heartened. That is good. Thank you.
Again, Professor Reidenberg, it looks like the way some of
this data is collected and used--and clearly, it does seem to
me that there is an intent here to share student data with
post-secondary education, and it seems that that might have
some value.
But it also looks like some of this may be just flatly in
violation of the law, the--of FERPA. Did you address that at
all? I missed that in your testimony. Are there instances here
where some states are just clearly violating the law?
Mr. Reidenberg. I addressed it more in my written
statement, but the answer to that is yes. We found in our
reviews of the state programs that in instances where states,
for example, did not have purpose limitations on the data, in
my judgment it is a violation of FERPA for the local school
district to give the state identifiable data.
The school district is permitted to give the state
educational authorities identifiable data for audit and
evaluation purposes. Unless there is some restriction, the
state can use that data for other reasons. The local school
district isn't permitted to give it to the state.
We found cases we obtained through Freedom of Information
Act requests--copies of vendor agreements between state
departments of education and their third-party vendors doing
the data processing, and we found agreements such as the one in
New Jersey that is not under the control of the department of
education. That is a violation of FERPA.
We found in most instances, the vendor agreements were
silent or said very little about privacy and how they were
going to be treating the data. I think there are--was evidence
that some of those agreements were not in compliance with
FERPA.
Mr. Kline. Okay.
Thank you, Mr. Chairman. It seems to me as we go forward
with this, we really are going to have to pay attention to how
the statute comes out and be mindful. It is, frankly, alarming
to me that some of that information is out there and too easily
accessible.
And I yield back.
Chairman Miller. Thank you.
Mr. Scott?
Mr. Scott. Thank you. Excuse me. Thank you, Mr. Chairman.
Mr. Reidenberg, the information that is gathered on
students--is the information gathered in such a way that the
data conforms across state and--across the state and across the
nation so that you can compare what is going on with one data
set in Virginia and one in California?
Mr. Reidenberg. What we focused on were the particular data
elements that were being collected, as opposed to, for example,
looking at is this type of school across state lines matching.
So I won't speak to that.
What I will speak----
Mr. Scott. I mean, is the data in conformity so that the
data in Virginia is the same data that is collected in
California?
Mr. Reidenberg. It depends. Some of the states--there are
groups of states that are using the same data protocols, so
the--the SIF organization has one common data dictionary. So if
the state is using the same data dictionary, the answer to that
is yes, they are using the same codes to report the same kinds
of information.
Mr. Scott. Now, one of the data points involves students
who do not take the test. It includes dropouts. This is
student-specific. One of the problems we have had with counting
dropouts is actually counting them, and there are different
mathematical formulas.
If you have student-specific data, will the dropout
calculation be easy to ascertain?
Mr. Reidenberg. Probably much easier than today, but what
we looked at was not just the state reporting--the local
district reporting that a student has dropped out, but their
reporting the particular reason that a student has dropped out.
So some of the reasons may be--and I am looking--these are
the disciplinary codes. The actual specific disciplinary codes
will report criminal damage to property, misappropriation with
violence to the person, possessed a pocket knife with a blade
of less than two-and-a-half inches. I mean, that is the kind of
detail that is being reported to the----
Mr. Scott. And so you could actually show that somebody
subjected to a simple-minded zero tolerance ended up dropping
out and on an aggregate basis some of these policies can become
counterproductive, so the reason can be extremely helpful.
Will there be information like an uptick in absences or
drop in grade where you can show a student all of a sudden got
into some trouble and might need intervention? Will that
information be available?
Mr. Reidenberg. It could be. It would depend on the nature
of the trouble the student got into and whether it triggers one
of the reporting requirements. We were not----
Mr. Scott. Would the school have that information?
Mr. Reidenberg. I am sorry?
Mr. Scott. Would the school have that information?
Ms. Hartley, if a school had information that a student--
student's grades dropped or has significant increase in
absences, is that information that would be useful to the
teacher?
Ms. Hartley. Oh, absolutely. You know, as a classroom
teacher, I can tell you there is a direct correlation between
student attendance and student achievement.
Mr. Scott. Okay.
Ms. Hartley. Students that are there on a consistent basis
do better. When they are not there, it is a very difficult
process to catch them up on instruction they have missed, fill
in assessment pieces and----
Mr. Scott. Now, you showed that the data can be aimed at--
per teacher and the different assessments for the teacher. One
of the problems--and I know in tennis, half your skill in
doubles is picking your partner.
You want to avoid a situation where the teacher starts
picking students and a teacher may be unwilling to take on a
slower student because it is going to mess up the average.
Can this be done in such a way that you can not discourage
the--a teacher from going across the hall and saying, ``Well,
let me try young Johnny, you are not having much success with
him?''
Ms. Hartley. Gosh, I don't work in a district where
teachers are allowed to choose students. I mean, that is
something that is done at the administrative level. I can't
speak to how it is done in other districts.
What I can say is that the exciting thing about value-added
data and actually measuring a student's growth is that you--
teachers no longer necessarily want to avoid those lower
achieving students. If we are going to be looking at how we
have grown students, as opposed to whether or not they pass or
test--pass or fail a state test--those low achieving students
have a much lower likelihood of passing the test.
If we can show growth with those students and, you know,
feel good about that and be recognized for that growth, as
opposed to whether or not they passed the state test, I think
as a teacher you are more motivated to take on the task of
working with those students.
Mr. Scott. Now, Mr. Reidenberg, is the data on the state
level--a lot of it can be valuable on an aggregate basis
without personalization.
For example, if pregnant teens are dropping out, nothing is
being done, or if a lot of low-weight babies are not doing well
because you missed an opportunity for early intervention
because you didn't take advantage of that, can--how can we make
sure that we make best use on an aggregate state-wide basis of
the information without violating individual privacy?
Mr. Reidenberg. That is a great question. That is exactly
what I was referring to when I said that we aren't challenging
that this information can be very important and used for very
legitimate reasons.
The issue is really who needs to know the identity of the
student and, you know, why does the state need to know, for
example, that Johnny or Sally dropped out of school for
religious reasons. That is one of the codings that states
report on.
The issue there is how do you structure the technical
architecture so that the----
Mr. Scott. You mean you might----
Mr. Reidenberg [continuing]. States get cohorts----
Mr. Scott. Wait a minute. You might not need that
particular student, but you might be interested in a lot of----
Mr. Reidenberg. In the cohort. You want to know what is
happening with the cohort. So it is certainly--I can easily
see--and the commissioner, I am sure, can speak to this, but
the cohort information the state needs to know, but does the
state need to know that it is Johnny or Sally in this
particular district who had this particular problem?
I think if the state needs to know it, then the state needs
to be able to publicly justify it through a rule-making
proceeding or some kind of enabling statute so that the public
can decide if that is really--or the public can make an
assessment if the government is accountable.
Mr. Scott. You look to me like you----
Mr. Wenning. Thank you, Mr. Chairman. Appreciate that.
Chairman Miller. And then we will go to Mr. Roe, so just
quickly.
Mr. Wenning. This issue of should the state have it, should
the state know it, gets down to an issue, again, of what
service do we need to provide to our schools and districts as a
state.
And we are all familiar with our federal system. We have
got a complex one. I have 180 school districts, ranging from
75,000 students to 25 students. All of them have the same legal
expectations from all of you here.
If every district were to keep its own data warehouse, we
would have many without them at all. There are basic
efficiencies for the state to manage a state-wide data
warehouse. Role-based access is essential. FERPA is clear.
Those that have a right to the information should have the
data.
But it is much more effective and, in fact, much more
secure for the state to do this effectively at the state level
and then provide access to our districts and schools, rather
than having, in our state, 180 data warehouses with 180 privacy
processes.
And this is simply a matter of effectiveness and efficiency
in the state doing this work. At the same time, states need
major investments to manage this data more responsibly. And I
think that has been pointed out very clearly. And it is a very
important issue.
There ought to be government--we have a government data
advisory board that manages how we govern this and provide
access to information. We have an education data advisory board
that reviews every data collection and makes recommendations to
the legislature or the state board to eliminate them. Those are
critical safeguards.
But state management of this information is essential if we
want to have the effectiveness that we are looking for in
education, and we have that tension, basically, to grapple
with.
Chairman Miller. Thank you.
Mr. Roe?
Mr. Roe. Thank you, Mr. Chairman.
And first, just a question toward the chair for
information. With 41 states having this data collection--we ran
across it in electronic medical records. You have one type of
system here, one type of system here. None of the systems talk
to each other. It is terribly expensive to do that.
Do we have in our bill, the stimulus package--have we
dedicated money so that a state can go in at a very low cost
and get a generic system to get the data we think they need? Is
that available?
Chairman Miller. I think Mr. Wenning can answer that as an
example, and then perhaps I can elaborate.
Mr. Wenning. Thank you, Mr. Chairman. Not yet. Most of our
collections are required by the federal government, and there
is a data dictionary and entities like the Data Quality
Campaign have been incredibly helpful in helping us get to
exactly that vision. But right now, we have multiple disparate
systems in our state----
Mr. Roe. But you have had to go--or you have had to--right,
and that is incredibly inefficient. And we have seen it with
DOD and V.A. where they spent $10 billion and can't talk to
each other.
Mr. Wenning. Right.
Mr. Roe. And----
Chairman Miller. Well, but, you know, I think, obviously,
we are--you know, we come along and--Mr. Kitchens, go ahead.
Let's have people with experience speak before those of us with
an opinion speak.
Mr. Kitchens. In our district, we use 12 disparate data
software programs built by different vendors, and it is a
hugely expensive issue. But there is a group called the School
of Interoperability Framework Association that has set a lot of
standards--not all of them--for data translation between
disparate systems.
And one of the things that I will tell you that we--3 years
I thought I am ready to distribute data to parents. For 3
years, ladies and gentlemen of the committee, we felt we were
ready. And then there would be another thing come up and, ``Oh,
my gosh, I can't distribute this data because I have got a
privacy issue with this because I don't know how many families
live in one residence.''
And there is an economic unit--there are four economic
units over here in this one address, and I didn't take that
into account. And my goodness, I cannot distribute this data
based on address. I can't distribute this data based on
something else.
There has to be business rules in place, and there have to
be very serious business rules. It took us 3 years before we
finally said we have family information management business
processes in place that we know that we can allow our parents
access, and they will see their data on their children, and
that data only.
And we even have a failsafe in the system that if the
status of a family changes--address, let's say one student
leaves out of three in a family, there is a divorce, there is
something else happening--then access to records is immediately
suspended. That is a business rule. Don't let it go on. Stop
and resolve it.
Those kind of business rules have to be in place at the
district level, site level, to protect the privacy----
Mr. Roe. I guess what I--and not to interrupt, but my time
is limited, but wouldn't it be simpler if we created--and much
less expensive--you have obviously gone to tremendous--at your
own expense, Mr. Kitchens, in Oklahoma to do that.
Just to make a point, I think we need----
Chairman Miller. That is a very important point. I think
the purpose of the grants to the states is that the states will
look across the state of California or Colorado and decide that
with the mobility of this population now that you can't have
systems that don't talk to one another, can't----
Mr. Roe. Exactly. And what we do, Mr. Chairman, is we
create systems that don't talk, and then we create another
whole business in between to make them talk to each other. I
mean, that is--I have seen that in an electronic medical record
it happens all the time. One system--anyway. Move on.
Ms. Hartley, thank you for your comments. I think,
obviously, data is important. I have used data my entire life
to treat patients. You obviously use data to evaluate students
and performance of your teachers.
And I think I could not agree more--without information you
can't make any meaningful change. I think the problem that I
have is certainly how this data is managed. Is there bias
involved in the data?
I will give you an example. Think about the--in ``The Blind
Side,'' this young fellow who is picked up off the street with
a supposedly 60 I.Q., and if look--if someone had looked at
that data and said, ``Well, he can't be taught.'' Well, it
turns out he is a college graduate and did very well.
And so I worry about data creating bias, too. And math is a
little different. Math is an absolute, but not necessarily so
with other things.
And just a comment from you all--I would like to know a
couple things. One, a comment was made--is how do you determine
the same starting date for the child. How do you know that a
child is on the same level when you made that? Because one kid
may be here and one here. How do you know that? How do you know
they are at the same level of learning, through testing, or----
Ms. Hartley. Are you talking about----
Mr. Roe. When you measured performance.
Ms. Hartley. Okay.
Mr. Roe. You said we started at the same level. How do you
know that?
Ms. Hartley. Well, not all students start at the same
level, and that is what----
Mr. Roe. Right.
Ms. Hartley [continuing]. Value-added does. It allows us to
say, ``Okay, you know, Johnny has''--in Ohio, 400 is passing on
the state test. So let's say that Johnny has a predicted score
of a 452. Johnny is a very bright student. He is predicted to
not only pass but do very well on the test.
Sally has a predicted score of a 385. Sally has not
performed well on tests in the past. Students like Sally have
not performed on the subsequent tests. Therefore, Sally is not
predicted to pass the state test.
What value-added does, then, is says okay, after Sally and
Johnny have taken the test in that grade level, you know,
Sally's predicted score was a 385, and Sally's score is a 395,
which is still not passing, it is still not at that 400 mark,
but Sally has made growth. And that is what----
Mr. Roe. I agree.
Ms. Hartley [continuing]. We need to look at. And your
reference to the movie ``The Blind Side''--you know, we have
students like that. I have students like that in my classroom
right now who are, you know, traditionally low-achieving
students.
And it is not necessarily my goal that they pass the state
test that year, but it is certainly my goal to do everything in
my power to, in the year that I have them, raise their level of
achievement so that maybe after two or 3 or 4 years of that
level of growth they are able to achieve that.
Mr. Roe. I believe you would achieve that.
I want to ask one question and then yield back my time, and
they can--if they have time later. With all the data we have
now, why do you think the U.S. system is failing?
We have got all this information. Why are we failing?
Chairman Miller. Mr. Wenning?
Mr. Wenning. Thank you, Mr. Chairman. Very quickly. You
know, data, data, everywhere, right? Data rich, information
poor. The key is we have not put this into a useful format for
educators so that it is information, and so that we can build
knowledge.
So this issue of data to information to knowledge is the
key sequence. We collect tons of data. It is useless because we
think that crunching it is useful. Well, no. Educators need
information in context so they can act on it.
And that is why we are not getting any breakthrough results
with the data we have currently.
Mr. Roe. Thank you, Mr. Chairman.
Chairman Miller. Mr. Polis?
Mr. Polis. Thank you, Mr. Chairman.
I thank the panel for their really important testimonie on
this topic as we go into the ESEA reauthorization process. The
data and the data systems are one of the most important
benefits, I think, that can come out of our federal education
reform.
Mr. Wenning, I want to congratulate you and Commissioner
Jones in Colorado and the rest of your colleagues for your
innovative efforts to create a accountability system that
focuses on all students reaching college and career readiness
by high school graduation.
I think the model that Colorado has come up with provides
something that we can learn from in this ESEA process, and I
applaud your hard work on developing the Colorado Growth Model.
My first question is regarding the potential of new
technology to foster the widespread understanding of
performance. And of course, the more people can understand
performance--families, parents, academics--it can promote new
and broad public conversation and motivate public pressure for
sustained reform.
Specifically, the new Web-based tools can provide a
critical way to empower parents through giving them information
about school choice. However, the digital divide is a major
obstacle, especially in low-income communities and some of the
communities most in need of the very empowerment that these
tools are attempting to provide.
Frequently, parents have limited or no Internet access or
aren't aware of where or how to find that useful information,
or have a language barrier or a literacy barrier to acquiring
that information.
Can you discuss what additional policies and strategies are
necessary to ensure that the information actually reaches and
empowers parents, particularly parents from an at-risk
background?
Mr. Wenning. Thank you. Thank you, Mr. Polis. Incredibly
important question. And I would say there are really two
issues. One--and this is where the federal government comes
in--we have got to close the digital divide and make sure that
we have broadband throughout the entire nation.
And as you know, we still have parts of our state that
don't have access to high-speed Internet. And that inability is
a major challenge. And of course, we have the technology to
overcome that if we have the right investment into that. That
is one step. That opens up the pipe, but it doesn't bring about
the understanding.
One of the things that we are doing is that anything we
provide on the Web has got to be available in a print format.
We have got to organize it in brochures. We have got to do
cable TV. We have got to make sure it is in multiple languages.
And basically, the role--and this is an incredibly
important role for the state, because we have the capacity to
reach an entire state, and we have to basically be very
deliberate in working with our parent groups, our educators and
making sure that we have community forums. And those are all
things that require investment and time.
States need to get into that role and recognize their most
important function is really to build the understanding of a
child's performance to their parents so they can engage
constructively in schools, and that is going to take a--really,
a multilayered approach.
But of course, the federal government can really be helpful
on the broadband issue, so that we do close that digital
divide.
Mr. Polis. I applaud the emphasis on reaching families.
Ironically, the powerful and compelling information that you
could provide if it was provided only to the information elite
and parents who already have a lot of advantages could actually
be used to perpetuate some of the learning gaps that have--that
exist by empowering parents on the positive side of that divide
to go to the schools or attend schools that are better.
I understand that Colorado is taking an open source
approach to SchoolView and the Colorado Growth Model. Can you
briefly just describe why that is important?
Mr. Wenning. Sure. Thank you. One, you know, think about
the iPhone and the apps that are emerging. We found that by
creating an open source data visualization platform it has
created a lot of room for for-profit and non-profit vendors to
work with us.
But what we are trying to do is build collaboration among
states to get to a common understanding of performance. You
know, for those of you that are in the private sector, we can
interpret a balance sheet in the same way from company to
company. In education, we have no ability to do that.
In fact, we constantly have debates about evidence rather
than how we are actually going to do better. And so what we are
trying to do is measure student progress in a common way to
understand the productivity of our education system, be able to
do return-on-investment analysis, do that with other states
like Massachusetts, and begin understanding really what works,
rather than just having debates about it.
And by having an open source approach, we found multiple
states now that are joining our effort to understand student
progress in the same way, how much growth a child is making,
and whether it is good enough for them to catch up, to keep up,
or to move up to higher levels of performance.
And the open source aspect of this is--means that we are
really leveraging public and private investment and makes it--
makes the barriers to entry really low for other states----
Mr. Polis. Briefly, by open source, you mean that
academics, amateur psychometricians and hackers can play around
with this information and create new ways of looking at it on
their own, and if they catch on, they catch on, and if they
don't, they don't?
Mr. Wenning. Yes. Now, that doesn't mean they can play
around with confidential student information.
But the visual tools that we are creating are essentially
trying to create an open market for application development
that is useful for teachers and parents and students. And that
is really what we are trying to motivate nationally through our
work.
Mr. Polis. Yes. Let me clarify that the word hackers has
several definitions, so the one I was alluding to is people who
like to code, not people who like to crack privacy firewalls.
I yield back.
Chairman Miller. Mr. Cassidy?
Mr. Cassidy. Mr. Reidenberg, Mr. Wenning, it seems
metaphorical that you are on either end of the spectrum,
because I really get different messages from the two of you.
I have a sense that although everybody is concerned about
privacy, Mr. Wenning, you see the potential in regression
analysis, put in a bunch of variables, some of which may seen
unrelated, do your regression analysis and come out with an R
squared which indicates that, ``Wow, this is significant. We
never thought it would be.''
Mr. Reidenberg, I have the sense that you are a little bit
more kind of, ``Wow, this is what we need, and this is why we
need it,'' and less exploring the possibilities in fear of
sacrificing privacy.
That said, let me ask specific questions of you both, and
then I will ask you to comment upon my premise there.
Mr. Reidenberg, I am struck that some of the things that
you point out as being kind of crazy may actually have a
rationale.
So when New Jersey contracts with Montana to take their
Medicaid-specific data to an out-of-state database, that may be
because the Montana database has experience with HIPAA
regulations and they are segregating that data because there is
a whole 'nother kind of legal set of rules for that database
activity. Is that a fair statement?
Mr. Reidenberg. No, I mean, New Jersey is contracting with
a private vendor to handle all of New Jersey's database. It is
not----
Mr. Cassidy. Well, that is what I saw in your testimony,
that it was specific for Medicaid information.
Mr. Reidenberg. I am sorry?
Mr. Cassidy. I thought I saw in your testimony that it
was----
Mr. Reidenberg. No, what----
Mr. Cassidy [continuing]. Specific for Medicaid
information.
Mr. Reidenberg. What you saw was the way New Jersey has
structured its system, it is using--it is paying for the
database on the general education student population using
money coming from the SEMI and MAC programs for Medicaid, which
are funds designed to reimburse states for health costs
associated with special education children in their classroom.
Mr. Cassidy. I see, so they are broadening that application
to include other educational data even though in the spirit of
the law it would be for medical aspects.
Mr. Reidenberg. In that case, New Jersey is diverting
Medicaid money.
Mr. Cassidy. Got you.
Mr. Reidenberg. I personally think it is Medicaid fraud.
Mr. Cassidy. Wait till the new health care bill hits. But
that is another story.
That said, some of this seems like it could be dealt with--
it seems like you have a specific issue with patients--excuse
me, student-specific indicators, so if you are using Social
Security, clearly you just want a unique identifier.
And you mentioned a dual architecture. I assume that you
want one database for research purposes and another database
for the most appropriate person to be able to see student-
specific data, your dual architecture. Can you----
Mr. Reidenberg. Look, it is not--no, the dual architecture
structure is one in which the state officials are being
structurally separated from the identity of individual
students--from identifying information for individual students.
Mr. Cassidy. Now, let me ask you--Mr. Wenning's comment,
how you have 180 school districts--if a kid commits a felony
and is expelled and goes from one end of the state to the
other, how is that data transferred? How would you see ideally
that data transferred?
Because under Mr. Wenning's--I could see that the data will
be transferred because you have a state-wide database in which,
``Wow, the kid is expelled. Why is the child expelled? He
pulled a gun on a teacher.''
Mr. Reidenberg. Well, that information has always been
transferred. The incoming school will--at least historically,
from my experience as a board member, if we were receiving a
child from out of district, we have the child's records from
the former district transferred to us.
That can be done today by electronic data interchange as
opposed to paper files, as it was in the old days. But that
information--you would want to get it directly from the school
system.
In Colorado's example, the state is centralizing the data
warehousing function. It may make sense in Colorado, not
necessarily in other states. When you centralize the data, you
magnify the risks of----
Mr. Cassidy. So then your----
Mr. Reidenberg [continuing]. Privacy----
Mr. Cassidy. Just because I am limited on time, so your
point isn't so much that you are absolutely against
centralizing data, it is just that you would want stronger
safeguards to avoid the spillage that you spoke of in
Nashville, et cetera.
Mr. Reidenberg. That is correct. It is not the
centralization. Again, I am not--this data, as we have heard,
is very valuable for determining school performance quality
measures.
Mr. Cassidy. Now, one more thing, and then I will go to
you, Mr. Wenning.
When someone decides to leave because of religious reasons,
for example, I could imagine that you might want to do a
specific initiative to reassure members of that religious
community that know the school is a safe place teaching your
values.
I guess--more information--not less, because it seems like
I am a physician, the more data you have, the more likely you
see relationships you never thought could exist. So what are
your thoughts on that?
Mr. Reidenberg. So why should the state have the list of
children, specific children, of a particular religion that are
leaving school? The fact that----
Mr. Cassidy. No, no, no----
Mr. Reidenberg [continuing]. You have a group, yes, but----
Mr. Cassidy [continuing]. No, no, no, not of the unique
identifier, but rather--again, you mentioned a cohort. At some
point, a cohort becomes an individual, and so you still want
someone to say, ``Wow, we have a uptick here on a regression
analysis that we have lost a bunch of kids because of X.''
Mr. Reidenberg. Yes, that can be valuable. But again, I
think that is a decision that needs to be made at a public
public-policy level rather than a technical behind closed doors
which, from our study, seemed to be the case.
Mr. Cassidy. I am just out of time, almost.
Mr. Wenning, your comments on all that?
Mr. Wenning. Thank you. You know, Mr. Reidenberg's
observations are extremely well founded. I think what I
disagree with, perhaps, are the conclusions.
Strong security safeguards are absolutely essential, and
the examples he cited are unfortunate and unconscionable. It is
essential that states do a better job with privacy.
My argument, though, would be that the likelihood of
building those types of safeguards in are going to be much more
likely done at the state level than in our 180 school districts
that have very limited capacity to do so. They are still
required to capture this information.
The focus on educational accountability and performance is
an imperative. We need to do much better for our children. And
this information is critical to bringing that about. But I do
not want in any way to diminish the concern over student
privacy and the need to have very solid safeguards.
And I think his analysis is spot on in terms of the kinds
of recommendations that he has got, but I don't want to make--I
want to make sure we are not throwing the baby out with the
bath water on state data systems. They have to be more secure.
They are incredibly useful.
Chairman Miller. Congresswoman Chu?
Mr. Cassidy. Thank you.
Ms. Chu. Thank you, Mr. Chair.
This question is for Mr. Wenning and/or Mrs. Hartley, and
it--I actually have two questions pertaining to distinguishing
between the growth or the value-added assessment models.
And do you distinguish between them? And one might be more
appropriate for a particular school district. My concern is
that it seems that since the value-added model relies on
aggregate data that it might be more appropriate for homogenous
student populations.
And I am wondering whether that model is as well suited to
school districts with highly diverse populations, say an urban
district where you have a large number of suburban students and
a very, very large number of poor inner-city school students
with many ESL students.
Mr. Wenning. Thank you. I will take that. So yes, these
terms are batted around a lot, and value-added models and
growth models are different things. The choice of them is
really a policy issue in terms of what questions we are trying
to answer. They are both applicable to any type of school or
student population.
And the main distinction is one of attribution and where
the attribution should be made. So the value-added model that
Ohio uses, that Tennessee uses, is designed specifically to
attribute a quantity of student progress to either a school or
to a teacher, so there is a calculation of a teacher effect.
That is modeled within a statistical model. And so for example,
how persistent is a teacher effect over time? There is an
assumption that goes in there.
A growth model like the Colorado Growth Model does not make
that attribution. Instead, we ask the educator or the user to
make the attribution.
And so for example, when we use this to look at educator
effectiveness, the question we ask is what are the growth rates
of students associated with this teacher, and we like to look
at that over three longitudinal cohorts and then, as a
principal evaluating that teacher, they would have that
information.
A value-added model would provide an effect coefficient for
that teacher that would say this much of the students' learning
was attributed to this teacher. And essentially, the
attribution is made in the statistical model. We choose to have
that attribution rest with the user.
And the reason we do that is because we want to make sure
that the quantity is useful to a teacher and a parent and a
student. And quite frankly, I find it very difficult to explain
an effect coefficient to mom and dad.
So there is a level of precision that is lost with a growth
model because of where the attribution lies, but we find that
the body of evidence, particularly using the data
visualizations we showed, we are able to move up and down from
a single child to an entire state with ease using the same
language throughout, a very straightforward language.
Value-added models are incredibly useful for research and
evaluation purposes, and so the choice of them really depends
on the questions the school or district or state is interested
in answering.
Both have a very important function, but they represent two
approaches that are not at all incompatible with one another.
Ms. Chu. Could you use both simultaneously?
Mr. Wenning. Sure, absolutely. And again, depending on--
what question you want to answer and what--with what level of
precision would dictate the kind of model you would like to
use. And both kinds are readily available in the open source
community and from specific vendors.
Ms. Chu. And what about my question about which would be
more appropriate for a large urban school district with a
widely variable population?
Mr. Wenning. Both are equally appropriate. Again, it just
depends on what questions one is trying to answer. For example,
if you want to answer with great precision how well a reading
program is improving student outcomes, and you want to control
for a variety of other variables and answer that with the
precision that a researcher would demand, I strongly recommend
a value-added model.
If you wanted to disclose this information so parents could
make good choices and you want to just sort of democratize all
the hypotheses, a growth model pushed out would be a great way
for the public to engage immediately without requiring a
researcher to answer that question.
So both are appropriate for a large system. And again,
whatever questions one wants to answer and with what level of
precision really dictates the kind of model you would like to
use.
Ms. Hartley. I just have one comment to add. As I work with
groups of teachers, it is very nice to have a growth model that
does factor out specifics like socioeconomic status, race, all
of those factors that, unfortunately, in my experience in
working with teachers, is one of the first things that teachers
like to point to for failures in students' achievement.
When you work with groups of teachers and students who are
failing, rarely do you see a teacher that stands up and says,
``Wow, that is my fault.'' So one of the first things that we
point to is, you know, home life, all kinds of factors that a
growth model completely takes out of the equation.
And from a professional development standpoint, that is
extremely important so that teachers are actually looking at
and understanding that the growth measurement that you are
looking at is actually what you as a teacher have attributed to
that child's learning.
Mr. Wenning. If I could just add, because I will share a
slightly different perspective, and this is an important one.
It is why it is important that states be able to make choices.
We actually firmly disagree with controlling for race or
socioeconomic status.
Instead, we have created a growth model that is unbiased
based on race or socioeconomic status, and we control for one
thing that we think is the most important gap, where a child
starts, regardless of their race or socioeconomic status.
And then we disaggregate those results by race and
socioeconomic status so we have a common growth measure. What
we found is that folks can make excuses very quickly about home
life or about socioeconomic status, so we don't control for it.
We disaggregate by it. But that was a choice our state made.
Chairman Miller. I am going to use the prerogative of the
chair for a second.
Mr. Kitchens, you started out your testimony talking about
mobility, what Mr. Wenning just finished saying. I assume the
questions of mobility, where that student started when they
entered my classroom is a very important factor in terms of how
we allocate--how we believe that classroom or that teacher or
that school is doing.
If a child came 2 years behind, and I am judged a failure
because they didn't make that--they didn't meet proficiency at
the fourth grade or the fifth grade, whatever, that can be
factored in this data also, as I understand it, and your use of
it.
Mr. Kitchens. That is absolutely true, and I believe that--
--
Chairman Miller. Because this is a major complaint among--
--
Mr. Kitchens. Yes.
Chairman Miller [continuing]. You know, highly mobile
districts where teachers say, ``Wait a minute, you know, the
raw material came to me in this form.''
Mr. Kitchens. All of the data that we have--and we hired
a--we are not a very big school district, but we hired a Ph.D.
research statistician and, you know, he came in, and I asked
him to do a study on behalf of the students in the district,
and to tell us what the top three issues were in our district
that impeded academic success.
And he came back with attendance is the number on issue. If
students attend more than 90 percent of the time, they have a
much greater chance of being successful academically. And yet I
will tell you that it troubles us deeply that we don't see
enough activity at our level, at the state level particularly,
backing us up to get students in to school, okay?
The next issue that he said that was extremely important
was mobility, and the third issue was discipline, and that if
these--if a child had attended school 90 percent or more of the
time, if a child had been with the school at least a year
academically, if the child had no more than three discipline
issues in a year, that they were almost universally successful
academically in that sort of----
And so you know, we really take those issues very
seriously. And this issue of mobility--I tell you, I think that
we have to standardize some information, because we have to
pass that information to people and to stakeholders and to
parents on behalf of their children.
I mean, if you look at the parent, the student and the
teacher, you have to be able to pass information between the
parent and the student and the teacher. And if you organize
your systems to do that--you know, I am not talking about
necessarily at a state-wide level, but at the functional level,
at the school level--we have to do this.
And we have to be able to inform the next district because
in 4 years, sir, 45 percent of my children will be mobile. And
they will drop out at twice the rate of all of the non-mobile
students.
Chairman Miller. Thank you.
Mr. Kitchens. And they will fail at twice the rate.
Chairman Miller. Thank you.
Mrs. McCarthy?
Mrs. McCarthy. Thank you, Mr. Chairman.
I thank the panel. It has been really fascinating to look
at the data. And this is something that a number of us on the
committee have been thinking about for quite a long time, and I
thank you for the hearing.
My colleague Rush Holt has been working on data collection
and matrix access. He is actually going to be introducing
today--that I have been working with him on.
With a lot of the questions that, you know, have been
coming up--the privacy issue, which I think everybody certainly
is concerned about--I mean, that is one of the things that we
are looking at even in the health care debate, is the privacy
issue.
But we also know that that information, that data, has to
get out there. But one of the things that I have been thinking
about as you have all been talking--when you get the data, and
you see on one of the graphs that you showed on the screens
that, all right, here is your--you saw a teacher doing very,
very well with a certain population, and you saw a teacher that
was just doing okay, and then looking at the growth of the
child, student, what--is that over a year's time?
I mean, when do you start making the corrections? Do we
lose a year of the time of the child's life? Is it possible
that in 3 months you are saying, ``You know, something is not
going on here?'' How do you make that change so we are not
looking at a year or 2 years or even 3 years?
Mr. Kitchens, when you say, you know, we have--you know, I
am sorry to say that a lot of my under served schools in my
district--there is a big dropout. It is probably 45, 50
percent. That is unacceptable.
And I know in the city, New York City, in Brooklyn, they
have been able to track these kids and they actually bring them
in to--I am not going to say a precinct, but pretty close to
it, and they have the teachers there. They are going to make
sure that these kids continue their education while they go to
an alternative school so they can get them back into the
regular school.
One of the things they said, though--the kids didn't want
to go to a regular school. They like the alternative school
because they got more attention.
So I will go back to my original question now. How long
does the data come in when you are tracking the teacher and the
child? And where do you see about that time--you know, what
has--needs to be moved to help both of them? I will throw that
open to everybody.
Mr. Kitchens. I would love to answer that question. I
fundamentally believe that we have to retool our schools at the
school site, at the classroom level, to deliver data in near
real time, that we have to become actively engaged in
developing formative assistance, that we assess our students on
a fairly regular basis within the year, that we point out where
they are having problems, that we establish a common vocabulary
for what we teach, so that we can articulate between
institutions and between classrooms and on behalf of the
students to make it easy to understand a common vocabulary that
we can apply.
We have actually received a grant from an institution in
Oklahoma, a foundation called The Inasmuch Foundation, to do
that very thing. I think that being able to understand in a
common way, for parents to understand where their children
needs help--or need help, in reading comprehension, in word
attack skills, and that sort of thing, and to make it that
simple for people and to support teachers as well.
We need to move data in near real time fashion to whomever,
to the teacher, to the student, to the parent, move data as
well--we have to set some standards for what we teach and how
we teach and articulate those to everybody concerned so that
everybody understands what is happening with the child.
Ms. Hartley. I will speak to the graphs that I put up
there. Those data are available typically in September, so in
September 2009 we looked at those data. They are based on--
primarily on tests that students took in May of 2009, the
state--the Ohio Achievement Test that they took.
And those graphs basically measured the 2009 results of
students with their predictions, and so their predictions are
based on their longitudinal test history. We see those once a
year, and that is unfortunate. If it were an ideal world, I
would have that data every week so that I can, you know,
constantly evaluate how I am teaching and the things going on
in my classroom.
Unfortunately, you know, the state test is given once a
year, and with the data system that we use, that is the type--
that is the level of test that is needed to calculate that type
of growth measurement.
Mrs. McCarthy. Okay.
Ms. Hartley. But you are right, there is a definite need
for more immediate timely information.
Chairman Miller. All right. Thank you.
Mr. Wenning. Oh, I am sorry. What Mr. Kitchens described is
exactly what could be, and the fact that that is not the
reality in most schools in the United States is probably the
single biggest reason that we have been flat for so long in
performance.
We are holding folks accountable, but we don't give them
the basic tools and information they need to actually get the
results we are expecting. And that fundamentally is what this
longitudinal data is--should be used for, not compliance, but
rather to provide the kind of support that educators need and
deserve in real time about their students to engage
constructively in education.
Chairman Miller. Thank you.
Mr. Reidenberg. If I may also just address the longitudinal
aspect of it, because I think your question was focused on the
immediate term, how do you bring it into the classroom right
away. One of the aspects that we observed with these databases
is they are being set up precisely to archive data over time.
How long? What happens with the old data?
For it to be valuable to come up with new curricula, new
standards, new ways of measuring performance, there needs to be
a commitment for a extended period of time. We are talking 20
years. And I think it is an important question that is not
really being asked.
Are states--is the federal government--willing to fund for
20 years the maintenance of these systems? If they are not, if
the commitment is not there now for doing that, then how will
the privacy questions be addressed when several years', 5-, 10-
years' worth of data is sitting there, and all of a sudden the
state decides, ``Oh, that didn't work. It didn't help us answer
these questions about why the school has been failing, so now
we are going to stop it.'' But the data is still there.
Chairman Miller. Mrs. Biggert?
Mrs. Biggert. Thank you, Mr. Chairman.
And I am sorry I have been in and out and--so I hope I am
not asking the same questions.
But, Mr. Wenning, in your testimony you demonstrated the
ability to examine the data by subgroups, including students
with IEPs, and I think and as you know, one of the biggest
challenges under the former NCLB and now the ESEA--one of the
biggest challenges is how to measure achievement for special
needs students.
And I know that we have--you know, we have heard all the
stories from some of the teachers about how concerned they were
having to give the kind of tests that they did to these
students. And given that your state's data system can measure
student growth, how do your districts use this data to improve
instruction for these children? And do you use the IEP?
Mr. Wenning. We do, and of course, the state assessment is
one data point for any student, including those with
disabilities. And so what we do is provide the information on
student growth rates for students that are on IEPs, and that
information can then be interpreted by schools and districts to
understand how well their children are doing based on where
they began and compare that to others and benchmarks so they
can understand whether their students with disabilities are
growing faster or more slowly than the next school next door or
in any district, and then allow them to benchmark where they
see the best performance happening.
That summative assessment that we administer at the end of
the year is only so useful for these purposes, but we want to
make sure that any assessment we use--and this is a nice
addition to the other evidence and learning objectives in a
child's IEP so it is one part of a body of evidence.
But importantly, it does allow any educator to understand
how well their child on an IEP is doing compared to any other
child on an IEP, controlling for where that child began. So
again, that initial starting point is the most important aspect
of this.
And so it is used to better inform whether their practices
are actually producing the results they expect.
Mrs. Biggert. Okay.
Then, Ms. Hartley, do you have anything to add from a
classroom perspective to this issue?
Ms. Hartley. Sure. Interestingly, this year I have a sixth
grade inclusion math class that I have. About half the students
in there are on IEPs. And they will take the state test along
with everyone else.
And you know, our level of expectation--when I say our, I
have a special education teacher that co-teaches the class with
me. We have high hopes for how they will perform on the Ohio
Achievement Test, but our true expectations is that we make
positive growth with every single one of those students--that,
you know, the student who is predicted to score 340, you know,
realistically could pass on a good day maybe, but if, you know,
he or she scores a 390, which is still not passing, we have
made a lot of positive growth with that child. That is going to
be a success for us.
And so as you look at some of these policies, that might be
something to look at when it comes to special needs students.
Where is it we really need for them to end up? Do we really
need for them to score that 400 on the state test, or do we
need for them to show growth?
Mrs. Biggert. I think that that is one question that I have
been asking, should we just use the IEPs in that case. But
then, Mr. Reidenberg, you know, there--you have been talking a
lot about the significant privacy concerns that you discovered
in looking at the student data systems across the country.
And do you think that this would put a kibosh or do you
think that this--using the IEPs rather than just the testing?
Mr. Reidenberg. I think including the IEPs on an
identifiable basis at the level of the state is a risky
proposition, because there will be a state where there is a
data leakage. It will happen. And when that happens, I think it
will alarm significantly the parents of classified children.
In my district when I was serving on the school board, we
saw experiences where parents were very reluctant to have their
children classified, because they didn't want them labeled. And
we saw instances where parents wanted their children classified
because they wanted the 504 accommodation for testing.
And if that kind of data is fed into a permanent record, so
to speak, I think it will certainly have a distorting effect on
the goals that the IEPs are designed to achieve and the goals
of the 504 accommodation in ways that I don't think we can
anticipate right now.
Mrs. Biggert. What kind of a breach would cause that
concern?
Mr. Reidenberg. All it will take is for a state official to
lose a laptop that has all the information on IEPs from kids
throughout the state, or 10,000 kids in a school district.
In Nashville, when Nashville had its data breach, I can
check whether Tennessee included--whether students had IEPs.
But when the data set was put on the Internet without security,
if it were a state that was recording IEP status in the state-
level database, that would have been out to the world.
Mrs. Biggert. Thank you.
Chairman Miller. Ms. Woolsey?
Ms. Woolsey. Thank you.
This has been so interesting. Thank you, witnesses. I am
not going to repeat the questions everybody else asked, so I am
going to get to--I decided that I was going to ask the
questions that parents would ask. So I think we need to know--
parents want to know how is my child doing, is my child
learning, is my child keeping up.
And they need to be, I believe--parents need to be taught
and trained how to read the data. I mean, this is just going to
be a lot of stuff to a lot of parents.
And there is a lot of parents that I know in my district--
they get a lot of their information from their teachers over
their computers, so we have to bring in a whole group of
parents that don't even have computers, if that is going to be
necessary.
So how are we going to treat these parents and--so that
they can be partners in this? I know we are talking about
finding out what are the better teachers, and of course every
parent wants their student in the better teacher's class and in
the better school, but our goal is to make all schools good.
But can you talk to me about that?
And another thing. Maybe each one of you could tell me what
social service data do you consider pertinent so that we can
actually educate the whole child. I mean, because there is
information that needs to be taken into account.
I see, Mr. Kitchens, you got excited to answer this.
Mr. Kitchens. I think it is extremely important to train
parents in this day and time, and I think common language
issues about having it easily understood, what we are trying to
teach their children, is extremely important.
I mean, we can get lost in ``educanese''----
Ms. Woolsey. Yes.
Mr. Kitchens [continuing]. So to speak. And we need not to
do that. We need to be very direct and be able to create
reporting, what I will call parent-friendly, student-friendly,
teacher-friendly reporting, about what we have taught, how
successful we have taught it.
And I think that, you know, when you think about a common
language, I want you to--I want to give you this thought, that
really the way we learn, when we learn, it is kind of like in a
hierarchy, if you kind of go back to the way people learn
issues when they are in school, how do students learn, in what
order, in what sequence.
And there is kind of an order and sequence in a lot of
ways, not totally, because we learn in differing ways. But
there is a common language, I think, that can be developed that
is hierarchical that could--people could follow--people--
parents could follow.
Ms. Woolsey. Right.
So, Mr. Wenning, do we think parents care if it is value-
added or growth model?
Mr. Wenning. No, I don't think----
Ms. Woolsey. What do they want to know?
Mr. Wenning. Parents are very clear about what questions
they want to answer, and you hit them right on the nail--how
much progress is my child making; is it good enough for them to
catch up or to keep up if they are already there, and if not,
what are we going to do about it; and how good is the school at
serving my child.
Those questions are of interest to every parent. And we are
very deliberate in stating that that is the single most
important customer for this information. And it is why we are
very deliberate of our language.
And Mr. Kitchens is exactly right. It has got to be clear.
It can't be 15,000 districts using different language for their
parents and every school in the country, and that is why it is
so important for the state to set the tone as well, so it is
not just a few districts that are able to do this, but we get
to a national conversation.
And of course, that means longitudinal data, and that means
some risk on privacy. But this is an incredibly important
leveler for our parents to engage, and we need to be leaders in
closing this digital divide so that every parent has access to
this high-quality information that allows them to
constructively engage in their schools.
Ms. Woolsey. Ms. Hartley?
Ms. Hartley. I guess I would like to come at this question
as a parent. I have a son who is in fifth grade who is a gifted
student, and my fear for him is always that he is not going to
be challenged enough.
Ms. Woolsey. Right.
Ms. Hartley. And so you know, as a value-added specialist,
I obviously have a whole wealth of knowledge and can look at
his information and come at it from that standpoint and support
his teachers and his learning and help him make the years with
the growth every year that he needs to make.
And that is important for those students as well. I think
sometimes we think about those kids that need to catch up and
engaging those parents, and those are definitely--all parents
are important to engage in the conversation.
From a teacher's standpoint, I have in the past used value-
added reports with parents. They tend to come at it--they are a
little confused, and I think the confusion comes from the
culture that they went to school in.
Education traditionally did not necessarily measure growth.
We give kids tests and how you score on the test is your
measure. It is an achievement measure. We didn't necessarily
measure, you know, this is where you came in to the chapter,
this is your understanding of this concept now and this was
your understanding at the end of the chapter.
We are not necessarily a culture that is used to looking at
growth measurements in schools, and so I think that that is
probably an important piece that we need to involve parents in,
understanding measurement of growth in schools and not just
achievement scores.
Chairman Miller. Mrs. Davis?
Mrs. Davis. Thank you, Mr. Chairman.
Thank you all for being here, and I certainly hope I didn't
miss this in being out of the room for a little while. I am
particularly interested in teacher evaluations.
And yet I do believe that we have not given teachers enough
of what they need to be able to make the adjustments in their
classrooms with the kind of information that they are provided.
I am wondering--and perhaps this is to you, Mr. Wenning--
what is it that is going on in addition to the data that is
making it possible for teachers to be able to make those
adjustments? Is it particular training that they are getting?
One of the things that we know about some school systems is
they have a very good strong collaborative model, and I guess
this goes internationally as well, where teachers mentor one
another. They talk about their students. They have time to do
that. They are not just in their classroom, you know, by
themselves all the time.
So what is it that you think is really critical that is
more dynamic so that the data systems actually work for
teachers in a way that they can make the changes that are
required to be certain that kids can be more successful?
Mr. Wenning. That is just an essential question. The keys
to this are providing that result very quickly, and it has to
be over time, because actually we are interested in progress,
not just a snapshot. And that is why the longitudinal aspect is
so important.
Immediately, though, that has to be connected to an
instructional strategy that works, and so that requires linking
it to a battery of--and a body of instructional strategies that
might be openly available in the public or might be recommended
by other educators.
And the amount of digital information that is emerging now
that can be connected through social collaboration, which
teachers are beginning to use--I receive the chairman's tweets
from this committee.
I am not sure if you are sending them, but, you know, we
are all communicating readily now. It is that connection
between the students' progress over time on multiple bodies of
evidence to specific instructional strategies, and then
allowing real-time collaboration both in schools and districts
and buildings but, more importantly, across school districts,
perhaps globally.
One of the things we have done in Colorado--and this is--I
will plug our Race to the Top proposal--is we have created a
number of ways for educators to get access to this, including
an open marketplace where an educator can contribute a lesson
plan or instructional strategy, and if it is highly rated by
another teacher as making a contribution to their practice, it
would trigger a $1,000 royalty from the state.
Now, that is a way of getting pay for performance that is
not tied to personnel evaluations but rather encouraging
teacher professionalism and contributions to one another's
practice.
Mrs. Davis. Some districts believe that in order for them
to bring this about, though, they need, you know, a lot more
resources and money. This is something that it seems to me
shouldn't overwhelm districts and their ability to actually
make this happen.
I mean, how do you--I will take California, for example,
and the chairman is very familiar with this. I mean, we are
aware of the budget today.
We know that we have been behind in terms of data
collection, and I would be curious in terms of whether you feel
that there has been the kind of communication out there and
strategies that states even as large as California and as
complex as California could pick some of this up.
What is it that--how do we talk this through with a
community that is a bit hysterical right now, and for some good
reasons, because the budgets have been so impacting on helping
kids, you know, move forward?
Mr. Wenning. It is a tough time. Thomas Friedman talks
about the great inflection happening during the great
recession.
We are cutting $260 million of state resources at the time
we are adding a whole new round of accountability requirements,
saying, ``Hey, we are going to make these great investments in
structural improvement systems, and let's do it,'' and try--and
trying to build that excitement is challenging in this tough
time.
But the Race to the Top investment and the state
longitudinal data system investment is incredibly well timed.
This type of really substantial investment in instructional
improvement systems, which is a key aspect of the data
assurance, is so well timed right now, because when I speak
with teachers and parents, they are so ready for this
information.
They are tired of the old way of holding people accountable
and not having the basic resources they need. And so the
customer is ready.
And with the investment coming from the--you know, the
federal level on these particular use--you know, tools for use,
not tools for compliance or accountability, but this emphasis
on instructional improvement is critical and is, again, well
timed and should produce the results we are expecting from them
if there--if these dollars are well invested.
But I see no resistance from educators, who are our biggest
supporters, or our parents for this kind of information,
because they are starving for it and haven't gotten it before.
Mrs. Davis. Thank you.
Chairman Miller. Thank you.
I would just add that the--and unfortunately, I represent
in some cases at various times the poorest performing schools
in the state, and when you audit those campuses, those sites,
sometimes they are awash in money but they are absolutely at
sea as to what is happening on campus.
They don't know anything about their students. They don't
have any communication with their parents. They are just
managing the site for 8 hours, 10 hours a day, and then--and
they are just--they just have no discussion of that.
This is really also about a very efficient use of those
dollars that this information can bring about in real time.
Mr. Hinojosa?
Mr. Hinojosa. Thank you, Mr. Chairman. I agree with much of
what you said in that statement, because I, too, come from an
area that is very, very poor, and with some of the highest
dropout rates that I have ever seen.
But by the same token, that area has several schools that
are in the top 100 best high schools in the whole country, and
so I identified with what Mr. Kitchens learned from that
counselor or that analyst that came in and gave you the three
reasons why students who have a high daily attendance and
mobility and good student discipline have the basis for
learning and succeeding, because that is the difference between
the schools that are having a very high dropout rate and the
ones that have 97 percent attendance, they stay with that
school ninth through the 12th grade, and they have a student
disciplined if they must, or they send him back to the sending
school district.
But together with student and parental involvement
emphasizing early reading and writing literacy, teachers
trained to teach in their major, and basic tools such as a good
library and good science lab--that gives us 97 percent
graduation rate, 97 percent going to college, unbelievable
response, even though we are very poor.
But this whole thing today has been so interesting that I
have missed other committee meetings because I wanted to stay
and chat with you. I want to learn more, because I think this
improves instruction and helps us close the achievement gap.
Mr. Wenning, my question to you is how important is it to
have this student-level data, especially for measuring growth
in subgroups such as English language learners and minority
students?
Mr. Wenning. It is essential. Ultimately, we have an
interest in equity, not just in opportunity, but now the
movement to getting really equity of results. It is essential
that we understand how different student groups are doing.
And in looking at that very carefully--and there is such an
important federal role, because without NCLB I don't think many
states would have done this. By looking particularly--I will
use the example of English language learners--we were able to
break some myths in Colorado.
Our old accountability system, which was just focused on
AYP and achievement, basically discouraged schools from wanting
students that had a low starting point.
When we began measuring growth using the Colorado Growth
Model, which only looks at where a child starts and then says,
``How much progress are they making based on all other children
with the same starting point,'' we learned something amazing
about our English language learners that folks didn't want to
believe until they saw the data.
And that is in Colorado, English language learners outgrow
their native language peers in every grade level in every
subject.
Mr. Hinojosa. If I may interrupt you----
Mr. Wenning. Yes.
Mr. Hinojosa [continuing]. That is very interesting. Tell
me, how can we afford in the school districts to be able to put
in this kind of educational accountability system? What does it
cost?
Mr. Wenning. To do what we did, the costs are relatively
low. Again----
Mr. Hinojosa. What is low?
Mr. Wenning. Everything I showed you for the Colorado
Growth Model and what we rolled out to across the state--it was
about $2 million. Now, the----
Mr. Hinojosa [continuing]. 200?
Mr. Wenning. Two million.
Mr. Hinojosa. Two million dollars.
Mr. Wenning. Yes, and that is state-funded. Now, there is a
large investment in the back end, and so we started talking
about these state longitudinal data systems, the back end data
warehouse that relates all the evidence. That is much more
expensive, and that is where the state longitudinal data system
dollars are going.
But in order to--our growth model--it is free to other
states. So when Massachusetts adopted it, it cost them zero. It
is just----
Mr. Hinojosa. But if the states were to make the
investment, then the school district could tap into that----
Mr. Wenning. Absolutely.
Mr. Hinojosa [continuing]. And thus not have to come up
with the 2 million.
Let me ask a question of Katie Hartley.
I really enjoyed your presentation. I love math. While many
teachers across my state of Texas recognize the value of using
data to improve teaching and learning, they would say that they
do not have enough time to review data during the school day.
So how often do they do it?
Ms. Hartley. That is a great question, and I completely
agree. There is just never enough time. The principal in my
building gives us a half day in the fall, and I work with each
department during that half day, 3-hour time, and we go
through--excuse me--achievement scores from the prior year as
well as value-added growth scores.
And we use that in conjunction with some other pieces of
information that we have gathered through some formative
assessments that we have the students take, and we write action
plans for the upcoming or present school year at that point.
And then we have a system within our schedule during the
day--all of our students have study hall at the same time, and
that allows teachers then to--one teacher might have, you know,
50 students in a study hall, but that frees up another teacher
to collaborate with teachers either in their grade level or in
their department.
And so grade levels meet at least once every 2 weeks and
departments meet at least once every 3 weeks during the school
day, during that structured time, to look at data, to work on
the steps in the action plan, and that is one way that our
school has very creatively and effectively gone about making
sure that teachers have that collaboration time.
Mr. Hinojosa. My time has run out, but it is all very
interesting. Thank you.
Chairman Miller. Mr. Holt?
Mr. Holt. Thank you, Mr. Chairman, and thank you for
bearing with us for the full hearing here.
This is something that a number of us have been interested
in for some time. Representative McCarthy mentioned that she
and I are introducing today a bill called the metrics bill.
It is not because we think that this is the end-all and be-
all of legislation, but we want to clearly make the point that
the NCLB mandate of infrequent high-stakes tests may have some
value for somebody somewhere--I am not sure--but it is pretty
much worthless in informing instruction.
And I visited a school district in New Jersey not too long
ago that showed me that data-driven instruction can work. This
was one of the--what we call Abbott school districts. Low
performing districts that fall under a Supreme Court case.
And in this district, they have in recent years now been
offering frequent tests, the results of which are communicated
to--throughout the school system and to the teachers almost
immediately, within hours, and the teachers use the data to
modify their instruction.
And rather than finding it threatening or intimidating, the
teachers--or intrusive--the teachers seem to love it. And so it
gets me to two questions. We have mandated these infrequent
high-stakes tests--I mean, Congress has, through NCLB.
In a revised elementary and secondary act, should we be
mandating more frequent collection of data? Should we be
mandating data systems that are built on frequent input of data
so that teachers can use that to modify their instruction?
Secondly, if we are going to specify a data system to be
used, how do we make sure that the first--that it is used
primarily to illuminate student instruction?
Now, it may also be used for teacher evaluation of teacher
performance and so forth, but it seems to me the greatest need
is student instruction, because going back to Ms. Woolsey's
comment, if we are talking about accountability in schools for
adequate progress, what every parent thinks, what every
taxpayer thinks, what every person would think is not how does
this year's fourth grade compare with last year's fourth grade,
but how are Tommy and Susie getting along.
Chairman Miller. The chair is eagerly awaiting the answer.
I think you have asked two great questions here, and I want to
make sure there is time for the answer.
Mr. Kitchens. I would like to take a try.
Chairman Miller. Take a shot.
Mr. Kitchens. I, number one, commend very highly what you
have suggested. I believe that we need to move toward formative
assessments, and that we need to move toward growth models that
are formative-based, and that we need to retool our schools to
do that.
Now, you know, from my perspective, NCLB as it exists right
now is about taking a look at differing students over the same
time frame, and it doesn't make any sense to moms and dads to
do that. It doesn't make any sense to teachers to do that.
What we need to be doing is looking at the same child over
time but through a formative assessment environment where we
are informing instruction and articulating instructional needs
to the mother and father and to the teacher, and having the
teacher able to look into a program and actually suggest to the
teacher maybe some--use data to suggest how to inform
instruction.
And I supported NCLB, and I think there were some good
things that came out of it. I think we just need to morph into
this more informative model that I think would be better and
would sustain us over time.
Mr. Holt. Mr. Wenning, could----
Mr. Wenning. Thank you. Our view is that we need to have a
comprehensive assessment system. We should continue the
requirement to have annual summative assessment. We need to
make sure we are investing in formative and interim assessment.
Summative assessment footprint can shrink and should not
crowd out the formative and interim practice and assessment
which is vital to provide that real-time information.
But we have both national accountability interests and
local performance management purposes. Both have to be
balanced. The balance is off right now.
But the annual summative assessment, the ability to measure
progress in a common way, to understand what kind of return on
investment we are getting from our tax dollars needs to be
maintained. But it can shrink in its role so that we leave much
greater room for outstanding formative practice to emerge and
allow that to be the focus of educators.
But we would urge a balanced approach on this to make sure
that we actually can have the understanding about how effective
we are nationally in reaching that goal.
By the way, that lets us--lets the federal government start
holding states accountable rather than reaching right into our
schools and districts. So that is important in terms of what
kind of roles we have between federal, state and local as well.
Mr. Reidenberg. May I say something? I share the view as a
former school official that continuing assessment is very
valuable. But be careful what you ask for, because if you make
that mandatory--are you talking about making it mandatory at
the district level, or making reporting mandatory back to the
states?
So a parent will ask you, ``Why does the state have Johnny
or Sally's biology test result from this week?'' Because that
is what happens in the state reporting systems, the test
results get--that are mandated are getting reported back to a
state database.
And to your second question, how do you assure that the
information is primarily used for student instruction, statute.
Regulation. Have these databases--the uses been defined
legally. Have the restrictions on their use been defined by law
so that there is a way to enforce that that is how the data is
used.
Ms. Hartley. I would just like, very briefly--I know we are
short on time--coming into teaching 10 years ago, I was a very
young teacher when No Child Left Behind was, you know,
implemented. And I would just like to say that----
Chairman Miller. You are still young, and I am still stuck
with No Child Left Behind. What the hell are we doing?
[Laughter.]
Ms. Hartley. Thank you. I would just like to say that I
think that sometimes we miss that, you know, No Child Left
Behind, at least in my state and in my school, put a lot of
emphasis on what was being taught in reading and math, and I
think that is a wonderful thing.
I don't think we need to necessarily throw out those
summative tests at the end of the year. I think they are
invaluable in measuring whether or not schools are--and
teachers are teaching the things they should be teaching and
students are learning the things that they should be learning.
But I also agree that that formative piece, those
assessments during the year that lead up to that summative
assessment at the end of the year, are extremely important and
probably would give us more valuable information that would
inform instruction than one summative assessment at the end of
the year.
Chairman Miller. Thank you.
Mr. Kucinich?
Mr. Kucinich. Thank you very much, Mr. Chairman, for
holding this hearing. I am sorry I am just joining you. I was
chairing a subcommittee meeting down the road.
I have read over some of the testimony, and I think that
this committee certainly has some contributions that we can
make in the area of privacy protections.
I was particularly struck by Mr. Reidenberg's testimony
where he talked about ways that we can make it possible to
protect children from disclosure of sensitive information that
really is unrelated to the educational environment.
I would like to just--and also, I was interested in the
value-added approach to data that one of the witnesses was
discussing.
Mr. Chairman, you know, it may be beyond the scope of this
hearing, but since we are talking about primarily a system
which relies on a quantitative approach, is anything being said
here or does anyone here have any thinking about a more
qualitative approach towards education?
I mean, No Child Left Behind was totally structured based
on a testing regimen. And I am interested in your experience,
even though all of you are here talking about rather discrete
quanta. What about a qualitative approach? And is there
anything that you would recommend based on your experience that
might lend itself to measurement of qualitative approaches?
Whoever would like to respond.
Mr. Wenning. Thank you, Mr. Chairman. That is an excellent
observation, and quantitative and qualitative are both
important. And we think it is important that as Congress
reauthorizes ESEA that you consider making an investment in
qualitative approaches of school evaluation.
Let me be specific about what I mean, and that is to use
models like the British inspectorate system and other
approaches of school reviews. In our state, we use this--you
know, the quantitative accountability system. It is a good
signaling device.
It tells us where there are strengths and where there are
weaknesses, where there is persistently low performance and
where there is persistently high performance that we can learn
from.
But then to intervene in a school that is low performing,
we need to send a team in of educators to really examine the
practices that are being used. Those reviews can be diagnostic
and they can also be summative. But they are essential if we
are to actually understand why a school is either succeeding or
failing.
Document that and share that information, that qualitative
evidence, along with quantitative--provides a much richer
perspective for educators to support improvement.
Mr. Kucinich. Anyone else like to try?
Mr. Kitchens. Could I respond there?
Mr. Kucinich. Please.
Mr. Kitchens. I think that it is extremely important if we
are going--you know, we have this data, and we are using data
to inform instruction. That means we need new management
practices instituted in schools.
So I think that leadership needs to be potentially
rethought, that there needs to be an investment in leadership
and in change considered that would really have us go back and
review.
Everywhere that we have seen data take hold of our economy
and improve our economy, inevitably there has been business
rule adaptations that had to be adopted, had to be instituted.
We are going to have to do this in education, I think, in a big
way.
Mr. Kucinich. Well, I want to thank the gentlemen.
Mr. Chairman, I am through asking the witnesses questions.
I just want to pose this to you. Our education system tends to
promote linear thinking. The data-oriented approach that
subsumes the educational system is ingrained with and conducive
to linear thinking.
And I am just wondering if--in our approach that this
committee uses that we shouldn't, particularly with a new
administration, expand our horizons to look at what--how do you
get out of this box that we are in. I am not rejecting the idea
that we need measuring, but how do you get out of this box that
we are in to move towards more creative, qualitative
approaches?
So I appreciate that, Mr. Chairman. Thank you.
Chairman Miller. Thank you.
Mr. Reidenberg. Mr. Chairman, could I interject just one
thing? Privacy, I think, is a critical piece for being able to
think about the qualitative side, because to the extent that
these data sets--the data collections that are being managed
without adequate attention to privacy--it de-emphasizes the
child's dignity. It de-emphasizes very important aspects of the
whole child and how that whole child is treated in the
educational system.
So I think if you want to address the qualitative side, you
have to have privacy as a piece of it, because that is going to
help assure it for you.
Chairman Miller. Any last comments?
Thank you very much. This has been a very good panel, I
think, a very helpful panel. And I am a very proud author of No
Child Left Behind, co-authored with others.
And I think that the--you know, we allowed people for 25
years to hide their failures within the systems. We knew a lot
about the top 15 percent, 20 percent of the students in this
country, and now we know a lot about the entire student
profile.
The question is now what are we doing with that
information, and this is really what this committee is working
on in a bipartisan fashion, is that next iteration.
And it really is about moving to a workplace that looks
more like a modern workplace, management that looks more modern
in terms of the management, which is a great deal of
collaboration, about sharing the responsibilities across work
forces, across customer bases--in this case, it would be
parents, families and the community--and seeing how you can
share that information to develop that quality, to develop
those tools and to develop those--what are our expectations for
young people, and to have them be able to realize them.
We recognized, obviously, in the middle of No Child Left
Behind, if you will, or from then to this date, that a growth
model started to make more sense, that we were holding people
accountable for things they had no control over, and we
continue to refine that idea in this legislation.
One of the ways we refine that is through information,
through data, because, again, many schools don't have a clue
about the populations. They don't know what has been going
wrong. They don't know what is happening in their classrooms.
And that sounds like maybe a very harsh indictment. Just visit
a lot of schools and you will find a lot of schools where
teachers are desperate for help but it doesn't come.
And I think the data--this kind of data that we have
discussed this morning really give us the best opportunity to
draw out the talents and to call upon the capital that exists
in schools today but doesn't necessarily--we don't polish that
diamond, we don't help that process, because we don't
understand the composition.
It is interesting that in 1-year's time this administration
has taken the two most recalcitrant uses of data, health care
and education, and moved them into a new century. And it has
great peril, has great concerns, but the fact of the matter
is--talk about quality. My colleague talks about quality.
If you have health insurance in my district, over 60
percent of you will probably have Kaiser. And the fact of the
matter is when you see how they manage caseloads, how they
manage families in the--in asthma epidemics and others--you now
see the story in the Wall Street Journal where hospitals, non-
profits, profits, and Kaiser are moving across to share data
systems because they recognize the mobility of their patients.
When you are in an emergency, you may not walk into your
home hospital, your home health plan or anything else. And that
information is critical--the medical errors problem. All of
these things, and now we look at this.
What we ended up with in No Child Left Behind, which is
unacceptable, is that on a single data point we take one of the
most complex organizations in American society and we make a
judgment on whether that school failed, the teacher failed, the
student failed, the family failed, the community failed and the
system failed.
There is no complex organization in our society that would
make those kinds of judgments if they were doing it for real
consequences. And we did it. What we have now is an opportunity
to use this data to help every component of that system to be
better informed and to target their talents and to strengthen
their weaknesses.
To me, that is the promise of data. And it has to be
carefully managed. It has to be protected in terms of privacy.
But the fact of the matter is what we are starting to see is
where teachers are exposed to the data in real time that is
designed to help them, it becomes their friend.
It just isn't about my pay, or my hiring or my firing, it
is about do I get to take my hopes and desires, wishes and
talents and utilize them to--the best that they can.
And I do this at a lot of teacher sites, and I am very--I
am fascinated when a teacher in California will ask a question
and a teacher in Arizona will answer it, referring that teacher
to a teacher in northern Michigan and to see what that response
is.
We are empowering, and we are providing this kind of
information. It is happening without us, but we are not getting
the full benefit of it, certainly not at a school site.
That teacher may be getting that benefit, but the school
site is not set up so that that teacher can then share with his
or her colleagues or with the principal to enlighten the
principal about a better practice or a better way for that
lesson----
And so if we--you know, as we move away from a system that
is very regimented to that one test day on that state test, and
everything else is disregarded, hopefully we do--we are able to
then realize the real potentials of the opportunity of
education.
I think we also expand the school day rather inexpensively
if we include after-school programs and what can be
accomplished in that time frame, what can be accomplished at
home if parents are informed as to what is--what the
expectation is for tomorrow in class.
These are real opportunities that do not exist in most
schools under the current system because of the lack of
information and knowledge about the student population and the
community resources that are available.
And I think over probably longer than my term in Congress,
we will also understand that education is very much more of a
process than a place, and data allows us--the students to take
themselves to other places to learn, whether it is the museum,
or whether it is an art gallery, or it is the girls' and boys'
club, or it is scouting and a merit badge and subjects in
school. All of a sudden, all of this becomes possible.
So thank you. Thank you. You have been out there riding on
the edge, and we appreciate that. And I think this is one of
the most important things we will do in this reauthorization.
And, Mr. Reidenberg, absolutely, you raise issues that I
think every member on this committee shares and is passionate
about, maybe from different ideological points of view, but we
are passionate about it, and--but I want to be very careful
that we don't start getting into mandates of what is or is not.
There may be a reason a state wants to know about this age
population for another reason. That is their decision, you
know, but for the educational components, we want it used for
this purpose, but I don't want to override what other decisions
the states made.
But with respect to this particular data, I think you are
right, we want to know how it is going to be used and for what
purpose is it being gathered, because, you know, it is like
people get excited about Web sites, and all of a sudden they
have 12,000 of them. They don't know why they have 12,000,
because they are only using four, but anyway, it is so exciting
to have access to all this information. It is also costly.
Thank you very, very much.
Thank you, Mr. Roe, for your participation this morning.
And we look forward to continuing to work with you as we
progress on the legislation.
And all members will have 14 days to submit additional
material and questions for the hearing.
And with that, the hearing stands adjourned. Thank you.
[Questions for the record submitted to Mr. Wenning follow:]
U.S. Congress,
[Via Facsimile],
Washington, DC, April 19, 2010.
Mr. Richard Wenning, Associate Commissioner,
Colorado Department of Education, 201 E. Colfax, Denver, CO.
Dear Mr. Wenning: Thank you for testifying at the Committee on
Education and Labor's hearing on, ``How Data Can be Used to Inform
Educational Outcomes,'' on April 14, 2010.
Committee Members have additional questions for which they would
like written responses from you for the hearing record.
Representative Dennis Kucinich (D-OH) has asked that you respond in
writing to the following questions:
1. Mr. Wenning, you acknowledge the importance of qualitative data
to the assessment of school performance and the value of the
``inspectorate'' model to the educational system in England; however I
note that in the Administration's blueprint for ESEA reauthorization,
the four intervention models for ``Challenge'' schools seem to lack any
sort of method for qualitative data collection (to complement the
quantitative data upon which such a judgment is based). In the context
of assessing school and student performance, what are the consequences
of giving too much weight to quantitative data relative to qualitative
data? Would it not make sense for underperforming schools to have a
method of school quality review that is based on qualitative data
collection? Additionally, would it not make sense to have such a method
of school quality review available to more than just the lowest-
performing schools?
2. Mr. Wenning, school- and district-level data is only one side of
the coin. When we talk about student/classroom assessment, we largely
mean standardized tests and other quantitative data collection
methods--NCLB has ensured that. Can you speak to the value of
qualitative data collection at the student/classroom level, and how
that might be used to assess student performance without subjecting
students to repeated, high-stakes standardized tests? How can Congress,
as it contemplates the reauthorization of ESEA, improve state and local
capacity to develop and conduct student/classroom assessments that
incorporate qualitative data collection methods?
Please send an electronic version of your written response to the
questions to the Committee staff. If you have any questions, please do
not hesitate to contact the Committee.
Sincerely,
George Miller, Chairman.
______
Responses to Mr. Kucinich's Questions From Mr. Wenning
1. Mr. Wenning, you acknowledge the importance of qualitative data
to the assessment of school performance and the value of the
``inspectorate'' model to the educational system in England; however I
note that in the Administration's blueprint for ESEA reauthorization,
the four intervention models for ``Challenge'' schools seem to lack any
sort of method for qualitative data collection (to complement the
quantitative data upon which such a judgment is based). In the context
of assessing school and student performance, what are the consequences
of giving too much weight to quantitative data relative to qualitative
data? Would it not make sense for underperforming schools to have a
method of school quality review that is based on qualitative data
collection? Additionally, would it not make sense to have such a method
of school quality review available to more than just the lowest-
performing schools?
The question sequence of what? so what? and now what? is useful in
considering the answer to your questions. Student and school
performance can be measured effectively using quantitative evidence
based on summative, interim, and formative assessments. That is, such
data is useful in answering the question of what is the academic
performance of the school.
Quantitative evidence is also useful in answering the so what
question given that such data directs our attention to inequities in
academic outcomes and subjects of strength and weakness.
Quantitative evidence falls short, however, in diagnosing root
causes of weaknesses in performance. Qualitative evidence of school
process and practice is essential in answering the question of now what
will we do to improve. Qualitative school reviews, informed by
quantitative evidence of performance strengths and weaknesses, play an
essential role to inform school improvement efforts. Failure to
understand root causes of performance problems can set educators on a
course of pursuing quick fixes that do not set a path for sustained
improvement. Qualitative school reviews are useful to all schools and
especially low-performing schools that will be the recipient of large
investments of Federal funding for improvement efforts.
As Congress contemplates the reauthorization of ESEA, it should
consider including a prominent role for qualitative school reviews to
inform school improvement efforts and to evaluate the efficacy of
school interventions.
2. Mr. Wenning, school-and district-level data is only one side of
the coin. When we talk about student/classroom assessment, we largely
mean standardized tests and other quantitative data collection
methods--NCLB has ensured that. Can you speak to the value of
qualitative data collection at the student/classroom level, and how
that might be used to assess student performance without subjecting
students to repeated, high-stakes standardized tests? How can Congress,
as it contemplates the reauthorization of ESEA, improve state and local
capacity to develop and conduct student/classroom assessments that
incorporate qualitative data collection methods?
The design of assessments is a function of funding availability and
the kinds of questions they are intended to answer. Performance
assessment that incorporate demonstrations of work or simulations can
provide timely and useful feedback to students and educators that
drives insight and action. Assessments of student work progression
through demonstrations, for example, still will yield a quantitative
score based on a rubric. So the quantitative vs. qualitative
distinction may be less important that the nature of the performance
task that is the subject of the assessment. Large scale performance
assessments that yield valid and reliable evidence are more complex and
expensive than many current state assessments.
As Congress contemplates the reauthorization of ESEA, it should pay
close attention to the kinds of summative assessments developed with
the Race to the Top (RTTT) assessment competition resources and the
formative assessments developed through RTTT phase 1 and 2 awards.
These resources present a major opportunity to invest in both large
scale and local assessments that incorporate richer perspectives on
student and classroom practice.
______
[Whereupon, at 12:33 p.m., the committee was adjourned.]