[House Hearing, 109 Congress]
[From the U.S. Government Publishing Office]
NO CHILD LEFT BEHIND:
CAN GROWTH MODELS ENSURE
IMPROVED EDUCATION FOR ALL STUDENTS?
=======================================================================
HEARING
before the
COMMITTEE ON EDUCATION
AND THE WORKFORCE
U.S. HOUSE OF REPRESENTATIVES
ONE HUNDRED NINTH CONGRESS
SECOND SESSION
__________
July 27, 2006
__________
Serial No. 109-50
__________
Printed for the use of the Committee on Education and the Workforce
Available via the World Wide Web: http://www.access.gpo.gov/congress/
house
or
Committee address: http://edworkforce.house.gov
_____
U.S. GOVERNMENT PRINTING OFFICE
28-839 PDF WASHINGTON : 2006
_________________________________________________________________
For sale by the Superintendent of Documents, U.S. Government
Printing Office Internet: bookstore.gpo.gov Phone: toll free
(866) 512-1800; DC area (202) 512-1800 Fax: (202) 512-2250 Mail:
Stop SSOP, Washington, DC 20402-0001
COMMITTEE ON EDUCATION AND THE WORKFORCE
HOWARD P. ``BUCK'' McKEON, California, Chairman
Thomas E. Petri, Wisconsin, Vice George Miller, California,
Chairman Ranking Minority Member
Michael N. Castle, Delaware Dale E. Kildee, Michigan
Sam Johnson, Texas Major R. Owens, New York
Mark E. Souder, Indiana Donald M. Payne, New Jersey
Charlie Norwood, Georgia Robert E. Andrews, New Jersey
Vernon J. Ehlers, Michigan Robert C. Scott, Virginia
Judy Biggert, Illinois Lynn C. Woolsey, California
Todd Russell Platts, Pennsylvania Ruben Hinojosa, Texas
Patrick J. Tiberi, Ohio Carolyn McCarthy, New York
Ric Keller, Florida John F. Tierney, Massachusetts
Tom Osborne, Nebraska Ron Kind, Wisconsin
Joe Wilson, South Carolina Dennis J. Kucinich, Ohio
Jon C. Porter, Nevada David Wu, Oregon
John Kline, Minnesota Rush D. Holt, New Jersey
Marilyn N. Musgrave, Colorado Susan A. Davis, California
Bob Inglis, South Carolina Betty McCollum, Minnesota
Cathy McMorris, Washington Danny K. Davis, Illinois
Kenny Marchant, Texas Raul M. Grijalva, Arizona
Tom Price, Georgia Chris Van Hollen, Maryland
Luis G. Fortuno, Puerto Rico Tim Ryan, Ohio
Bobby Jindal, Louisiana Timothy H. Bishop, New York
Charles W. Boustany, Jr., Louisiana [Vacancy]
Virginia Foxx, North Carolina
Thelma D. Drake, Virginia
John R. ``Randy'' Kuhl, Jr., New
York
[Vacancy]
Vic Klatt, Staff Director
Mark Zuckerman, Minority Staff Director, General Counsel
C O N T E N T S
----------
Page
Hearing held on July 27, 2006.................................... 1
Statement of Members:
McKeon, Hon. Howard P. ``Buck,'' Chairman, Committee on
Education and the Workforce................................ 1
Prepared statement of.................................... 2
Miller, Hon. George, Ranking Minority Member, Committee on
Education and the Workforce................................ 3
Prepared statement of.................................... 5
Statement of Witnesses:
Haycock, Kati, director, the Education Trust................. 32
Prepared statement of.................................... 34
Klein, Chancellor Joel, New York City Department of Education 20
Prepared statement of.................................... 22
Sanders, William L., Ph.D., senior manager, value-added
assessment and research, SAS Institute, Inc................ 37
Prepared statement of.................................... 39
Shaul, Marnie S., Director, Education, Workforce, and Income
Security Issues, U.S. Government Accountability Office..... 7
Prepared statement of.................................... 9
Weaver, Reg, president, National Education Association....... 24
Prepared statement of.................................... 27
NO CHILD LEFT BEHIND:
CAN GROWTH MODELS ENSURE
IMPROVED EDUCATION FOR ALL STUDENTS?
----------
Thursday, July 27, 2006
U.S. House of Representatives
Committee on Education and the Workforce
Washington, DC
----------
The committee met, pursuant to call, at 10:03 a.m., in room
2175, Rayburn House Office Building, Hon. Howard McKeon
[chairman of the committee] presiding.
Present: Representatives McKeon, Castle, Biggert, Platts,
Keller, Wilson, Kline, Inglis, Miller, Kildee, Owens, Scott,
Woolsey, Hinojosa, Tierney, Kind, Kucinich, Wu, Holt, Davis of
California, McCollum, Van Hollen, and Bishop.
Staff present: Amanda Farris, Professional Staff Member;
Steve Forde, Communications Director; Jessica Gross, Press
Assistant; Richard Hoar, Professional Staff Member; Lindsey
Mask, Press Secretary; Chad Miller, Coalitions Director for
Education Policy; Deborah L. Emerson Samantar, Committee Clerk/
Intern Coordinator; Alice Cain, Legislative Associate/
Education; Denise Forte, Legislative Associate/Education;
Lauren Gibbs, Legislative Associate/Education; David Hartzler,
Junior Technology Assistant; Lloyd Horwich, Legislative
Associate/Education; Thomas Kiley, Communications Director;
Ricardo Martinez, Legislative Associate/Education; Joe Novotny,
Legislative Assistant/Education; and Mark Zuckerman, Staff
Director/General Counsel.
Chairman McKeon [presiding]. A quorum being present, the
Committee on Education and the Workforce will come to order.
We are holding this hearing today to hear testimony on ``No
Child Left Behind: Can Growth Models Ensure Improved Education
for All Students?''
With that, I ask unanimous consent for the hearing record
to remain open 14 days to allow members' settlements and other
extraneous material referenced during the hearing to be
submitted in the official hearing record. Without objection, so
ordered.
Good morning. Today marks the latest in our series of
hearings on the No Child Left Behind Act, as we work to lay the
foundation for next year's reauthorization of this landmark
law.
As always, I would like to take a moment to thank my
colleagues for taking part in this important hearing. I would
like to extend a special note of gratitude to our committee's
senior Democrat, Mr. Miller, and the Education Reform
Subcommittee's chairman, Mr. Castle, and ranking member, Ms.
Woolsey, for providing leadership throughout the worthwhile
series of hearings.
We have only just begun exploring the various aspects of No
Child Left Behind, yet I think we can all say that we have
learned a great deal already. Each hearing has been
constructive and informative, and I believe they will prove to
be invaluable as we forge ahead into next year's
reauthorization.
Today's hearing will evaluate the implications of using
growth models to determine if schools are making adequate
yearly progress under No Child Left Behind.
Additionally, we will be discussing a new Government
Accountability Office report on the benefits and challenges of
using growth models for accountability purposes under NCLB.
The reliability and utility of growth models is the focus
of an ongoing debate, and for this committee to gather input on
the subject is both reasonable and responsible as part of our
ongoing series. We are not necessarily here to embrace the
concept nor to refute it. Instead, we are simply here to listen
and to learn.
Under current No Child Left Behind guidelines, school
districts use a status model to compare the performance of
students in a specific grade against the performance of the
students of that same grade in the previous year. This is done
to determine if schools and districts are meeting adequate
yearly progress.
Some have raised concerns about the reliability of the
status model and have suggested that a growth model would be
more useful. To be clear, growth models differ from status
models by comparing the achievement of the same students over
time.
Today, we will be hearing from witnesses on their views of
growth models, whether they are effective monitors of school
performance and progress, and whether or not growth models meet
or can be tailored to meet the objectives of No Child Left
Behind.
We will have an opportunity to hear from expert witnesses
on whether growth models determine more than just improvement,
and if they can be tailored to determine if all students are,
in fact, reaching proficiency.
Additionally, we will be considering whether or not growth
models can ensure that achievement gaps between groups of
students are closing. After all, that is the fundamental
principle of the No Child Left Behind Act.
I believe today's hearing will be very insightful and will
help us better understand the benefits and challenges of using
growth models for NCLB accountability purposes. I am looking
forward to this hearing and the additional hearings we will be
having in this series.
And I now yield to my friend, Mr. Miller, for his opening
statement.
[The prepared statement of Mr. McKeon follows:]
Prepared Statement of Hon. Howard P. ``Buck'' McKeon, Chairman,
Committee on Education and the Workforce
Good morning. Today marks the latest in our series of hearings on
the No Child Left Behind Act, as we work to lay the foundation for next
year's reauthorization of this landmark law. As always, I'd like to
take a moment to thank my colleagues for taking part in this important
hearing.
I'd also like to extend a special note of gratitude to our
Committee's senior Democrat, Mr. Miller, and the Education Reform
Subcommittee's Chairman, Mr. Castle, and ranking Member, Ms. Woolsey,
for providing leadership throughout this worthwhile series of hearings.
We have only just begun exploring the various aspects of No Child
Left Behind, yet I think we can all say that we have learned a great
deal already. Each hearing has been constructive and informative, and I
believe they will all prove to be invaluable as we forge ahead into
next year's reauthorization.
Today's hearing will evaluate the implications of using growth
models to determine if schools are making adequate yearly progress
under No Child Left Behind. Additionally, we will be discussing a new
Government Accountability Office report on the benefits and challenges
of using growth models for accountability purposes under NCLB.
The reliability and utility of growth models is the focus of an
ongoing debate, and for this Committee to gather input on the subject
is both reasonable and responsible as part of our ongoing series. We're
not necessarily here to embrace the concept, nor to refute it. Instead,
we're simply here to listen--and to learn.
Under current No Child Left Behind guidelines, school districts use
a ``status model'' to compare the performance of students in a specific
grade against the performance of the students of that same grade in the
previous year. This is done to determine if schools and districts are
meeting adequate yearly progress.
Some have raised concerns about the reliability of the status model
and have suggested that a growth model would be more useful. To be
clear, growth models differ from status models by comparing the
achievement of the same students over time,
Today, we'll be hearing from witnesses on their views of growth
models, whether they are effective monitors of school performance and
progress, and whether or not growth models meet or can be tailored to
meet the objectives of No Child Left Behind.
We'll have an opportunity to hear from expert witnesses on whether
growth models determine more than just improvement--and if they can be
tailored to determine if all students are, in fact, reaching
proficiency. Additionally, we'll be considering whether or not growth
models can ensure that achievement gaps between groups of students are
closing. After all, that's the fundamental principle of No Child Left
Behind.
I believe today's hearing will be very insightful and will help us
better understand the benefits and challenges of using growth models
for NCLB accountability purposes. I am looking forward to this hearing
and the additional hearings we'll be having in this series. And I would
now like to yield to my friend, Mr. Miller, for his opening statement.
______
Mr. Miller. Thank you very much, Mr. Chairman, and thank
you so much for not only scheduling this hearing on, I think,
one of the most important subjects we will be considering
during reauthorization, but also for the series of hearings
that you have scheduled to help the committee better understand
the challenge we confront with the reauthorization and
important consideration of various subject matters that we will
have to take into account.
I believe that today's hearing focuses on one of the most
important decisions we will face in reviewing the No Child Left
Behind law, whether or not to reform the law's current
accountability system.
I can think of no question more central to the
reauthorization goals of the law. As one of the original
authors of No Child Left Behind, I am often asked how would I
like to see the law changed.
The short answer is that I would like to see us be
responsive to the legitimate concerns while maintaining the
core values of the law--that is, providing an equal educational
opportunity for all children and an excellent education to
every single child.
There are some well-founded concerns with the current
accountability system. One widespread concern relates to
schools that are not making adequate yearly progress under the
law, even though their students are making impressive academic
progress.
For example, take a 5th-grader who reads at the 1st-grade
level. Their school could make great strides in helping the
student read and, over the course of a year, improve enough to
read at 3rd-grade level, but the school would miss making
adequate yearly progress under the law if the student and
others were not reading at grade level.
The second major concern is that the different students are
measured each year, so the achievement of this year's 5th-
graders is measured against the achievement of last year's 5th-
graders. As a result, a gain or loss of a percentage of
students who are proficient could be the result of factors
beside the school.
We need to carefully weigh and address these and other
concerns. It is important that any accountability system
identify schools that need extra help in the most fair and
accurate way possible, so that they can qualify for the
additional resources, so that children can qualify for the
additional academic opportunities such as tutoring or
transferring to another public school.
It is absolutely necessary that Congress appropriate the
funds promised to make these services available to the children
who need them.
Here is the question we must start answering today: Are
growth models a feasible alternative to the current
accountability system? I have an open mind on growth models. I
came to this idea reluctantly, but I think we have three basic
questions that we have to deal with.
First, do states have the data capacity and the expertise
they need to ensure the information gathered to determine
whether a school has made adequate progress is both valid and
reliable?
Second, do growth models appropriately credit improving
schools, or do they overstate academic progress? In other
words, are they a step forward in offering a fairer, more
reliable means of accountability, or are they a step backwards,
simply another loophole that hinders accountability?
Third, and most importantly from my point of view, is are
growth models consistent with No Child's Left Behind ultimate
goal of assuring that all children can read and do math and
science at grade level by 2014? It is imperative that the
growth be pegged to proficiency.
No Child Left Behind's goal of an excellent education for
all children and equality of educational opportunity for all
children are goals that our nation has been pursuing for at
least 40 years, ever since the signing of the Elementary and
Secondary Education Act of 1966.
We have yet to achieve them. Poor and minority children are
still often assigned to less challenging classes with less
qualified teachers. As a result, fewer than half of the
minority children can read at grade level according to the
National Assessment of Educational Progress.
And black and Latino 17-year-old students, on the average,
taught math at the same level as white 13-year-old students.
This is unacceptable.
For these children, a good education is often their best
and only hope for a prosperous future. That is why we must stay
true to No Child Left Behind's promise to provide opportunity
and an excellent education to every child, even as we make the
necessary adjustments to the law.
I want to thank our panelists for being here today.
I think you have assembled a wonderful panel, Mr. Chairman.
And I would like to extend a special thanks to Marlene Shaul
from the GAO, who I understand is now retiring.
One lousy report on NCLB and you are retiring, right? My
gosh, it is that difficult? Maybe we will all think about
joining you. But thank you so much for all your work in the GAO
office.
And with that, I look forward to the hearing.
And again, thank you, Mr. Chairman, for starting this
process this year before we get into reauthorization next year.
Prepared Statement of Hon. George Miller, Ranking Minority Member,
Committee on Education and the Workforce
Good morning. I want to thank Chairman McKeon for scheduling
today's hearing on one of the most important decisions we face in
reviewing the No Child Left Behind law: whether or not to reform the
law's current accountability system. I can think of no question more
central to the reauthorization and goals of the law.
As one of the original authors of No Child Left Behind, I am often
asked how I would like to see the law changed. The short answer is that
I would like to see us be responsive to legitimate concerns while
maintaining the core values of the law--providing equal educational
opportunities for all children and an excellent education to every
single child.
There are some well-founded concerns with the current
accountability system.
One widespread concern relates to schools that are not making
adequate yearly progress under the law even though their students are
making impressive academic progress.
For example, take a fifth grader who reads at the first grade
level. Their school could make great strides in helping the student
read and, over the course of a year, improve enough to read at the
third grade level. But the school could miss making adequate yearly
progress under the law if that student, and others, are still not
reading at grade level.
A second major concern is that different students are measured each
year, so the achievement of this year's fifth graders is measured
against the achievement of last year's fifth graders. As a result, a
gain or loss in the percentage of students who are proficient could be
a result of factors besides the school.
We need to carefully weigh and address these and other concerns.
It is important that any accountability system identify schools
that need extra help in the most fair and accurate way possible, so
they can qualify for additional resources, and so their children can
qualify for extra academic opportunities, such as tutoring or the
transferring to another public school.
It is absolutely necessary that Congress appropriate the funds
promised to make these services available to the children who need
them.
Here's the question we need to start answering today: are growth
models a feasible alternative to the current accountability system?
I have an open mind about growth models, and have three basic
questions.
First, do states have the data capacity and expertise they
need to ensure that information gathered to determine whether a school
has made adequate progress is both valid and reliable?
Second, do growth models appropriately credit improving
schools, or do they overstate academic progress? In other words, are
they a step forward in offering a fairer, more reliable means of
accountability? Or are they a step backward--simply another loophole
that hinders accountability?
Third, and most importantly, are growth models consistent
with No Child Left Behind's ultimate goal of ensuring that all children
can read and do math and science at grade level by 2014? It is
imperative that growth be pegged to proficiency.
No Child Left Behind's goals--of an excellent education for all
children and equality of educational opportunity for all children--are
goals that our nation has been pursuing for at least 40 years, ever
since the signing of the Elementary and Secondary Education Act in
1966. But we have yet to achieve them.
Poor and minority children are still often assigned to less-
challenging classes and less qualified teachers. As a result, fewer
than half of minority children can read at grade level, according to
the National Assessment of Educational Progress.
And black and Latino 17-year-old students are, on average, taught
math at the same level as white 13-year-old students.
This is unacceptable.
For these children, a good education is often their best and only
hope for a prosperous future.
That's why we must stay true to No Child Left Behind's promise to
provide opportunity and an excellent education to every child, even as
we make the necessary adjustments to the law.
I thank our panelists for being with us today and for the light you
will shed on these important questions. I would like to extend a
special thank you to Marnie Shaul--who I understand is retiring--for
her outstanding work at the Government Accountability Office, including
all of her work on the very useful report that we will hear about
today.
I also thank Chairman McKeon and his staff for the bipartisan
process that led to this hearing. I hope that today's hearing will help
us make great progress towards our reauthorization of No Child Left
Behind.
______
Chairman McKeon. Thank you, Mr. Miller. I hope that wasn't
your retirement you were announcing.
Mr. Miller. Not me, Mr. Chairman. Not as long as you are
here.
[Laughter.]
Chairman McKeon. We work well together. I would hate to see
him leave, most of the time.
We have a distinguished panel here today, and I would like
to introduce them now at this time.
First we will hear from Dr. Marlene Shaul, the director of
education, workforce and income security for the Government
Accountability Office. Dr. Shaul is responsible for the studies
GAO undertakes on early childhood programs and elementary and
secondary education programs. Before her work in the Federal
Government, she worked for the state of Ohio on community and
business development issues and at the Kettering Foundation.
Dr. Shaul holds a Ph.D. in economics from the Ohio State
University.
I would also like to take a moment to recognize Dr. Shaul
for all of her work on behalf of GAO. And as Mr. Miller has
pointed out, she will be retiring next month, and I want to
thank her for all her efforts on what you have done.
Then we will hear from Mr. Joel Klein, the chancellor of
the New York City Department of Education. Chancellor Klein
oversees more than 1,450 schools with over 1.1 million
students. When I served on the school board in California, our
district had 10,000 students, and the way they divided this up
at the state level was small districts, large districts, and
Los Angeles. And Los Angeles is much smaller than New York when
it comes to the number of children that you have.
Since becoming chancellor, Mr. Klein has enacted his reform
program, Children First, which provides academic support for
students who are struggling as well as new supports for
parents, in addition to improving school safety. A native of
New York, he is active in community service work and has a
strong interest in educational issues. He studied at New York
University School of Education and later taught math to 6th-
graders at a public school in Queens.
Then we will hear from Dr. Reg Weaver, the president of the
National Education Association. Dr. Weaver is currently serving
his second term as president of the NEA. Prior to his position
at the NEA here in Washington, Dr. Weaver served as president
of his local NEA in Harvey, Illinois and as president of the
NEA affiliate in Illinois.
A 30-year classroom veteran, Dr. Weaver also serves on a
number of boards, including the executive board of the National
Council for the Accreditation of Teacher Education and the
board of directors of the National Board of Professional
Teaching Standards.
Then we will hear from Ms. Kati Haycock, who is the
director of The Education Trust. Established in 1992, Ed Trust
provides a voice for young people, especially those who are
poor or members of minority groups. Ed Trust also provides
hands-on assistance to educators who want to work together to
improve student achievement in pre-Kindergarten through
college. Prior to coming to The Education Trust, Ms. Haycock
served as executive vice president of the Children's Defense
Fund, the nation's largest child advocacy organization.
And finally, we will hear from Dr. William Sanders,
research fellow with the University of North Carolina system
and a manager of value-added assessment and research for the
SAS Institute Inc in Cary, North Carolina. Over the last 20
years, Dr. Sanders and his colleagues have developed and
refined a methodology to measure the influence that school
systems, schools and teachers have on the academic progress of
students.
He has served as an adviser to policymakers at the Federal
level and has worked with many states and school districts
interested in developing a value-added component to leverage
their testing data into more precise and reliable information.
Many of his suggestions concerning measurement of student
outcomes were incorporated into Tennessee's Educational
Improvement Act of 1992.
Thank you all for being here.
I would like to remind you, as you begin your testimony,
there is a little light that will come on in front of you.
Green means go. Yellow means you have a minute left. And red
means wrap it up if you haven't already done so.
And your full written testimony will be included in the
record.
Dr. Shaul?
STATEMENT OF MARLENE S. SHAUL, DIRECTOR, EDUCATION, WORKFORCE
AND INCOME SECURITY ISSUES, U.S. GOVERNMENT ACCOUNTABILITY
OFFICE
Ms. Shaul. Mr. Chairman and members of the committee, thank
you for the opportunity to present the findings of the report
we did for this committee on growth models and how they might
be used to meet the goals of the No Child Left Behind Act, such
as having all students proficient by 2014.
Our findings are based on GAO's March 2006 survey of the 50
states and the District of Columbia about their use of growth
models. We also analyzed academic performance data in several
states.
As the committee requested, we have used a broad definition
of growth models that includes year-to-year comparisons of
schools, cohorts of students and individual students.
My remarks today focus on three topics: How many states are
currently using some form of growth models and why; how growth
models can measure progress toward achieving key goals of the
law; and the challenges states face in using growth models to
measure schools' progress, especially to meet the law's
requirements.
Before I go over our findings on each of these, let me give
you our bottom line. It is possible for states to use growth
models to meet the goals of the law, but they face considerable
challenges in doing so.
Now I would like to turn to each of the topics. As you
know, states use growth status models to measure AYP and many
of them use growth models for their own purposes such as
grading schools and rewarding teachers.
As the map shows on the screen, in March of 2006 26 states
were using growth models. Those are shown in the dark green.
Another 22 states were either considering using growth models
or in the process of implementing them. And those are shown in
the medium green.
Most states are currently using school level growth models.
They compare changes in proficiency levels over time at a
particular school. Using a hypothetical example, let's see how
this works. It is a three-step process.
In year one, 50 percent of the students assessed were
proficient or above. In year two, 60 percent of the students
assessed were proficient or above. Growth is measured as that
10 percent difference, and states could use this measure of
growth, for example, to compare schools that are similarly
situated.
Seven states currently track individual students and can
compare growth for a cohort of students--that is the same group
of students over time--or for individual students. These models
require substantially more data and are more involved than
models that measure results at the school level.
In these models, the state may compare actual student
achievement to prior student achievement to determine if
progress is sufficient.
We examined two state models that measure growth over time
and measure progress toward the law's goals. Massachusetts'
model sets school targets that are achieved when the number of
students at various levels of achievement increases.
The model sets different targets for the school and for
each subgroup in the school, and this model has been approved
by the Department of Education for use in meeting AYP.
Tennessee's model uses individual student data and projects
future academic achievement using information from past
performance. It was recently approved by the department for its
new growth model pilot program.
Let's take a look at a very simplified example. My
apologies to Dr. Sanders here for a very simplified example. In
4th grade, we just have three students on the screen, Students
A, B and C, and none of them are shown as meeting the 4th-grade
proficiency target.
Next, a projection is made based on past performance, and
you can see the projection to 7th grade shows that two of these
three students are projected to make the state's 7th-grade AYP
target.
So two of these three students can be counted as
contributing to meeting the 4th-grade AYP target. Now, you may
wonder why two students with similar scores would have
different projections. That is because the projection is based
on prior assessments, and those could be quite different.
As you might gather, using growth models can be
complicated. To undertake growth models, states must have
assessments that are comparable across grades, and at least 2
years of assessment data and the capacity to analyze this
greater amount of data.
States need personnel who have technical skills and can
explain the results of growth models to teachers and to
parents.
Using growth models for AYP purposes may pose a risk for
some lower performing schools. Lower performing schools that
initially make good growth will need to sustain high levels of
growth rates to meet future targets.
If Title I schools do not sustain high growth, they may be
disadvantaged by not receiving federally required assistance
during the years that the growth target was met.
On the other hand, if fewer schools need federally required
assistance, resources could be concentrated in the lowest
performing schools.
Mr. Chairman, this concludes my remarks. Thank you.
[The prepared statement of Ms. Shaul follows:]
Prepared Statement of Marlene S. Shaul, Director, Education, Workforce,
and Income Security Issues, U.S. Government Accountability Office
Mr. Chairman and Members of the Committee: I am pleased to be here
today to discuss our report, which describes how states use growth
models to measure academic performance and how these models can measure
progress toward achieving key goals of the
No Child Left Behind Act of 2001 (NCLBA). With annual expenditures
approaching $13 billion dollars for Title I alone, NCLBA represents the
federal government's single largest investment in the education of the
48 million students who attend public schools. The NCLBA-the most
recent reauthorization of the Elementary and Secondary Education Act of
1965-requires states to improve academic performance so that all
students are proficient by 2014 and achievement gaps among groups such
as economically disadvantaged students close. The upcoming
reauthorization of the law presents an opportunity to discuss some key
issues associated with the act.
To measure whether schools are making adequate yearly progress
(AYP) toward having all students proficient by 2014, states set annual
proficiency targets using an approach known as a status model, which
calculates test scores 1 year at a time. With status models, states or
districts determine whether schools make AYP based on performance for
the year while generally not taking into account how much better or
worse the school did than during the previous year. Thus, a school that
is showing significant improvement in student achievement but has too
few students at the proficient level would not likely make AYP.
In addition to determining whether schools meet proficiency
targets, some states have interest in also recognizing schools that
make progress toward NCLBA goals. Growth models can measure progress in
achievement or proficiency over time and vary in complexity, such as
calculating annual progress in a school's average test scores from year
to year; estimating test score progress while taking into account how
factors such as student background may affect such progress; or
projecting future scores based on current and prior years' results.
While growth models are sometimes defined as tracking the same students
over time, because of the committee's interest in the range of models
states are using to measure academic improvement, we define a growth
model as a model that measures changes in proficiency levels or test
scores of a student, group, grade, school, or district for 2 or more
years. We included models that track schools and student groups in
order to provide a broad assessment of options that may be available to
states.
My testimony today will focus on how growth models may provide
useful information on academic performance. Specifically, I will
discuss (1) how many states are using growth models and for what
purposes, (2) how growth models can measure progress toward achieving
key NCLBA goals, and (3) what challenges states face in using growth
models especially to meet the law's key goals.
My written statement is drawn from our recent report on growth
models, which we completed for the committee.\1\ For this report, we
conducted a survey of all states to determine whether they were using
growth models. We conducted telephone interviews with state and local
education agency officials in eight states that collectively use a
variety of growth models, and conducted site visits to California,
Massachusetts, North Carolina and Tennessee. For Massachusetts and
Tennessee we analyzed student-level data from selected schools to
illustrate how their models measure progress toward key NCLBA goals. We
conducted this work in accordance with generally accepted government
auditing standards.
Summary
In summary, nearly all states were using or considering growth
models for a variety of purposes in addition to their status models as
of March 2006. Twenty-six states were using growth models, and another
22 were either considering or in the process of implementing them. Most
states that used growth models did so for schools as a whole and for
particular groups of students, and 7 also measured growth for
individual students. For example, Massachusetts measured growth for
schools and groups of students but does not track individual students'
scores, while Tennessee set different expectations for growth for each
student based on the student's previous test scores. Seventeen of the
states that used growth models had been doing so prior to passage of
the NCLBA, while 9 began after the law's passage. States used their
growth models for a variety of purposes, such as targeting resources
for students that need extra help or awarding teachers bonus money
based on their school's relative performance.
Certain growth models are capable of tracking progress toward the
goals of universal proficiency by 2014 and closing achievement gaps.
For example, Massachusetts uses its model to set targets based on the
growth that it expects from schools and their student groups. Schools
can make AYP if they reach these targets, even if they fall short of
reaching the statewide proficiency targets set with the state's status
model. Tennessee designed a model, different from the one used for
state purposes described above, that projects students' test scores and
whether they will be proficient in the future. In this model, if 79
percent of a school's students are predicted to be proficient in 3
years, the school would reach the state's 79 percent proficiency target
for the current school year (2005-2006).
States face challenges in developing and implementing growth models
that would allow them to meet NCLBA goals. Technical challenges include
creating data and assessment systems to meet the substantial data
requirements of growth models and having personnel that can analyze and
communicate growth model results. For example, states need to have
tests that are comparable from one year to the next to accurately
measure progress. Further, some models require sophisticated data
systems that have the capacity to track individual student performance
across grades and schools. Using growth models can present risks for
states if schools are designated as making AYP while still needing
assistance to progress. For example, one school in Tennessee that did
not make AYP under the status model would make AYP under the state's
proposed growth model. This school is located in a high-poverty, inner-
city neighborhood and has been receiving federal assistance targeted to
improving student performance. If the school continues to make AYP
under the growth model, its students would no longer receive federally
required services, such as tutoring or the option of transferring to a
higher performing school. On the other hand, the school's progress may
result in its making AYP in the future under the state's status model.
U.S. Department of Education (Education) initiatives may help states
address these challenges. For example, Education started a pilot
project for states to use growth models that meet the department's
specific criteria to determine AYP. Education also provided grants to
states to support their efforts to track individual test scores over
time.
By proceeding with a pilot project with clear goals and criteria
and by requiring states to compare results from their growth model with
status model results, Education is poised to gain valuable information
on whether or not growth models are overstating progress or whether
they appropriately give credit to fast-improving schools. In comments
on a draft of our recent report, Education expressed concern that the
use of a broader definition of growth models would be confusing. GAO
used this definition in order to reflect the variety of approaches
states have been taking to measure growth in academic performance.
Background
The NCLBA \2\ requires states to set challenging academic content
and achievement standards in reading or language arts and
mathematics\3\ and determine whether school districts and schools make
AYP toward meeting these standards.\4\ To make AYP, schools generally
must:
show that the percentage of students scoring at the
proficient level or higher meets the state proficiency target for the
school as a whole and for designated student groups,
test 95 percent of all students and those in designated
groups, and
meet goals for an additional academic indicator, such as
the state's graduation rate.
The purpose of Title I Part A is to improve academic achievement
for disadvantaged students. Schools receiving Title I federal funds
that do not make AYP for 2 or more years in a row must take action to
assist students, such as offering students the opportunity to transfer
to other schools or providing additional educational services like
tutoring.
States measure AYP using a status model that determines whether or
not schools and students in designated groups meet proficiency targets
on state tests 1 year at a time. States generally used data from the
2001-2002 school year to set the initial percentage of students that
needed to be proficient for a school to make AYP, known as a starting
point. From this point, they set annual proficiency targets that
increase up to 100 percent by 2014. For example, for schools in a state
with a starting point of 28 percent to achieve 100 percent by 2014, the
percentage of students who scored at or above proficient on the state
test would have to increase by 6 percentage points each year, as shown
in figure 1.\5\ Schools that do not reach the state target will
generally not make AYP.
figure 1: hypothetical example of annual proficiency targets set under
a status model
The law indicates that states are expected to close achievement
gaps, but does not specify annual targets to measure progress toward
doing so. States, thus, have flexibility in the rate at which they
close these gaps. To determine the extent that achievement gaps are
closing, states measure the difference in the percentage of students in
designated student groups and their peers that reach proficiency. For
example, an achievement gap exists if 40 percent of a school's non-
economically disadvantaged students were proficient compared with only
16 percent of economically disadvantaged students, a gap of 24
percentage points. To close the gap, the percentage of students in the
economically disadvantaged group that reaches proficiency would have to
increase at a faster rate than that of their peers.
If a school misses its status model target in a single year, the
law includes a ``safe harbor'' provision that provides a way for
schools that are showing significant increases in proficiency rates of
student groups to make AYP. Safe harbor measures academic performance
in a way that is similar to certain growth models do and allows a
school to make AYP by reducing the percentage of students in designated
student groups that were not proficient by 10 percent, so long as the
groups also show progress on another academic indicator. For example,
in a state with a status model target of 40 percent proficient, a
school could make AYP under safe harbor if 63 percent of a student
group was not proficient compared to 70 percent in the previous year.
Nearly All States Reported Using or Considering Growth Models to
Measure Academic Performance
Twenty-six states reported using growth models in addition to using
their status models to track the performance of schools, designated
student groups, or individual students, as reported in our March 2006
survey. Additionally, nearly all states are considering the use of
growth models (see fig. 2).
figure 2: states that reported using or considering growth models, as
of march 2006
Of the 26 states using growth models, 19 states reported measuring
changes for schools and student groups, while 7 states reported
measuring changes for schools, student groups, and individuals, as
shown in table 1.
table 1: types of growth models and states using them, as of march 2006
For example, Massachusetts used a model that measures growth for
the school as a whole and for designated student groups. The state
awards points to schools in 25-point increments for each student,\6\
depending on how students scored on the state test. Schools earn 100
points for each student who reaches proficiency, but fewer points for
students below proficiency. The state averages the points to award a
final score to schools. Growth in Massachusetts is calculated by taking
the difference in the annual scores that a school earns between 2
years. Figure 3 illustrates the growth a school can make from one year
to the next as measured by Massachusetts model.
figure 3: illustration of school-level growth
Tennessee reported using a growth model that sets different goals
for each individual student based on the students' previous test
scores. The goal is the score that a student would be expected to
receive, and any difference between a student's expected and actual
score is considered that student's amount of yearly growth,\7\ as shown
in figure 4.
figure 4: example of higher-than-expected growth for a fourth-grade
student under tennessee's model
In addition, Tennessee's model, known as the Tennessee Value-Added
Assessment System, estimates the unique contribution-the value added-
that the teacher and school make to each individual student's growth in
test scores over time.\8\ The state then uses that amount of growth,
the unique contribution of the school, and other information to
determine whether schools are below, at, or above their level of
expected performance. The model also grades schools with an A, B, C, D,
or F, which is considered a reflection of the extent to which the
school is meeting its requirements for student learning.
Seventeen of the 26 states using growth models reported that their
models were in place before the passage of the NCLBA during the 2001-
2002 school year, and the remaining 9 states implemented them after the
law was passed. States used them for purposes such as rewarding
effective teachers and designing intervention plans for struggling
schools. For example, North Carolina used its model as a basis to
decide whether teachers receive bonus money. Tennessee used its value-
added model to provide information about which teachers are most
effective with which student groups. In addition to predicting
students' expected scores on state tests, Tennessee's model was used to
predict scores on college admissions tests, which is helpful for
students who want to pursue higher education. In addition, California
used its model to identify schools eligible for a voluntary improvement
program.
Certain Growth Models Can Measure Progress toward Key NCLBA Goals
Certain growth models can measure progress in achieving key NCLBA
goals of reaching universal proficiency by 2014 and closing achievement
gaps. While states developed growth models for purposes other than
NCLBA, states such as Massachusetts and Tennessee have adjusted their
state models to use them to meet NCLBA goals. The Massachusetts model
has been used to make AYP determinations as part of the state's
accountability plan in place since 2003. Tennessee submitted a new
model to Education for the growth models pilot that differs from the
value-added model described earlier. This new model gives schools
credit for students projected to reach proficiency within 3 years in
order to meet key NCLBA goals. Our analysis of how models in
Massachusetts and Tennessee can measure progress toward the law's two
key goals is shown in table 2.
table 2: how a status model and certain growth models measure progress
in achieving key nclba goals*
*Note: Additional requirements for schools to make AYP are
described in the background section of our report. Massachusetts refers
to proficiency targets as performance targets and refers to growth
targets as improvement targets.
\a\ The information presented in this table reflects the model
Tennessee proposed to use as part of Education's growth model pilot
project, as opposed to the value-added model it uses for state
purposes. The information is based on the March 2006 revision of the
proposal the state initially made in February 2006.
Massachusetts designed a model that can measure progress toward the
key goals of NCLBA by setting targets for the improvement of schools
and their student groups that increase over time until all students are
proficient in 2014. Schools can get credit for improving student
proficiency even if, in the short term, the requisite number of
students has yet to reach the state's status model proficiency targets.
For example, figure 5 illustrates a school that is on track to make AYP
annually through 2014 by reaching its growth targets. While these
growth targets increase at a faster pace than the state's annual
proficiency target until 2014, they do provide the school with an
additional measure by which it can make AYP.
figure 5: targets for a selected school in massachusetts compared to
state status model targets
The model also measures whether achievement gaps are closing by
setting targets for designated student groups, similar to how it sets
targets for schools as a whole. Schools that increase proficiency too
slowly-that is, do not meet status or growth targets-will not make AYP.
For example, one selected school in Massachusetts showed significant
gains for several designated student groups that were measured against
their own targets. However, the school did not make AYP because gains
for one student group were not sufficient. This group-students with
disabilities-fell short of its growth target, as shown in figure 6.
figure 6: results for a selected school in massachusetts in mathematics
Tennessee developed a different model that can also measure
progress toward the NCLBA goals of universal proficiency and closing
achievement gaps. Tennessee created a new version of the model it had
been using for state purposes to better align with NCLBA.\9\ Referred
to as a projection model, this approach projects individual student's
test scores into the future to determine when they may reach the
state's status model proficiency targets.
In order to make AYP under this proposal, a school could reach the
state's status model targets by counting as proficient in the current
year those students who are predicted to be proficient in the future.
The state projects scores for elementary and middle school students 3
years into the future to determine if they are on track to reach
proficiency, as follows:
fourth-grade students projected to reach proficiency by
seventh grade,
fifth-grade students projected to reach proficiency by
eighth grade, and
sixth-, seventh-, and eighth-grade students projected to
reach proficiency on the state's high school proficiency test.
These projections are based on prior test data and assume that the
student will attend middle or high schools with average performance (an
assumption known as average schooling experience).\10\ At our request,
Tennessee provided analyses for students in several schools that would
make AYP under the proposed model. To demonstrate how the model works,
we selected students from a school and compared their actual results in
fourth grade (panel A) with their projected results for seventh grade
(panel B) (see fig. 7).
figure 7: results for selected students in mathematics from a school in
tennessee*
*Note: The same students are presented in both panels (for example,
student A in panel A is the same student as student A in panel B).
While these data reflect the scores of individual students, Tennessee
provided data to GAO in such a way that student privacy and
confidentiality were ensured. Data are illustrative and are not meant
to be a statistical representation of the distribution of students in
this school.
Tennessee's proposed model can also measure achievement gaps. Under
NCLBA, a school makes AYP if all student groups meet the state
proficiency target. In Tennessee's model, whether the achievement gap
is potentially closed would be determined through projections of
students' performance in meeting the state proficiency target.
States Face Challenges in Implementing Growth Models
States generally face challenges in collecting and analyzing the
data required to implement growth models including models that would
meet the law's goals. In addition, using growth models can present
risks for states if schools are designated as making AYP while still
needing assistance to progress. Education has initiatives that may help
states address these challenges.
States must have certain additional data system requirements to
implement growth models, including models that would meet NCLBA
requirements.
First, a state's ability to collect comparable data over at least 2
years is a minimum requirement for any growth model. States must ensure
that test results are comparable from one year to the next and possibly
from one grade to the next, both of which are especially challenging
when test questions and formats change. Second, the capacity to collect
data across time and schools is also required to implement growth
models that use student-level data. This capacity often requires a
statewide system to assign unique numbers to identify individual
students. Developing and implementing these systems is a complicated
process that includes assigning numbers, setting up the system in all
schools and districts, and correctly matching individual student data
over time, among other steps. Third, states need to ensure that data
are free from errors in their calculations of performance. While
ensuring data accuracy is important for status models, doing so is
particularly important for growth models, because errors in multiple
years can accumulate, leading to unreliable results.
States also need greater research and analysis expertise to use
growth models as well as support for people who need to manage and
communicate the model's results. For example, Tennessee officials told
us that they have contracted with a software company for several years
because of the complexity of the model and its underlying data system.
Florida has a contract with a local university to assist it with
assessing data accuracy, including unique student identifiers required
for its model. In addition, states will incur training costs as they
inform teachers, administrators, media, legislators, and the general
public about the additional complexities that occur when using growth
models. For example, administrators in one district in North Carolina
told us that their district lacks enough specialists who can explain
the state's growth model to all principals and teachers in need of
guidance and additional training.
Using growth models can present risks for states if schools are
designated as making AYP while still needing assistance to progress. On
the basis of growth model results, some schools would make AYP even
though these schools may have relatively low-achieving students. As a
result, some students in Title I schools may be disadvantaged by not
receiving federally-required services.
In two Massachusetts districts that we analyzed, 23 of the 59
schools that made AYP did so based on the state's growth model, even
though they did not reach the state's status model proficiency rate
targets in 2003-2004.\11\ Consequently, these schools may not be
eligible to receive services required under NCLBA for schools in need
of improvement, such as tutoring and school choice. Because these
schools would need to sustain high growth rates in order to achieve
universal proficiency by 2014, it is likely that their students would
benefit from additional support.
In Tennessee, 47 of the 353 schools that had not made AYP in the
2004-2005 school year would do so under the state's proposed projection
model. One school that would be allowed to make AYP under the proposed
model was located in a high-poverty, inner-city neighborhood. That
school receives Title I funding, as two-thirds of its students are
classified as economically disadvantaged. The school was already
receiving services required under NCLBA to help its students. If the
school continues to make AYP under the growth model, these services may
no longer be provided.
Education's initiatives, such as the growth model pilot project,
may facilitate growth model implementation. In November 2005, Education
announced a pilot project for states to submit proposals for using a
growth model-one that meets criteria established by the department-
along with their status model, to determine AYP. While NCLBA does not
specify the use of growth models for making AYP determinations, the
department started the pilot to evaluate how growth models might help
schools meet NCLBA proficiency goals and close achievement gaps.
For the growth model pilot project, each state had to demonstrate
how its growth model proposal met Education's criteria, many of which
are consistent with the legal requirements of status models. In
addition to those requirements, Education included criteria that the
proposed models track student progress over time and have an assessment
system with tests that are comparable over time. Of the 20 proposals,
Education approved 2 states-North Carolina and Tennessee-to use growth
models to make AYP determinations in the 2005-2006 school year. States
may submit proposals for the pilot again this fall.
In addition to meeting all of the criteria, Education and peer
reviewers noted that Tennessee and North Carolina had many years of
experience with data systems that support growth models. These states
must report to Education the number of schools that made AYP on the
basis of their status and growth models. Education expects to share the
results with other states, Congress, and the public after it assesses
the effects of the pilot.
In addition to the growth model pilot project, Education awarded
nearly $53 million in grants to 14 states for the design and
implementation of statewide longitudinal data systems-systems that are
essential for the development of student-level growth models. While
independent of the pilot project, states with a longitudinal data
system-one that gathers data such as test scores on the same student
from year to year-will be better positioned to implement a growth model
than they would have been without it. Education intended the grants to
help states generate and use accurate and timely data to meet reporting
requirements, support decision making, and aid education research,
among other purposes. Education plans to disseminate lessons learned
and solutions developed by states that received grants.
Conclusion
While status models provide a snapshot of academic performance,
growth models can provide states with more detailed information on how
schools' and students' performance has changed from year to year.
Growth models can recognize schools whose students are making
significant gains on state tests but are still not proficient.
Educators can use information about the academic growth of individual
students to tailor interventions to the needs of particular students or
groups. In this respect, models that measure individual students'
growth provide the most in-depth and useful information, yet the
majority of the models currently in use are not designed to do this.
Through its approval of Massachusetts' model and the growth model
pilot program, Education is proceeding prudently in its effort to allow
states to use growth models to meet NCLBA requirements. Education is
allowing only states with the most advanced models that can measure
progress toward NCLBA goals to use the models to determine AYP. Under
the pilot project, which has clear goals and criteria that requires
states to compare results from their growth model with status model
results, Education is poised to gain valuable information on whether or
not growth models are overstating progress or whether they
appropriately give credit to fast-improving schools.
While growth models may be defined as tracking the same students
over time, GAO used a definition that also includes tracking the
performance of schools and groups of students. In comments on our
report, Education expressed concern that this definition may confuse
readers because it is very broad and includes models that compare
changes in scores or proficiency levels of schools or groups of
students. GAO used this definition of growth to reflect the variety of
approaches states are taking to measure academic progress.
Mr. Chairman, this concludes my prepared statement. I would be
happy to respond to any questions that you or members of the committee
may have.
endnotes
---------------------------------------------------------------------------
\1\ GAO, No Child Left Behind Act: States Face Challenges Measuring
Academic Growth That Education's Initiatives May Help Address, GAO-06-
661 (Washington, D.C.: July 17, 2006).
\2\ Pub. L. No. 107-110 (Jan. 8, 2002).
\3\ The law also requires content standards to be developed for
science beginning in the 2005-2006 school year and science tests to be
implemented in the 2007-2008 school year.
\4\ States determine whether schools and school districts make AYP
or not. For this report, we will discuss AYP determinations in the
context of schools.
\5\ States were able to map out different paths to universal
proficiency subject to certain limitations. For example, states must
increase the targets at least once every 3 years and those increases
must lead to 100 percent proficiency by 2014. See GAO, No Child Left
Behind Act: Improvements Needed in Education's Process for Tracking
States' Implementation of Key Provisions, GAO-04-734, (Washington,
D.C.: Sept. 30, 2004).
\6\ Students with disabilities are generally included in these
calculations. The state is allowed to give different tests to students
with significant cognitive impairments and to count them differently
for calculating points awarded to schools.
\7\ Tennessee's growth model mentioned here is not used to make AYP
determinations under NCLBA. However, Tennessee developed a different
growth model to determine AYP for Education's growth model pilot
project. That model is discussed later in this testimony.
\8\ The state calculates the unique contribution of schools and
teachers by using a multivariate, longitudinal statistical method where
results are estimated using data specific for students within each
classroom or school.
\9\ Tennessee continues to use its original model to rate schools
based in part on the unique contributions-or the value added-of school
to student achievement.
\10\ While Tennessee's model estimates future performance, other
models are able to measure growth without these projections. For
example, Florida uses a model that calculates results for individual
students by comparing performance in the current year with performance
in prior years.
\11\ Another 11 schools also met the growth target, but these 11
schools made AYP under NCLBA's safe harbor provision.
---------------------------------------------------------------------------
______
Chairman McKeon. Thank you very much.
Chancellor Klein?
STATEMENT OF JOEL I. KLEIN, CHANCELLOR, NEW YORK CITY
DEPARTMENT OF EDUCATION
Mr. Klein. Thank you, Mr. Chairman, Mr. Miller, members of
the committee. I appreciate the opportunity to be before you
today.
I want to acknowledge my friend, Major Owens, who has
served this country with distinction for the past 24 years, who
will be leaving this Congress at the end of the term. And it is
a loss to our city, and you will be missed, sir. I just want
you to know that.
You have my testimony, so let me highlight a few of the
points that I think are critical, because I agree with the
committee. I think this is a key issue on No Child Left Behind.
Mr. Miller, I think you are right. I think we have many
problems in K-12 education, none greater than the achievement
gap between black and Latino kids on the one hand and white and
Asian kids on the other. It is the shame of this nation. I used
to think it was merely a moral crisis. Now it is an economic
crisis as well.
And I think NCLB, which is an act that needs to be
improved, nevertheless put in place an accountability system
that forced our nation to come to grips, after decades of
reform in public education that had not moved the needle--to
come to grips with this fundamental issue of a massive
achievement gap.
And to me, whatever else happens, this Congress and this
nation should not throw out the baby with the bath water when
it comes to the fundamental framework and the accountability
that is built in NCLB.
People like you should be very tough on people like me.
Excuses for why our kids are not succeeding in public education
are not going to help us solve the problem.
That said, my view is an accountability system need not
only be tough, it needs to be fundamentally credible. And the
current NCLB system I don't think is ultimately long-term
credible and sustainable.
We are almost a decade out from 2014 so its impact is not
fully clear yet, but let me tell you what I think the behaviors
it is leading to in the trenches where people like I am working
with those 1.1 million kids.
By focusing on proficiency, what we do is we create
perverse incentives in terms of those kids that the school
system focuses on. The kids who are in my system--level three
is proficient. Kids who are close to the line get the most
attention, because you are going to get the credit for bumping
them over.
You take a kid from level 2.1 to 2.9--that is a huge gain.
You don't get any credit for it. You take a kid from 2.9 to
three--that is a big credit event.
People out in the field get that, because AYP and city
status matters to them. If you take a kid from level three to
level four, which is where we want our kids, future leaders of
this country--we want our highest performing kids to be moving
forward, too.
Level three to level four--you get no credit for it in the
system. We group them together. It is a mistake. It is a major
mistake. Every kid has got to be moving forward.
The derivative effect of that is we often engage in what I
call zip code educational policy, because in my school system
we have some communities where kids are coming to school ahead
of the game and are performing at level three and four before
they got to school. And yet that school is perceived to be a
good school whether it is moving those kids forward or not.
We have got other kids--schools where there are high
concentrations of English language learners, special ed kids
and so forth. Those kids may have a longer road to travel, but
they may be moving along that road and still not getting the
kind of recognition they deserve.
It is incredibly important to close this gap, but you can't
leap across this gap in one fell swoop. And schools that are
bringing over those years a sustained growth deserve credit at
every end of the spectrum.
I don't want us to leave those level threes at level three
in New York, and I want those schools to get credit. That is
precisely what we are doing in our city. We are putting in
place--building on NCLB an accountability system that will give
real credit for schools that are moving forward on growth.
We are going to do this in a way that looks at exactly the
kind of thing that Dr. Shaul said. Year to year, how did this
year's 3rd-graders do in the 4th grade, do in the 5th grade, do
in the 6th grade?
Now, we are going to take into account where schools start
and how they move forward. And we are not going to allow the
kind of what I think is perverse responsive behavior of using
gimmicks to try to move the number of kids to level three. We
are going to look at every single kid in the system.
We are going to give extra credit to those people who are
moving the most challenging, the hardest-to-serve kids in the
system. But we are going to look at every single kid.
And we are going to provide the data and the information to
the schools. An accountability system that is a gotcha game is
not going to be sufficient for what we need. The information
has got to be used by the schools.
Schools that use assessment take the data and use that to
improve their teaching practices and to make sure that we build
the solid foundation for kids before they move forward.
So we are creating a massive data bank and investing
enormous amounts of resources in professional development, so
our principals, assistant principals and teachers can use the
data to drive accountability.
At some point, an accountability system has to be a
spotlight, not a hammer. We can do that.
[The prepared statement of Mr. Klein follows:]
Prepared Statement of Chancellor Joel Klein, New York City Department
of Education
Good morning. Thank you Chairman McKeon and Congressman Miller for
inviting me to testify today. I would also like to thank Congressman
Owens for his many years of service to New York City. You will be
missed.
This morning, I will discuss the promise of growth models from the
vantage point of New York City, where we are implementing a hybrid
system that tracks both growth and proficiency levels. But first, I'd
like to step back and put my comments into context.
For decades, school reformers have tried and failed to fix
education in America, to ensure that American students do not fall
behind their international peers and to ensure that all students in
this country, no matter what their race or socio-economic status, are
receiving the high-quality educations they deserve. The law that we're
discussing today, No Child Left Behind, might not be perfect, but it is
incredibly valuable because it recognizes that the achievement gap-the
gap that separates our African-American and Latino students from their
white and Asian peers-is the chief, though certainly not the only,
problem in American schooling. When Congress passed NCLB, it helped
America finally take responsibility for the fact that white and Asian
students are performing four years ahead of African-American and Latino
students in high school. Four years. And this law finally puts muscle
behind the attempt to close that gap. It forces us to report student
performance in grades three through eight by race. We can no longer
mask the deficiencies of some students with outsized gains by others.
Now, NCLB has problems, and I will talk about those today. But at
the same time, it is critical that we all remember that NCLB is not
just important. It's fundamental. I've never met a law that couldn't be
improved. But to criticize the heart of No Child Left Behind is to
refuse to take responsibility for the achievement gap-the most serious
civil rights, social, and economic crisis facing America today. We
should, of course, learn from our experiences and make a good thing
better, but we should not consider diluting or destroying a law that
forces us to confront our problems head on. We must not yield to the
critics of NCLB because, I believe, their complaints are missing the
law's broader significance.
Now, to the topic of the day. For a long time, we have heard that
we face a choice between absolute standards, like those in NCLB, and
value-added approaches, like the ones we are discussing today. I don't
think we have to choose. We need to keep our eye on the ultimate goal,
which is ensuring that every child is at least proficient in reading,
math, and the other core academic areas; if a school is not helping all
children achieve high standards, the school still has work to do. But
we also need to recognize that schools sometimes succeed in helping
kids make real, substantial progress without boosting them all the way
to proficiency. We all know it is wrong to call a school an unqualified
success when children are not meeting standards in reading and math,
but it is also wrong to call a school an unqualified failure when it is
helping its students grow academically. Both absolute achievement and
growth should count when we are judging schools.
While it's important to strive for 100% proficiency, focusing on
proficiency alone and ignoring gains can lead to serious negative
consequences.
First, the current the law, which shortchanges growth, motivates
educators to help certain types of students and not others. The way the
law is designed, we get credit if we move a child across the threshold
of ``proficiency.'' So, if a school wants to succeed under No Child
Left Behind, it makes sense to dedicate more time and energy helping
students who are just on the cusp of meeting standards than to students
who are far away from proficiency or to students who are already safely
above the threshold. Gains achieved by students who make progress but
fail to reach proficiency are not rewarded. If children are so far
behind that they have little prospect of achieving proficiency, it may
make more sense under NCLB to encourage them to drop out than to help
them achieve at the highest levels possible. Similarly, it's not worth
giving the brightest kids-the students who could become the future
leaders of this nation-the extra support and attention they need to
achieve their potential because boosting children further above
proficiency brings no added benefit under the law.
Second, the law can give a misleading picture of how well a school
is doing. We have schools in New York filled with kids who score above
the proficiency threshold even though the schools are doing very little
to help students progress beyond where they started on the first day of
school. These schools pass under NCLB. At the other extreme, we have
other schools that make real contributions to student learning and
consistently help students move in the right direction, but don't get
them all the way to proficiency. These schools do not get credit under
NCLB.
I know some people worry that growth models will give weak
educators an excuse to give weak schools passing grades just because
they are showing marginal gains. I say it depends on the growth model.
It's true that a growth model could be designed as a smokescreen, but
it could also be crafted to accurately gauge student learning. The
latter is what we are creating in New York.
Mayor Bloomberg and I have been very tough on accountability. We
eliminated social promotion in our elementary and middle schools. We
shut down failing high schools. We created ``Empowerment Schools,''
schools whose principals receive greater autonomy in exchange for
entering specific performance contracts agreeing to be held accountable
for results. More than 300 principals volunteered, knowing that they
could lose their jobs if they were unable to raise student achievement.
The sophisticated growth model we are crafting is another way we
hold our schools accountable for providing New York City schoolchildren
with the educations they need and deserve. Our new measure of progress
will be more precise than what is required under No Child Left Behind.
We will measure all year-to-year gains, even those that don't boost
students to proficiency, and all losses. I think this will be a vast
improvement that will start placing incentives where they belong: on
educating all students. Here's how it will work: According to NCLB
Student A's progress from a 2.0 to a 2.9 on New York's four-point scale
counts for nothing and Student B's drop from a 2.8 to a 2.2 does not
count against his school. Student C's progress by a tenth of a point,
however, from a 2.9 to the point of proficiency, a 3.0, matters. If
progress is our goal, NCLB's incentives system is out of whack. New
York City's new accountability system will provide more appropriate
incentives. We want teachers to focus on all students, not just those
who are most likely to jump across an arbitrary threshold.
Our new value-added information will feed into annual school
progress reports, which will be completely transparent and publicly
available for all of our 1,400 schools. These reports will measure
schools on three quantitative factors: School Environment, Performance,
and Progress. School Environment is a combination of attendance and
safety data, as well as student, teacher, and parent survey results.
Performance is what NCLB now relies on-snapshots of student performance
in a given year. Progress is our ``value added'' measurement. It
measures aggregated individual student performance over time. Based on
these three factors, with Progress counting most, our schools will
receive grades of A, B, C, D or F.
These grades will tell parents how well their students' schools are
performing-and whether they are making progress. Grades will also help
administrators take responsibility for the schools, ensuring that
children don't remain in failing schools. We will intervene to make
improvements, and if we need to, we will shut down failing schools or
change school leadership. Under our growth model, we expect that about
15% of schools will receive Ds or Fs.
We need to use data not only to evaluate how schools are
performing, but also to enable principals and teachers to identify
their strengths and weaknesses, to share the practices that work and
improve the ones that don't. So, starting next year, we will also
implement a comprehensive achievement reporting system, which will
include all the information from the state standardized exams as well
as information from periodic in-class assessments, which are no-stakes
diagnostics that teachers will use to check in on their students. Using
these systems, teachers will be able to give mini-assessments to learn
whether students have grasped what they've been taught. If they have,
teachers will move to the next subject. If they haven't-or if certain
students are still struggling-teachers can intervene. These assessments
will help educators measure progress over the course of the year and
make mid-course corrections for students or classes with difficulties.
They will no longer have to wait for end-of-year standardized exams to
learn if students needed extra help.
Some of our schools are already using data to drive performance,
but too many of our educators are estimating and guessing, even though
we're working in an age of technology, an age when educators don't have
to guess what the problem is and experiment until they find the right
solution. In its current form, NCLB does not motivate educators to help
all children achieve at the highest levels possible. I believe New York
City's new system's will help to make this possible.
Thank you. I welcome your questions.
______
Chairman McKeon. Thank you very much.
Dr. Weaver?
STATEMENT OF REG WEAVER, PRESIDENT, NATIONAL EDUCATION
ASSOCIATION
Mr. Weaver. Thank you very much. Thank you for the
opportunity to join you this morning to share the views of 2.8
million members of the National Education Association.
The NEA is the largest professional association in the
country, and our membership is diverse, but we have a common
mission and values based on our belief that a great public
school is a basic right for every child.
NEA and our members have long supported the Elementary and
Secondary Education Act. And despite what some say, we have
always supported the laudable goals of No Child Left Behind,
closing the achievement gaps, raising overall student
achievement and ensuring all students have a qualified teacher.
NEA did not oppose annual testing, nor did we oppose the
passage of No Child Left Behind. We did, however, make a clear
statement as to our objection to accountability systems based
solely upon test scores.
And I hope that this committee understands that expressions
of concern about the impact of this law are not a rejection of
the goals of No Child Left Behind, nor do we want to do away
with the law.
In fact, I just returned from our representative assembly
where almost 9,000 delegates voted on NEA's priorities for
elementary and secondary act reauthorization. And they didn't
vote to repeal nor do away with No Child Left Behind.
Instead, they voted on a comprehensive set of proposals
designed to fix what is not working with the law and add to it
the kind of initiatives that will make our goals a common
reality.
And our report spells out seven key components of a great
public school: Quality programs and services; high expectations
and standards; quality conditions for teaching and lifelong
learning; qualified, caring, diverse and stable staff; shared
responsibility for appropriate school accountability by
stakeholders at all levels; parental, family and community
involvement and engagement; and adequate, equitable and
sustainable funding.
Growth models--well, we applaud Secretary Spellings'
decision to pilot a growth model project, and we thank her for
listening to educators and being willing to explore new
options.
We believe that accountability systems should reward
success and support educators to help students learn. And
measuring student growth over time will be more helpful than
the current snapshot approach which measures student
achievement on 1 day at a time throughout the year.
I taught middle school science. And as a veteran classroom
teacher, I would welcome the opportunity to use my students'
test results to guide my instructional practice.
But I seriously question the logic of any system that
mandates tests but does not also mandate that the results of
those tests be given in time to make any adjustments in
instruction.
And if someone had told me that my class the next year
would be tested in the spring, and their scores would be
compared to my students' scores from last year, I would have
said that there is something inherently wrong with that system.
An accountability system designed to measure performance
cannot compare apples and oranges. And as a science teacher, I
know that such a system will not yield any meaningful data.
The implementation phase of No Child Left Behind has
highlighted a critical void in assessing student progress,
measuring student progress over time, and providing the
resources and tools that educators need to get the job done.
In our opinion, it does not recognize that children are
human beings and not cardboard cutouts and that teaching them
is both an art and a science.
Including growth models in No Child Left Behind's
accountability system would not mean abandoning requirements
that all students read and complete math problems on grade
level. Quite the opposite is true. Growth models hold greater
promise to demonstrate whether a student is learning and
provides data to educators in order to inform their
instruction.
Better data will show when instructional techniques are not
working and will allow teachers to make adjustments to meet
student needs.
Growth models also reward success in teaching and learning
by giving schools credit both for moving a child from below
basic to basic, as well as moving a child from proficient to
advanced.
I would like to offer one caution, however. Including a
growth model as part of adequate yearly progress--it is not a
panacea. Complexities will continue to arise for some English
language learners and certain students with disabilities who
take alternate assessments.
And we will also continue to need much more research about
growth models as well as technical assistance to states and
local districts and educators, but the ultimate goal should be
to help classroom educators use the data to inform instruction.
And I have made closing the achievement gap one of our
organization's highest priorities. And it is not only something
that I care deeply about, but I believe it is the right thing
to do.
My written testimony highlights the many ways in which the
NEA has marshaled our resources to assist our state and local
affiliates to help close the achievement gaps. In short, we
have devoted millions of dollars and thousands of hours of
staff time to that effort.
And I would like to conclude by sharing a story from one of
our members. It is a story about a boy named Cesar and his ESL
teacher, Mary Beth Solano in Fort Collins, Colorado.
And she writes: ``Cesar, a 3rd-grade student, came to me in
August with not a word of English. And together with his
classroom teacher, significant work from me and a fantastic
group of peers, he learned English amazingly well.
``And I was almost going to exempt him from the test, but
two things stopped me: One, part of No Child Left Behind that
says that any child who doesn't take the test counts as a zero
on our school's report card, and Cesar himself, who set as his
personal goal knowing enough English to pass Colorado State's
mandated No Child Left Behind test.
``And reluctantly but with Cesar's terrific desire to
exceed coaxing us on, we had him take the test. And he
struggled and he struggled, reading every word, and read it
over and over again until he understood and thought he had
understood well enough to answer the question.
``And he worked so hard and so long on the test, it brought
tears to my eyes. And he was so proud of himself after he
completed it. You should have seen his face.
``Well, the scores came out. While we celebrated, his
family was devastated. Cesar earned a score that was only two
tiny points below the cutoff for partially proficient, but he
was labeled an unsatisfactory learner. Nothing could be farther
from the truth, for he had gone from basically zero to almost
proficient in just 6 months.
``And I tried to explain that to his parents and tried even
harder to lift Cesar's spirits. But I am not sure how much
success that he had.
``So without changes in the structure and the process of
reporting scores, stories like Cesar's will continue to deflate
and demoralize the best and the brightest.''
So I encourage every one of you, every one of the members
of the committee, to talk to your local educators about their
experiences and ask them about their frustrations. But more
importantly, ask them about their successes.
And they all have a Cesar story, and they all want every
one of their students to succeed. And you know what? They go
every day above and beyond the call of duty, time and time
again, to make that goal a realty for America's children.
Thank you.
[The prepared statement of Mr. Weaver follows:]
Prepared Statement of Reg Weaver, President, National Education
Association
Good morning Chairman McKeon, Congressman Miller, and Members of
the Committee.
Thank you for the opportunity to join you this morning and share
the views of the 2.8 million members of the National Education
Association (NEA).
NEA is the largest professional association in the country,
representing public school educators-teachers and education support
professionals, higher education faculty, educators teaching in
Department of Defense schools, students in colleges of teacher
education, and retired educators across the country. While our
membership is diverse, we have a common mission and values based on our
belief that a great public school is a basic right for every child.
Our members go into education for two reasons-because they love
children and they appreciate the importance of education in our
society. We want all students to succeed. Our members show up at school
every day to nurture children, to bring out their full potential, to be
anchors in children's lives, and to help prepare them for the 21st
century world that awaits them. It is their passion and dedication that
informs and guides NEA's work as we advocate for sound public policy
that will help our members achieve their goals.
I. NEA Principles for Great Public Schools
Today's hearing focuses primarily on the use of growth models in
the No Child Left Behind (NCLB) accountability framework. Before I
address that specific point, however, I would like to take a moment to
make a few broader points about NEA's principles and goals for ensuring
great public schools.
NEA and its members have long supported the Elementary and
Secondary Education Act (ESEA). During the last reauthorization of
ESEA, we supported the laudable goals of No Child Left Behind-closing
achievement gaps, raising overall student achievement, and ensuring all
students have a qualified teacher. We also supported a number of
specific elements in the new law, including the targeting of Title I
funds to the neediest schools and students; disaggregation of test data
by subgroup; and programs for dropout prevention, after-school learning
opportunities, and math and science education. We continue to support
all of these elements.
NEA did not at any time oppose annual testing nor did we oppose
passage of the No Child Left Behind Act. We did, however, make clear to
Congress our objection to accountability systems based solely upon test
scores. We also made clear that any tests used in an accountability
system had to be valid and reliable, aligned with the curriculum, and
designed to inform instruction, and that the system had to allow for
accurate and fair measurement of test results.
During the debate on No Child Left Behind, we suggested two ways to
craft a more workable, accurate, and fair accountability system. First,
we suggested the use of multiple measures of student achievement and
school quality to determine school effectiveness. Second, we suggested
creating accountability systems that not only required certain
proficiency levels, but that also measured growth in student
achievement over time. We supported, and continue to support, these
polices because, while we deeply believe that all children can learn,
we know that not all children learn at the same rate or in the same
way.
Let me be perfectly clear that our criticisms during initial debate
and our continuing expressions of concern over implementation of the
law are not rejections of the goals of No Child Left Behind. Nor do
they reflect a desire to do away with the law.
In fact, I have made closing the achievement gaps one of NEA's
highest goals. It is not only something about which I care personally;
it is the right thing to do. As someone who taught for 30 years, I know
that change doesn't happen overnight. But, I also know that if we are
to achieve the change we seek, we cannot ignore the experiences of
those working in our classrooms every day. Rather, we must translate
the lessons we learn from our nation's educators into sound, workable
policies that will help us meet our goals.
I just returned from NEA's annual meeting where almost 9,000
delegates voted on NEA's priorities for ESEA reauthorization. They
didn't vote to repeal or do away with NCLB. Instead, they voted on a
comprehensive set of proposals designed to fix what's wrong with the
law and add to it the kinds of initiatives that will make our common
goals a reality. A copy of that report is attached as Appendix I. I
hope it will help guide the committee as you approach reauthorization.
Our report spells out what we believe to be the seven key
components of a great public school:
Quality programs and services that meet the full range of all
children's needs so that they come to school every day ready and able
to learn;
High expectations and standards with a rigorous and comprehensive
curriculum for all students;
Quality conditions for teaching and lifelong learning;
A qualified, caring, diverse, and stable workforce;
Shared responsibility for appropriate school accountability by
stakeholders at all levels;
Parental, family, and community involvement and engagement; and
Adequate, equitable, and sustainable funding.
The priorities detailed in the report are very consistent with the
views of our general membership. In fact, our recent member poll found:
57 percent of our members want major changes to No Child Left
Behind, 21 percent want minor changes, 17 percent favor repeal, and
just 4 percent want to keep the law as it is;
95 percent of our members want the Association to be active in
working to change the law, while only 4 percent disagree;
85 percent of our members believe there is too much reliance on
standardized testing.
Our members rated highest the following proposals to improve NCLB:
Use multiple measures instead of just tests (71% strongly favor);
Measure student achievement over time instead of just the day of
the test (70% strongly favor);
Ensure that employee rights under contracts and laws are respected
(65% strongly favor);
Fully fund mandates (64% strongly favor);
Restore the class size reduction program (63% strongly favor).
II. Growth Models and Effective Accountability Systems
I would now like to turn to the focus of today's hearing, the use
of growth models in measuring student progress and school
effectiveness. As I have stated, we believe that accountability systems
should be based upon multiple measures, including local assessments,
teacher-designed classroom assessments collected over time, portfolios
and other measures of student learning, graduation/dropout rates, in-
grade retention, percentage of students taking honors/advanced classes
and Advanced Placement exams, and college enrollment rates. We strongly
believe that the current one-size-fits-all system is unacceptable and
that states need the flexibility to design systems that produce
results, including deciding in which grades to administer annual
statewide tests.
Accountability systems should reward success and support educators
to help students learn. To this end, any improved accountability system
should allow for use of growth models and other measures that assess
student learning over time and recognize improvement on all points of
the achievement scale. These measures should then be used as a guide to
revise instructional practices and curricula, provide individual
assistance to students, and tailor appropriate professional development
for teachers and other educators. They should not be used to penalize
teachers or schools.
We applaud Secretary Spellings' decision to pilot a growth model
project. Her decision signaled that she has heard what our members have
had to say, and we thank her for that. We also applaud her decision to
allow states to propose their own growth models for peer review, rather
than prescribing a certain type of model. This flexibility was
particularly welcome given that all states were testing prior to
enactment of NCLB and 15 states were already testing annually in grades
three though eight. We have recently completed a policy brief on the
growth model pilot program and the process used by the Department of
Education to approve proposals by two states (NC and TN). This policy
brief is attached as Appendix II to my statement.
Our members believe that measuring student growth over time will be
more helpful than the current snapshot approach, which measures student
achievement on one day out of the year. A growth model approach will
allow for a more accurate reflection of student learning and will help
inform instruction.
I taught middle school science for 30 years. If someone had told me
that my students would be given a state standardized test in the spring
and that I would not receive the results of those assessments in time
to make any instructional adjustments, I would have seriously
questioned the logic of the central testing office. If someone had then
told me that my class the next year would be tested in the spring and
that their scores would be compared to my students from last year, I
would have said there was something inherently wrong with the system.
An accountability system designed to measure performance cannot
compare apples and oranges. As a science teacher, I know that such a
system simply will not yield any meaningful data. The children I teach
in any given year will have completely different educational needs than
the children I teach in the following year. NCLB fails to recognize
that children learn in different ways and at different rates. It fails
to recognize that children are human beings, not widgets in a factory,
and that teaching them is both an art AND a science.
One of our members from Rockford, IL has noted the illogical
consequences of the current system:
``Jackson Elementary School teachers worked tirelessly in the first
year of corrective action to bring up scores to the level set by NCLB.
The students made incredible gains, unfortunately they missed AYP by
less than one percent. This translates to one or two students that made
gains, but not enough to bring them to the prescribed level. Therefore,
they are in their second year of corrective action and labeled as a
failing school.''
The current system simply fails to provide useful, timely data for
diagnosing learning problems and facilitating instructional changes.
Rather, students who are tested in one grade move on to the next grade,
and their new teacher receives their test results-results that have
virtually no relevance to the choices that new teacher will make in
instructional strategies.
Not only is the current underlying system flawed, but
implementation is also troubling. NCLB requires assessments to be built
upon states' content standards, which in turn are to be aligned with
statewide assessments. Yet, four and a half years into the law, only
ten states have received full approval from the Department of Education
for their content standards and assessment systems. To educators, this
translates as a lack of interest in what is tested and whether the test
content has actually been taught in the classroom. It appears that the
goal is simply to administer tests and assign accountability labels.
This is demoralizing to educators and contradictory to sound
educational practice.
NEA is not alone in supporting an improved accountability system
that allows for use of more accurate measures. We have led an effort to
develop consensus on a broad set of principles for ESEA
reauthorization. To date, 87 organizations have endorsed these
principles, one of which calls for use of growth models as part of an
accountability framework (See Appendix III for the complete Joint
Organizational Statement on NCLB with the list of signatories). In
addition, nine bills that would allow use of growth models are
currently pending in Congress. Some of those bills were introduced by
members of this committee, including Representatives McCollum, Wu, and
Andrews. Several other committee members, including Representatives
Grijalva, Ryan, and Woolsey, have cosponsored bills that would allow
states to utilize growth models.
Governors and state legislators have also called on a bipartisan
basis for more flexibility to use growth models. The National Governors
Association's (NGA) proposals for the ESEA reauthorization, issued in
March 2006, state that, ``Maximum flexibility in designing state
accountability systems, including testing, is critical to preserve the
amalgamation of federal funding, local control of education, and state
responsibility for system-wide reform.''
Similarly, the National Conference of State Legislatures issued a
report in February 2005 calling on Congress to make substantial changes
to the law. The report states:
``Administrators at the state, local and school levels are
overwhelmed by AYP because it holds schools to overly prescriptive
expectations, does not acknowledge differences in individual
performance, does not recognize significant academic progress because
it relies on absolute achievement targets, and inappropriately
increases the likelihood of failure for diverse schools.''
By allowing inclusion of growth models in NCLB's accountability
system, Congress would not have to abandon the requirement that all
students read and complete math problems on grade level. Quite the
opposite is true. We believe that growth models hold greater promise to
demonstrate whether a student is learning. They would provide a more
accurate measure by giving schools credit both for moving a child from
below basic to basic as well as moving a child from proficient to
advanced. They would also offer a way to recognize highly effective
schools that have an influx of students who are not performing at grade
level.
Growth models will also help overcome the all-or-nothing approach
of measuring Adequate Yearly Progress (AYP). Currently, a typical
school has to meet 37 criteria to make AYP. A school that falls short
on just one of the 37 is treated in the same manner as a school that
fails all 37 criteria. (See tables below) Growth models that offer more
common sense ways to measure student achievement, in particular for
students with disabilities and English Language Learners (ELL), will
ameliorate this problem.
CURRENT AYP FAILURE SCHOOL: MISS 1 OF 37
----------------------------------------------------------------------------------------------------------------
Other
Reading 95% Math 95% indicator
----------------------------------------------------------------------------------------------------------------
All Students...................... Yes Yes Yes Yes Yes
Black/African American............ Yes Yes Yes Yes
Hispanic.......................... Yes Yes Yes Yes
Native American................... Yes Yes Yes Yes
Asian............................. Yes Yes Yes Yes
White............................. Yes Yes Yes Yes
ELL............................... Yes Yes Yes Yes
Poverty........................... Yes Yes Yes Yes
Disability........................ Yes Yes Yes No
----------------------------------------------------------------------------------------------------------------
CURRENT AYP FAILURE SCHOOL: MISS ALL
----------------------------------------------------------------------------------------------------------------
Other
Reading 95% Math 95% indicator
----------------------------------------------------------------------------------------------------------------
All Students...................... No No No No No
Black/African American............ No No No No
Hispanic.......................... No No No No
Native American................... No No No No
Asian............................. No No No No
White............................. No No No No
LEP............................... No No No No
Poverty........................... No No No No
Disability........................ No No No No
----------------------------------------------------------------------------------------------------------------
However, the use of growth models will not completely eliminate
this problem. If one subgroup makes AYP using a growth model, while
another subgroup in the school does not make AYP (even using the growth
model), the school will still be designated as failing AYP. In this
instance, the use of the growth model doesn't eliminate the ``all or
nothing'' approach.
I would also note that there are differences among growth models,
with varying levels of complexity. Some states, like North Carolina,
have had to implement a different growth formula for students not
already proficient than for students who are proficient. This was
necessary because of the federal requirement that proficiency be the
end result of any growth trajectory. Obviously, such a model does not
work for students who already are proficient.
I would like to close my comments on growth models by reminding the
committee that having a growth model as part of the AYP process is an
improvement but it will not be a panacea. Getting certain students on
track to proficiency within a four-year timeline, as is required under
North Carolina's approved model, will still be a challenge for many
schools. In addition, complexities will continue to arise for some ELL
students or certain students with disabilities who take alternate
assessments. We will also continue to need much more research about
growth models as well as technical assistance to states, local
districts, and educators to evaluate and use data, evaluate the models
themselves, and replicate successful efforts. The ultimate goal should
be to help classroom educators use data to inform instruction.
III. NEA's Work to Close Achievement Gaps
It has been a majority priority of mine to marshal NEA resources to
assist our state and local affiliates in seeking policy changes at the
state and local level to help close achievement gaps. Our work has
included:
Committing more than $6 million through NEA Foundation grants to
close achievement gaps in urban school districts. Those grants fund
programs with clear goals of improving literacy and math and science
achievement; helping stabilize quality staff; and involving families
and communities involved in the learning process. In two of the grant
sites, Hamilton County, Tennessee and Milwaukee, Wisconsin, school
faculties use growth data to assess progress in reading and mathematics
and determine appropriate interventions for instructional improvement.
Reaching ambitious growth targets provides confidence and positive
reinforcement to teachers and students who have large gaps to overcome
and helps teachers and administrators set continuous benchmarks for
progress and observe what works in changing instructional practice. For
example, last year in Hamilton County, Tennessee, the five schools
targeted under the NEA grant set and achieved a goal of 115 percent of
the expected growth according to state standards per annum in reading
and mathematics achievement. While these schools have not yet all
reached high levels of achievement compared to the state's affluent
schools, they have made greater gains than many of the top-ranked
schools. By significantly accelerating the rate of achievement, low-
performing schools can close achievement gaps, while all schools
continue to make progress.
Delivering trainings and products on a variety of instructional
issues, including closing the achievement gaps, to our members and
leaders across the country.
Sponsoring statewide National Board Certified Teacher summits
focused on recruiting and retaining accomplished teachers in high-need,
high-poverty schools with low student achievement.
Developing and sharing with all NEA affiliates our Closing
Achievement Gaps: An Association Guide-a blueprint for closing the
gaps.
Awarding grants to ten states (Colorado, Minnesota, Illinois,
Indiana, Michigan, Missouri, New Mexico, Nebraska, Oklahoma, and
Pennsylvania) focused on closing gaps through changes in state policies
such as early childhood education, coaching for new principals,
statewide teacher induction programs, and revised state professional
development guidelines. We have also provided grants to eight states
for their work to encourage highly skilled teachers to move to and stay
in high-need schools.
Continuing the partnership established in 2005 with the Tom Joyner
Foundation to increase the percentage of highly qualified Black
teachers in targeted high-needs communities. The Teacher Licensure
Scholarship Program, funded by the Tom Joyner Foundation, provides
financial assistance to historically Black colleges and universities to
assist minority teachers in preparation for licensure exams. More than
250 scholarships have already been awarded nationally through this
program.
Developing training modules on closing the achievement gaps for use
in community conversations. This work has been piloted in Mississippi
and Florida and will be used by the end of this program year in three
of the seven states that have been awarded grants to convene community
conversations.
Surveying, through the Center for Teacher Quality, teachers in
three states (Kansas, Arizona, Ohio,) to identify the necessary working
conditions to achieve optimum teaching and learning environments.
Developing online professional development focused on helping
teachers become more effective with a diverse student body.
Awarding grants to recognize model teacher retention program
through a Saturn-UAW-NEA partnership.
Launching and maintaining an easily accessible, interactive Website
to help our affiliates and the general public research and locate
resources about the achievement gaps (www.achievementgaps.org).
In the near future, NEA will develop a program guide outlining
effective support strategies for minority candidates pursuing National
Board Certification, with the goal of increasing the percentage of
minority National Board Certified teachers in high-need schools. In the
next school year, we will convene community conversations in seven
states as part of our Public Engagement Project (PEP) initiative. We
will also develop additional educational materials for state affiliates
on teaching and working conditions, and we will be announcing
additional grants for 2006-07. Finally, NEA is building a state-by-
state database to identify policies, practices and programs that help
close the achievement gaps. I would be happy to share additional
information on any of our projects with the committee.
Before I conclude, I would like to share just one of the many
stories from countless NEA members about the impact of ESEA on them and
their students. It is a story about a boy named Cesar and his ESL
teacher, Mary Beth Solano in Fort Collins, CO. She writes:
``One of my recent student's stories is a prime example of how NCLB
legislation, by labeling students unfairly, is demoralizing and needs
to be changed. Cesar, a third grade student, came to me in August with
not word one of English. Together with his classroom teacher,
significant work with me, and a fantastic group of peers, he learned
English amazingly quickly.
I was almost going to exempt him from the test, but two things
stopped me. One, the part of NCLB that says that any child who doesn't
take the test counts as a zero for our school's report card, and Cesar
himself, who set as his personal goal knowing enough English to pass
CSAP (Colorado's state mandated NCLB test). Reluctantly, but with
Cesar's terrific desire to succeed coaxing us on, we had him take the
test. He struggled and struggled, reading every word * * * over and
over again until he thought he understood well enough to answer each
question.
He worked so hard and so long on that test, it brought tears to my
eyes. He was so proud of himself after he completed it, you should have
seen his face. Remember, he had only been working in English since
August and the reading test was given in February * * * six short
months with a new language, and he took the same test as native
speakers did. He kept asking what his score was, and actually looked
forward to the day his parents would get `the letter'. Well, the scores
came out, and while we celebrated (understanding statistics and
scoring), he and his family were devastated. Cesar earned a score that
was only two tiny points below the cut off for partially proficient on
CSAP, but below the cut off it was, so he was labeled an
`Unsatisfactory' learner.
Nothing could be farther from the truth for he had gone from
basically zero to almost proficient in just six months (something no
politician has ever done), but the federal government didn't care about
that effort or progress. To the feds, the state and the public he was
unsatisfactory. I tried explaining it all to his parents, and tried
even harder to lift Cesar's spirits, but I'm not sure how much success
I had. Without changes in the structure and process of reporting
scores, stories like Cesar's will continue to deflate and demoralize
the best and brightest students.''
On behalf of all 2.8 million members of the National Education
Association, I want to thank you for this opportunity. We look forward
to working with you throughout the reauthorization process. I encourage
every member of this committee to talk to your local educators about
their experiences. Ask them about their frustrations. But more
importantly, ask them about their successes. When you do, you'll get as
clear a sense as I have. They all have a ``Cesar'' story. They all want
every one of their students to succeed. And they go above and beyond
the call of duty time and time again to make that goal a reality for
America's public school students.
Thank you. I will be happy to answer any questions.
______
Chairman McKeon. Thank you.
Ms. Haycock?
STATEMENT OF KATI HAYCOCK, DIRECTOR, THE EDUCATION TRUST
Ms. Haycock. Mr. Chairman, Mr. Miller, and members of the
committee, thank you for inviting me to come speak with you
this morning about whether the addition of a growth component
can improve the NCLB accountability system.
As you know, I head the Education Trust, and in that role I
spend an awful lot of time with educators around the country,
including on accountability matters. I also serve as one of 10
members of the peer review committee assembled by the secretary
of education to review the proposals from states.
My simple answer to your overarching question this morning
is yes, incorporating growth models can improve accountability
systems. Such improved systems won't necessarily guarantee, in
and of themselves, more quality education, but they can help.
The reason for that is because accountability systems that
grant appropriate credit for growth can help in four ways.
First, they can help local and state education leaders to
prioritize the schools that need the most help by
distinguishing between schools where the kids may not yet be at
proficient, but they are on their way to being proficient in a
few years, and those schools where kids are on a trajectory to
get nowhere at all.
Second, they can help ensure that the problem that
Chancellor Klein talked about, schools focusing just on kids
right below the proficient bar, doesn't happen as often, that
they are working on growth for all kids.
Third, they can help education leaders set stretch goals
for all schools, including the higher achieving schools that my
friend Bill Sanders here talks about as the slide-and-glide
schools.
And finally, they can help to get rid of the perception by
some people at least that the NCLB accountability system is
arbitrary and unfair. That said, suggesting that the inclusion
of those measures can improve accountability systems is
different from guaranteeing that the inclusion of such
components will improve the system.
So let me focus, if I can, on the principles and conditions
that will help to maximize impact. But before I do that, just a
word or two on the peer review process. I think you know that
the secretary assembled a quite diverse group of peer
reviewers.
It included representatives from state education agencies,
local school districts, assessment experts and child advocates.
We were a diverse group in every way, including politically.
Despite that, however, we came to full agreement on every
single substantive issue that we have faced. And in fact, the
conversations among the peer reviewers were so substantive that
we actually wrote up a public summary of that that we all
signed in order to help states assemble future growth
proposals.
I do want to share with you, though, the two most
fundamentally important findings from that group. First,
despite perceptions that the current AYP system is
unnecessarily rigid and crude, the current system actually gets
it right most of the time.
It turns out that schools that don't make AYP in a system
that does not include a growth component actually most often
don't make AYP in a system that does include it. In both of the
states that we approved, fewer than 50 schools that didn't make
AYP actually made AYP with a growth component included.
Second, it is really important to understand that growth to
proficiency in most cases is actually a higher bar, a tougher
standard, than the current status system.
In North Carolina, for example, if accountability systems
decisions had been made entirely on growth, actually, fewer
schools would have made AYP instead of more. Why is that?
Because lots of schools that are now over the status bar are
actually not growing their kids very much.
So growth isn't an easier standard at all, but done right
it can help. Let me talk about four or five principles for
doing it right. No. 1, as Mr. Miller suggested, it is really
important and you insist on growth to proficiency. Any old
growth won't do.
No. 2, once a student's growth trajectory is established
with a goal of proficiency within 4 years, you don't want to
allow it to be reset and reset again.
Third, growth models need to set goals for proficient
students as well. The reason for that is very simple. A lot of
students who are proficient 1 year won't be proficient the next
year or the year after unless they get some focused attention
as well.
Fourth, just as in the current law you have not allowed
schools to average and mask the low performance of some kids
with the high performance of others, it is also important that
you not allow schools to mask the low growth of some students
with the high growth of others.
Finally, it is always important to ask the question growth
to what. Just as averaging performance can mask the
underachievement of some groups of kids, too low a standard can
mask problems with the performance of students.
We need to have high and rigorous standards if growth is
something--is actually to matter at all. We need to be focused
more on where kids are going than simply where they have been.
One final comment, though. Improving NCLB's accountability
system is an important undertaking. But we need to keep it in
perspective. When you look overseas, what you find is most
other countries sort of got about the business of standards and
accountability fast and then moved on to improvement.
We Americans, however, keep obsessing about how to get that
accountability system a little bit better instead of focusing
on what is really important, and that is improving teaching and
learning, helping low performance schools get better, making
sure we have quality teachers and that they are distributed
fairly, and also providing the curriculum and benchmark
assessment that teachers need to make sure they are doing the
job right.
In the end, that is what is going to get us to higher
achievement, not simply a better accountability system. Thank
you.
[The prepared statement of Ms. Haycock follows:]
Prepared Statement of Kati Haycock, Director, the Education Trust
Mr. Chairman, Mr. Miller, and Members of the Committee, thank you
for the opportunity to testify this morning regarding the potential of
growth measures to improve school accountability determinations.
As you know, I head the Education Trust, an independent, non-profit
organization focused on improving achievement and closing gaps between
groups, pre-kindergarten through college. In that role, I and my
colleagues do a lot of work with both states and school districts,
including frequent opportunities to advise on and observe the effects
of accountability systems. I also served as one of ten members of the
Peer Review Team assembled by Secretary Spellings to advise her on
proposals for the Growth Model Pilot program.
My simple answer to your overarching question is, yes,
incorporating growth measures can improve accountability systems. They
don't in and of themselves ensure improved education for all students,
but growth measures can help.
First, using growth measures can help education leaders to
prioritize the schools that need the most help. Giving schools credit
for growth in students' learning can help distinguish between schools
where, though students are not yet proficient, they are on a trajectory
to become proficient soon-and those where students are on a trajectory
to nowhere. These more nuanced determinations can help to ensure that
help and resources are targeted to the schools and students with the
most acute needs.
Second, establishing credit for growth can help to ensure that
schools don't focus inordinate attention on students who are just below
proficient, but rather seek to grow the knowledge and skills of
students at all ends of the achievement spectrum--including the middle-
achieving low-income and minority students who could become high
achievers with more challenging instruction.
Third, the inclusion of growth measures can help education leaders
to set stretch goals for all schools, including the higher achieving
schools that Bill Sanders often calls the ``Slide and Glide'' Schools.
Finally, establishing credit for growth can reduce the perception
of some that the NCLB accountability system is arbitrary and unfair,
especially to schools that receive large numbers of underprepared
students.
That said, suggesting that the inclusion of growth measures can
improve accountability systems is different from guaranteeing that it
will. So rather than spending my time with you extolling the virtues of
growth measures, let me focus instead on what I think I've learned
about the principles and conditions that must undergird these or any
other changes in accountability systems.
But first a word about the peer review process.
The Peer Review Process
In November 2005, the U.S. Secretary of Education announced a
growth model pilot program, with the stated goal of informing the
reauthorization of NCLB. Several months later, she named a panel of ten
peer reviewers.
The peer reviewers represented a broad cross section of experts and
stakeholders, including practitioners from state departments of
education and school districts, as well as outside researchers,
assessment experts and child advocates. The group included both
liberals and conservatives.
Despite our different perspectives, however, we reached consensus
on every significant aspect in our review of the states' growth model
proposals. Indeed, our conversations were so constructive and
substantive that the peer review panel took the extraordinary step of
publicly releasing a statement of principles-signed by every member of
the team--to help inform subsequent discussions of these issues. I'll
draw from that statement in just a moment.
I do, however, want to share what were perhaps the two most
important findings from our work:
First, despite perceptions that it is unnecessarily rigid
and crude, the current AYP system actually gets it right most of the
time. In both states that were approved to implement growth models,
most of the schools that didn't make AYP without a growth component
also did not make AYP with a growth component. As it turns out, schools
with low achievement also tend to be schools with low growth. In each
state, fewer than 50 schools that made growth targets that had not made
AYP under the current rules.
Second, in most cases, growth to proficiency is actually a
higher bar than the current status system. In North Carolina, for
example, if accountability determinations were based solely on growth,
fewer schools would have met their goals. Why? Because a lot of schools
that are now over the status bar are actually not growing their
students' knowledge and skills very much.
So growth isn't an easier standard at all. But, done right, it can
both enhance the fundamental fairness of the system and provide the
more nuanced information necessary to help leaders target resources and
assistance to the schools and students with the most acute needs.
Principles
So what are the principles that should undergird this or any other
change in the NCLB accountability system?
First and foremost, we must be very clear that any old growth won't
do. Congress must insist on growth to proficiency. It's important to
recall why you focused the nation on the goal of student proficiency in
the first place. The reality is that there are absolute standards
against which students will be judged, whether they go right to work or
into postsecondary education. In the real world, there won't be
allowances based on family background or parents' education level.
If public education is going to serve as an engine of upward
mobility, then expectations need to be pegged not just to where
students come from, but where they need to go. A growth model can
provide incentives to focus on students at all levels of achievement,
but the goal must still be proficiency for all students.
Some argue that this is somehow unfair to schools that serve
concentrations of poor children. Our collective responsibility,
however, must be to the students themselves. And frankly it's unfair to
them not to require schools and districts to take responsibility for
student achievement.
Second, once a student's growth trajectory is established, it's the
school system's responsibility to catch the student up within the
designated time frame. If the expectation is that a student will reach
proficiency in three or four years, the targets should not be reset
downwards nor should the time frame generally be extended. Otherwise,
students will never actually be expected to be proficient.
Third, as the peer review committee suggested, growth models should
set goals for proficient students. This is especially important because
many students who perform at the proficient level one year will not be
proficient in subsequent years without explicit attention to their
needs. This is one of the reasons why, in recent years, we've observed
a pattern on both NAEP and state tests in which more students are
proficient on elementary tests than in the middle or high school
grades. Growth models should orient educators toward students' success
in successive years.
Fourth, any growth models should retain NCLB's historic focus on
individual students. If, as some suggest, we look at the average growth
for a whole school, high growth with some students will mask the
stagnant or slow growth--and even the academic decline--of other
students. One of NCLB's strengths is that it does not allow schools to
compensate for the under-education of low-achieving students by having
a greater number of advanced students. Likewise, schools should not
compensate for some students' stagnant growth by showing greater growth
with higher-achieving students.
Looking at the growth of individual students over time requires
assessments that are aligned from year to year and longitudinal data
systems. Many states still do not have these in place, limiting their
ability to implement growth models. The quality of assessments and data
systems is critically important to the accuracy and validity of growth
measures, so focusing on state capacity and resources in this area
needs to be a priority.
Finally, it is important to ask the question, ``growth to what?''
Just as averages can mask under-achievement by some groups of students,
so too can standards that are not sufficiently rigorous. If schools can
meet their goals not only based on students that are meeting standards,
but also on growth toward these standards, it becomes even more
important to have meaningful, high-level standards. As Congress
considers allowing states to incorporate growth into accountability, it
is important to revisit the hands-off approach that has ignored the
rigor of state standards.
Accountability Alone Is Not Adequate
Improving NCLB's accountability system is an important undertaking,
and the inclusion of a growth component guided by these principles can
help.
But we need to keep all this in perspective. Even the best
accountability system is essentially a signaling system, helping
educators, policymakers, and the public-at-large to understand whether
schools are meeting their goals.
I'm the last person to dismiss the power of accountability systems
to improve coherence and focus in systems of public education that
often lack both. Certainly, there is evidence that standards and
accountability help: This year's edition of Education Week's Quality
Counts report concluded that states with stronger implementation of
accountability have seen appreciably bigger gains on NAEP, bolstering
earlier findings from a study by the RAND Corporation. And NCLB's focus
on accountability for different groups of students has provided
critically important leverage to get systems of public education
responding to the needs of low-income and minority students.
What's really important, however, is improving teaching and
learning. No amount of tweaking the accountability system will solve
the serious, systemic problems plaguing our public schools. Indeed, I
worry that, unlike in other countries, where leaders adopt standards
and accountability systems and then move quickly to improvement
activities, we Americans keep focusing the bulk of our energy on making
accountability systems ever better.
There are some critical improvement issues that demand their own
attention:
Capacity to turn around struggling schools: As a country, we've
made a policy decision to no longer tolerate widespread failure in
public education. This marks a historic shift, and one that was long
overdue. But the old state and district systems were built when low-
performing schools were considered acceptable and even inevitable, and
weren't set up to diagnose problems and intervene in struggling
schools.
We need to rapidly expand the expertise and the resources focused
on turning around persistently low-performing schools. The current
budget proposal to dedicate $200 million to the school improvement fund
is a good idea, in part because it sends a signal to the states that
this is a priority. But right now states aren't investing enough of
their own resources in this area, and demand far outpaces ability to
respond.
Teacher Quality: We cannot close achievement gaps without closing
gaps in access to teacher quality. Recent research from Illinois
documented that students who studied all the way through Calculus in
schools with the lowest teacher quality learned less math than students
who only went through Algebra 2 in schools with just average teacher
quality. Yet Congress has continued to pour billions of dollars into
systems, ostensibly to help educate poor kids, only to have systems
provide these students with the least access to qualified teachers and
high-quality teaching.
Recently, for the first time in the four-year history of NCLB's
implementation, the U.S. Department of Education required states to
develop equity plans to ensure poor and minority students get their
fair share of teacher talent. And Congress has encouraged innovation in
teacher assignment, evaluation, and compensation by creating the
Teacher Incentive Fund. These are important steps, but raising teacher
quality and ensuring equal access to effective teachers must remain a
bipartisan priority.
Title 2 of NCLB is by far the biggest investment in raising teacher
quality, especially in school districts with high proportions of low-
income students. With Title 2, Congress gave districts virtually
unfettered discretion to use the money as they saw fit. The result has
been general programs that diluted the targeted support envisioned by
Congress, with no discernable impact on teacher quality distribution
despite a $3 billion annual investment. Congress should conduct an
intense inquiry into how Title 2 is being implemented and stipulate
better targeting, but should not cut $300 million from the program, as
is currently proposed.
Curriculum development with aligned benchmark assessments: In the
early days of standards-based reform, leaders thought that standards
themselves would provide sufficient guidance to teachers about what to
teach and to what level. They believed that teachers themselves would
figure out how to get students to those standards.
It turns out that most teachers neither want to develop their own
curriculum nor have the skills to do it. Rather, they need coherent,
well-designed lessons, units and assignments that they can use day to
day. Fortunately, states and districts are doing more and more along
these lines, in part to try to reach their accountability goals.
They are also doing more with the kinds of regular, teacher-
friendly, ``benchmark'' assessments that teachers need to gauge their
students' progress toward state standards.
While states and districts have increased their activity and
funding in this arena, the federal government could play an important
role in bringing these practices to scale by targeting grants in this
area to jurisdictions that have good data systems and want to submit
these activities to rigorous evaluation, so that lessons can be learned
and widely disseminated.
Conclusion
I wish I could report to you that the culture of accountability and
continuous improvement had permeated public education, so that you
could hand back more discretion to the states to set accountability on
their own terms. But nothing in our history or the current climate
suggests that we have made enough progress on this front. In fact, some
of the push for growth models is a ruse to distract attention from the
stark reality that many of our schools themselves must grow a lot, and
fast. Several of the states that clamored the loudest for growth models
did not even apply for the pilot because they do not have the requisite
assessments or data systems in place. For some of the most outspoken
critics, the focus on growth amounts to little more than an attempt to
diminish public support for meaningful accountability.
Incorporating growth is a good idea and Congress should not be
deterred because some of its boosters have mixed motives. But it does
mean that you must be vigilant in scrutinizing proposals to ensure that
core principles are preserved and strengthened.
Strong accountability is the most important leverage we have to
focus public education on continuous improvement and the quest for
equal educational opportunity. The consequences of weakening
accountability will reverberate in the nation's military preparedness,
economic vitality, and social cohesion.
Basing school accountability determinations on measures of
individual students' growth over time can improve accountability, and
those improvements can help ensure that public education targets its
resources to the students who need the most help. I do not in any way
want to diminish the importance of getting accountability systems to be
as good as they can be.
But we have got to get beyond this never-ending quest for the
perfect accountability system and turn to the hard work of curriculum
development, teacher professional development, and leadership training
for principals.
______
Chairman McKeon. Thank you.
Dr. Sanders?
STATEMENT OF WILLIAM L. SANDERS, PH.D., SENIOR MANAGER, VALUE-
ADDED ASSESSMENT AND RESEARCH, SAS INSTITUTE, INC.
Mr. Sanders. Thank you, Mr. Chairman, Mr. Miller, members
of the committee. I am honored by your invitation.
My letter of invitation asked me to comment on my
experiences with value-added and growth models, how growth or
value-added models might fit into state-developed
accountability systems under No Child Left Behind, benefits and
challenges of implementation, as well as some of our major
research findings that we have accumulated over the past 24
years.
In my remarks I will refer to each of these requests.
However, the total intent of my remarks will be to make the
case to Congress that the addition of a properly constructed
growth component to the adequate yearly progress measure will
make NCLB fairer to schools and will provide positive benefits
to a greater percentage of their student populations.
My experiences--I am a statistician that fortuitously got
involved with educational research 24 years ago. At that time
in Tennessee, there was considerable discussion how to improve
the effectiveness of public schooling for that state's
populations.
These discussions inevitably led to the question of how to
quantitatively measure the impact of schooling on measures of
academic performance, especially academic growth.
In fact--and I think this is important for the committee to
recognize--in fact, some of the same quantitative issues that
are now being raised relative to the growth model discussions
were indeed being raised in that era.
After learning of these issues, and my personal knowledge
of statistical mixed model theory and methodology, I felt then,
and I certainly believe even more deeply today, that there
exist solutions to many of the pertinent questions that are
often cited as impediments to using student test data to
provide quantitative, reliable, robust measures of schooling
influences.
Our work over the past 24 years has certainly been about
refining the statistical methodology for application to the
educational measurement arena.
Now, next, Mr. Chairman, what I would like to do is to try
to make a distinction between value-added models and growth
models in the context of No Child Left Behind.
Value-added models and growth models--and I often refer to
them as projection models--all use longitudinal data, as
Chancellor Klein has mentioned--in other words, following the
progress of each student as an individual.
In value-added models like the Tennessee value-added
assessment system, the purpose is to essentially get a measure
of the impact, a summative measure of the impact that that
school is having on the rate of progress of all students.
In the context of No Child Left Behind and in the context
of what I would certainly recommend Congress consider, the
projection models use longitudinal data but for a different
purpose. The purpose is to essentially take the data for an
individual child and develop a projection of whether or not
that child is on a trajectory to meet a meaningful academic
standard in the future.
Let me give you a specific example. In Memphis, there is a
school, and I could certainly point to many other schools,
where the average entering 3rd-grader profiles at about the
25th percentile point relative to all 3rd-graders in the state
of Tennessee--very low-achieving kids entering that school.
Yet by the end of 5th grade in that school, the average kid
is scoring at the 45th percentile point relative to all kids in
that school. That school in the past has failed AYP because
under existing rules you have got to aggregate the 3rd-grade
scores, the 4th-grade scores, the 5th-grade scores.
In my view, that is not a failing school. That is an
excellent, outstanding, effective school. So to me, the
addition of the growth measure could enable schools like that
to be recognized proudly for their effectiveness.
Now, even with the additional augmentation of AYP, as two
states have, as was mentioned, now been approved in a pilot
project by the Department of Education, there is another
consideration that Congress should give prior to
reauthorization of No Child Left Behind. And basically, I will
be restating what Chancellor Klein said.
There are a lot of schools, and I certainly could show you
many, that have presently passed AYP, presently passed adequate
yearly progress, yet when you look in those schools you will
often see that those students have a projection whereby that
they will not be proficient in the future.
And this notion that unwittingly we put this perverse
incentive in too many places to teach to the bubble kids, the
kids just below proficiency such that if we get them over the
bar, then we will tend to raise our percent proficient.
And often, this is at the exclusion of those children that
are so far behind we are not going to get them over the bar
this year, and also at the exclusion of those children that are
already above the bar because they are going to be proficient
anyway.
This is the kind of thing that if there is a projection
component included in the reauthorization of No Child Left
Behind, I think some of that disincentive will be removed.
Now, NCLB, in my view, is beginning to yield many positive
results and raising the nation's academic achievement for a
large segment of the student population. The suggested tweaking
with addition of a projection component will be an improvement.
And certainly our research accumulated over the past 24
years certainly has documented that effective schooling
sustained over time will trump socioeconomic influences.
Thank you, Mr. Chairman.
[The prepared statement of Mr. Sanders follows:]
Prepared Statement of William L. Sanders, Ph.D., Senior Manager, Value-
Added Assessment and Research, SAS Institute, Inc.
Thank you, Mr. Chairman. My name is William L. Sanders; I am
presently Senior Manager, Value-added Research and Assessment, SAS
Institute, Inc. Additionally, I hold the honorary title of Senior
Research Fellow with the University of North Carolina. Previously, I
was Professor and Director of the Value-Added Research and Assessment
Center with the University of Tennessee.
My letter of invitation asked me to comment on: my experience with
value-added and growth models, how growth or value-added models might
fit into State developed accountability systems under No Child Left
Behind (NCLB), benefits and challenges of implementation, as well as
some of our major research findings. In my remarks I will refer to each
of these requests. However, the total intent of my remarks will be to
make the case to Congress that the addition of a properly constructed
growth component to the adequate yearly progress measure (AYP) will
make NCLB fairer to schools and will provide positive benefits to a
greater percentage of their student populations.
My experiences. I am a statistician that fortuitously got involved
with educational research 24 years ago. At that time in Tennessee,
under the leadership of Governor Lamar Alexander, there was
considerable discussion on how to improve the effectiveness of public
schooling for that state's population of students. These discussions
inevitably lead to the question of how to quantitatively measure the
impact of schooling on measures of academic performance, especially
measures of academic growth attributable to various schooling entities.
In fact, some of the same quantitative issues that are now being raised
relative to the growth model discussions were indeed being raised in
that era. After learning of these issues, and being knowledgeable of
statistical mixed model theory and methodology, I felt that there
existed solutions to many of the pertinent questions being cited as
impediments to using student test data to provide quantitative,
reliable robust measures of schooling influences on the rate of
academic progress of student populations.
Using this methodology, my colleagues and I built the quantitative
system on which the Tennessee Value-Added Assessment System is based.
Perhaps I was not the first, but one of the first to apply the term
``value added assessment'' to measurement of educational outcome.
Value-added assessment provides measures of the influence that
educational entities, (i.e. districts, schools and classrooms) have on
the rate of student academic progress. All value-added procedures use
longitudinal data (i.e. follow the progress of individual students over
grades) to get measures of these influences. These measures provide
information as to the effectiveness schools or districts in providing
the opportunity for academic progress for all students. Tennessee has
had value-added measures as part of its accountability system since
1993.
However, all value-added modeling efforts do not give equivalent
results. Some of the more simplistic value-added approaches should be
rejected because of serious biases and/or unreliable estimates that
they provide. If value-added models are to be used as part of an
accountability system, then there are some minimal criteria that must
be required. To dampen the error of measurement associated with a
single test score for an individual student, all test data over grades
and subjects for each individual student must be used in the analysis.
However, all students do not have the same quantity of test data.
Disproportionately low scoring students have more missing longitudinal
data than higher scoring students. Thus, any value-added model approach
must be sophisticated enough to provide unbiased, reliable measures
using all data for each student no matter how sparse or complete.
Simple posttest minus pretest averages and simple regression
approaches, which use only the previous year's score as a predictor
variable, are examples of value-added attempts that should not be used.
Next, I would like to make a distinction between the use of value-
added models (like the Tennessee Value-Added Assessment System) and
growth, or projection models to be used as part of NCLB. In
accountability systems, value-added models use longitudinal data to
provide a summative measure of the aggregate progress of all students
attending a school. The projection (growth) model recently approved for
Tennessee to augment AYP uses longitudinal data to ascertain if a
student is on a trajectory to reach a proficiency standard three years
in the future. The same data structure is used for two different
purposes. However, the same statistical issues (fractured student
records, using all data for each student, etc.) are just as important
and must be accommodated in the projection (growth) models as with the
value-added models. Again some of the more simplistic approaches to
measurement of growth should not be used because of the resulting
innate biases and greater unreliability. On this topic, I concur with
the U.S. Department of Education's peer review team's comment that all
of each student's prior data should be used; not just two data points.
Why should NCLB be augmented to allow projection (growth) models?
Students enter a school with a wide range of achievement levels. Under
existing rules if a school has an entering population whose achievement
level is extremely low, then regardless of the magnitude of progress of
these students, it is most difficult for the school to make its AYP
targets. For those schools which are eliciting superior academic growth
for its student population, this additional measure can clearly
differentiate these schools and enable them to be recognized proudly
for their effectiveness.
Even with the additional augmentation of AYP, as two states have
now been approved in a pilot project by the Department of Education,
there is another consideration that Congress should give prior to
reauthorization of NCLB. With the existing AYP rules, schools can now
be meeting their AYP targets, yet within those schools the progress
rates of students who are currently proficient can be so relatively
modest that their likelihood of not meeting proficiency in the future
is greatly enhanced. Unfortunately, it appears that in too many
schools, which are in jeopardy of not meeting their AYP targets, more
instructional effort is focused on the ``bubble kids'', (i.e. those
kids who are perceived to be near proficiency) with less effort
extended for other students. The mistaken belief is that some students
are so far behind that regardless of effort they will not reach
proficiency this year, while other students are going to meet the
proficiency requirements without much curricular attention. The focus
on ``bubble'' students leaves two groups of students vulnerable, those
most behind academically and those barely proficient.
To provide a disincentive for this practice, I would recommend that
Congress give serious consideration to replacing the existing ``safe
harbor'' component of the AYP rules with a projection component so that
schools will receive credit for keeping all students on trajectories to
be proficient in the future. This should tend to alleviate the problem
of not focusing appropriate instructional effort on all students. All
states will now have the annual testing in place to allow for the
projection approach to be applied. At a minimum, states should be
allowed to substitute this better approach for the existing ``safe
harbor'' rules.
In summary, the impact of NCLB is beginning to yield many positive
results in raising the academic achievement level for a large segment
of the nation's student population. The suggested tweaking with the
addition of a projection (growth) component will be an improvement. Our
research accumulated on the past 24 years, certainly has documented
that effective schooling will trump socio-economic influences if
effective schooling is sustained over time for each student. The data
resulting from the implementation of NCLB, if wisely used, can
certainly lay the information base for insuring that all students will
have the opportunity to learn, consistent with their achievement level.
______
Chairman McKeon. Well, thank you very much.
This has been very interesting. I was listening to Mr.
Miller's opening comments and my opening comments, and there is
not a lot of difference.
I think one thing that is exciting to me as we move
forward--we have had now--this is the fourth hearing on getting
ready for the reauthorization. Each one of them has been good.
This one has been excellent.
I have no agenda. It doesn't sound like Mr. Miller has any
agenda, other than improving on what we did when we wrote the
No Child Left Behind. We are not trying to say this is
something we want to emphasize.
We want to listen to experts like you find out what can be
done to make it better. And as we go through this process, we
are going to be listening very carefully. That sounds like the
ultimate goal, and I am hopeful that we are going to be able to
continue to work in a bipartisan way to really make this
happen, because it is too important to not do that.
I forget now which one of you mentioned--it was probably
Chancellor Klein--about ``gotcha.'' You know, I have some
concerns about government at all levels--or not just
government, but people that have that attitude, of trying to
find people doing something wrong rather than trying to
encourage people to do what is right.
And I hope that we don't have that attitude, that all we
want to do is find something wrong. And I think that is a poor
management model. Better to find what people are doing right
and focus on that. And government's role should be how can we
help you do better, not can we find what you are doing wrong.
And I think several of you mentioned the importance of
principals, assistance principals, teachers, parents, because
we are all in this together. There has been a lot of focus on
teachers, but I know I go to school sometimes and I can see--
just as you walk on the campus, you can feel the influence of a
great principal.
You can feel the influence of a great superintendent. You
can feel the influence of an assistant principal. You go into
the classroom, you can feel the influence of a great teacher.
You can also feel the influence of a teacher that is not
measuring up.
And I am hopeful that from your testimony today--and I
think this is important. We have to do this. But I would rather
get away from this kind of a setting and sit around in a circle
and really have some give and take and some real meaningful
discussion about--you are all experts.
Really, you can teach us a lot. And maybe we can also do
some of those kind of settings as we move forward, just have
some real round-table discussions and go into some of this
stuff in depth.
One thing, though. I think one of the problems we have had
with No Child Left Behind is people have put, you know,
different labels on things, and sometimes we have gotten blamed
for things that are part of state government. We say that
states have standards, and then we get blamed for the standards
the states set.
Dr. Weaver, in the conclusion of your testimony you told
the story about this young student, and I think that grabbed
all of us. He was devastated when he didn't pass an exam and he
was labeled an unsatisfactory learner.
What responsibility do you think the teachers have for
ensuring that students really understand the role of state-
developed assessments and the fact that NCLB judges the
progress of schools, not individual students?
And do teachers have a role in ensuring the public
understands that nothing in Federal law labels schools as
failing?
Mr. Weaver. I think that if we understand the rules, if we
understand the law, if we understand what it is we are supposed
to do, then I think that we can be good messengers.
However, there have been so many mixed messages. There have
been many stops and many starts. And so it has left a number of
people, not only teachers--but it has left a number of people
uncertain as to what it is that we are supposed to do.
Also, when it comes to the involvement of people, teachers,
regardless of what position, involvement of those individuals
in implementation, it makes it much easier if, in fact, we have
been involved or included in terms of determining what that
outcome is going to be.
However, in many instances, we have had decisions made for
us, without us, in spite of us, without even talking with us.
And then the expectation is that we implement it. That is not a
good way to encourage good implementation. I don't think it
works at our level. I don't think it works at your level. I
don't think it works at any level.
So the more that we are----
Chairman McKeon. Or any business.
Mr. Weaver. Absolutely. The more that we are involved in
helping to shape the circumstances that impact us, I think the
better off we all will be.
Chairman McKeon. Thank you. I have a whole lot of other
stuff I would like to talk about, but my time is up.
Mr. Miller?
Mr. Miller. Thank you very much.
Thank you for participating this morning. We look forward
to spending a lot of time with you all as we move forward in
this process. I want to make sort of a comment and then maybe
get your reaction, because I won't have time for individual
questions this morning.
But back in the days of yesterday when I was a kid, I
brought home a report card at the end of my 4th-grade year, and
it said either 4th-grade reading, 4th-grade math, 4.5, 3.5,
whatever, so my parents said, OK, you are doing all right.
And if I brought home the next one and it said--at the end
of my 5th grade and it said 4.5, my parents said, you are
falling behind here, you are going to go to summer school, you
are going to do something like that.
I won't even go to the deportment questions, but there was
an implied bargain. You put your kid in the school, and you get
a year's growth for a year's schooling, that they would be at
grade level. That was sort of the idea.
That is where you were moving, and that is--I think when we
talk about proficiency, we all understood that we were talking
about grade level.
And, Mr. Klein, you talked about, you know, the mass of
investments you are making in data, and one of the things as we
kind of look around the country--districts that have decided
that they are going to--and states that decide they are going
to embrace the data seem--and, Kati, you referenced this in
your discussion--and try to figure out accountability and use
this data to get a road map seem to be doing somewhat better.
But you have to--I think in this version of trying to close
the gap, you have got to embrace the data. You have got to know
something about your students. You have to know on a real-time
basis what is happening in that classroom.
And we didn't invent high-stakes testing. We didn't do any
of that. We took the stakes as we found them. In fact, the
president and I, one of the few times we have ever been in
unison, talked about diagnostic data. We really wanted real-
time interventions, hopefully small interventions that could
make a large difference for students.
But as we now start to look at the data, another thing is
happening. Some of the urban myths and legends are falling
apart: But for those English learners, we would have made AYP.
The data from the Aspen Institute starts to suggest that is not
the case.
If we had only had a growth model, we would have made AYP.
Your data suggested in some cases that is not the case.
And I think what we need to understand here is that in
achieving the growth and the proficiency that we want, there is
a lot of work that needs to be done. It is not just changing
the standard.
And I think you sort of say that, Kati, in your testimony.
You know, Dr. Sanders, you are kind of a legend, you know,
to me, in the sense that how you took this problem apart of
value-added.
But it starts to suggest that if we really make the
investment in our teacher corps, if we really, you know,
understand the need for that core investment, we can, in fact,
get these years of growth in succession and not have the
student fall back.
So the data becomes--it is troublesome, because it
describes the magnitude of what we have to do and the resources
that are going to have to be necessary. Some states don't have
a clue what is going on inside that classroom. Therefore, they
can't make changes that help those students.
And I just--I don't know if there is a comment there for
you to make, but I am concerned that we think if we just tinker
with this, all will be right, because I have gone into I don't
know how many school rooms in how many schools all across this
nation, and I have always been told but for those English
learners we could have--just two students missed the mark, we
were going to be a great school.
Now, I don't know if AYP would have told me that was a
great school or not, because we understand that it is not a
fair measurement in some instances. But that is not the
problem, I don't think.
It is what we have--we have to hold that as a standard, but
getting to those high standards, getting to those aligned
curriculums, getting to tests that are useful and impart
information on a timely basis is also a big part of the
problem, and I don't--you know, goes to some question of
resources.
That is my comment here, because--if you want to comment, I
would certainly appreciate it. If you can't decipher it, send
me a note 10 days from now. But I just want to--I want us to
understand the magnitude of the problem we are engaging here.
I mean, there has been some alignment, and I think we have
got a lot of people headed in the right direction over the last
5 years, but there is a big unfinished part of this painting
here.
Mr. Klein. Mr. Miller, I would like to comment. I will also
send you a letter, but I would like to comment.
I think what you say is absolutely true, and I don't think
anybody on the panel would disagree, that if you--you can have
a perfect accountability system, but if you don't have the
proper investments in terms of your human resources, your
talent and so forth, you are not going to get the job done.
However, it seems to me accountability, which is a core
thing, and trying to change the culture of a school system in
which you are saying to your folks, which I am saying all the
time, we are responsible for the educate of our kids--the days
of excuses are over--it is the kids, it is poverty, it is the
fact that they are immigrants. I don't want to hear that.
What I want to hear is how we are going to educate each and
every one of our kids. And accountability is critical to that
equation. And so what I am trying to say to you today is the
message that I try to send to my school system is this is our
collective responsibility in the city of New York.
For far too long, far too many of our kids have not gotten
the education that they need to succeed. But I need to be able
to convince those people that the measurements we use are
really very powerful and fair.
And I agree with you. The fact that there is a growth model
is going to lead to schools that don't pass the average yearly
progress. There is no question about it.
And I think Bill Sanders' point is a very important one.
Some of those schools are going to be schools that people now
call good schools, because they are not moving their kids. And
what you want to see is really almost a kind of kinetic motion
in the system of all the kids moving, not a strategic motion.
I walked into a school not far from Major Owens' district
and the principal said--I said you had such great results, you
went up 20 points last year, how did you do it.
You know what he did? He took out a little chart, and he
said here are all my kids. I looked at all the ones that were
closest to level three and just above level three.
The ones above level three I wanted to make sure they don't
fall below level three, so I focused on them. The ones below
level three, I boosted them up level three, and we raised it
then by 20 points. I said what happened to the other kids. He
said they probably did OK.
And I think it is very important that we don't breed
cynicism about accountability, and that is why I think what Dr.
Sanders has done has really enabled us to move it to a
different level.
I agree with Kati. In the end, the solution is not to
simply grow to no end. It is to grow to proficiency, but I
would say even beyond proficiency. Proficiency is a minimalist
standard.
Our nation isn't going to succeed if our kids are not
proficient--many of them--at an entirely higher level. So I
think what you say is right, but I wouldn't discount the power
of what we measure in terms of the behaviors in the school
system like mine.
Mr. Miller. I was just going to say I don't want to be
unfair to my colleagues who are waiting here for questions, so
I will take my answers on the air later, because I am afraid I
am going to run over time, and you had better recognize that.
But I am sorry, Reg. We will talk about this. I don't know
what to do. I took too much time with the question, is the
problem.
Mrs. Biggert [presiding]. Thank you.
The gentleman from Delaware, Mr. Castle, is recognized.
Mr. Castle. Thank you very much.
I think this is an extremely important hearing, which I
have been pushing for, and it is an extremely important but
very difficult subject. I agree with chairman's suggestion that
we have round table discussions at some point.
I don't know if a hearing, 5 minutes of testimony and a few
questions, lends itself to what we need. Plus, this happens to
be an extraordinarily busy morning, which is unfortunate, too.
You know, this is an aside from everything, but in dealing
with education over many years now, I just have learned that
you need sort of a systemic component--and Dr. Shaul is
retiring, and I say this to her now, but you need a systemic
component to all this.
I see so many charismatic type programs and individuals who
do things in education, but it doesn't translate to other
people being able to do it.
I think any time we have a report, any time we have a
suggestion for something, you need to have some sort of a
spelling out of how it could be done systemically and not just
by those individuals who sort of embrace it, or it is what they
do, or how they get their foundations of whatever the heck it
may be. Some of these things just fall when you get into that.
I also worry about what Ms. Haycock said, that we worry too
much about accountability definition and not improvement. I am
just seeing way too much of that. I am from Delaware, and I
have seen just incredible improvement in these scores.
And I have been in those schools, and I have seen what they
are doing, and they are really breaking it down and really
analyzing it, and that is systemic. That is something I think
would translate to other schools.
But I have been in other schools where everybody is
throwing their hands up and saying oh, we can't do this, we
need something else, or whatever it may be, No Child Left
Behind is no good, and that is just not teachers. That is
administrators, even some parents in some cases, even state
officials in some cases, as we have seen with lawsuits or
whatever.
So that concerns me a lot, too. We need to really focus on
getting this job done, and I think it can be done.
I also worry about the different standards and assessments,
and I realize that all 50 states have gone through the
Department of Education, but I still think there is a
tremendous variance between standards and assessments.
And I am not about to suggest national standards and
assessments, but it goes through my mind every now and then. It
would sure straighten out some of those problems that exist out
there. You know, maybe it is worthy of discussion at some
point.
But I think this is important. I think this is the most
important aspect of No Child Left Behind, what we are talking
about today, in terms of what may change in the next
reiteration of this particular legislation.
And I am not suggesting I understand it, and I do
appreciate the study, which is just being issued now, and which
I haven't had a chance to really review myself, so I have a
long ways to go, too.
But my question, if anyone wants to take a stab at it
before the red light goes on, is what are we talking about here
in terms of the--is this an either/or circumstance, or is it a
blended circumstance?
I think, Chancellor Klein, you just made a statement--I
think I got it correct--that the growth model will lead to
schools that don't make AYP. Well, that would mean to me that
it is not either/or. That would lead me to believe that it is
some sort of a blended, you know, situation.
And perhaps it should be. Why shouldn't a school that is
doing very well also be able to show growth as well as the
schools doing poorly, which, by showing growth, can get to AYP?
I am not sure what the answer to that is, but I am
interested in any quick comments you might have about the--you
know, whether we are dealing with an either/or or blend
situation with respect to growth models or the pure testing.
Ms. Haycock. Well, I can tell you, Mr. Castle, that the two
states that got approval for a growth model are, in fact,
blended systems. About each school there are essentially three
questions asked.
First, do they make AYP under the straight status model. If
the answer to that is yes, then they make AYP. If the answer is
no, though, then the question is did you make it under the safe
harbor or sort of improvement model. If the answer to that is
yes, they make AYP. If the answer to that is no, then the
question is do they make it under a growth model.
So in both cases it is essentially a blended system. You
essentially have several options in reauthorization. You could
ask for blended systems, you could allow states to go with a
growth only system, or any combination thereof.
But the two experiments at least that you have under way
now are blended models.
Mr. Castle. Dr. Sanders?
Mr. Sanders. I would just make one additional comment on
that. Based upon all of the analysis that we have done, I would
certainly recommend a blended system.
I would allow, just as Kati has pointed out, that--schools
to apply the existing AYP rules. What I would do is substitute
the projection growth model, if you would, component for the
safe harbor part that is present.
In other words, instead of going the way it is right now,
even for the two states that were approved, this was
essentially a pilot to augment what existed, so you remember
what Chancellor Klein mentioned and what I attempted to
mention.
One of the things that concerns me the most is particularly
schools serving a disproportionate number of poor and minority
kids. That is when those early above-average kids too often are
allowed to slide.
That is, to me, one of the biggest negatives associated
with the present system. And that has not been that--it is not
recognized under the existing system. And that is the sliding
of above-average kids.
So if you substituted a growth component for the existing
safe harbor, that would be my preference.
Mr. Castle. But just in closing, I hope when it comes time
to write all this that you all will be around to help us with
it, because it sounds to me like it is going to be complicated.
I yield back the balance of my time.
Mrs. Biggert. The gentleman yields back.
The gentlelady from California, Ms. Woolsey, is recognized.
Ms. Woolsey. Thank you, Madam Chairwoman.
First of all, I agree with Chancellor Klein. No Child Left
Behind is extremely valuable and important to us, but it does
need to be improved.
What I say is it needs to be fixed, and it can be fixed
without giving up accountability, improving teaching and
learning, and without giving up on high-quality schools and
their teachers.
So I am not going to say any more than that. I am going to
ask two questions. I have done quite a bit of research and
discussion with my local educators and with national education
groups over the last year, and two major questions that
continue to come up fit right in with what we are talking about
today.
One, on growth. The question of can we educate our most
challenged students--English learners, those that are
economically disadvantaged--within a growth model that, over
time, will not leave them behind.
And the second one is--my second question is accountability
based solely on test scores is not very popular in my area.
What would you recommend? What would you recommend that would
include teaching the whole child?
Can we provide growth models that include, yes, math and
reading, of course, but also P.E., history, art, and music, and
all of the other things that kids need to be whole people when
they are finished with school?
So I am going to leave it there and answer as you wish.
Let's start with Dr. Sanders. He looks like he is ready.
Mr. Sanders. I am sorry, I couldn't hear the last point.
Ms. Woolsey. Pardon?
Mr. Sanders. I am sorry, I missed your last comment. I am
sorry.
Ms. Woolsey. Well, the last comment was a growth model that
includes educating the whole child, meaning in addition to math
and reading.
Mr. Sanders. Well, first of all, let me refer to some
comments that Dr. Weaver made earlier. To me, one of the really
huge advantages of states creating the longitudinal data
structure--the accountability part is important, but what is
even more important is the diagnostic information that is now
available.
And when you begin to start looking at projections for
every single kid as an individual to various standards in the
future--if we have got a child that is very low achieving, the
first standard is to get that kid--the first objective is to
get that kid on a trajectory to meet the proficiency standards
1 year or 2 years in the future.
But once we have got a kid like that, then let's talk about
getting that kid on a trajectory to meet high school graduation
requirements. Once we have got kids on that trajectory, let's
talk about getting those kids on trajectories to have more
opportunities for more college majors.
So in other words, once you begin to create the
longitudinal data structure, you have got the opportunity to
begin to start focusing and planning for the needs for a
diversity of students.
Now, we worked very closely with the Tennessee Department
of Education in preparing their proposal that was--and we are
very proud of that, that was approved to be one of the two
states.
Now, back to your thing of all children and all subgroups,
in that proposal, all subgroups with regard to their
projections have got to meet it or they don't get credit from
this alternative approach--ESL kids, et cetera, et cetera, et
cetera.
Now, I will make one comment on ESL kids. We have had the
opportunity to look at a lot of data, a lot of years.
If you are a principal or a classroom teacher, looking at
it from a value-added or growth project, you often want those
kids in your classroom because as they accumulate more language
skills, often their gain rates is higher, not lower. So
consequently, the data strongly suggest that that is not a
problem.
Now, for kids that are significantly learning-impaired,
what we believe and was included in that proposal--those
students take an alternative assessment. You include that data
in the overall computations, but you do that relative to the
appropriate assessment for their needs.
So consequently that has got to be incorporated in total.
But it can all be done within the overriding spirit of No Child
Left Behind.
Ms. Woolsey. Thank you.
Dr. Weaver?
Mr. Weaver. I cannot tell you how pleased I am to be a part
of this discussion. I cannot tell you how pleased I am.
But you know what I want to do? I want to speak to you as a
parent, and many parents with whom I have spoken with. And I
also want to speak to you as a teacher representing many with
whom I have spoken with.
And I also want to speak to you as an African-American,
many of whom I speak with consistently and whose children are
not receiving what they consider to be the kind of education
that they want.
And when I talk with them and I tell them that I was a part
of this panel, and we were talking about growth models, you
know what they are going to say? They are going to look at me--
what is that? I could care less, because how is that going to
impact what happens to my kid.
I want my kid to be able to have the opportunity to have
science and math and technology. I want my kid to be able to
have a qualified and certified teacher. I want my kid to be
able to go to a school that is safe and orderly.
I want my kid to be able to have counselors. I want my kid
to be able to have smaller class sizes, because I know that
that is what it is going to take for my kid to be successful.
And the growth model--I think it can lead to that, but the
parents don't understand that. And when we go out and talk
about well, we think that we need to have growth model added to
No Child Left Behind, we understand that, but how do we get the
parents to understand that?
How do we get the average teacher, administrator ad other
educator to understand that? So I am saying a lot of times we
talk to ourselves in a code that the parents don't understand,
the public doesn't understand, and as a result it is difficult
to get their support.
I go to these parents and I talk about--and teachers--well,
I believe that no one test should be used to determine the
future of a child. I believe that there needs to be multiple
assessments, such as college entrance, such as retention--
graduation rates, such as portfolios.
Those are the kinds of things that I do believe that we can
begin to utilize in addition to having the test. A test should
be a unit to measure, not the unit.
And also, I believe that we need to look at how people view
outputs and inputs. When we think about outputs, we are talking
about a test score as if the test score is the only unit to
determine whether or not a kid is successful.
And we get focused on the output which is the test score,
and we forget about the input, input such as class size,
qualified and certified teachers, safe and orderly environment,
parental involvement, because I believe that the input helps to
determine the output.
And so again, I am speaking for those individuals who are
not here to speak for themselves.
Mrs. Biggert. The gentlelady's time has expired.
You might have noticed that there is rather a dearth of
committee members over on this side of the aisle, and I know
that they would really like to be here, but there was an
important meeting called. I just wanted the witnesses to know
that.
I will recognize myself for 5 minutes.
Dr. Shaul, in your GAO report you mention that 26 states
were using growth models and another 22 states were either
considering using them or in the process of implementing them,
even though they are not under the--they are not part of the
pilot but just on their own.
Do you believe that all of these states are in a position
to implement the growth models that comply with the goals of No
Child Left Behind and the guidelines set by the Department of
Education in their pilot program?
Ms. Shaul. Not all the states are in a position to be able
to implement the department's growth model pilot, which set out
some very high standards, seven core principles that models
needed to meet, that used individual student data.
Of the states that we looked at, seven of the states had
models that were based on individual student data. The rest
were using school-level data. So they would not have been
eligible.
As you know, 20 states chose to apply. Only eight states
were peer-reviewed as being close enough to actually having
systems in place that would meet the department's goals.
Mrs. Biggert. When you look at these growth models and we
are looking at the proficiency and having to reach 100 percent
by 2013--and in the growth models, if they don't go up, you
know, at a--in other words, a line that goes up instead of
going like this at the end, then it is going to have to go like
this.
Do you think that this will work?
Ms. Shaul. I think that states can design growth models
that set a trajectory that will allow all students to be
proficient by 2014. But for some students, those trajectories
will be pretty high if they are starting at lower levels.
But I think it is possible. But then, of course, setting
the accountability systems--it goes back to points other
panelists were making. Then you would have to have tailored
instruction, the other things in place that will allow those
students to reach those high standards.
Mrs. Biggert. Thank you.
And I will yield back.
Mr. Van Hollen from Maryland is recognized for 5 minutes.
Mr. Van Hollen. Thank you. Thank you very much, Madam
Chairman.
And let me thank all of you for your testimony. As others
have said, I think this is a very valuable hearing for all of
us, and I think we are approaching this in a bipartisan spirit
at this point as to what is best for the kids.
And I think that all of you have made the case that the
current system of AYP does create some of these perverse
incentives in the program. You talked about the kids between--
you know, on the bubble between 2.9 and 3.1, and a lot of
emphasis and focus is put on them to the exclusion of people
who are way below and probably won't make it up to that
standard, or people who are above that standard.
Now, clearly, I think that we need to eliminate those kind
of perverse incentives. And I think that we need to come up
with something different. In doing that, we need to make sure
we don't replace it with another standard that creates
different perverse incentives.
And Mr. Castle said he hopes you are all here while we do
that. I do, too. But I also hope that that doesn't mean we come
up with a Rube Goldberg-type machine with, you know, three
different things that no one can understand and does also
substitute a system for this one and creates other perverse
incentives.
And that is why the quality--and the quality of the data
and the ability of these school systems to assemble the data
necessary for the growth model is something that concerns me
greatly.
As you just said, Dr. Shaul, I mean, you looked at about 25
states or whatever the number is that are currently using it
somewhat--most for their own purposes, and whether they are in
a position to even begin to implement a growth model.
And it seems to me that it is clear that we are not there
yet. And if we really want to move them there, we are going to
have to put some resources and some Federal Government money
and some guidelines and standards for how we move in this
direction. And let me just ask for the starting point.
And, Chancellor Klein, you mentioned the difference between
spotlight and hammer. And whatever system we come up with in
terms of what the standard is for measuring AYP, it seems to me
collecting some of this data is important for the purposes of
spotlight anyway.
Let's put aside the consequences, which can create these
perverse--would you all agree that it makes sense to put all
these school systems in a position where, whether or not we go
to a full growth model, that we have the data necessary to
determine whether progress is being made?
Would you all agree that we should at the very least do
that?
Mr. Klein. Absolutely. And it is not just the school
system. It is the people who are teaching our children who need
the information, because if you get the test score at the end
of the year, almost invariably that student is in another class
at another time.
And what happens is you don't have the information to
create the kind of positive feedback loop. Those schools that
are using effective assessment strategies are able to intervene
much more quickly on the ground at the time, and that is one of
the key things.
And I admit this is a huge technological and data
challenge, but if you don't address this challenge, it is like
that old Thorndike experiment at Yale. You can throw darts at a
target forever. If you are blindfolded, you are not going to
get any better at it.
Now, we need the information to help the people who are
teaching our kids do the work that they need to do to make the
adjustments, because one thing that worries me enormously about
assessing systems is you can always have different assessing
systems, but in my school system when I got there, thousands
and thousands of kids were in high school, unable to read.
Now, let me tell you something. That is because the system
was unprepared to be serious about what it is to educate a kid.
And we just passed them along to inevitable failure.
So to me, strong benchmarks along the way are one aspect--
and then rich data to the people teaching our kids, so that
they can actually do the transformative work with the kids.
Mr. Van Hollen. Right. At some point--and time is running
out, and I have another question--but it seems to me we need to
get some kind of idea about what resources would be required to
bring these school systems up to a point where they can have
the--I mean, because bad data is not--bad results.
And perverse incentives to gather certain kinds of data is
also a problem.
Let me just as you, Dr. Sanders--what I am trying to--and I
commend you for what you have done in Tennessee. As I
understand the system, under the waiver that you have got, you
can achieve AYP in one of three different ways, is that right?
Mr. Sanders. That is correct.
Mr. Van Hollen. And is that the kind of model you want to--
I guess my concern at making that national is that it would
allow a school system to game which of the three it is going to
go for.
I am not saying that is happening in Tennessee, but just as
a national model--in other words, we talked about the problems
with the more static--the sort of status quo model, what we are
using now.
But as I understand it, in Tennessee, you could hit AYP
using that model. And if you miss AYP using that model, you can
use the other model. But as you very aptly pointed out, there
are schools that--your slide and glide, or whatever, rule.
I mean, there are schools that can make it under the
current rules that would not be making it under the growth
model. And if you allow sort of the school system to target
one, two or three, you may get a situation where they go for
the strategy to go for the one they are making now, but they
would miss it under another.
And it seems to me you don't want that kind of gamesmanship
going on.
Mr. Sanders. Working with the Tennessee Department of
Education, for the proposal to be considered for the pilots, we
perceived the guidelines to be to talk about an augmentation of
the existing process.
What I attempted to do in my previous remarks was basically
say if I were doing this from scratch, I would leave the
existing AYP in place but I would change the safe harbor part
to essentially be the growth projection model.
And then I want to very briefly say I didn't--it is in my
written statements but not here. I want the committee to
recognize that all of these growth models are not the same and
do not yield the same rigor and the same properties.
And so consequently some of these more simplistic
approaches to measures of growth I think should be seriously
frowned on, like, for example, merely subtracting last year's
score from this year's score, et cetera, because those things
are extremely unstable.
And everybody in the room would agree that in schools
serving high concentrations of low-scoring kids, you have got
more missing scores. And so any of the--I don't want to spin
off on a whole bunch of quantitative mumbo-jumbo here, but any
of these growth models--and this is where I really appreciate
what the peer team--you folks need to look very carefully at
what the peer reviewers said about that.
So if a state is proposing this, there should be some
minimal quantitative standards placed. But your specific
question, sir--I would blend it. I would leave exactly what you
have got. Then if they want to go under safe harbor, I would
definitely go to that.
And then finally, the other comment--I would not hold all
states to the same capacity as the minimum state. In other
words, states that have invested and were further down the road
with regard to capacity to do this, you should build in the
legislation flexibility where those states could take advantage
of it.
In other words, don't wait till all states have this
capacity. Allow the state--and actually create an incentive for
the states, because as Dr. Weaver says, the big--actually, the
byproducts could be greater than the product.
The byproducts is this wealth of diagnostic information
that is going to flow to principals and teachers and so forth
relative to the progress and the trajectories of all these kids
as individuals.
Mr. Van Hollen. Thank you.
Mrs. Biggert. The gentleman's time has expired.
The gentlewoman from California, Ms. Davis, is recognized
for 5 minutes.
Mrs. Davis of California. Thank you. Thank you, Madam
Chairwoman.
And thank you to all of you for being here. I wanted to
raise an issue that we haven't talked about this morning that I
raised when we were discussing the end sizes a while back and
how we structure or how we count children in specific
subgroups.
And that is the whole idea of AYP and whether, in fact, we
have a number of students who aren't counted because they enter
the school after the yearly progress counts have been
designated. And I don't know across the country the extent of
that problem.
And we know that it is not that kids who move around from
school to school aren't counted, but when it comes to AYP, in
fact, they are not necessarily included in that data. So how do
we get at that issue in the growth models? And how can we begin
to think about that?
You all have talked about the importance of, you know,
intervening on the ground. It worries me that we have a number
of students who, in fact, for a number of reasons do move
around. Some school districts have made great accommodations so
kids can move in and out within clusters fairly easily. But the
reality is I think we are missing these kids.
Dr. Sanders, you are smiling. Could you comment on that?
And how do we use the growth models to support that so that we
do have the data, we do understand what impact that has on
them?
Mr. Sanders. Let me attempt to answer your question, first
of all, from a statistical theory methodological point of view.
If I had the data to know what percentage of time this child
was in that school, then with this longitudinal record you
would certainly have a way to apportion a part of that kid's
time according to the amount of time that the kid was in this
school compared to another school, et cetera.
We probably have constructed more longitudinal data
structures to date than anybody. We presently have the data
from at least one district from over 20 states. Presently, I
have yet to see any data system that would have that kind of
information that would enable one to go so practically, even
though the theory is there.
The analytical capacity is there. The data structures are
not there to know that little Suzie was in School A for 38 days
and in School B for 61 days, et cetera, et cetera.
That is a concern. But it is something that I think, as
data structures mature over time--there are kids falling
through the cracks. There I don't think is any question about
that, particularly kids that move around.
Now, in our value-added modeling efforts, where we are
doing the summative measures--this is in my written remarks--
you will see this is one of the very points that I am making.
Any of these structures have got to be sophisticated--any
of these accountability systems have got to be sophisticated
enough to utilize the data for all kids, not just for the kids
that are not moving around.
So I am attempting to answer your question from a
theoretical conceptual point of view, and then move to the
practical reality under No Child Left Behind with regard to
that.
Right now, I have yet to see any of the data structures
that will enable you to have those sorts of things that would
enable you to do that. The percentage of the kids that are
falling through the cracks I think perhaps is lower than some
people would guess it to be, because we can do things like--we
know how many kids in a large urban district took the test
somewhere.
Then we know how many kids that we had previous scores on
somewhere. And then we can--in fact, that was one of the
questions the peer review team had. How does those ratios of
kids with prior data compare across their socioeconomic groups?
There is a difference. Minority kids, ESL kids, will have a
lower ratio, but it is not as big as some would have guessed it
to be. I think Congressman Miller's comment earlier--sometimes
the things that we have heard merely being horror stories, when
you really go in and look at the data, are not as big as
sometimes people think it is.
Ms. Haycock. Ms. Davis, if I could just say so, I hope you
will stick with this issue. While it may not be our topic here
today, it is a very important question that you are raising.
As you know, in an effort to be sort of fair to schools,
your decision last go 'round was to take kids who were not in
the school for a full academic year out of the equation, put
them into the district, so somebody is responsible, but as you
know, if it is not really the school, then that doesn't work
very well.
And yes, in fact, that may feel fairer to schools, but we
have got to really ask the question, given the amount of
mobility there is in many communities, like certainly yours,
whether it is fair to kids.
And we have got a number of places, like Kentucky, that
actually had always had all kids in the accountability system
and were essentially forced to take them out in order to become
compliant with NCLB. And there is a lot of dissatisfaction in
Kentucky with that right now.
So coming back to that issue in reauthorization and ask the
question what is fair to both schools and kids, and how do we
really make sure that the mobile kids get the attention they
need to advance, because there is too many mobile kids to shut
out to the side.
Mrs. Biggert. The gentlelady's time has----
Mrs. Davis of California. Thank you. My time is up.
Mrs. Biggert [continuing]. Expired.
Mrs. Davis of California. Can I get Dr. Weaver to make a
very brief comment? Would that acceptable?
Mrs. Biggert. Well, no.
Mrs. Davis of California. No? OK.
Mrs. Biggert. I think that we have got so many members
waiting. We will have votes at noon, so we would like to try
and finish by then and make sure everybody can have an
opportunity. Otherwise we could come back.
But the gentleman from New York, Mr. Owens, is recognized.
Mr. Owens. I want to begin by thanking Mr. Klein for his
kind remarks.
And I also want to salute my friend Reggie Weaver. He heads
a collective bargaining unit that doesn't represent my city,
but I regret that very much. He is very outspoken on the
national macro issues in education as well as the micro issues
that concern his members, and I salute him for that.
This panel has been very informative, starting with
Director Shaul. I have got a good idea how to easily clearly
explain this to parents from your presentation.
I also very much appreciate the remarks of Dr. Sanders.
Standard school efforts--I mean sustained school efforts can
trump socioeconomic influences. I think you said something like
that.
I think that links with something that Reggie Weaver said,
qualified, diverse and stable staff are very important as part
of this process of achieving the goals that we have set.
You have been very disciplined, all of you, in keeping the
discussion within the context of what I call a grand
accountability hypocrisy that No Child Left Behind forces us
all to operate within, or at least a grand incomplete
accountability system, because it holds the teachers and the
children accountable, but it does not hold the system--the
decisionmakers who promulgate the budgets are not held
accountable.
And just a quick piece of history. Ronald Reagan and the
Governors started the whole business of let's have national
standards. Under Bush we codified it--national standards for
curriculum, national standards--voluntary, by the way--national
standards for curriculum, national standards for testing.
This committee, Democrats on this committee, fought very
hard to add another set of national standards, national
standards for opportunities to learn under leadership of now
Senator Jack Reed and the late Patsy Meek--they were the great
troublemakers. I just held their coats.
We went up against the Clinton administration and Secretary
Riley because the Governors, both Democratic and Republican,
didn't want national standards for accountability because
national standards--national standards for opportunity to
learn.
National standards for opportunity to learn said you know,
as you--before you measure how well the kids and the teachers
are doing in their schools, measure what you are doing to
provide them with the resources that they need.
Are you doing the things that will allow us to have
qualified teachers, you know, libraries that are sufficient,
school laboratories, et cetera?
I don't think you have very many physics teachers in your
physics classes in high school. Last time I looked there were
few actual physics teachers who majored in physics teaching
physics in high school.
In the junior high schools, we don't even have laboratories
in most of our junior high school science departments in New
York.
I want to say that we cannot continue to run away from
opportunity to learn standards as being a part of the
accountability model. We have to have--address this issue, not
just focus on one part of it. That is qualified teachers.
We do deal with that sort of as a footnote in No Child Left
Behind. There is money in there that got reduced first when
they started making cuts. But the teachers are bitter. The
administrators are bitter. I am sure New York City is not the
only place where they are bitter, because they think they are
being forced into a system where they are not provided with
appropriate resources.
They need the books. They need the libraries. They need
modern technology. So we need to zero in on, let's just say,
qualified, diverse and stable staff.
How can we accomplish that within the context of the
present system, where not enough money is going to be made
available, say, for New York City, which operates in
competition with the surrounding suburbs and the rest of the
nation?
We train qualified teachers and they go off to higher
salaries and higher opportunities.
I want to focus on you, Mr. Chancellor, and say is there
one--is there a possible way that we can combat this loss of
qualified, diverse and stable staff? Because I don't think much
is going to happen in improving these models.
You know, I think Secretary Spellings pointed out that New
York City is one of the places where we have still a great
problem of certified teachers--still a problem. And the least
certified teachers, non-certified teachers, and the substitutes
are concentrated in low-income communities.
Community leaders told you that 40 years ago. Nothing has
changed. Or maybe something has changed. I hope it has. But
basically, that is a truth that you have to leave with. Without
the qualified teachers, we are not going to make the progress.
Growth model, whatever you say--it is not going to happen.
So, Chancellor, do you have any plans--I know that your
administration talks about a conglomerate system willing to
experiment, diversify. In order to hold teachers, I would take
any experiment--I won't take but one experiment off the table.
We don't want vouchers.
But charter schools----
Mrs. Biggert. I am sorry.
Mr. Owens [continuing]. Non-profit schools--I don't talk
fast enough.
Mrs. Biggert. No, you didn't. I am sorry. Your time has
expired.
Mr. Owens. I will talk to you later about combining some
kind of model to hold the teachers----
Mrs. Biggert. The gentleman from Texas is recognized.
Mr. Owens [continuing]. Have a stake--have them have a
stake in what happens in the community.
Mrs. Biggert. The gentleman from Texas is recognized, Mr.
Hinojosa.
Mr. Hinojosa. Thank you, Madam Chair.
I want to say that it is--this is one of the best panels I
have ever seen come before Congress to address one of the
concerns that I have in education.
And seeing the audience, the size of the audience, the
makeup of the audience, pleases me very much, because it shows
me that there are people out there wanting as much as I do to
hear from experts like you on how we should address a problem
that was uncovered yesterday by the Government Accountability
Office.
And I want to thank Marlene Shaul for you and your staff
responding to our letter asking to give us the accountability
of how No Child Left Behind was affecting our minority
students.
And I say that I have lots of concerns about education for
the last three or four decades that I have been involved as a
policymaker. But I want to say that that top concern is the
persistently low high school graduation rate for minority
students.
And I like all the stories that each one of the panelists
have given.
And, Reg, you certainly did captivate those of us here as
congressmen and congresswomen when you told the story about
Cesar, because that reminded me of another young man who came
from a math and science academy where more than 40 percent of
the makeup of that student population are on the free lunch
program. Eighty percent are Hispanic.
And we have a young man who graduated from that school and
went on to graduate from the school of engineering at the
University of Texas-Pan American at Edinburgh at the age of 18.
Unheard of.
But the point is there are many limited English proficient
students of low-income families who have risen and been able to
function in our system.
But what I hear from all of you and this gentleman, Mr.
Sanders, as a statistician, is that we seem to have testing
that is like a square peg being forced through a round hole,
and it doesn't fit.
And those tests obviously do not measure the progress that
the children have made. And there are many schools throughout
the country like each one of you has pointed out that are
showing that something is working with many of the minority
students.
And it just seems like we in Congress don't have the
political will to really invest the amount of money that it
takes to be able to reach such large populations and to be able
to bring the parental involvement that is one of the secrets of
those schools that you all mentioned.
Every single one of those schools, like mine in south
Texas--there is parental involvement at a much higher level
than in most of the other schools. So how do you do that? How
do you get the certified teachers and all of that?
So, Reggie, I want to start with you. Where does real
accountability for high school graduation rates fit into the
growth model of accountability?
Mr. Weaver. Well, I think the accountability fits, again,
when you give the people the opportunity to participate in
helping to shape the circumstances that impact their work
environment.
As it relates to the point that you made about dropouts, it
is absolutely critical that we have some kind of legislation or
something that speaks to high school dropouts, that speaks to
literacy coaches, that enable us to be able to have a better
understanding with what these young people need in order to be
successful.
If, in fact, you really want to have the teachers to remain
in the profession and to come into the profession, then I
believe that you need to make sure that they have the respect
that they deserve, that they have the opportunity to
participate in the decisionmaking process, that you pay them,
that they are involved in an atmosphere that is conducive to
good teaching and learning. The schools have to be safe.
And so these are the kinds of things that I do know will
attract and keep the kind of people that we want in our
profession to be able to work with these young people.
Mr. Hinojosa. Kati, I am sure that being here in
Washington, D.C., you must have participated and heard about
the National Association of Governors meeting here in
Washington on one issue, and that was the high school dropout
rate, addressing it, and they brought in Margaret Spellings and
all the experts we could bring, and we addressed it.
So the question to you is why do you think that Congress
has not been able to address the need for the resources that
are needed after seeing that 50 Governors came, all concerned
about this problem, and yet a year has gone by and----
Mrs. Biggert. The gentleman's time from Texas has expired.
Mr. Hinojosa. Thank you, Madam Chair.
Mrs. Biggert. The gentleman from Michigan, Mr. Kildee, is
recognized.
Mr. Kildee. Thank you very much, Madam Chair.
First, I apologize for being tardy for the meeting. I was
down at the White House for the signing of the Voting Rights
Act, another very important milestone in our country's history.
I have seen most of you, know most of you.
Reg, I see you all over the country, including my home town
of Flint. Good to see you again.
You know, the chairman made a comment that it would be
great to have you in a better forum, a better setting with us,
because we really--as has been mentioned, you are the most
knowledgeable, helpful panelists that we have had.
And it would be kind of nice if we had you as adjunct
members of this committee as we work our way through the
reauthorization, because I think you would be very, very
helpful.
Let me ask you this question. Under some growth model
proposals, a school could have declining scores in one group,
say economically disadvantaged group, and still make AYP. In
our current system of disaggregated data, tell us this.
How can we implement growth models and assure that the
additional resources triggered by our current AYP system are
still made available to that subgroup that is struggling? And
if I may add another question to that, how would a blended
system take care of the needs of that subgroup?
Dr. Sanders?
Mr. Sanders. When you--work with the Tennessee Department
of Education in the proposal is that first of all, you start
looking at all of the students as individuals. You don't pay
any attention to subgroups first.
You start looking at all the kids as individuals. Then
after you have done that--of whether or not all these
individuals are on trajectories or not, then you aggregate it
into all of the existing subgroups required under No Child Left
Behind.
And for a school, then, to get credit for having passed AYP
with this projection approach, all subgroups have got to meet
it. So consequently, there will be no reason why that if a
school is not hitting the trajectory for its ESL kids or its
free and reduced-price lunch children, et cetera--if that
subgroup doesn't meet it, they don't pass AYP with regard to
the projection approach.
Ms. Haycock. Let me, if I could, add one more point to
that. And I very much agree with Bill's response. Under the
current system, schools essentially make AYP or they don't. And
you don't really have really good information on the sort of
gradations of problem below that.
If you add a growth component to that, you can distinguish
between two schools where--that didn't meet the status bar but
one of them just barely didn't meet it, and the kids are on a
trajectory that is quite clear within the next couple of years
they are going to be proficient.
That school doesn't need nearly the extra intervention and
resources that the school that is below that but not on a
trajectory needs, and what the addition of the growth
information does is allow you to make essentially more nuanced
choices, decisions both about schools and how to categorize
them, but also about what kinds of interventions are necessary.
Mr. Kildee. Back again to the blended system, how would a
blended system take care of the needs of those subgroups that
may have special needs?
Ms. Haycock. Bill is exactly right. In the blended system,
the test is still both an overall test and a test for each
subgroup, so you must make either the status, the safe harbor
or the growth for every single subgroup.
So there is still very clear subgroup accountability. That
doesn't change at all.
Mr. Kildee. Well, thank you very much. This is a very
helpful group. I know you will be helpful as we begin the
process of reauthorizing this bill. Thank you very much.
Mrs. Biggert. Thank you.
The gentleman from Virginia, Mr. Scott, is recognized.
Mr. Scott. Thank you.
And I want to add my voice to the comments you have heard
about this panel as being one of the most helpful we have had
on a difficult subject.
We have heard about the perverse incentives that occur when
you don't align the rewards with the behavior that you want.
And, Dr. Sanders, you mentioned focusing just on the bubble
and ignoring everybody else. And that same perverse incentives
works around dropouts. There is a perverse incentive not to
reach after people who have dropped out.
It was my understanding that when the bill passed we put a
provision in there that punished you for a high dropout rate.
That was the intent of the provision. Do you know what has
happened to that? Anybody?
Ms. Haycock. I do.
Mr. Scott. OK. Thank you.
Ms. Haycock. Yes. In the case of high school
accountability, you actually included a requirement that there
be a second measure beyond performance on the assessment and
that that measure be 4-year high school completion rates.
However, by contrast to what you did on the achievement
side, where you actually required the bar to be set at certain
levels and to grow over time, you made no such requirement for
the dropout level. You essentially left to states the
definition of what is adequate progress in reducing dropouts.
And the truth of the matter is that most states decided
that not falling backwards very far was good enough.
Mr. Scott. Well, that is an area that we have to,
obviously, deal with to get that perverse incentive back in
line.
There is another problem we have had with special ed
students. If someone qualifies for special education, obviously
they are going to have a harder time achieving, and we want to
align good teaching with reality.
Unfortunately, if you do it too well, there is a perverse
incentive to over-identify people as special ed. You have got
some students that aren't doing so hot, you just call them
special ed and they all of a sudden don't have to achieve.
How can we avoid the incentive to over-identify and give a
reasonable measure of how to measure special ed students?
Ms. Shaul. The flexibilities the department provided only
allow a certain percentage of students who have disabilities to
be counted or excluded from the calculations for AYP.
Mr. Scott. So if you over-identify, you still have to--only
a certain percentage get excluded?
Ms. Shaul. The students, for example--with the severely
cognitively disabled students, up to 1 percent of those--those
students--when they take those tests, they can be--1 percent of
those students can be excluded.
But they can be included if they pass the test. One percent
can be excluded. But if there are more students who do not pass
the test, they are still included in the overall count.
So for that subgroup, they would not count as passing.
Mr. Scott. The growth model helps if you are having a--if
you have a high school and the middle schools from which you
receive students are failing schools, you have a tougher job
than if the students you receive were not failing.
And yet the incentive there is to try to talk the school
board into redistricting, so you get some better middle
schools. If you don't have a growth model, how do you deal with
that? Because if you receive all failing students and get half
of them to pass, that is better than somebody who just
maintains the status quo.
Without a growth model--would the growth model help deal
with that problem?
Mr. Weaver. I don't know whether a growth model--I would
think that it would, but I--you know, you mentioned special
education, and then there was mention of the dropout.
I believe that what we need are programs that help these
individual students to feel, No. 1, that they can be successful
and that that leads to them staying in school. It leads to them
believing that they can be successful and attain a college
degree or a trade degree.
And as we talk about growth models, I believe that I want
to inject in here something that I think is crucial, and that
is pre-K, early childhood education, because if we get them
earlier--the earlier we get them, the much better chance we
have of----
Mr. Scott. Well, let me get to that again in a quick
question. You mentioned input and output. Can you see how that
works from--are we moving the achievement gap? If you have got
a gap, don't you have to improve the input on the students for
whom--who are victims of the gap?
Mr. Weaver. Well, see, the input to me means things that
are important to achieving the--closing the gap--qualified and
certified teachers, class size, adequate and equitable funding,
safe and orderly schools, parental involvement.
Those are the kinds of inputs that I believe are important
to help the----
Mr. Scott. And if you have an achievement gap you would
have to emphasize those inputs for that subgroup.
Mr. Weaver. I believe you need to emphasize that regardless
of what the population of students are. The question is will
the key policymakers have the commitment to make it happen.
Mrs. Biggert. Does the gentleman yield back?
Mr. Miller. Just on that point, it would seem to me that if
the growth model is constructed correctly, and assuming a
decent level of resources, that the growth model should in some
ways equate expanded opportunities for those students who are
not on that trajectory that you want.
I mean, you know, we--under the schools in need of
improvement, theoretically additional resources would flow. In
some cases, it has. In some cases, it hasn't.
But in theory, if you had that information, and the earlier
you had that information and that trajectory, you should be
able to make an intervention, so that information in itself
will help those teachers, that school principal and that
district understand the deployment of those resources.
I am assuming that you would get additional resources, but
even with existing resources you may make a more efficient use
of those resources than you otherwise would when you didn't
have that information.
Mr. Klein. Absolutely. But it is important to understand
that--I think everyone said it, but I want to underscore it. It
means that each kid has real value in the accountability
system, and that there is no reason for every kid--special ed
kid, and language proficient kid--there is no reason for every
kid not to be moving forward in a steady, coherent way.
And the theory of a growth model is to say now, if you are
not moving those kids forward, whatever the resource
constraints--there is not a person who doesn't want to see more
resources.
But whatever the constraints are, you are able to then both
target the resources in terms of differentiated strategies and
also to look, frankly, in terms of the professional development
of your workforce, so that you can make sure--why are these
kids who started in the same place in one class--they moved
forward a full grade.
These kids who started in the same place in another class
did not. What is that about? And you need to be able to address
that day in and day out in the work we do, because--and that is
what is wrong with NCLB right now.
I have got kids in so-called high-performance schools who
are not moving forward. And nobody is going to bother them
because they meet AYP and everyone sees them as high
performing.
So if you want the kind of precision you are looking for,
Mr. Miller, then I think you need a heavy dose of growth.
And I want to end this way. Everybody wants to see
proficiency. But if you look at New York City public school
system right now, we have vast numbers of kids, and we have
had, for as far back as anybody can remember, vast numbers of
kids who are not close to proficient.
And we have got to devise a system that gets them there
over the time that they are in the school system. And we can
pretend we have a national standard that says they are all
going to get there on a certain date.
But the truth of the matter is if they don't get there
incrementally, year by year, they are not getting there. And we
can deal with that issue in a multitude of ways--changing the
proficiency standard, all sorts of other goofy things.
Or we can get serious about the fact that the dimensions of
the problem require each and every kid growing all of the time
in the 12 years, not kids who are in the 12th grade in New York
City who can't read. Those kids didn't grow, period, in the
time they were in the school system.
Mrs. Biggert. And that would be the last word. Thank you.
I remind members that they have 14 days to submit questions
for written answers from the witnesses.
I would like to thank the witnesses for their valuable time
and testimony, and both the witnesses and members for their
participation.
If there is no further business, the committee stands
adjourned.
[Whereupon, at 12:01 p.m., the committee was adjourned.]