No Child Left Behind Act: States Face Challenges Measuring
Academic Growth (27-JUL-06, GAO-06-948T).
The No Child Left Behind Act (NCLBA) requires that states improve
academic performance so that all students reach proficiency in
reading and mathematics by 2014 and that achievement gaps close
among student groups. States set annual proficiency targets using
an approach known as a status model, which calculates test scores
1 year at a time. Some states have interest in using growth
models that measure changes in test scores over time to determine
if schools are meeting proficiency targets. The Chairman of the
Committee on Education and the Workforce asked GAO to testify on
its recent report on measuring academic growth. Specifically,
this testimony discusses (1) how many states are using growth
models and for what purposes, (2) how growth models can measure
progress toward achieving key NCLBA goals, and (3) what
challenges states face in using growth models especially to meet
the law's key goals. While growth models may be defined as
tracking the same students over time, GAO used a definition that
also included tracking the performance of schools and groups of
students. In comments on the report, Education said that this
definition could be confusing. GAO used this definition of growth
to reflect the variety of approaches states were taking.
-------------------------Indexing Terms-------------------------
REPORTNUM: GAO-06-948T
ACCNO: A57539
TITLE: No Child Left Behind Act: States Face Challenges
Measuring Academic Growth
DATE: 07/27/2006
SUBJECT: Academic achievement
Educational standards
Educational testing
Elementary school students
Elementary schools
Federal/state relations
Performance measures
Secondary school students
Secondary schools
Strategic planning
Pilot programs
******************************************************************
** This file contains an ASCII representation of the text of a **
** GAO Product. **
** **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced. Tables are included, but **
** may not resemble those in the printed version. **
** **
** Please see the PDF (Portable Document Format) file, when **
** available, for a complete electronic file of the printed **
** document's contents. **
** **
******************************************************************
GAO-06-948T
* Summary
* Background
* Nearly All States Reported Using or Considering Growth Model
* Certain Growth Models Can Measure Progress toward Key NCLBA
* States Face Challenges in Implementing Growth Models
* Conclusion
* GAO Contact and Staff Acknowledgments
* GAO's Mission
* Obtaining Copies of GAO Reports and Testimony
* Order by Mail or Phone
* To Report Fraud, Waste, and Abuse in Federal Programs
* Congressional Relations
* Public Affairs
Testimony
Before the House Committee on Education and the Workforce
United States Government Accountability Office
GAO
For Release on Delivery Expected at 10:00 a.m. EDT
Thursday, July 27, 2006
NO CHILD LEFT BEHINDACT
States Face Challenges Measuring Academic Growth
Statement of Marnie S. Shaul, Director, Education, Workforce, and Income
Security Issues
GAO-06-948T
Mr. Chairman and Members of the Committee:
I am pleased to be here today to discuss our report, which describes how
states use growth models to measure academic performance and how these
models can measure progress toward achieving key goals of the No Child
Left Behind Act of 2001 (NCLBA). With annual expenditures approaching $13
billion dollars for Title I alone, NCLBA represents the federal
government's single largest investment in the education of the 48 million
students who attend public schools. The NCLBA-the most recent
reauthorization of the Elementary and Secondary Education Act of
1965-requires states to improve academic performance so that all students
are proficient by 2014 and achievement gaps among groups such as
economically disadvantaged students close. The upcoming reauthorization of
the law presents an opportunity to discuss some key issues associated with
the act.
To measure whether schools are making adequate yearly progress (AYP)
toward having all students proficient by 2014, states set annual
proficiency targets using an approach known as a status model, which
calculates test scores 1 year at a time. With status models, states or
districts determine whether schools make AYP based on performance for the
year while generally not taking into account how much better or worse the
school did than during the previous year. Thus, a school that is showing
significant improvement in student achievement but has too few students at
the proficient level would not likely make AYP.
In addition to determining whether schools meet proficiency targets, some
states have interest in also recognizing schools that make progress toward
NCLBA goals. Growth models can measure progress in achievement or
proficiency over time and vary in complexity, such as calculating annual
progress in a school's average test scores from year to year; estimating
test score progress while taking into account how factors such as student
background may affect such progress; or projecting future scores based on
current and prior years' results. While growth models are sometimes
defined as tracking the same students over time, because of the
committee's interest in the range of models states are using to measure
academic improvement, we define a growth model as a model that measures
changes in proficiency levels or test scores of a student, group, grade,
school, or district for 2 or more years. We included models that track
schools and student groups in order to provide a broad assessment of
options that may be available to states.
My testimony today will focus on how growth models may provide useful
information on academic performance. Specifically, I will discuss (1) how
many states are using growth models and for what purposes, (2) how growth
models can measure progress toward achieving key NCLBA goals, and (3) what
challenges states face in using growth models especially to meet the law's
key goals.
My written statement is drawn from our recent report on growth models,
which we completed for the committee.1 For this report, we conducted a
survey of all states to determine whether they were using growth models.
We conducted telephone interviews with state and local education agency
officials in eight states that collectively use a variety of growth
models, and conducted site visits to California, Massachusetts, North
Carolina and Tennessee. For Massachusetts and Tennessee we analyzed
student-level data from selected schools to illustrate how their models
measure progress toward key NCLBA goals. We conducted this work in
accordance with generally accepted government auditing standards.
Summary
In summary, nearly all states were using or considering growth models for
a variety of purposes in addition to their status models as of March 2006.
Twenty-six states were using growth models, and another 22 were either
considering or in the process of implementing them. Most states that used
growth models did so for schools as a whole and for particular groups of
students, and 7 also measured growth for individual students. For example,
Massachusetts measured growth for schools and groups of students but does
not track individual students' scores, while Tennessee set different
expectations for growth for each student based on the student's previous
test scores. Seventeen of the states that used growth models had been
doing so prior to passage of the NCLBA, while 9 began after the law's
passage. States used their growth models for a variety of purposes, such
as targeting resources for students that need extra help or awarding
teachers bonus money based on their school's relative performance.
Certain growth models are capable of tracking progress toward the goals of
universal proficiency by 2014 and closing achievement gaps. For example,
Massachusetts uses its model to set targets based on the growth that it
expects from schools and their student groups. Schools can make AYP if
they reach these targets, even if they fall short of reaching the
statewide proficiency targets set with the state's status model. Tennessee
designed a model, different from the one used for state purposes described
above, that projects students' test scores and whether they will be
proficient in the future. In this model, if 79 percent of a school's
students are predicted to be proficient in 3 years, the school would reach
the state's 79 percent proficiency target for the current school year
(2005-2006).
1 GAO, No Child Left Behind Act: States Face Challenges Measuring Academic
Growth That Education's Initiatives May Help Address, GAO-06-661
(Washington, D.C.: July 17, 2006).
States face challenges in developing and implementing growth models that
would allow them to meet NCLBA goals. Technical challenges include
creating data and assessment systems to meet the substantial data
requirements of growth models and having personnel that can analyze and
communicate growth model results. For example, states need to have tests
that are comparable from one year to the next to accurately measure
progress. Further, some models require sophisticated data systems that
have the capacity to track individual student performance across grades
and schools. Using growth models can present risks for states if schools
are designated as making AYP while still needing assistance to progress.
For example, one school in Tennessee that did not make AYP under the
status model would make AYP under the state's proposed growth model. This
school is located in a high-poverty, inner-city neighborhood and has been
receiving federal assistance targeted to improving student performance. If
the school continues to make AYP under the growth model, its students
would no longer receive federally required services, such as tutoring or
the option of transferring to a higher performing school. On the other
hand, the school's progress may result in its making AYP in the future
under the state's status model. U.S. Department of Education (Education)
initiatives may help states address these challenges. For example,
Education started a pilot project for states to use growth models that
meet the department's specific criteria to determine AYP. Education also
provided grants to states to support their efforts to track individual
test scores over time.
By proceeding with a pilot project with clear goals and criteria and by
requiring states to compare results from their growth model with status
model results, Education is poised to gain valuable information on whether
or not growth models are overstating progress or whether they
appropriately give credit to fast-improving schools. In comments on a
draft of our recent report, Education expressed concern that the use of a
broader definition of growth models would be confusing. GAO used this
definition in order to reflect the variety of approaches states have been
taking to measure growth in academic performance.
Background
The NCLBA2 requires states to set challenging academic content and
achievement standards in reading or language arts and mathematics3 and
determine whether school districts and schools make AYP toward meeting
these standards.4 To make AYP, schools generally must:
o show that the percentage of students scoring at the proficient
level or higher meets the state proficiency target for the school
as a whole and for designated student groups,
o test 95 percent of all students and those in designated groups,
and
o meet goals for an additional academic indicator, such as the
state's graduation rate.
The purpose of Title I Part A is to improve academic achievement for
disadvantaged students. Schools receiving Title I federal funds that do
not make AYP for 2 or more years in a row must take action to assist
students, such as offering students the opportunity to transfer to other
schools or providing additional educational services like tutoring.
States measure AYP using a status model that determines whether or not
schools and students in designated groups meet proficiency targets on
state tests 1 year at a time. States generally used data from the
2001-2002 school year to set the initial percentage of students that
needed to be proficient for a school to make AYP, known as a starting
point. From this point, they set annual proficiency targets that increase
up to 100 percent by 2014. For example, for schools in a state with a
starting point of 28 percent to achieve 100 percent by 2014, the
percentage of students who scored at or above proficient on the state test
would have to increase by 6 percentage points each year, as shown in
figure 1.5 Schools that do not reach the state target will generally not
make AYP.
2 Pub. L. No. 107-110 (Jan. 8, 2002).
3 The law also requires content standards to be developed for science
beginning in the 2005-2006 school year and science tests to be implemented
in the 2007-2008 school year.
4 States determine whether schools and school districts make AYP or not.
For this report, we will discuss AYP determinations in the context of
schools.
Figure 1: Hypothetical Example of Annual Proficiency Targets Set under a
Status Model
The law indicates that states are expected to close achievement gaps, but
does not specify annual targets to measure progress toward doing so.
States, thus, have flexibility in the rate at which they close these gaps.
To determine the extent that achievement gaps are closing, states measure
the difference in the percentage of students in designated student groups
and their peers that reach proficiency. For example, an achievement gap
exists if 40 percent of a school's non-economically disadvantaged students
were proficient compared with only 16 percent of economically
disadvantaged students, a gap of 24 percentage points. To close the gap,
the percentage of students in the economically disadvantaged group that
reaches proficiency would have to increase at a faster rate than that of
their peers.
5 States were able to map out different paths to universal proficiency
subject to certain limitations. For example, states must increase the
targets at least once every 3 years and those increases must lead to 100
percent proficiency by 2014. See GAO, No Child Left Behind Act:
Improvements Needed in Education's Process for Tracking States'
Implementation of Key Provisions, GAO-04-734 , (Washington, D.C.: Sept.
30, 2004).
If a school misses its status model target in a single year, the law
includes a "safe harbor" provision that provides a way for schools that
are showing significant increases in proficiency rates of student groups
to make AYP. Safe harbor measures academic performance in a way that is
similar to certain growth models do and allows a school to make AYP by
reducing the percentage of students in designated student groups that were
not proficient by 10 percent, so long as the groups also show progress on
another academic indicator. For example, in a state with a status model
target of 40 percent proficient, a school could make AYP under safe harbor
if 63 percent of a student group was not proficient compared to 70 percent
in the previous year.
Nearly All States Reported Using or Considering Growth Models to Measure
Academic Performance
Twenty-six states reported using growth models in addition to using their
status models to track the performance of schools, designated student
groups, or individual students, as reported in our March 2006 survey.
Additionally, nearly all states are considering the use of growth models
(see fig. 2).
Figure 2: States That Reported Using or Considering Growth Models, as of
March 2006
Of the 26 states using growth models, 19 states reported measuring changes
for schools and student groups, while 7 states reported measuring changes
for schools, student groups, and individuals, as shown in table 1.
Table 1: Types of Growth Models and States Using Them, as of March 2006
Measures growth of schools,
Measures growth of schools and groups groups, and individual students
Compares the change in scores or Compares the change in scores or
proficiency levels of schools or groups proficiency levels of schools,
of students over time. groups of students, and individual
students over time.
Data requirements, such as measuring
proficiency rates for schools or Data requirements, such as
groups, are similar to those for status tracking the proficiency levels or
models. test scores for individual
students, are typically more
involved than those for status
models.
Arizona California Colorado Connecticut Florida Mississippi North Carolina
Delaware Indiana Kentucky Louisiana South Carolina Tennessee Texas
Massachusetts Michigan Minnesota Utah
Missouri New York Ohio Oklahoma Oregon
Pennsylvania Vermont Washington
Source: GAO survey.
For example, Massachusetts used a model that measures growth for the
school as a whole and for designated student groups. The state awards
points to schools in 25-point increments for each student,6 depending on
how students scored on the state test. Schools earn 100 points for each
student who reaches proficiency, but fewer points for students below
proficiency. The state averages the points to award a final score to
schools. Growth in Massachusetts is calculated by taking the difference in
the annual scores that a school earns between 2 years. Figure 3
illustrates the growth a school can make from one year to the next as
measured by Massachusetts model.
6 Students with disabilities are generally included in these calculations.
The state is allowed to give different tests to students with significant
cognitive impairments and to count them differently for calculating points
awarded to schools.
Figure 3: Illustration of School-Level Growth
Tennessee reported using a growth model that sets different goals for each
individual student based on the students' previous test scores. The goal
is the score that a student would be expected to receive, and any
difference between a student's expected and actual score is considered
that student's amount of yearly growth,7 as shown in figure 4.
7 Tennessee's growth model mentioned here is not used to make AYP
determinations under NCLBA. However, Tennessee developed a different
growth model to determine AYP for Education's growth model pilot project.
That model is discussed later in this testimony.
Figure 4: Example of Higher-than-Expected Growth for a Fourth-Grade
Student under Tennessee's Model
In addition, Tennessee's model, known as the Tennessee Value-Added
Assessment System, estimates the unique contribution-the value added-that
the teacher and school make to each individual student's growth in test
scores over time.8 The state then uses that amount of growth, the unique
contribution of the school, and other information to determine whether
schools are below, at, or above their level of expected performance. The
model also grades schools with an A, B, C, D, or F, which is considered a
reflection of the extent to which the school is meeting its requirements
for student learning.
Seventeen of the 26 states using growth models reported that their models
were in place before the passage of the NCLBA during the 2001-2002 school
year, and the remaining 9 states implemented them after the law was
passed. States used them for purposes such as rewarding effective teachers
and designing intervention plans for struggling schools. For example,
North Carolina used its model as a basis to decide whether teachers
receive bonus money. Tennessee used its value-added model to provide
information about which teachers are most effective with which student
groups. In addition to predicting students' expected scores on state
tests, Tennessee's model was used to predict scores on college admissions
tests, which is helpful for students who want to pursue higher education.
In addition, California used its model to identify schools eligible for a
voluntary improvement program.
8 The state calculates the unique contribution of schools and teachers by
using a multivariate, longitudinal statistical method where results are
estimated using data specific for students within each classroom or
school.
Certain Growth Models Can Measure Progress toward Key NCLBA Goals
Certain growth models can measure progress in achieving key NCLBA goals of
reaching universal proficiency by 2014 and closing achievement gaps. While
states developed growth models for purposes other than NCLBA, states such
as Massachusetts and Tennessee have adjusted their state models to use
them to meet NCLBA goals. The Massachusetts model has been used to make
AYP determinations as part of the state's accountability plan in place
since 2003. Tennessee submitted a new model to Education for the growth
models pilot that differs from the value-added model described earlier.
This new model gives schools credit for students projected to reach
proficiency within 3 years in order to meet key NCLBA goals. Our analysis
of how models in Massachusetts and Tennessee can measure progress toward
the law's two key goals is shown in table 2.
Table 2: How a Status Model and Certain Growth Models Measure Progress in
Achieving Key NCLBA Goals
Status model Growth models
Massachusetts Tennesseea
(school-level and (student-level)
group-level)
Universal Sets same annual Sets biennial Sets same annual
proficiency by proficiency target growth targets for proficiency target
2014 for all schools in each school/group for all schools in
the state in the state the state
State proficiency School/group State proficiency
targets increase growth targets targets increase
incrementally to increase incrementally to 100%
100% by 2014 incrementally to by 2014
100% proficiency
by 2014; Projects future test
increments may be scores to determine
different by if students may be
school/group proficient
School makes AYP School makes AYP School makes AYP if
if it reaches the if it reaches the it reaches the state
state proficiency state proficiency proficiency target
target target or its own based on students
growth model projected to be
targets proficient in the
future
Closing State proficiency Each student group State proficiency
achievement target applies to in a school has target applies to
gaps each student group its own growth each student group in
in all schools target all schools
School makes AYP School makes AYP School makes AYP if
if each student if each student each student group
group reaches the group reaches the reaches the state
state proficiency state proficiency proficiency target
target target or its own based on students
growth model projected to be
target proficient in the
future
Source: GAO analysis of NCLBA and of information provided by the states of
Massachusetts and Tennessee.
Note: Additional requirements for schools to make AYP are described in the
background section of our report. Massachusetts refers to proficiency
targets as performance targets and refers to growth targets as improvement
targets.
aThe information presented in this table reflects the model Tennessee
proposed to use as part of Education's growth model pilot project, as
opposed to the value-added model it uses for state purposes. The
information is based on the March 2006 revision of the proposal the state
initially made in February 2006.
Massachusetts designed a model that can measure progress toward the key
goals of NCLBA by setting targets for the improvement of schools and their
student groups that increase over time until all students are proficient
in 2014. Schools can get credit for improving student proficiency even if,
in the short term, the requisite number of students has yet to reach the
state's status model proficiency targets. For example, figure 5
illustrates a school that is on track to make AYP annually through 2014 by
reaching its growth targets. While these growth targets increase at a
faster pace than the state's annual proficiency target until 2014, they do
provide the school with an additional measure by which it can make AYP.
Figure 5: Targets for a Selected School in Massachusetts Compared to State
Status Model Targets
The model also measures whether achievement gaps are closing by setting
targets for designated student groups, similar to how it sets targets for
schools as a whole. Schools that increase proficiency too slowly-that is,
do not meet status or growth targets-will not make AYP. For example, one
selected school in Massachusetts showed significant gains for several
designated student groups that were measured against their own targets.
However, the school did not make AYP because gains for one student group
were not sufficient. This group-students with disabilities-fell short of
its growth target, as shown in figure 6.
Figure 6: Results for a Selected School in Massachusetts in Mathematics
Tennessee developed a different model that can also measure progress
toward the NCLBA goals of universal proficiency and closing achievement
gaps. Tennessee created a new version of the model it had been using for
state purposes to better align with NCLBA.9 Referred to as a projection
model, this approach projects individual student's test scores into the
future to determine when they may reach the state's status model
proficiency targets.
In order to make AYP under this proposal, a school could reach the state's
status model targets by counting as proficient in the current year those
students who are predicted to be proficient in the future. The state
projects scores for elementary and middle school students 3 years into the
future to determine if they are on track to reach proficiency, as follows:
9 Tennessee continues to use its original model to rate schools based in
part on the unique contributions-or the value added-of school to student
achievement.
o fourth-grade students projected to reach proficiency by seventh
grade,
o fifth-grade students projected to reach proficiency by eighth
grade, and
o sixth-, seventh-, and eighth-grade students projected to reach
proficiency on the state's high school proficiency test.
These projections are based on prior test data and assume that the student
will attend middle or high schools with average performance (an assumption
known as average schooling experience). 10 At our request, Tennessee
provided analyses for students in several schools that would make AYP
under the proposed model. To demonstrate how the model works, we selected
students from a school and compared their actual results in fourth grade
(panel A) with their projected results for seventh grade (panel B) (see
fig. 7).
10 While Tennessee's model estimates future performance, other models are
able to measure growth without these projections. For example, Florida
uses a model that calculates results for individual students by comparing
performance in the current year with performance in prior years.
Figure 7: Results for Selected Students in Mathematics from a School in
Tennessee
Note: The same students are presented in both panels (for example, student
A in panel A is the same student as student A in panel B). While these
data reflect the scores of individual students, Tennessee provided data to
GAO in such a way that student privacy and confidentiality were ensured.
Data are illustrative and are not meant to be a statistical representation
of the distribution of students in this school.
Tennessee's proposed model can also measure achievement gaps. Under NCLBA,
a school makes AYP if all student groups meet the state proficiency
target. In Tennessee's model, whether the achievement gap is potentially
closed would be determined through projections of students' performance in
meeting the state proficiency target.
States Face Challenges in Implementing Growth Models
States generally face challenges in collecting and analyzing the data
required to implement growth models including models that would meet the
law's goals. In addition, using growth models can present risks for states
if schools are designated as making AYP while still needing assistance to
progress. Education has initiatives that may help states address these
challenges.
States must have certain additional data system requirements to implement
growth models, including models that would meet NCLBA requirements.
First, a state's ability to collect comparable data over at least 2 years
is a minimum requirement for any growth model. States must ensure that
test results are comparable from one year to the next and possibly from
one grade to the next, both of which are especially challenging when test
questions and formats change. Second, the capacity to collect data across
time and schools is also required to implement growth models that use
student-level data. This capacity often requires a statewide system to
assign unique numbers to identify individual students. Developing and
implementing these systems is a complicated process that includes
assigning numbers, setting up the system in all schools and districts, and
correctly matching individual student data over time, among other steps.
Third, states need to ensure that data are free from errors in their
calculations of performance. While ensuring data accuracy is important for
status models, doing so is particularly important for growth models,
because errors in multiple years can accumulate, leading to unreliable
results.
States also need greater research and analysis expertise to use growth
models as well as support for people who need to manage and communicate
the model's results. For example, Tennessee officials told us that they
have contracted with a software company for several years because of the
complexity of the model and its underlying data system. Florida has a
contract with a local university to assist it with assessing data
accuracy, including unique student identifiers required for its model. In
addition, states will incur training costs as they inform teachers,
administrators, media, legislators, and the general public about the
additional complexities that occur when using growth models. For example,
administrators in one district in North Carolina told us that their
district lacks enough specialists who can explain the state's growth model
to all principals and teachers in need of guidance and additional
training.
Using growth models can present risks for states if schools are designated
as making AYP while still needing assistance to progress. On the basis of
growth model results, some schools would make AYP even though these
schools may have relatively low-achieving students. As a result, some
students in Title I schools may be disadvantaged by not receiving
federally-required services.
In two Massachusetts districts that we analyzed, 23 of the 59 schools that
made AYP did so based on the state's growth model, even though they did
not reach the state's status model proficiency rate targets in
2003-2004.11 Consequently, these schools may not be eligible to receive
services required under NCLBA for schools in need of improvement, such as
tutoring and school choice. Because these schools would need to sustain
high growth rates in order to achieve universal proficiency by 2014, it is
likely that their students would benefit from additional support.
In Tennessee, 47 of the 353 schools that had not made AYP in the 2004-2005
school year would do so under the state's proposed projection model. One
school that would be allowed to make AYP under the proposed model was
located in a high-poverty, inner-city neighborhood. That school receives
Title I funding, as two-thirds of its students are classified as
economically disadvantaged. The school was already receiving services
required under NCLBA to help its students. If the school continues to make
AYP under the growth model, these services may no longer be provided.
Education's initiatives, such as the growth model pilot project, may
facilitate growth model implementation. In November 2005, Education
announced a pilot project for states to submit proposals for using a
growth model-one that meets criteria established by the department-along
with their status model, to determine AYP. While NCLBA does not specify
the use of growth models for making AYP determinations, the department
started the pilot to evaluate how growth models might help schools meet
NCLBA proficiency goals and close achievement gaps.
For the growth model pilot project, each state had to demonstrate how its
growth model proposal met Education's criteria, many of which are
consistent with the legal requirements of status models. In addition to
those requirements, Education included criteria that the proposed models
track student progress over time and have an assessment system with tests
that are comparable over time. Of the 20 proposals, Education approved 2
states-North Carolina and Tennessee-to use growth models to make AYP
determinations in the 2005-2006 school year. States may submit proposals
for the pilot again this fall.
11 Another 11 schools also met the growth target, but these 11 schools
made AYP under NCLBA's safe harbor provision.
In addition to meeting all of the criteria, Education and peer reviewers
noted that Tennessee and North Carolina had many years of experience with
data systems that support growth models. These states must report to
Education the number of schools that made AYP on the basis of their status
and growth models. Education expects to share the results with other
states, Congress, and the public after it assesses the effects of the
pilot.
In addition to the growth model pilot project, Education awarded nearly
$53 million in grants to 14 states for the design and implementation of
statewide longitudinal data systems-systems that are essential for the
development of student-level growth models. While independent of the pilot
project, states with a longitudinal data system-one that gathers data such
as test scores on the same student from year to year-will be better
positioned to implement a growth model than they would have been without
it. Education intended the grants to help states generate and use accurate
and timely data to meet reporting requirements, support decision making,
and aid education research, among other purposes. Education plans to
disseminate lessons learned and solutions developed by states that
received grants.
Conclusion
While status models provide a snapshot of academic performance, growth
models can provide states with more detailed information on how schools'
and students' performance has changed from year to year. Growth models can
recognize schools whose students are making significant gains on state
tests but are still not proficient. Educators can use information about
the academic growth of individual students to tailor interventions to the
needs of particular students or groups. In this respect, models that
measure individual students' growth provide the most in-depth and useful
information, yet the majority of the models currently in use are not
designed to do this.
Through its approval of Massachusetts' model and the growth model pilot
program, Education is proceeding prudently in its effort to allow states
to use growth models to meet NCLBA requirements. Education is allowing
only states with the most advanced models that can measure progress toward
NCLBA goals to use the models to determine AYP. Under the pilot project,
which has clear goals and criteria that requires states to compare results
from their growth model with status model results, Education is poised to
gain valuable information on whether or not growth models are overstating
progress or whether they appropriately give credit to fast-improving
schools.
While growth models may be defined as tracking the same students over
time, GAO used a definition that also includes tracking the performance of
schools and groups of students. In comments on our report, Education
expressed concern that this definition may confuse readers because it is
very broad and includes models that compare changes in scores or
proficiency levels of schools or groups of students. GAO used this
definition of growth to reflect the variety of approaches states are
taking to measure academic progress.
Mr. Chairman, this concludes my prepared statement. I would be happy to
respond to any questions that you or members of the committee may have.
GAO Contact and Staff Acknowledgments
For more information on this testimony, please call Marnie S. Shaul at
(202) 512-7215. Individuals making key contributions to this testimony
include Blake Ainsworth, Karen Febey, Harriet Ganson, Shannon Groff,
Andrew Huddleston, Jason Palmer, and Rachael Valliere.
(130595)
This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed in
its entirety without further permission from GAO. However, because this
work may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this material
separately.
GAO's Mission
The Government Accountability Office, the audit, evaluation and
investigative arm of Congress, exists to support Congress in meeting its
constitutional responsibilities and to help improve the performance and
accountability of the federal government for the American people. GAO
examines the use of public funds; evaluates federal programs and policies;
and provides analyses, recommendations, and other assistance to help
Congress make informed oversight, policy, and funding decisions. GAO's
commitment to good government is reflected in its core values of
accountability, integrity, and reliability.
Obtaining Copies of GAO Reports and Testimony
The fastest and easiest way to obtain copies of GAO documents at no cost
is through GAO's Web site ( www.gao.gov ). Each weekday, GAO posts newly
released reports, testimony, and correspondence on its Web site. To have
GAO e-mail you a list of newly posted products every afternoon, go to
www.gao.gov and select "Subscribe to Updates."
Order by Mail or Phone
The first copy of each printed report is free. Additional copies are $2
each. A check or money order should be made out to the Superintendent of
Documents. GAO also accepts VISA and Mastercard. Orders for 100 or more
copies mailed to a single address are discounted 25 percent. Orders should
be sent to:
U.S. Government Accountability Office 441 G Street NW, Room LM Washington,
D.C. 20548
To order by Phone: Voice: (202) 512-6000 TDD: (202) 512-2537 Fax: (202)
512-6061
To Report Fraud, Waste, and Abuse in Federal Programs
Contact:
Web site: www.gao.gov/fraudnet/fraudnet.htm E-mail: [email protected]
Automated answering system: (800) 424-5454 or (202) 512-7470
Congressional Relations
Gloria Jarmon, Managing Director, [email protected] (202) 512-4400 U.S.
Government Accountability Office, 441 G Street NW, Room 7125 Washington,
D.C. 20548
Public Affairs
Paul Anderson, Managing Director, [email protected] (202) 512-4800 U.S.
Government Accountability Office, 441 G Street NW, Room 7149 Washington,
D.C. 20548
www.gao.gov/cgi-bin/getrpt? GAO-06-948T .
To view the full product, including the scope
and methodology, click on the link above.
For more information, contact Marnie S. Shaul (202) 512-7215 or
[email protected].
Highlights of GAO-06-948T , a testimony before the House Committee on
Education and the Workforce
July 27, 2006
NO CHILD LEFT BEHIND ACT
States Face Challenges Measuring Academic Growth
The No Child Left Behind Act (NCLBA) requires that states improve academic
performance so that all students reach proficiency in reading and
mathematics by 2014 and that achievement gaps close among student groups.
States set annual proficiency targets using an approach known as a status
model, which calculates test scores 1 year at a time. Some states have
interest in using growth models that measure changes in test scores over
time to determine if schools are meeting proficiency targets.
The Chairman of the Committee on Education and the Workforce asked GAO to
testify on its recent report on measuring academic growth. Specifically,
this testimony discusses (1) how many states are using growth models and
for what purposes, (2) how growth models can measure progress toward
achieving key NCLBA goals, and (3) what challenges states face in using
growth models especially to meet the law's key goals.
While growth models may be defined as tracking the same students over
time, GAO used a definition that also included tracking the performance of
schools and groups of students. In comments on the report, Education said
that this definition could be confusing. GAO used this definition of
growth to reflect the variety of approaches states were taking.
Nearly all states were using or considering growth models for a variety of
purposes in addition to their status models as of March 2006. Twenty-six
states were using growth models, and another 22 were either considering or
in the process of implementing them. Most states using growth models
measured progress for schools and for student groups, and 7 also measured
growth for individual students. States used growth models to target
resources for students that need extra help or award teachers bonuses
based on their school's performance.
States That Reported Using or Considering Growth Models, as of March 2006
Certain growth models are capable of tracking progress toward the goals of
universal proficiency by 2014 and closing achievement gaps. For example,
Massachusetts uses its model to set targets based on the growth that it
expects from schools and their student groups. Schools can make adequate
yearly progress (AYP) if they reach these targets, even if they fall short
of reaching the statewide proficiency targets set with the state's status
model. Tennessee designed a model that projects students' test scores and
whether they will be proficient in the future. In this model, if 79
percent of a school's students are predicted to be proficient in 3 years,
the school would reach the state's 79 percent proficiency target for the
current school year (2005-2006).
States face challenges measuring academic growth, such as creating data
and assessment systems to support growth models and having personnel to
analyze and communicate results. The use of growth models to determine AYP
may also challenge states to make sure that students in low-performing
schools receive needed assistance. U.S. Department of Education
(Education) initiatives may help states address these challenges.
Education started a pilot project for states to use growth models that
meet the department's specific criteria, including models that track
progress of individual students, to determine AYP. Education also provided
grants to states to track individual test scores over time.
*** End of document. ***