No Child Left Behind Act: Improvements Needed in Education's	 
Process for Tracking States' Implementation of Key Provisions	 
(30-SEP-04, GAO-04-734).					 
                                                                 
The No Child Left Behind Act of 2001 (NCLBA) has focused national
attention on improving the academic achievement of the nations'  
48 million students by establishing a deadline--school year	 
2013-14--for public schools to ensure that all students are	 
proficient in reading and math. Accordingly, states, the District
of Columbia, and Puerto Rico developed plans that set goals for  
increasing the numbers of students who attain proficiency on	 
state tests each year, with all meeting goals by 2014. To provide
information about states' efforts, GAO determined (1) what goals 
states established for student proficiency and their implications
for whether schools will meet these goals; (2) what factors	 
facilitated or impeded selected state and school district	 
implementation efforts; and (3) how the Department of Education  
(Education) supported state efforts and approved state plans to  
meet student proficiency requirements.				 
-------------------------Indexing Terms------------------------- 
REPORTNUM:   GAO-04-734 					        
    ACCNO:   A12852						        
  TITLE:     No Child Left Behind Act: Improvements Needed in	      
Education's Process for Tracking States' Implementation of Key	 
Provisions							 
     DATE:   09/30/2004 
  SUBJECT:   Education or training				 
	     Elementary school students 			 
	     Elementary schools 				 
	     Secondary school students				 
	     Secondary schools					 
	     Educational standards				 
	     Educational testing				 
	     Academic achievement				 
	     Public schools					 
	     Performance measures				 
	     Federal/state relations				 
	     School districts					 
	     Strategic planning 				 

******************************************************************
** This file contains an ASCII representation of the text of a  **
** GAO Product.                                                 **
**                                                              **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced.  Tables are included, but    **
** may not resemble those in the printed version.               **
**                                                              **
** Please see the PDF (Portable Document Format) file, when     **
** available, for a complete electronic file of the printed     **
** document's contents.                                         **
**                                                              **
******************************************************************
GAO-04-734

United States Government Accountability Office

GAO

                       Report to Congressional Committees

September 2004

NO CHILD LEFT BEHIND ACT

 Improvements Needed in Education's Process for Tracking States' Implementation
                               of Key Provisions

GAO-04-734

Contents

Letter

Results in Brief
Background
States Varied in the Goals Established for Student Progress, and

That Variation May Have Implications for How Many Schools Meet State Goals
over Time

Leadership and Technical Assistance Facilitated Implementation Efforts,
but Data Accuracy Problems and Tight Timelines Impeded Efforts

Education Has Aided States in Developing Their Plans and Assessment
Systems but Did Not Have Written Plans to Help States Meet NCLBA
Provisions

Conclusions
Recommendations for Executive Action
Agency Comments and Our Evaluation

                                       1

                                      2 5

10

23

27 35 36 37

Appendix I Methods to Establish Starting Points

Appendix II	Percentage of Schools That Met State Goals in 2002-03

Appendix III State Plan Requirements

Appendix IV Comments from the Department of Education

          Appendix V          GAO Contacts and Staff Acknowledgments       50 
                                           GAO Contacts                    50 
                                      Staff Acknowledgments                50 
     Related GAO Products                                                  51 

Tables

Table 1: Primary Methods Education Used To Support State Planning and
Implementation Efforts 28 Table 2: Calculating a Starting Point Using the
School at the 20th Percentile in Cumulative Enrollment 40

Figures

Figure 1: Example of States' Discretion to Develop Their Own Content
Standards and Tests and to Determine What Constitutes Proficiency on Each
of the Tests 8

Figure 2: Percentage of Students in Each State Expected to Demonstrate
Proficiency on the Reading Tests in the First Year 12

Figure 3: Minimum Size of Student Groups by Number of States 15 Figure 4:
Three Variations in State Projected Rates of Progress from 2002 to 2014 17
Figure 5: A Majority of States Used Confidence Intervals to Determine
Student Progress in School Year 2002-03 19 Figure 6: Student Progress
Measures and Potential Effects on Whether Schools Meet Proficiency Goals
22 Figure 7: Example of a Timeline to Determine School's Proficiency
Status 26 Figure 8: Approval Status of State Plans as of July 31, 2004 32

Abbreviations

ESEA Elementary and Secondary Education Act
NCLBA No Child Left Behind Act
LEA Local Educational Agency

This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed in
its entirety without further permission from GAO. However, because this
work may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this material
separately.

United States Government Accountability Office Washington, DC 20548

September 30, 2004

The Honorable Judd Gregg

Chairman

The Honorable Edward M. Kennedy

Ranking Minority Member

Committee on Health, Education, Labor, and Pensions United States Senate

The Honorable John A. Boehner
Chairman
The Honorable George Miller
Ranking Minority Member
Committee on Education and the Workforce
House of Representatives

The No Child Left Behind Act of 2001 (NCLBA) has focused national
attention on increasing academic achievement and closing achievement
gaps among the nation's 48 million school-aged children by establishing a
deadline-school year 2013-14-for public schools to bring all of their
students to an achievement level deemed "proficient" in reading and math
by their state. This includes students in total and in NCLBA-designated
student groups-students who are economically disadvantaged, are
members of major racial or ethnic groups, have disabilities, or have
limited
English proficiency. As a condition for receiving federal funds, NCLBA
required that each state submit a plan to the Department of Education
(Education) that describes how the state will ensure that all students are
proficient in reading and math by the deadline, as measured primarily by
tests each state used. To provide information about the current status of
states' efforts to implement student proficiency requirements, GAO
determined (1) what goals states established for student proficiency and
their implications for whether schools will meet these goals, (2) what
factors facilitated or impeded selected state and school district
implementation efforts, and (3) how Education supported state efforts and
approved state implementation plans to meet student proficiency
requirements.

To address these issues, we analyzed data from plans which all states
submitted to Education and that Education approved by June 2003. We
extracted detailed information from each plan and developed a database

of that information to facilitate analysis. We also contacted officials in
50 states, the District of Columbia, and Puerto Rico to obtain information
about the number of schools they had identified as meeting annual progress
goals and their school and district characteristics in 2002-03.1 We
visited 4 states (California, Illinois, North Carolina, and Rhode Island)
and 6 school districts within these states, and conducted phone interviews
with officials in another 17 states to obtain information about factors
that facilitated and impeded implementation of student proficiency
requirements. The states and districts were selected to achieve variation
in geography and size and to explore variation among the states in such
areas as their starting points, first-year goals, and successive annual
student proficiency goals. We reviewed documentation Education provided
the states, reviewed regulations and guidance issued by Education, and
interviewed Education officials about their efforts to assist states in
developing plans and their process for approving plans. We also reviewed
the status of Education's approval of states' standards and assessments
systems that were required to comply with the 1994 Elementary and
Secondary Education Act (ESEA). In July 2004, in response to our requests,
Education provided us with updated and new information related to the
approval status of states' plans, grant award conditions, assessment
system enforcement efforts, and assistance provided to improve the quality
of state data. Finally, we interviewed officials from national education
organizations and other experts in the area. We conducted our work between
August 2003 and August 2004 in accordance with generally accepted
government auditing standards.

States varied in how they established proficiency and measured student
progress, and this variation in state approaches could affect how many
schools meet their annual goals over time. NCLBA permits such variability
for each state to address its unique circumstances, thus differences are
not unexpected. First, states varied in their starting points-the 2001-02
assessment levels that were used to set first-year proficiency goals-and
also varied in their first-year goals. NCLBA prescribed a statutory
formula for determining starting points based on each state's 2001-02
assessment data. State starting points reflected the differences in
decisions states had previously made in choosing content standards,
determining the rigor of

  Results in Brief

1Hereinafter, the term states will refer collectively to the 50 states
plus the District of Columbia and Puerto Rico.

tests developed or chosen to measure student performance, and setting
proficiency levels. Consequently, the percentage of students expected to
meet proficiency goals in the first year varied widely. For example, in
California's schools, 14 percent of elementary school students were
expected to be proficient in reading in the first year, while Colorado
expected that 78 percent of its elementary students would be proficient.
States also varied in the minimum size of designated groups, such as
economically disadvantaged and ethnic minority students, whose progress
must be measured separately. In determining whether schools met
proficiency goals, states were not required to include results for these
groups if the number of students was too small to yield statistically
reliable information. For example, in the state of Washington, which has a
minimum group size of 30, schools would not be required to include
separately the test scores for any group of fewer than 30 students. States
also varied in the percentage of students they expected to be proficient
annually to meet NCLBA's requirement that all students be proficient by
2014. For example, some states expected schools to show steady progress
every year and others every 3 years. Finally, states varied in how they
planned to determine whether their schools met state goals. The majority
of states used statistical techniques that they believed improved the
accuracy of their determinations, such as determining that a school had
made adequate progress if the percentage of students scoring at the
proficient level or above came within a statistical range (i.e.,
confidence intervals) of the state goal. The approaches states used to
establish goals and determine student proficiency, such as confidence
intervals, could have implications over time for the number of schools
that meet their goals.

State officials we interviewed cited factors that facilitated
implementation of student proficiency requirements, such as the commitment
of their state leadership to the goals of NCLBA and technical assistance.
However, factors such as data problems and tight timelines for determining
school progress impeded implementation. Officials reported that state
leadership, by providing administrative and legislative support, had been
influential in facilitating the goals of NCLBA. They also reported that
technical assistance from the Council of Chief State School Officers,
under contract with Education, had been an important factor in
facilitating states' firstyear implementation. On the other hand, more
than half of the state and school district officials we interviewed
reported being hampered by poor and unreliable student data. Reliable data
are essential for implementing the requirements of the law. For example,
officials in Illinois reported that about 300 of their 1,055 districts had
problems with data accuracy. Education is working on efforts to help
states improve their data systems,

such as monitoring state data quality policies and establishing a common
set of data definitions. Officials from about half of the 21 states also
said that tight timelines impeded implementation of student proficiency
requirements. For example, because tests were often given late in the
school year, it was difficult for states to make final determinations
about whether schools had met progress goals prior to the next school
year.

Education assisted states in developing their plans for improving student
proficiency in several ways and approved all plans, fully or
conditionally, by June 10, 2003. To help states, Education asked experts
familiar with student assessments to review all plans and provide them
with on-site evaluation. Education also allowed states some flexibility
with certain requirements, such as granting all states greater flexibility
in determining how students with limited English proficiency could be
assessed. On June 10, 2003, when Education announced it had approved all
plans, 11 state plans met all NCLBA requirements. The remaining 41 plans
were approved by Education with conditions that needed to be met to
satisfy all NCLBA requirements. As of July 31, 2004, 28 states had plans
that met all NCLBA requirements, and 24 states, including the District of
Columbia, had plans with conditions that needed to be met before receiving
full approval from Education. According to Education, states approved with
conditions had sufficient information in their plans to demonstrate that
the requirements of NCLBA could be met in the future if certain actions
were taken. Although Education officials said that they are continually
monitoring states whose plans have not been fully approved, the department
does not have a written process that delineates how and when each state
will meet the conditions. In addition, in July 2004 some states did not
have approved academic standards and assessment systems in place to meet
the requirements for the 1994 education law, even though they are the
primary means by which the law requires states to determine student
proficiency. By school year (2005-06), all states are required by NCLBA to
increase the current level of testing. Given the difficulties states
experienced meeting the 1994 requirements, developing new standards and
assessment systems to meet the expanded assessment requirements may be
challenging for states. Education has developed guidance for its review
and approval of states' expanded standards and assessment systems.
However, it has not established a written plan that clearly identifies the
steps required, interim goals, review schedules, and timelines. Without
such a plan, states may be challenged to meet NCLBA standards and
assessment systems requirements by the 2005-06 school year deadline.

We are recommending that the Secretary of Education delineate in writing
the process and time frames that are appropriate for each state's
particular

circumstances to meet conditions for full approval, develop a written plan
that includes steps and time frames so that all states have approved NCLBA
standards and assessment systems by the 2005-06 school year, and further
support states' abilities to gather accurate student data used to
determine whether schools met state goals.

In its comments on a draft of this report, Education expressed support for
the recommendations we made on developing a written plan to help states
meet the 2005-06 NCLBA requirements for standards and assessment systems
and indicated the department has begun to take steps to develop such a
plan. Education also supported our recommendation to provide additional
assistance to the states to improve their abilities to gather accurate
student performance data. Education disagreed with our recommendation that
it delineate in writing the process and time frames for states to meet
conditions needed to receive Education's full approval of their plans.
Education indicated that it has a process to monitor states' progress,
although not in writing, and that this process has resulted in additional
plans being fully approved. We recognize the efforts the department has
taken to support states' implementation of NCLBA. However, Education has
not fully approved almost half (24) of state plans, meaning that
conditions still exist for some states in order for them to be able to
meet NCLBA provisions. A written delineation of conditions that these
plans need to meet and the time frames appropriate for each state's
circumstances would provide the necessary documentation and assurance to
Education, Congress, and the public that the steps states need to take and
the timeframes for their actions are clear and understood.

                                   Background

Prior Federal Reform Over the past 40 years, the Elementary and Secondary
Education Act

Efforts 	(ESEA) has authorized billions of dollars in federal grants to
states and school districts to improve educational opportunities for
economically disadvantaged children.2 ESEA was reauthorized in 1994, with
requirements designed to hold states accountable for student progress.3
Specifically, as a condition for receiving federal financial assistance
under

2Title I, Part A, of the ESEA is the largest program of federal aid for
elementary and secondary education, allocating almost $12 billion in
fiscal year 2003 to serve disadvantaged children in approximately 90
percent of the nation's school districts.

3ESEA was reauthorized and amended as the Improving America's Schools Act
in 1994.

Title I, Part A, of the act, states were required to develop academic
standards, develop tests and measure student proficiency in certain
grades, and determine whether schools were meeting proficiency goals. As
ESEA neared reauthorization in 2001, however, only 17 states had received
Education's approval of their systems for standards and testing, and
Congress was concerned that student performance was not improving as
quickly as it should have, specifically among some student groups, such as
the economically disadvantaged.

New Test Requirements and Standards and a Goal for 2014

In part to address these issues, the No Child Left Behind Act of 2001
enhanced the federal government's role in kindergarten-12th grade (K-12)
education by taking steps to ensure that all students reach the
"proficient" level of achievement within 12 years of the enactment of the
law, that is, by school year 2013-14. NCLBA strengthened the 1994
reauthorization requirements in several ways. NCLBA increased the amount
of testing in future school years. Beginning in the 2005-06 school year,
tests in math and reading must be administered every year in grades 3
through 8 and once in high school, and by 2007-08, states must also
measure students' science achievement. NCLBA requires that these tests
serve as the primary means of determining the annual performance of
schools and that states provide Education with evidence from the test
publisher or other relevant sources that these assessments are of adequate
technical quality and consistent with nationally recognized professional
and technical standards. States are to show that increasing numbers of
students are reaching the proficient level on state tests over time so
that by 2014, every student is proficient.

Similar to the 1994 law, NCLBA also designated specific groups of students
for particular focus. These four groups are students who (1) are
economically disadvantaged, (2) represent major racial and ethnic groups,
(3) have disabilities, and (4) are limited in English proficiency. 4
States and school districts are required to measure the progress of all
students in meeting proficiency goals, as well as to measure separately
the progress of these designated groups. To be deemed as having made
adequate progress, each school must show that each of these groups, as
well as the school as a whole, met the state proficiency goal. Schools
must also show that at

4Students with disabilities refers to students covered under the
Individuals with Disabilities Education Act, the primary law that
addresses the unique needs of children with disabilities.

least 95 percent of students in grades required to take the test have done
so.5 Further, schools must also demonstrate that they have met state
targets on another measure of progress-graduation rates in high school or
attendance or other measures in elementary or middle schools.6

Finally, NCLBA requires that additional actions be taken if schools that
receive funding under Title I, Part A, of the act do not meet state goals.
Schools that have not made progress for 2 consecutive years or more are
"identified for improvement" and must take certain actions such as
offering parents an opportunity to transfer students to another school
(school choice) and providing supplemental services (e.g., tutoring).7
States and school districts are required to provide funding up to a
maximum amount specified in law for such actions, including
transportation, tutoring, and training.

Although NCLBA placed many new requirements on states, states have broad
discretion in many key areas. States develop their own tests to measure
the content students are taught in their schools. States set their own
standards for what constitutes "proficiency" (see fig. 1). NCLBA does,
however, require states to set two standards for high achievement-
"advanced" and "proficient," to reflect a degree of mastery-and to set
another standard for "basic" achievement to indicate the progress of the
lower-achieving children toward mastering their state standards. As part
of its monitoring, Education reviews any changes states may make to their
tests and academic and proficiency requirements, and the law requires
states to notify Education of any significant change.

5The department issued guidance in March 2004 indicating that states-if
they request to do so-may average participation rates over 2 or 3 years.

6These other measures may include, but are not limited to, grade-to-grade
retention rates, and changes in the percentage of students completing
gifted and talented and advance placement or college preparatory courses.

7Schools designated as in need of improvement under the Improving
America's Schools Act had their designation carry over after NCLBA took
effect.

Figure 1: Example of States' Discretion to Develop Their Own Content
Standards and Tests and to Determine What Constitutes Proficiency on Each
of the Tests

                             Source: GAO analysis.

State Plans for Setting Goals and Measuring Student Progress

Under NCLBA, each state requesting federal financial assistance was
required to submit a plan to Education that, among other things,
demonstrated how the state will meet the law's requirements for setting
annual goals and measuring student progress.8 The law required that plans
demonstrate that the state has developed and is implementing a statewide
system that will be effective in ensuring that schools make adequate
yearly progress toward the 2013-14 goal. The law also required that state
plans

8We use the term plan to refer to a state's "accountability workbook," a
format developed by Education. Education required states to use the
accountability workbooks to detail the basic elements of the state's
system to demonstrate meeting NCLBA requirements.

demonstrate what constitutes adequate yearly progress, and required that
plans establish:

o  	Starting points for measuring the percentage of students who meet or
exceed the state's proficient level of academic achievement using
assessment data from the 2001-02 school year. The methods for computing
starting points, as specified in the law, take into account such factors
as scores from designated student groups and how schools rank in their
state.9 Separate starting points were to be developed for reading/language

arts and math.

o  	Annual goals, including first year goals, establishing the single
minimum percentage of students who will be required to score at or above
the proficient level on the state assessment in each year until 2013-14.
The goals are based on the starting points for each state's reading and
math assessments. 10

o  	The minimum number of students in a designated student group necessary
for their test results to be used as a separate group in determining
whether a school met state goals. Each state was allowed to determine the
minimum number required to ensure that the group size was sufficient to
produce statistically reliable results.11

o  	Graduation rates for high schools and another indicator of progress of
the state's choosing for elementary and middle schools, such as attendance
rates. Graduation rate is defined in NCLBA as the percentage of students
who graduate from secondary school with a regular diploma in the standard
number of years.

Following states' submission of their plans in January of 2003, Education
was statutorily required to conduct a peer review process, identifying
federal and state officials and outside experts to meet as a team with
each state, review its plan, and provide assistance. Subsequently, the
teams were to provide their assessment of the extent to which state plans
met

9See appendix I for a description of the methods the law required states
to use to develop starting points.

10The first-year goals may or may not be the same as the starting points.

11Measuring the achievement of a group of students is not required if the
number of students in that group is insufficient to yield statistically
reliable information or would reveal personally identifiable information
about an individual student.

NCLBA requirements, such as having starting points, first-year goals and
annual goals to ensure that every student would become proficient, and
minimum student group sizes for measuring the achievement of designated
students. The law required Education to review the plans and approve them
within 120 days of a state's submission. If the Secretary determined that
a state plan did not meet all requirements, he was required to notify the
state and offer technical assistance, among other actions, before
disapproving the plan.

NCLBA also requires the Secretary to report to Congress annually regarding
state progress in implementing various requirements, including the number
of schools identified as needing improvement. While NCLBA requires
accurate and reliable data on student test scores and valid systems for
identifying designated student groups, GAO, along with other auditors, has
reported that states and school districts face serious challenges in this
regard. GAO has proposed several recommendations for improving the
collection and reporting of student data.12 Additionally, Education's
Inspector General has reported that the lack of procedures and controls on
student data is a continuing challenge for the department.13

States varied in how they established goals and measured student progress;
this variation may affect how many schools meet their annual
goals-adequate yearly progress-each year. NCLBA permits variability in a
number of areas, allowing states to address their unique circumstances.
States varied in the percentage of students they expected to demonstrate
proficiency on their tests in the first year of NCLBA's implementation and
in the number of students in designated groups whose proficiency had to be
measured separately. They also varied in the annual rates they set to
increase student proficiency. Finally, they differed in how they measured
student progress. These variations may have implications for the number of
schools that meet their goals each year.

12GAO, Title I: Education Needs to Monitor States' Scoring of Assessments,
GAO-02-393, (Washington, D.C.: Apr. 1, 2002), and Title I Program:
Stronger Accountability Needed for Performance of Disadvantaged Students,
GAO/HEHS-00-89, (Washington, D.C.: June 1, 2000).

13U. S. Department of Education, Office of Inspector General, Department
of Education Management Challenges, Feb. 2004.

  States Varied in the Goals Established for Student Progress, and That
  Variation May Have Implications for How Many Schools Meet State Goals over
  Time

States' Starting Points and First-Year Goals Varied

States' starting points-based on the percentage of students proficient in
reading and math on state tests in 2001-02-varied widely, as did their
first-year performance goals. NCLBA specified that states were to use
their 2001-02 test data to calculate their starting points and instructed
states on how the starting point was to be set from these data. After
states computed their starting points, they specified performance goals
for each year that would result in all children being proficient by
2013-14. As figure 2 illustrates, the percentage of students expected to
be proficient in reading in the 2002-03 school year differed widely among
the states.

Figure 2: Percentage of Students in Each State Expected to Demonstrate
Proficiency on the Reading Tests in the First Year

Percent

80

70

60

50

41

40

30

20

10

0 Calif. Mo. S.C. D.C. Hawaii Wyo. Fla. Ark. Nev. Maine La. Mich. Ill.
Ohio

Source: GAO analysis of state plans approved by Education in June 2003.

Notes: Thirty-six states both provided data on first-year goals and used
"percent proficient" as their measure for the goals. Six states used a
different measure (proficiency index) that allowed them to incorporate
other data in determining school progress, while 10 states did not provide
first-year goal data.

When states set different first-year goals (e.g., separate goals for
elementary, middle, and high schools), we used goals set at the lowest
grade span or level (e.g., elementary) for this chart.

Oreg. Ariz. Pa. Tex. Ky. Kans. Wash. Conn. Ga. N.H. Va. Wis. Miss. Nebr. Alaska
                   Iowa N. Dak. S. Dak. Utah N.J. N.C. Colo.

For example, in order for an elementary school to meet the state reading
goal in California, at least 14 percent of its students had to score at
the proficient level on the state test, whereas in Colorado, at least 78
percent of the students had to score at the proficient level.14

14First-year goals for math also varied substantially across states.

Variation in states' starting points and first year-goals reflected the
differences in decisions states had previously made in choosing content
standards, developing tests to measure student performance, and setting
proficiency levels, among other factors. For example, the score required
to be proficient on a similar type of test might be higher in one state
than in another, potentially affecting the percentage of students that
demonstrate proficiency on the test.

In addition to establishing widely varying first-year goals, states
differed in whether they set the same goal for all of their schools or
whether they set different goals by grade level. Given that each state has
its own system and structure, decisions about setting the same or
different goals for schools was generally within the states' discretion.
Some states established different first-year goals for each grade; others
for elementary, middle, and high schools; and some established the same
first-year goals for all schools. Vermont, for example, had distinct goals
for different grade configurations: schools that had elementary, middle,
and high school grades had different goals than schools with just
elementary grades.

States Set Different Size Requirements for Measuring the Progress of
Designated Groups

The size of the designated groups (the economically disadvantaged, ethnic
minorities, students with disabilities, and students with limited English
proficiency) whose progress must be measured separately also varied among
states.15 NCLBA specified that to make adequate yearly progress, the
school overall and each of these individual groups must reach the
performance goal unless the number of students in the group is small
enough to reveal personally identifiable information on an individual
student or to yield statistically unreliable information. States decided
the minimum number of students in such groups, and the resulting group
sizes varied from state to state and sometimes within a state. As figure 3
shows, the majority of states (36) set the minimum group size between 25
and 45 students.16

15This statement refers to the minimum size of designated student groups
for measuring proficiency and not for reporting test scores.

16Some states had more than one group size. When states reported multiple
group sizes, we report the lower size.

Figure 3: Minimum Size of Student Groups by Number of States

Number of states

40

36

35

30

25

20

15

10

5

0 Fewer 25-45 More than 25 than 45 Student group size

Source: GAO analysis.

Note: This figure does not include Montana or North Dakota, which used a
statistical model to determine the minimum group size so that the number
may be different for each school and designated student group.

For example, in Washington state, with a minimum group size of 30, schools
were not required to include the results of any student group with fewer
than 30 students in determining whether they met the state's proficiency
goals. In this case, if a school had fewer than 30 students of a
particular ethnic group, for instance, the scores of this student group
would not be considered separately. These students' individual scores
would still be considered, however, in determining whether their school as
a whole had met its goal. A few states used different group sizes,
depending on other factors. For example, California set its group size at
50 but allowed the minimum size to be 100, depending on the size of the
school's enrollment. Ohio, among other states, used a larger group size
for its students with disabilities than the one used for other student
groups. According to its state plan, one of the reasons Ohio set a larger
size for this group was to account for the fact that students in that
group have a wide variety of conditions and results for small groups could
be unreliable.

States Set Different Rates States also varied in the annual rate at which
they expected their students for Annual Student to progress toward full
proficiency by 2014. Using the flexibility in the law, Progress some
states set different proficiency goals each year, while others set

goals for 3-year intervals. 17 Some states used a combination of staggered

and steady progress. (See fig. 4.)

17NCLBA states that when states increase their goals from one year to the
next, those increases must occur in equal increments, that the first
increase must occur by 2004-05, and that future increases occur no later
than every 3 years thereafter.

Figure 4: Three Variations in State Projected Rates of Progress from 2002
to 2014 Panel A: equal annual increases

Percentage of students proficient 100

75

50

25 02 0304 05060708091011121314

School year ending

Percentage of students proficient Percentage of students proficient

                                      100

                                       75

                                       50

                                       25

100

75

50

2502030405060708091011121314 020304050607080910111213

School year ending School year ending

Source: GAO analysis.

Note: Graphs in this figure are hypothetical and do not reflect particular
states.

Three states assumed generally equal annual increases in student progress.
(See panel A.) For example, Arkansas's first-year goal was that 32 percent
of the elementary students in each of its schools would be proficient in
reading, followed by an increase of about 6 percent of its students
annually until all of its students were proficient by 2014. In contrast,
14 states staggered improvement over 2-or 3-year periods rather than in

1-year increments. (See panel B.) For example, North Carolina's first-year
goal was that about 69 percent of elementary students in each of the
state's schools would be proficient in reading in the first year and the
state set goals for subsequent increases every 3 years: 77 percent by
2005, 84 percent by 2008, 92 percent by 2011, and 100 percent by 2014.
Finally, 18 states used a combination of progress rates. (See panel C.)
Nevada, for example, staggered improvement goals in 2-and 3-year
increments until 2011, at which point the state planned for annual
increases in percentages of students that were proficient up to 2014.18

States Varied in How They States also used different approaches for
determining whether schools, Measured Annual Student and designated groups
of students within schools, met their annual Progress performance goals.
In the 2002-03 school year, a majority of states used

statistical measures such as confidence intervals in which schools were
deemed to have made adequate yearly progress if they came within a range
of the state proficiency goal, as shown in Figure 5. 19

18As of June 10, 2003, 17 states either did not report an annual rate or
used some other method.

19When using confidence intervals, upper and lower limits around a
school's or district's percentage of proficient students are calculated,
creating a range of values within which there is "confidence" the true
value lies. For example, instead of saying that 72 percent of students
scored at the proficient level or above on a test, a confidence interval
may show that percentage to be between 66 and 78, with 95 percent
confidence.

States that used confidence intervals constructed an estimate of student
performance that included a range of scores, which was then compared with
the state goal. For example, 68 percent of students making the goal might
be represented by a confidence interval of 64-72 percent. If the state
goal was 70 percent, it would be included in the confidence interval, thus
the school or designated group would be classified by the state as having
made its performance goal.

Education officials told us that states used such statistical procedures
to improve the reliability of determinations about the performance of
schools and districts. According to some researchers, such methods may
provide more valid results because they account for the effect of small
group sizes20 and year-to-year changes in student populations.21

The Way States Measured Student Progress Has Implications for Schools
Meeting State Goals Over Time

Variations in states' approaches may influence whether schools will meet
annual state goals. First, schools in states that established smaller
annual increases in their initial proficiency goals may be more likely to
meet state goals in the earlier years compared with schools in states that
set larger annual increases. For example, Iowa projected moderate annual
increases in student proficiency in the first 8 years, followed by more
accelerated growth. Nebraska, however, projected a different
scenario-steady increases in student proficiency. Although schools in
states such as Iowa may be more likely to meet state goals in the first
few years, they may find it more challenging to meet state goals in
subsequent years to ensure that all students are proficient by 2013-14.

Second, schools with a large number of designated student groups may be
less likely to meet state goals than schools with few such groups, all
other factors being equal.22 In order for a school to meet its state goal,
both the school as a whole and each designated student group must meet
proficiency goals. Some schools may have few student groups that must

20Theodore Coladarci, Gallup Goes to School: The Importance of Confidence
Intervals for Evaluating "Adequate Yearly Progress" in Small Schools, the
Rural School and Community Trust Policy Brief, Oct. 2003.

21Thomas J. Kane and Douglas O. Staiger, "Volatility in School Test
Scores: Implications for Test-Based Accountability Systems," in Diane
Ravitch, ed., Brookings Papers on Education Policy 2002, pp. 235-283.
Washington, D.C.: Brookings Institution.

22Other factors, such as the application of certain statistical
procedures, can affect this result.

demonstrate progress because they do not have the state-prescribed minimum
number of such students needed for their results to be considered
separately. Several state officials told us that many of their schools
were not meeting state goals because one or two student groups did not
meet their annual proficiency goals.

Finally, the approach states used in determining whether schools met
proficiency goals may influence the number of schools meeting goals. Some
states used statistical methods, such as confidence intervals, which may
result in more of their schools reaching proficiency goals than states
that do not. For instance, Tennessee-a state that initially did not use
confidence intervals but later received approval to do so-re-analyzed its
data from 2002-03, applying confidence intervals. The application of
confidence intervals substantially decreased the number of schools not
meeting state goals. The number of elementary and middle schools not
making state goals was reduced by over half-47 percent to 22 percent. The
application of confidence intervals can produce such differences because
the computed ranges can be large, especially when small numbers of
students make up groups or when scores vary significantly among students.
For example, in a Kentucky high school, 16 percent of students with
disabilities scored at the proficient level a state test in 2004, and the
goal was 19 percent. However, when the state applied confidence intervals,
the computed interval associated with 16 percent was 0 to 33 percent.
Because the state goal-19 percent-was within the confidence interval, the
state considered this group to have met the goal.23 (See fig. 6 for
potential effects of different student progress measures on whether
schools meet proficiency goals.)

23These results were preliminary at the time we obtained them and were
calculated at the 99 percent confidence level.

  Figure 6: Student Progress Measures and Potential Effects on Whether Schools
                             Meet Proficiency Goals

     Source: GAO analysis and Copyright @ Corel Corp. All rights reserved.

  Leadership and Technical Assistance Facilitated Implementation Efforts, but
  Data Accuracy Problems and Tight Timelines Impeded Efforts

State officials we interviewed cited factors that facilitated
implementation of student proficiency requirements, such as their states
leadership's commitment to the goals of NCLBA and technical assistance
provided by the Council of Chief State School Officers, through a contract
with Education. Officials also cited factors that impeded implementation,
such as problems with the data they use to determine student proficiency,
tight timelines, and a lack of timely guidance from Education.

Officials Cited Leadership Commitment and Technical Assistance as Key
Factors That Facilitated Implementation of Student Proficiency
Requirements

Officials in 10 of the 21 states we interviewed said that their
leadership's commitment to improving student achievement facilitated their
efforts to implement student proficiency requirements. For example, one
state's Commissioner of Education said he supported holding schools
accountable for the progress of all students, a sentiment echoed by other
state officials. Officials in three of the school districts where we
interviewed expressed their commitment to NCLBA's focus on raising the
proficiency of all students. For example, one district official said the
law has been helpful in demonstrating achievement gaps to school
officials. Another told us that NCLBA has focused the state's attention on
the importance of annually tracking student proficiency. Leadership's
commitment facilitated implementation in many ways, such as helping
schools and school districts focus on improving student proficiency and
enabling state education staff in different offices to share information.

Officials from 7 states also reported that the assistance provided by the
Council of Chief State School Officers facilitated implementation of NCLBA
requirements. Through its contract with Education, the council has
provided states technical assistance in implementing NCLBA requirements
and issued many publications about the law's requirements. The council has
also held meetings where state officials have discussed common challenges
and strategies and received advice and assistance from national experts
and Education officials. For example, of the seven officials citing the
council's work, two said that their meetings assisted them in developing
their state plans. Officials from another state said they turned to the
council for information when they were unable to obtain answers about
implementation from other sources.

Data Quality Issues and Tight Timelines Were Cited as Impeding
Implementation Efforts

Concern about the quality and reliability of student data was the most
frequently cited impediment to implementing student proficiency
requirements. More than half of the state and school district officials we
interviewed cited this concern. For example, officials in California
indicated that they could not obtain racial and ethnic data-used to track
the progress of designated student groups-of comparable quality from their
school districts.24 Officials in Illinois reported that about 300 of its
1,055 districts had problems with data accuracy, resulting in those
schools' appealing their progress results to the state. Similarly,
officials in Indiana acknowledged data problems but said addressing them
would be challenging. Inaccurate data may result in states incorrectly
identifying schools as not meeting annual goals and incorrectly trigger
provisions for school choice and supplemental services. GAO, Education's
Inspector General, and other auditing groups have also reported the
challenges states face in gathering and processing accurate and reliable
student data. For example, in a 2004 report, Education's Inspector General
reported that many states lacked procedures and controls necessary to
report reliable student data. Another auditing group reported that some
states were not reporting accurate student data to Education and
recommended that Education take steps to help states address data accuracy
problems.25

Although NCLBA focuses primarily on the state's responsibility to ensure
data reliability and validity, Education also has a critical role in
assisting states to improve the quality of data used for assessment and
reporting. NCLBA requires the Secretary of Education to provide an annual
report to Congress that includes national and state-level data on states'
progress in implementing assessments, the results of assessments, the
number of schools identified as needing improvement, and use of choice
options and supplemental services. Education officials acknowledged the
need to share responsibility with the states to improve data quality so
data provided to Congress are valid and reliable. According to Education
officials, they are working with states to monitor state data quality
policies and establish a common set of data definitions. Education also
has begun a multiyear pilot project related to data reporting. However,
while one of the primary goals of this effort is to improve the quality of
state data, this

24California officials told us that a bill had recently passed in its
state legislature that may address this issue.

25Texas State Auditor's Office, A Joint Audit Report on the Status of
State Student Assessment Systems and the Quality of Title I School
Accountability Data, SAO Report No. # 02-064, (Austin, Texas: Aug. 2002).

long-term project will not address states' problems with the data they are
now using to report on student progress. In addition to reporting data
quality concerns, officials from about half of the 21 states said that
tight timelines impeded implementation of student proficiency
requirements, even though many of those requirements built upon provisions
in the previous reauthorization of ESEA. That previous law required states
to test students in reading and math in three grades to measure if schools
were making progress. However, a majority of states did not have approved
assessment systems in place when NCLBA was enacted.26 NCLBA set specific
time frames, because many states had not been taking the necessary steps
to position themselves to meet requirements. Those states that had taken
steps to meet the earlier requirements were generally better positioned to
meet NCLBA requirements.

Officials we interviewed from 5 states said that they had very little time
to develop their state plans. They said that developing a system that
meets NCLBA requirements for measuring student proficiency for all
students and selected subgroups was complicated, and they had to resolve
many issues before their systems could be up and running. States that
already had a state system for measuring school progress in place prior to
NCLBA faced other challenges. These states had to determine how they would
reconcile parts of their existing systems with NCLBA's requirements in
order to submit their plans to Education on time.

Officials from 6 states said it was difficult for the state to notify
schools of their status in meeting proficiency goals in a timely fashion.
Many states test students in the spring, and NCLBA requires that test
results be processed before the beginning of the next school year in order
for districts to identify which schools did not make progress, as
illustrated in figure 7. However, many factors may make it difficult to
meet these deadlines, such as identifying and correcting errors in student
data.

26According to Education, shortly after NCLBA was enacted and prior to the
statutory deadline, all states without approved assessment systems were
under either a timeline waiver or a compliance agreement with specific
deadlines for full compliance.

Figure 7: Example of a Timeline to Determine School's Proficiency Status

2003

       April           May               July           August     September  
      Testing                                        District                 
    period for    Test booklets   District receives  notifies     School year 
      school       transported    preliminary        schools of       starts.
                 from district to test results from  their        
year 2002-03.    vendor for        the state.     results.     
                                                     Schools may  
                                                        appeal    
                                                       testing    
                     scoring.                          results.   

July-August

District reviews test results to verify and
reconcile data discrepancies such as lost or
duplicate test booklets and to ensure that students
that have moved are assigned to the correct school.

Using preliminary test results, district calculates
whether each school met state progress goals.

Source: GAO analysis of public school documents.

Note: This example reflects a sample timeline for a public school
district. Different states and districts may and do have different
timelines for these steps.

Officials in 12 of the 21 states where we interviewed said that the lack
of clear and timely guidance and information from Education has impeded
their efforts to implement NCLBA's student proficiency requirements.
Several officials said that Education's communications with them were not
timely and sometimes changed. Other officials said that Education was not
timely in resolving issues Education had with their plans. In response,
Education officials told us they provided states draft guidance on plan
requirements, and subsequent changes were made in order to be responsive
to the concerns of state officials. Education officials told us that it
was challenging to provide the support states needed to implement NCLBA's
proficiency requirements so that states could begin assessing students in
the 2002-03 school year. They also said it was challenging, because the
support often needed to be tailored, given the varied ways states chose to
measure student proficiency.

  Education Has Aided States in Developing Their Plans and Assessment Systems
  but Did Not Have Written Plans to Help States Meet NCLBA Provisions

Education aided states in developing their plans in several ways,
including having peer review teams evaluate plans on site and allowing
states flexibility in implementing some NCLBA requirements. As of July 31,
2004, Education had fully approved 28 plans as meeting all NCLBA
requirements; the remaining states had approval with conditions. In
addition, 17 states did not have approved academic standards and testing
systems in place to meet the requirements of the 1994 law, even though
they are the primary means by which the law requires states to determine
student proficiency. According to Education officials, the department has
been continually monitoring states progress in meeting conditions and has
been working with states to meet prior and NCLBA requirements for
standards and assessment systems. However, Education officials told us
that they did not have a written process to track that states are taking
steps toward meeting the conditions set for full approval of their plans
or to document states' progress in meeting NCLBA requirements for the
expanded standards and assessment systems required under NCLBA.

Education Aided States in Education aided states in developing their plans
for measuring student Developing Their Plans for progress and provided
technical assistance for implementing them. The Measuring Student
department helped states by having peer review teams examine and

provide suggestions about the plans, allowing states flexibility in
adheringProgress to certain NCLBA requirements and issuing guidance to
clarify key aspects of the law. (See table 1.)

Table 1: Primary Methods Education Used To Support State Planning and
Implementation Efforts

Method Education used
to support states Purpose

Peer review	To review and provide on-site suggestions to state officials
as they were developing their plans

Technical assistance 	To assist states in developing state plans as well
as implementing other aspects of NCLBA

Guidance 	To clarify requirements in NCLBA so that states understood their
roles and responsibilities with respect to NCLBA

Flexibility 	To help states deal with challenges they faced in
implementing some proficiency requirements, both in general and on a
caseby-case basis

Source: GAO analysis of Education's processes for supporting state
efforts.

As required by NCLBA, Education assembled a team of experts, consisting of
Education officials and external members drawn from state education
agencies and other organizations familiar with student assessments and
accountability, to review and provide states with advice on their plans.
In reviewing them, the peer review teams identified areas where states
were not meeting NCLBA requirements and closely examined areas that were
particularly complex, such as their methods for measuring student progress
goals. Peer reviewers also met with state officials on-site to discuss
their plans and to suggest ways to improve them. Following the reviews,
the teams presented the results to Education. The department then used
this information to determine the extent to which state plans met
requirements. Education also established a 12-member National Oversight
Panel to review state plans and advise Education of the extent of their
completeness. This panel, which met monthly, was composed of parents,
teachers, local education agency officials, and state education officials
with knowledge about a range of areas, including standards and assessments
and the needs of low-performing schools.

Education also provided states with technical assistance to implement
their plans. The department hosted conferences where it provided
information on requirements for state plans. Education also contracted
with the Council of Chief State School Officers to provide technical
assistance to states. The council has held meetings and workshops as well
as issued instructional publications about implementing different NCLBA
requirements.

Additionally, Education issued guidance in a number of areas to assist
states in their implementation efforts. For example, Education issued
guidance explaining state responsibilities for monitoring NCLBA
implementation and for providing schools with technical assistance,
including the kinds of assistance they must provide to schools identified
as needing improvement.27 Education also issued guidance addressing
actions states should take if schools do not meet their goals and
explaining the purpose of supplemental educational services and state
responsibility for providing and monitoring the receipt of such services.

Education also allowed all states flexibility to address difficulties they
experienced implementing some requirements and granted additional
flexibility to states on a case-by-case basis. For example, Education
granted all states greater flexibility in determining how students with
limited English proficiency could be assessed. Education no longer
required states to include the reading test results during students' first
year in school. Further, on a case-by-case basis, Education allowed
several states to vary the sizes they set for designated student groups.
For example, Ohio and other states were allowed to use a larger group size
for students with disabilities than for other student groups.

In February 2004, Education granted additional flexibility to states by
establishing a process whereby states could propose amendments to their
plans.28 Forty-seven states proposed amendments; for example, some states
proposed to use a 3-year average to calculate the percentage of students
taking state tests, rather than use the annual percentage. This
flexibility may lessen the effects of year-to-year fluctuations in how
many students take the tests. At the conclusion of our review, Education
officials told us they had responded to every state and approved many of
their proposals. Many of these amendments were in response to recently
announced flexibility options. Other amendments were responses to specific
conditions that Education had placed on some state plans before it would
grant full approval. For example, one state amended its plan to resolve
with Education how it would calculate its graduation rate for high school
students.

27See Department of Education, No Child Left Behind: LEA and School
Improvement (Non-Regulatory Guidance), Jan. 2004.

28Several states exercised this authority prior to Education establishing
a process.

Education Approved All State Plans by June 10, 2003, although Most States
Were Approved with Conditions to Meet All NCLBA Requirements

Education's review and approval of state plans included discussions with
state officials and ongoing exchanges of drafts of state plans because of
the uniqueness of each state's educational system. According to Education
officials, the review process was particularly challenging for those
states that did not have existing assessment systems that could provide a
basis for meeting NCLBA requirements. Education also noted that some
states with long-standing assessment systems needed to change their
processes to develop an assessment system that conformed to NCLBA
provisions.

NCLBA required each state's plan to demonstrate that the state had
developed and was implementing a single statewide accountability system,
had determined what constitutes adequate yearly progress for public
schools (e.g., starting points, graduation rates), and had established a
timeline for meeting state proficiency levels by 2014. The law
specifically identified that elements-such as the method for determining
adequate yearly progress-be demonstrated. Thus, a state's assurance that
an element will be implemented in the future would not be sufficient to
meet plan requirements.

NCLBA establishes that the Secretary of Education is responsible for
approving plans. Education developed guidance for states that lists state
plan requirements. (See app III) In reviewing state plans, Education
established two levels of approval: "fully approved," and "approved."
Education designated a plan as "fully approved" if it met all NCLBA
requirements, and "approved" if additional conditions had to be met to
fulfill requirements. Education described an approved plan as one that
demonstrated that, when implemented, the state, its school districts, and
its schools could meet NCLBA provisions. A state was required to have an
approved plan before it could receive its Title I funding for the 2003-04
school year, with release of funds scheduled for July 1, 2003. Education
included the conditions for approval in states' 2003 grant awards.29

On June 10, 2003, Education announced that it had approved all state
plans. Education fully approved 11 of the state plans (Connecticut,
Hawaii, Illinois, Kansas, Mississippi, Missouri, New Jersey, North Dakota,
Oregon, Texas, and Washington) as meeting all NCLBA requirements. Between
June 11, 2003 and July 31, 2004, Education fully approved plans for an
additional 16 states and Puerto Rico. The remaining 23 states and the

29If a state does not meet the conditions cited in the grant award, it is
subject to withholding of administrative funds.

District of Columbia had plans that met some, but not all, requirements,
and were approved with conditions. (See fig 8 for the approval status of
state plans as of July 31, 2004.)

confidence intervals and providing state report card examples. Further, of
these 23 states and the District of Columbia, 4 had to obtain final state
action from their state boards of education or legislatures as their only
condition for receiving full approval from Education.

According to Education officials, these approved plans provided sufficient
assurance that when implemented they would meet NCLBA requirements. For
example, some states provided Education with definitions for how they
would calculate their goals and targets and assurances that the
information would be forthcoming, but did not include the rates and
percentages required by the law. Education officials said that some of
these states did not have enough data to report graduation rates, but that
the states defined how they would do so once they began collecting such
data. Education approved these state plans with the condition that states
collect data on graduation rates and define them in a manner consistent
with their plans.

Education officials told us that they were in frequent communication with
states regarding unmet plan requirements. However, the department did not
have a written process to track interim steps and document that states
meet the identified conditions within a specified time frame. In the
followup letters Education sent to most states, it did not indicate
specific time frames for when it expected states to demonstrate that they
had met all NCLBA requirements. Education officials told us that they did
not have a written process to ensure states are taking steps toward
meeting the conditions set for full approval or what actions the
department would take if states do not meet them.

States Face Challenges in Meeting the 2005-06 NCLBA Requirements for
Standards and Assessment Systems

Standards and assessments are the primary means by which states gauge
student progress. States' current testing is governed by requirements
first enacted by the 1994 ESEA, which required that states assess students
once in each of three grade spans-elementary (3-5), middle (6-9), and high
school (10-12). Under this law, state standards and assessments systems
must meet certain requirements, such as measuring how well students have
learned the academic content taught in school.

As of March 2002, Education had not approved most states' (35) standards
and assessment systems required by the 1994 ESEA. 30 Education granted

30GAO, Title I: Education Needs to Monitor States' Scoring of Assessments,
GAO-02-393, (Washington, D.C.: Apr.1, 2002).

timeline waivers or compliance agreements for those states that did not
demonstrate that they could meet the 1994 ESEA requirements within the
statutory time frame.31 According to Education, enforcement efforts have
included close monitoring of states' progress, for example, agreements
included interim steps to ensure that states are making progress and
submitting quarterly reports to the department. Further, Education's
enforcement efforts have included withholding funds from one state that
did not fulfill its commitments under its timeline waiver. In accordance
with the law, the department withheld 25 percent of Title I administrative
funds from this state for fiscal year 2003. As of July 31, 2004, 35 states
had approved standards and assessment systems and 17 states did not.

By the next school year (2005-06) states will be required to increase the
current level of testing, as required by NCLBA. For example, states will
be required to test students annually in grades 3 through 8 and once in
high school in reading and math. Given the difficulties states experienced
meeting the 1994 requirements, developing new standards and assessment
systems to meet the expanded assessment requirements may be challenging
for states. All states will have to undergo a review and approval process
for these tests to ensure that state standards and assessment systems meet
NCLBA requirements.

Education has taken some steps to guide its review and approval process of
states' standards and assessments systems to meet the 2005-06 time frame.
It issued regulations on implementation in July 2002 and nonregulatory
guidance on the standards and assessments requirements in March 2003. In
April 2004, Education issued guidance to inform states about the
information they will need to demonstrate that their systems meet NCLBA
requirements and help peer reviewers determine whether state systems are
consistent with NCLBA. Finally, Education officials told us that they are
planning to train state Title I directors and to provide additional
outreach to states. Education officials said that they do not intend to
grant any waivers or extensions of time to states that fail to meet the
NCLBA standards and assessment requirements.

31The NCLBA gave states 90 days to show how they would address any aspect
of their standards and assessment systems that did not meet the 1994
requirements. After that 90day window expired, the NCLBA prohibited
Education from granting additional waivers of deadlines for meeting these
requirements. States failing to meet deadlines established by the 1994 law
(or under a waiver or compliance agreement) are subject to a mandatory
withholding of 25 percent of administrative funds. For states that do not
comply with NCLBA requirements, the law authorizes, but does not require,
Education to withhold Title I state administrative funds.

Conclusions

Although Education has undertaken several initiatives to prepare for the
review of state systems to meet the 2005-06 NCLBA deadline, it has not
established a written plan that clearly identifies the steps required,
interim goals, review schedules, and timelines. The assessment systems are
likely to be complex, given the increased number of tests required under
NCLBA. Given the complexity of developing such systems, the department may
find that, similar to its experience with states' compliance with the 1994
law, some states may be challenged to meet NCLBA standards and assessment
system requirements by the 2005-06 school year deadline.32

NCLBA seeks to make fundamental changes in public education. For the first
time, Congress has specified a time frame for when it expects all students
to reach proficiency on state tests showing that they know their state's
academic subject matter. It has also focused attention on closing the
learning gap between key groups of students that have historically not
performed well by also requiring that they be proficient. Achieving the
goal of having all students proficient will be a formidable challenge for
states, school districts, schools, and students. NCLBA provides a
framework to help states achieve this goal and has required states to plan
how they intend to do so. Education has undertaken numerous efforts to
assist states with meeting this challenge. For example, it promulgated
regulations, provided guidance, and reviewed state plans within fairly
tight time frames to meet NCLBA requirements.

Education approved all state plans by June 10, 2003. However, many of
these plans lacked key information regarding how states measure student
proficiency, such as graduation rates. Education approved these plans
conditionally, with the states' assurances that conditions could be met in
the future. As of July 31, 2004, the plans for 23 states and the District
of Columbia had not been fully approved. Although Education officials said
that they have been in frequent communication with these states, the

32In Title I Program: Stronger Accountability Needed for Performance of
Disadvantaged Students (GAO/HEHS-00-89) issued in June 2000, GAO concluded
that most states were not positioned to meet the 1994 ESEA requirement to
collect and report on student assessment by designated subgroups. In
Education's response to the report, it noted that states were not required
to publicly report these data until the 2000-01 school year. Specifically,
Education commented, "the Department is reviewing State final assessment
systems (using external peer reviewers) to ensure compliance with Title I
assessment requirements, including the requirement that States publicly
report disaggregated assessment data." Although Education devoted efforts
to ensure that deadlines were met, only 17 states had approved assessment
systems by the 2000-01 deadline.

department does not have written procedures and specified time frames for
monitoring states' progress for these 24 plans still needing to meet
conditions. Without such tracking mechanisms, Education may not be able to
ensure that required actions are taken in a timely way.

State assessment systems are the foundation for determining whether
students are proficient. NCLBA has significantly increased the amount of
testing, and states are required to have approved NCLBA standards and
assessments by the 2005-06 school year. Education does not have a written
plan that delineates steps and time frames to facilitate its review of
plans to ensure NCLBA time requirements are met. Given Education's recent
experience of a significant number of states that did not meet the 1994
ESEA requirements for standards and assessments systems, the lack of a
written plan could hinder Education's efforts to better position states to
meet the NCLBA requirements.

Furthermore, many state officials indicated they have concerns about the
accuracy of student demographic and test data. Education has also noted
these issues and has undertaken several initiatives to assist states with
their data systems. States and districts have routinely collected student
demographic and test data. However, the need to ensure the data's accuracy
is even more important with the introduction of NCLBA's accountability
requirements. The number of schools that are identified as in need of
improvement has implications for states and school districts, especially
when provisions for school choice and supplemental services become
applicable, as they have for schools in a number of states. Measuring
achievement with inaccurate data is likely to lead to poor measures of
school progress, with education officials and parents making decisions
about educational options on the basis of faulty information.

For those states that have plans that did not meet all NCLBA requirements
and still have conditional approval, we recommend that the Secretary of
Education delineate in writing the process and time frames that are
appropriate for each state's particular circumstances to meet conditions
for full approval.

Further, we recommend the Secretary of Education develop a written plan
that includes steps and time frames so that all states have approved NCLBA
standards and assessment systems by the 2005-06 school year.

To improve the validity and reliability of state data used to determine
whether schools are meeting state goals, we recommend that the

  Recommendations for Executive Action

  Agency Comments
  and Our Evaluation

Secretary of Education further support states' abilities to gather
accurate student data through activities such as disseminating best
practices and designating technical specialists who can serve as resources
to help states.

We provided a draft of this report to Education for review and comment.
Education agreed with our recommendation that it develop a written plan
that includes steps and time frames so all states have approved NCLBA
standards and assessment systems by the 2005-06 school year. Education
noted that such actions are consistent with current departmental efforts
and should help NCLBA implementation. Similarly, the department agreed
with our recommendation to further support states' abilities to gather
accurate student data. Education provided new information in its comments
on efforts to support states' improvements in their data collection
capacities. Consequently, we modified the report to reflect Education's
comments. Education officials also provided technical comments that we
incorporated into the report where appropriate. Education's comments are
reproduced in appendix IV.

Education disagreed with our recommendation that it delineate in writing
the process and time frames that are appropriate for each state's
particular circumstances to meet conditions for full approval of their
state plans. In its comments, Education cited several reasons for
disagreeing with this recommendation. Education stated it has a process of
continuous monitoring, although not in written form, and cited as evidence
of success of its process that all states have used their plans to make
annual progress determinations. However, experience under the 1994 ESEA
has shown that school progress determinations can be made without meeting
all plan requirements. As of July 31, 2004, plans from 23 states and the
District of Columbia have not received full approval, and according to
Education officials, these plans need to meet conditions to be able to
meet NCLBA requirements. We recognize the significant efforts the
department has taken to support states' implementation of NCLBA and its
plans to continue assisting states to improve the performance of their
districts, schools, and students. However, a written delineation,
appropriate to each state's circumstances, of the process and time frames
necessary for the remaining states to meet all conditions would provide
the necessary documentation and assurance to Education, Congress, and the
public that the steps states need to take and the timeframes for their
actions are clear and understood.

In its comments, Education also questioned our statement that it approved
plans without the states meeting all plan requirements. The department

states that no plan was approved unless it demonstrated that when
implemented the state, its districts and schools could meet the
accountability requirements of the law. Thus, Education asserted that GAO
narrowly interpreted approval. We do not disagree with the department's
interpretation of its authority to conditionally approve plans. Instead,
our focus was on whether plans contained all the elements required by
NCLBA and not merely on whether the plan contained an assurance that in
the future it would meet the requirements of the law. We found that many
plans that were conditionally approved did not meet all NCLBA requirements
for what states were to have in their plans, and Education did not dispute
this finding.

We will send copies of this report to the Secretary of Education, relevant
congressional committees, and other interested parties. We will also make
copies available to others upon request. In addition, the report will be
made available at no charge on GAO's Web site at http://www.gao.gov.

Please contact me at (202) 512-7215 if you or your staff have any
questions about this report. Other contacts and major contributors are
listed in appendix V.

Marnie S. Shaul Director, Education, Workforce, and Income Security Issues

Appendix I: Methods to Establish Starting Points

To establish goals for schools to reach in the first year of No Child Left
Behind Act (NCLBA) implementation, states were to set starting points
using student test performance data from the 2001-02 school year. They
computed results for each designated student group and for each school. To
set the starting points, states were required to choose the higher
percentage of students scoring at the proficient level or higher of the
following: 1

(a) the student group with the lowest 2001-02 test performance from among:

1. economically disadvantaged students,

2. students from major racial and ethnic groups,

3. students with disabilities,

4. students with limited English proficiency,

or

(b) the score of the school at the 20th percentile of enrollment when all
schools in the state were ranked according to 2001-02 test performance.

To identify the student group with the lowest 2001-02 test performance,
states had to determine what percentage of students in each of the
designated groups scored at the proficient level on state tests. For
example, a state may have found that 15 percent of students with
disabilities scored at the proficient level, whereas all other groups had
more students do so. In this case, the state would identify the students
with disabilities group as the lowest-performing student group.

To identify the score of the school at the 20th percentile of enrollment,
states had to follow the following process. First, they had to determine
the enrollment and percentage of students that were proficient for each of
their schools. Then, they would rank the schools based on how many
students were proficient in each school. For example, the state may list
schools as shown in the following table.

1For the rest of this appendix, we will refer to scoring at the proficient
level to mean scoring at the proficient level or higher.

Appendix I: Methods to Establish Starting Points

Table 2: Calculating a Starting Point Using the School at the 20th
Percentile in Cumulative Enrollment

                                 Percent scoring at                Cumulative 
                 School name   the proficient level  Enrollment    enrollment 
              Roosevelt H.S.                   25.0          110        1,875 
               Madison Elem.                   21.2           90        1,765 
             Jefferson Elem.                   15.0           75        1,675 
                 Adams Elem.                    9.1          350        1,600 
                Lincoln H.S.                    7.5          700        1,250 
            Washington Elem.                    7.2          550          550 

Source: Cowen, Kristen Tosh. 2004. The New Title I: The Changing Landscape
of Accountability. Washington, D.C.: Thompson Publishing Group, (used with
permission of the publisher).

Beginning with the school at the lowest rank, the state would add the
number of students enrolled until it reached 20 percent of the state's
enrollment. If the state's total student population is 9,375, then the
20th percentile cutoff is 1,875, (9,375 x 20 percent), or Roosevelt H.S.
in this example. At Roosevelt H.S., 25 percent of students were
proficient.

The state would compare the two results. Since the percentage of students
at the proficient level at Roosevelt H.S. (25 percent proficient) was
higher than the results for students with disabilities (15 percent
proficient), the state would set its starting point at 25 percent. For a
school to meet the state's proficiency goal in 2002-03, at least 25
percent of its students would have to score at the proficient level;
however, a first year goal could be set higher.2

2This example draws extensively from Cowan, Kristen Tosh. 2004. The New
Title I: The Changing Landscape of Accountability. Washington, D.C.:
Thompson Publishing Group.

Appendix II: Percentage of Schools That Met State Goals in 2002-03

Percentage of schools that met state Name of state proficiency goals in
2002-03

Alabama

Alaska

Arizona

Arkansas

California

Colorado

                           Connecticut Not available

Delaware

District of Columbia

Florida

Georgia

Hawaii

Idaho

Illinois

Indiana

Iowa

Kansas

Kentucky 60 Louisiana 92 Maine 88 Maryland 65 Massachusetts 76 Michigan 76
Minnesota 94 Mississippi 75 Missouri 51 Montana 80 Nebraska 51 Nevada 60
New Hampshire 69 New Jersey 88 New Mexico 79 New York 76 North Carolina 47
North Dakota 91

Appendix II: Percentage of Schools That Met State Goals in 2002-03

Percentage of schools that met state Name of state proficiency goals in
2002-03

                                    Ohio 78

Oklahoma

Oregon

Pennsylvania

Puerto Rico

Rhode Island

South Carolina

South Dakota

Tennessee

Texas

Utah

Vermont

Virginia

Washington

West Virginia

Wisconsin

Wyoming

Source: GAO analysis of state plans and other information reported by
states.

Note: Connecticut has not yet released figures for the 2002-03 school
year; and Iowa reported only schools that received funds through Title I,
Part A.

                     Appendix III: State Plan Requirements

                      State accountability system element

Accountability system includes all schools and districts in the state.

Accountability system holds all schools to the same criteria.

Accountability system incorporates the academic achievement standards.

Accountability system provides information in a timely manner.

Accountability system includes report cards.

Accountability system includes rewards and sanctions.

The accountability system includes all students

The accountability system has a consistent definition of full academic
year.

2.3 The accountability system properly includes mobile students.

Principle 3: method of adequate yearly progress determinations

3.1 	Accountability system expects all student subgroups, public schools,
and LEAs to reach proficiency by 2013-14.

Accountability system has a method for determining whether student
subgroups, public schools, and LEAs made adequate yearly progress.

3.2a Accountability system establishes a starting point.

3.2b 	Accountability system establishes statewide annual measurable
objectives.

3.2c Accountability system establishes intermediate goals.

The accountability system determines annually the progress of schools and
districts.

The accountability system includes all the required student subgroups.

The accountability system holds schools and LEAs accountable for the
progress of student subgroups.

The accountability system includes students with disabilities.

  5.4 	The accountability system includes students with limited English
  proficiency.

5.5 	The state has determined the minimum number of students sufficient to
yield statistically reliable information for each purpose for which
disaggregated data are used.

5.6 	The state has strategies to protect the privacy of individual
students in reporting achievement results and in determining whether
schools and LEAs are making adequate yearly progress on the basis of
disaggregated subgroups.

                     Appendix III: State Plan Requirements

                      State accountability system element

Accountability system is based primarily on academic assessments.

Accountability system includes graduation rate for high schools.

Accountability system includes an additional academic indicator for
elementary and middle schools.

7.3 Additional indicators are valid and reliable.

Principle 8: separate decisions for reading/language arts and mathematics

8.1 	Accountability system holds students, schools and districts
separately accountable for reading/language arts and mathematics.

Accountability system produces reliable decisions.

Accountability system produces valid decisions.

State has a plan for addressing changes in assessment and student
population.

Accountability system has a means for calculating the rate of
participation in the statewide assessment.

Accountability system has a means for applying the 95 percent assessment
criteria to student subgroups and small schools.

Source: U.S. Department of Education.

Note: Italics in original.

Appendix IV: Comments from the Department of Education

Appendix IV: Comments from the Department of Education

Appendix IV: Comments from the Department of Education

Appendix IV: Comments from the Department of Education

Appendix IV: Comments from the Department of Education

Appendix V: GAO Contacts and Staff Acknowledgments

GAO Contacts 	Harriet C. Ganson, (202) 512-7042, [email protected] Jason S.
Palmer, (202) 512-3825, [email protected]

Staff In addition to those named above, Deborah Edwards, Gilly Martin,
Sherri Doughty, Richard Burkard, Luann Moy, and Sheranda Smith-Campbell
Acknowledgments made key contributions to the report.

Related GAO Products

No Child Left Behind Act: Additional Assistance and Research on Effective
Strategies Would Help Small Rural Districts. GAO-04-909. Washington, D.C.:
September 23, 2004.

Special Education: Additional Assistance and Better Coordination Needed
among Education Offices to Help States Meet the NCLBA Teacher
Requirements. GAO-04-659. Washington, D.C.: July 15, 2004.

No Child Left Behind Act: More Information Would Help States Determine
Which Teachers Are Highly Qualified. GAO-03-631. Washington, D.C.: July
17, 2003.

Title I: Characteristics of Tests Will Influence Expenses; Information
Sharing May Help States Realize Efficiencies. GAO-03-389. Washington,
D.C.: May 8, 2003.

Title I: Education Needs to Monitor States' Scoring of Assessments.
GAO-02-393. Washington, D.C.: April 1, 2002.

Title I Funding: Poor Children Benefit Though Funding Per Poor Child
Differs. GAO-02-242. Washington, D.C.: January 31, 2002.

Title I Program: Stronger Accountability Needed for Performance of
Disadvantaged Students. GAO/HEHS-00-89. Washington, D.C.: June 1, 2000.

  GAO's Mission

Obtaining Copies of GAO Reports and Testimony

The Government Accountability Office, the audit, evaluation and
investigative arm of Congress, exists to support Congress in meeting its
constitutional responsibilities and to help improve the performance and
accountability of the federal government for the American people. GAO
examines the use of public funds; evaluates federal programs and policies;
and provides analyses, recommendations, and other assistance to help
Congress make informed oversight, policy, and funding decisions. GAO's
commitment to good government is reflected in its core values of
accountability, integrity, and reliability.

The fastest and easiest way to obtain copies of GAO documents at no cost
is through GAO's Web site (www.gao.gov). Each weekday, GAO posts newly
released reports, testimony, and correspondence on its Web site. To have
GAO e-mail you a list of newly posted products every afternoon, go to
www.gao.gov and select "Subscribe to Updates."

                             Order by Mail or Phone

The first copy of each printed report is free. Additional copies are $2
each. A check or money order should be made out to the Superintendent of
Documents. GAO also accepts VISA and Mastercard. Orders for 100 or more
copies mailed to a single address are discounted 25 percent. Orders should
be sent to:

U.S. Government Accountability Office 441 G Street NW, Room LM Washington,
D.C. 20548

To order by Phone: 	Voice: (202) 512-6000 TDD: (202) 512-2537 Fax: (202)
512-6061

  To Report Fraud, Contact:

Waste, and Abuse in Web site: www.gao.gov/fraudnet/fraudnet.htm

E-mail: [email protected] Programs Automated answering system: (800)
424-5454 or (202) 512-7470

Gloria Jarmon, Managing Director, [email protected](202)
512-4400Congressional U.S. Government Accountability Office, 441 G Street
NW, Room 7125 Relations Washington, D.C. 20548

Public Affairs 	Jeff Nelligan, Managing Director, [email protected] (202)
512-4800 U.S. Government Accountability Office, 441 G Street NW, Room 7149
Washington, D.C. 20548
*** End of document. ***