Reading First: States Report Improvements in Reading Instruction,
but Additional Procedures Would Clarify Education's Role in
Ensuring Proper Implementation by States (28-FEB-07, GAO-07-161).
The Reading First program was designed to help students in
kindergarten through third grade develop stronger reading skills.
This report examines the implementation of the Reading First
program, including (1) changes that have occurred to reading
instruction; (2) criteria states have used to award sub-grants to
districts, and the difficulties, if any, states faced during
implementation; and (3) the guidance, assistance, and oversight
the Department of Education (Education) provides states. GAO's
study is designed to complement several studies by Education's
Inspector General (IG) in order to provide a national perspective
on some of the specific issues being studied by the IG. For this
report, GAO administered a Web-based survey to 50 states and the
District of Columbia, and conducted site visits and interviews
with federal, state, and local education officials and providers
of reading programs and assessments.
-------------------------Indexing Terms-------------------------
REPORTNUM: GAO-07-161
ACCNO: A66348
TITLE: Reading First: States Report Improvements in Reading
Instruction, but Additional Procedures Would Clarify Education's
Role in Ensuring Proper Implementation by States
DATE: 02/28/2007
SUBJECT: Education
Education program evaluation
Educational standards
Educational testing
Eligibility criteria
Federal aid to states
Federal/state relations
School districts
Surveys
Program implementation
Reading First State Grants program
******************************************************************
** This file contains an ASCII representation of the text of a **
** GAO Product. **
** **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced. Tables are included, but **
** may not resemble those in the printed version. **
** **
** Please see the PDF (Portable Document Format) file, when **
** available, for a complete electronic file of the printed **
** document's contents. **
** **
******************************************************************
GAO-07-161
* [1]Results in Brief
* [2]Background
* [3]States Reported Changes as Well as Improvements in Reading I
* [4]States Reported That Reading Instruction Changed or Improved
* [5]Publishers Reported Making Few Changes to Reading Programs,
* [6]States Awarded Reading First Sub-Grants to School Districts
* [7]States Varied in How they Exercised Their Flexibility to Set
* [8]Some States Reported Difficulty Implementing Key Aspects of
* [9]Education Provided a Wide Range of Guidance, Assistance, and
* [10]Education Provided a Wide Range of Guidance and Assistance
* [11]States Were Generally Satisfied with the Guidance and Assist
* [12]Education Lacked Written Procedures to Guide Its Interaction
* [13]Conclusions
* [14]Recommendations
* [15]Agency Comments
* [16]Survey of States
* [17]GAO Contacts
* [18]Acknowledgments
* [19]GAO's Mission
* [20]Obtaining Copies of GAO Reports and Testimony
* [21]Order by Mail or Phone
* [22]To Report Fraud, Waste, and Abuse in Federal Programs
* [23]Congressional Relations
* [24]Public Affairs
United States Government Accountability Office
Report to Congressional Requesters
GAO
February 2007
READING FIRST
States Report Improvements in Reading Instruction, but Additional
Procedures Would Clarify Education's Role in Ensuring Proper
Implementation by States
GAO-07-161
Contents
Letter 1
Results in Brief 5
Background 7
States Reported Changes as Well as Improvements in Reading Instruction
Since the Inception of Reading First 13
States Awarded Reading First Sub-Grants to School Districts Using a
Variety of Different Criteria, and Some States Reported Difficulties with
Implementation 20
Education Provided a Wide Range of Guidance, Assistance, and Oversight,
Generally Satisfying State Officials, but Education Lacked Controls to
Guard against Mandating or Endorsing Curricula and Did Not Provide Written
Monitoring Procedures 24
Conclusions 35
Recommendations 36
Agency Comments 36
Appendix I Objectives, Scope, and Methodology 38
Appendix II Descriptions of Assessment Types 41
Appendix III Comments from the Department of Education 42
Appendix IV GAO Contacts and Staff Acknowledgments 53
Tables
Table 1: Reading Program Publishers Most Frequently Represented on States'
Approved Lists 18
Table 2: Approaches States Used to Develop Their Approved Lists 19
Table 3: Guidance and Assistance Provided to States during Application and
Implementation Processes 27
Table 4: States' Reported Sources of Guidance for Key Reading First
Implementation Components 31
Figures
Figure 1: The Process for Awarding State Sub-grants to School Districts 11
Figure 2: Improvements in Professional Development for Reading First
Teachers 16
Figure 3: All School Districts Nationwide That Were Eligible for, That
Applied for, and Were Awarded Reading First Sub-grants in the First School
Year of Funding 22
Figure 4: States Level of Ease or Difficulty Implementing Key Reading
First Program Aspects 23
Figure 5: Issues Education Raised About States' Reading First Applications
25
Figure 6: Geographic Regions for Reading First Regional Technical
Assistance Centers 26
Figure 7: States' Assessment of Whether Written Guidance Provided
Information Needed to Adequately Address Application Components 30
Abbreviations
AIR American Institutes for Research
CRRFTAC Central Regional Reading First Technical Assistance
Center
DEOA Department of Education Organizing Act
DIBELS Dynamic Indicators of Basic Early Literacy Skills
Education Department of Education
ERRFTAC Eastern Regional Reading First Technical
Assistance Center
IG Inspector General
Interim Report Education's Reading First Implementation
Evaluation: Interim Report
K-3rd kindergarten through third grade
NCLBA No Child Left Behind Act
NIFL National Institute for Literacy
RMC RMC Research Corporation
SEA state education agency
SBRR scientifically-based reading research
TAC Technical Assistance Center
WRRFTAC Western Regional Reading First Technical
Assistance Center
This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed in
its entirety without further permission from GAO. However, because this
work may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this material
separately.
United States Government Accountability Office
Washington, DC 20548
February 28, 2007
The Honorable Edward M. Kennedy
Chairman
The Honorable Michael B. Enzi
Ranking Minority Member
Committee on Health, Education, Labor, and Pensions
United States Senate
The Honorable George Miller
Chairman
Committee on Education and Labor
House of Representatives
The Honorable Jeff Bingaman
United States Senate
The Honorable Richard G. Lugar
United States Senate
The Reading First program, signed into law in 2002 as part of the No Child
Left Behind Act (NCLBA), was designed based on reading research to help
children in kindergarten through third grade develop stronger reading
skills. Since 2002, Congress has appropriated about $1 billion a year for
Reading First, more than for any other federal reading program, to fund
6-year grants to states. States, in turn, have awarded sub-grants to
school districts to establish reading programs. These sub-grants are
targeted to districts and schools with the highest percentage or numbers
of students in kindergarten through third (K-3rd) grade reading below
grade level and to districts and schools with large numbers of low-income
students. However, in awarding sub-grants, states do have some discretion
to establish priorities, such as giving priority to districts that
demonstrated leadership and commitment to improving reading. The Reading
First program, signed into law in 2002 as part of the No Child Left Behind
Act (NCLBA), was designed based on reading research to help children in
kindergarten through third grade develop stronger reading skills. Since
2002, Congress has appropriated about $1 billion a year for Reading First,
more than for any other federal reading program, to fund 6-year grants to
states. States, in turn, have awarded sub-grants to school districts to
establish reading programs. These sub-grants are targeted to districts and
schools with the highest percentage or numbers of students in kindergarten
through third (K-3rd) grade reading below grade level and to districts and
schools with large numbers of low-income students. However, in awarding
sub-grants, states do have some discretion to establish priorities, such
as giving priority to districts that demonstrated leadership and
commitment to improving reading.
The Reading First program places a number of requirements on state and
local grant recipients to ensure they adopt reading programs and methods
that are effective, while, at the same time, NCLBA provisions restrict the
ability of Department of Education (Education) officials to mandate or
direct state and local decisions on reading curriculum. In particular,
Reading First requires states and participating school districts to adopt
scientifically-based reading programs containing key instructional
components identified in the law. In some instances, state officials have
The Reading First program places a number of requirements on state and
local grant recipients to ensure they adopt reading programs and methods
that are effective, while, at the same time, NCLBA provisions restrict the
ability of Department of Education (Education) officials to mandate or
direct state and local decisions on reading curriculum. In particular,
Reading First requires states and participating school districts to adopt
scientifically-based reading programs containing key instructional
components identified in the law. In some instances, state officials have
required prospective Reading First districts to choose their reading
program from a state-approved list. In contrast, other states have opted
to vest district applicants with responsibility for researching and
selecting the reading programs to be used in their Reading First schools.
States and participating school districts are also required to provide
professional development in reading that is based on scientific research
and to track students' progress in reading using valid and reliable
assessments. However, NCLBA and various other statutory provisions place
limits on what Education officials can require states, districts, and
schools to do. For example, under NCLBA, Education officials are not
authorized to mandate, direct, or control state, school district or school
curriculum or program of instruction.1 In other words, these provisions
are intended to preserve state and local control over key aspects of the
public school system.
During the implementation of the Reading First program some concerns were
raised by publishers of reading programs to Congress and others about
Education's interactions with state and local grantees regarding reading
programs and assessments. Specifically, several groups representing
reading programs filed complaints with Education's Inspector General
alleging that Education officials, contractors, and consultants pressured
state and local applicants to choose specific reading programs and
assessments--actions that are expressly prohibited by NCLBA.
In response to these allegations, Education's Inspector General (IG)
undertook several separate investigations and audits of various aspects of
the program's administration. For the first of these reports, issued in
September 2006, the IG reviewed various aspects of the application process
for awarding Reading First grants to states, including the selection and
composition of the expert panel that reviewed state applications,
application review and feedback procedures, and interactions between
Education and state-level officials.2 During its review, the IG, among
other things, interviewed federal and selected state officials and
reviewed departmental guidance, internal e-mail correspondence, and state
applications from 11 states and 1 territory. Among its findings, the IG
reported that the Department intervened to influence a state's and several
school districts' selection of reading programs, actions that call into
question whether program officials violated statutory prohibitions against
directing or controlling state and local curricular decisions. 3 The IG
made a number of related recommendations, including that the Department
should (1) develop internal management policies and procedures to ensure
that programs are managed in compliance with applicable laws and
regulations, (2) review the management and staff structure of the Reading
First program office, and (3) develop guidance for program officials on
statutory prohibitions on intervening in state and local curricular
decisions. The Secretary of Education concurred with all of the report's
recommendations. 4 In their written comments to the IG report, Education
officials detailed numerous action steps they planned to undertake in
response to the report's findings and recommendations, including the
appointment of new internal leadership for the Reading First program,5 the
development of annual training on internal controls, and the drafting of a
memorandum for all program managers regarding the importance of impartial
job performance.
1 20 U.S.C. S 7907.
2 U.S. Department of Education Inspector General, The Reading First
Program's Grant Application Process: Final Inspection Report (Washington
D.C.: September 2006).
GAO's study is designed to complement the IG's work by providing a
national perspective on some of the specific issues being studied by the
IG, as well as providing some information on how the program has been
implemented at the state level, primarily. To obtain a better
understanding of how the Reading First program is being implemented at the
state level and to determine how Education conducts its interactions with
states and the kind of guidance and oversight it provides, GAO agreed to
answer the following questions:
1. What changes have occurred to reading instruction since the
inception of Reading First?
2. What criteria have states used to award Reading First
sub-grants to districts, and what, if any, difficulty did states
face in implementing the program?
3. What guidance, assistance, and oversight did Education provide
states related to the Reading First program?
3 Section 103(b) of the Department of Education Organizing Act (DEOA)
states that "No provision of a program administered by the Secretary or by
any other officer of the Department shall be construed to authorize the
Secretary or any such officer to exercise any direction, supervision, or
control over the curriculum, program of instruction, administration, or
personnel of any educational institution, school or school system....over
the selection or content of...textbooks, or other instructional materials
by any educational institution or school system, except to the extent
authorized by law." 20 U.S.C. S 3403(b).
4 The Office of Management and Budget, in its Circular A-50, requires
federal agencies to establish audit follow-up systems that ensure that
audit findings and recommendations are addressed. Under Circular A-50,
audit follow-up is a shared responsibility of agency management and the
auditors.
5 The Reading First Program Director resigned following publication of the
Final Inspection Report.
To answer these questions, we collected information about the Reading
First program from a variety of sources. We conducted a Web-based survey
of the Reading First Directors in all 50 states and the District of
Columbia, and 100 percent responded. We also obtained and analyzed data
from the Department of Education for each state on Reading First
districts' eligibility, applications, and awards for states' first school
year of funding. To assess the reliability of this data, we talked to
agency officials about data quality control procedures and reviewed
relevant documentation. We also conducted semi-structured follow-up
interviews with Reading First Directors in 12 states and conducted site
visits to 4 of the 12 states. During the site visits, we met with state
and district officials, as well as school officials, including teachers,
principals, and Reading First coaches. We selected these 12 states to
represent diversity in terms of geographic distribution, grant size,
poverty rates, and student reading achievement, as well as the presence or
absence of a statewide list of approved reading programs. In our selection
of the 12 states, we also took into consideration whether or not state
officials reported that they had received guidance from Education
officials advocating for or against particular reading programs or
assessments. For both the survey and follow-up interviews, we agreed to
protect the respondents' confidentiality to encourage candid responses:
therefore; we will not identify states by name in this report. We also
interviewed administrators from each of the three regional Reading First
Technical Assistance Centers (TAC), officials from RMC Research--which
provided technical assistance to states during the application process and
oversaw the TACs, and the American Institutes for Research (AIR)--the
contractor responsible for conducting monitoring visits in each state. In
addition, we interviewed several publishers and other providers of reading
curricula and assessments, as well as Reading First officials and
attorneys at Education. We also reviewed state grant files, monitoring
reports, and related correspondence for the 12 states where we conducted
follow-up interviews, as well as legislation and guidance related to
Reading First. We also interviewed Education officials about the
implementation of the Reading First program. Finally, we reviewed
Education's study "Reading First Implementation Evaluation: Interim
Report" to identify information collected at the district and school level
that related to the findings from our state survey and also the two
relevant reports that had been issued by Education's Inspector General at
the time our audit work was completed.
See appendix I for additional information on the survey, as well as our
other data collection methods. We conducted our work from December 2005
through January 2007 in accordance with generally accepted government
auditing standards.
Results in Brief
Since the inception of Reading First, states reported changes as well as
improvements in reading instruction due to increased emphasis on reading
instruction, assessments, and professional development, despite limited
changes to instructional material. In our survey, 69 percent of all states
reported great or very great improvement in reading instruction. One
specific area in which states reported an improvement is the degree to
which classroom instruction explicitly incorporates the key instructional
components of the program. In addition, state and local officials we
interviewed reported that the use of assessments changed after Reading
First, especially in the way that teachers use data from these assessments
to better inform reading instruction. All states reported improvement in
professional development with more than 80 percent of states reporting
that professional development for reading teachers improved greatly or
very greatly. Further, over 75 percent of states reported great or very
great increases in the frequency of and resources devoted to teacher
professional development. At the same time, the publishers we interviewed
reported making few significant changes to instructional materials as a
result of Reading First. While three of the four major publishers reported
that they had not made significant changes, some of these publishers
stated that they had made minor changes to their reading materials to
highlight the key instructional components of Reading First.
States awarded Reading First sub-grants to districts using a variety of
different eligibility and award criteria, and some states reported
difficulties with implementing key aspects of the program. States varied
in how they defined and established eligibility and award criteria as
permitted under Reading First. After applying eligibility criteria,
Education reported that over 3,400 school districts--or about 20 percent
of all districts nationwide--were eligible to apply for Reading First
sub-grants in the states' first school year of funding. After applying
award criteria to the nearly 2,100 eligible districts that applied,
Education reported that nearly 1,200 school districts--or about 34 percent
of all eligible districts and nearly 7 percent of all districts
nationwide--received Reading First funds. With respect to implementation
difficulties, 22 states reported that it was difficult or very difficult
to help districts with reading scores that had not improved sufficiently.
Furthermore, 17 states reported that it was either difficult or very
difficult to assess how districts applied scientifically-based reading
research in choosing their reading program.
Education officials made a variety of resources available to states during
the application and implementation process, including written guidance and
criteria, workshops, conferences, dedicated contractors and regional
technical assistance centers, as well as written feedback from both the
application review panel and annual monitoring visits. States were
generally satisfied with the various forms of Reading First guidance and
assistance they received, but Education failed to establish controls to
guard against mandating or directing curricula and clear monitoring
procedures. For example, state officials were generally positive in
characterizing the assistance they received from Education officials and
contractors, with officials from 48 states reporting in our survey that
Education staff were helpful or very helpful in addressing states'
implementation-related questions and officials from 41 states reporting
the same for the TACs. However, Education developed no written or other
formal guidance outlining policies and procedures to guide the behavior of
Education officials or contractors in providing Reading First guidance and
assistance to states. Responses from several state officials suggest that
Education officials may have violated the statutory prohibition against
mandating or directing local curricular decisions by effectively endorsing
or directing the selection of particular Reading First programs. According
to responses to our survey, Reading First officials or contractors made
specific recommendations about reading programs and assessments to
officials in several states. Specifically, officials from 10 states
reported receiving suggestions that they eliminate specific programs or
assessments, and officials from 4 states reported receiving suggestions to
adopt specific programs or assessments. Five states also reported
receiving recommendations from Reading First officials to change some of
their proposed professional development providers. In addition, while
Education officials laid out an ambitious plan for annually monitoring
every state's implementation, Education did not develop transparent
procedures to guide its monitoring visits and, as a result, states did not
always understand monitoring procedures, timelines, and expectations. For
example, officials from one state said they were unaware of Education's
expectation that states respond to monitoring findings, and officials from
another state told us that Education officials never responded to the
justification they submitted in response to specific monitoring findings.
We are making two recommendations to Education. First, in addition to
addressing the IG's recommendations to develop internal (1) policies and
procedures to guide program managers on when to solicit advice from
General Counsel and (2) guidance on the prohibitions imposed by section
103(b) of the Department of Education Organizing Act (DEOA), we recommend
that Education also establish control procedures to guide departmental
officials and contractors in their interactions with states, districts,
and schools. We also recommend that Education develop and distribute
guidelines regarding its monitoring procedures so that states and
districts are aware of their roles, responsibilities, and timelines. In
its comments on a draft of this report, Education agreed with our
recommendations and indicated that it will take actions to address them.
Background
Reading First, which was enacted as part of NCLBA, aims to assist states
and local school districts in establishing reading programs for students
in kindergarten through third grade by providing funding through 6-year
formula grants. The goal of the program is to ensure that every student
can read at grade level or above by the end of third grade. To that end,
Reading First provides funds and technical assistance to states and school
districts to implement programs supported by scientifically-based reading
research (SBRR), increase teacher professional development based on this
research, and select and administer reading assessments to screen,
diagnose, and monitor the progress of all students. NCLBA defines SBRR as
research that (1) uses systematic, empirical methods that draw on
observation or experiment; (2) involves rigorous data analyses that test
stated hypotheses and justify general conclusions; (3) relies on
measurements or observational methods that are valid; and (4) has been
accepted by a peer-reviewed journal or approved by a panel of independent
experts. Further, NCLBA requires states to adopt reading programs that
contain the five essential components of reading-- (1) phonemic awareness;
(2) phonics; (3) vocabulary development;
(4) reading fluency, including oral reading skills; and (5) reading
comprehension strategies.6
While Education has responsibility for overseeing the Reading First
program and states' implementation and compliance with statutory and
program requirements, NCLBA places restrictions on what Education
officials can require states to do. Specifically, Education is not
authorized to mandate, direct, control, or endorse any curriculum designed
to be used in elementary or secondary schools.7 Further, when Education
was formed in 1979, Congress was concerned about protecting state and
local responsibility for education and, therefore, placed limits in
Education's authorizing statute on the ability of Education officials to
exercise any direction, supervision, or control over the curriculum or
program of instruction, the selection of textbooks or personnel, of any
school or school system.8
Every state could apply for Reading First funds, and states were required
to submit a state plan for approval that demonstrates how they will ensure
that statutory requirements will be met by districts. Education, working
in consultation with the National Institute for Literacy (NIFL), as
required in NCLBA, established an expert review panel composed of a
variety of reading experts to evaluate state plans and recommend which
plans should be approved. In these plans, states were required to describe
how they would assist districts in selecting reading curricula supported
by SBRR, valid and reliable9 reading assessments, and professional
development programs for K-3rd grade teachers based on SBRR. The law does
not call for Education to approve or disapprove particular reading
programs or curricula identified in state plans. When appropriate, the
peer review panel was to also recommend clarifications or identify changes
it deemed necessary to improve the likelihood of a state plan's success.
NCLBA requires that Education approve each state's application only if it
meets the requirements set forth in the law.
6 These components were identified as integral parts of effective reading
programs by the National Reading Panel, or NRP. Phonemic awareness refers
to the ability to hear, identify, and manipulate the individual
sounds--phonemes--in spoken words. Phonics refers to the relationship
between the letters of written language and the sounds of spoken words.
Vocabulary development refers to the development of stored information
about the meanings and pronunciations of words necessary for
communication. Reading fluency refers to the ability to read text
accurately and quickly. It provides a bridge between word recognition and
comprehension. Reading comprehension refers to the understanding,
remembering, and communicating with others about what has been read.
7 20 U.S.C. S 7907.
8 20 U.S.C. S 3403.
9 Data collected must be valid and reliable to be considered accurate.
Valid refers to the degree that it actually measures what it claims to
measure. Reliable refers to the degree to which it consistently measures
what it claims to measure.
Reading First allows states to reserve up to 20 percent of their funds for
professional development; technical assistance; and planning,
administrative, and reporting activities.10 For example, states can use
their funds to develop and implement a professional development program to
prepare K-3rd teachers in all essential components of reading instruction.
One model for supporting teachers' reading instruction involves hiring a
Reading Coach who works with teachers to implement reading activities
aligned with SBRR. Almost all states require Reading First schools to have
a Reading Coach tasked with supporting teachers and principals with
instruction, administering assessments, and interpreting assessment data.
States that receive Reading First grants are required to conduct a
competitive sub-grant process for eligible school districts and must
distribute at least 80 percent of the federal Reading First grants they
receive to districts. NCLBA and Education guidance provides states with
flexibility to set eligibility criteria for school districts so that
eligible districts are among those in the state that have the highest
number or percentage of K-3rd grade students reading below grade level and
(1) have jurisdiction over an empowerment zone or enterprise community,11
(2) have a significant number or percentage of schools identified as in
need of improvement,12 or (3) are among the districts in the state that
have the highest number or percentages of children counted as poor and
school-aged for the purposes of Title I.13 NCLBA establishes priorities
that states must consider when awarding a Reading First sub-grant, while
also allowing states to establish other priority areas. For instance,
NCLBA requires that the state sub-grant process give priority to districts
with at least 15 percent of students or 6,500 children from families with
incomes below the poverty line, but states also have some flexibility to
establish additional priorities, such as a demonstrated commitment to
improving reading achievement. The sub-grant process along with the
criteria at each stage is summarized in figure 1.
10 From this 20 percent, states may spend up to 65 percent on professional
development, up to 25 percent for technical assistance for districts and
schools, and up to 10 percent for planning, administration, and reporting
activities.
11 Empowerment zones and enterprise zones are defined in 26 U.S.C. S 1392.
Both are local areas of high poverty that meet certain eligibility
requirements to receive specified forms of aid or regulatory flexibility.
12 Under NCLBA, each state creates its own content standards, academic
achievement tests, and proficiency levels. States are required to test all
children for reading and mathematics achievement annually in grades 3-8 to
determine whether schools are making adequate yearly progress (AYP).
Districts must identify as a school in need of improvement any school that
has not made AYP for 2 consecutive years.
13 Title I of Elementary and Secondary Education Act, as amended and
reauthorized by NCLBA, authorizes federal funds to help elementary and
secondary schools establish and maintain programs that will improve the
educational opportunities of economically disadvantaged children. Title I
is the largest federal program supporting education in kindergarten
through 12th grade.
Figure 1: The Process for Awarding State Sub-grants to School Districts
Districts are required to use their sub-grant funds to carry out certain
activities identified in NCLBA. For example, districts must use these
funds to select and implement reading programs based on SBRR that include
the essential components of reading instruction, to select and implement
diagnostic reading assessment tools, and to provide professional
development opportunities for teachers. Additionally, districts are
permitted to use Reading First funds in support of other activities, such
as training parents and tutors in the essential components of reading
instruction.
States are required to report to Education annually on the implementation
of Reading First, including their progress in reducing the number of
students who are reading below grade level. Additionally, states are
required to submit a mid-point progress report to Education at the end of
the third year of the grant period. These mid-point progress reports are
subject to review by the same expert peer review panel that evaluated
state applications. If Education determines, after submission and panel
review of a state's mid-point progress report and on the basis of ongoing
Education monitoring, that a state is not making significant progress,
Education has the discretion to withhold further Reading First grant
payments from that state. While these state reports to Education are
intended to provide information on the effectiveness of Reading First,
Education is also required to contract with an independent organization
outside Education for a rigorous and scientifically-valid, 5-year,
national evaluation of the program, with a final report scheduled to be
issued in 2007.
The Reading First program has relied on several key contractors to perform
a number of program functions. For example, Education officials hired RMC
Research Corporation, a company that provides research, evaluation, and
related services to educational and human services clients, to provide
technical assistance to states and districts that have received Reading
First funding. According to Education officials, RMC contractors were
tasked initially with providing specific, individualized guidance on the
application process to state officials who requested it. RMC later became
the national coordinator for the contract overseeing the National Center
for Reading First Technical Assistance and its three regional
subsidiaries: the Eastern Regional Reading First Technical Assistance
Center (ERRFTAC) in Tallahassee, Florida; the Central Regional Reading
First Technical Assistance Center (CRRFTAC) in Austin, Texas; and the
Western Regional Reading First Technical Assistance Center (WRRFTAC) in
Eugene, Oregon. In this role, RMC staff provides support to the TACs and
their employees, as well as weekly coordination among the TACs, and
regular training seminars. Operated out of universities recognized by
Education officials for their expertise in SBRR and related areas, the
centers began operations in 2003 and are responsible for providing an
array of technical assistance activities to states, including national and
regional conferences, training and professional development, products and
materials, and liaisons to national reading experts. Education officials
also contracted with Learning Point Associates to provide technical
assistance to states as they launched their sub-grant competitions.
Once Reading First sub-grants had been awarded to local districts,
Education contracted with the American Institutes for Research (AIR), a
behavioral and social science research organization, to conduct annual
monitoring visits to each state. These visits incorporate sessions with
state officials, as well as visits to a few districts in each state and
are designed to assess states' and districts' compliance with their
approved plans. After each monitoring visit, AIR representatives submit a
report, including any findings of non-compliance, to Reading First
officials. Reading First officials are to forward these reports to the
cognizant state officials.
States Reported Changes as Well as Improvements in Reading Instruction Since the
Inception of Reading First
States reported that there have been a number of changes and
improvements14 in reading instruction since the implementation of Reading
First. There has been an increased emphasis on the five key components of
reading,15 assessments, and professional development with more classroom
time being devoted to reading activities. However, according to publishers
we interviewed, there have been limited changes to instructional material.
Similarly, many states that approved reading programs for districts to
choose from report few changes to their lists of approved programs.
14 Education has contracted with an independent research firm to conduct a
5-year, quantitative evaluation of the Reading First program, evaluating
the effect of Reading First activities on improving reading instruction.
In addition, in their annual reports and mid-point evaluations, states
must include their progress in reducing the number of students in grades
K-3 reading below grade level.
15 The five key components of reading include (1) phonemic awareness; (2)
phonics; (3) vocabulary development; (4) reading fluency, including oral
reading skills; and (5) reading comprehension strategies.
States Reported That Reading Instruction Changed or Improved Following the
Implementation of Changes under Reading First
In responding to our survey, 69 percent of all states reported great or
very great improvement in reading instruction since inception of Reading
First. One area in which states16 reported a change that may have
contributed to improvement of reading was the degree to which classroom
instruction explicitly incorporated the five key components. In our
survey, at least 39 states reported that Reading First schools had
incorporated each of the five required components of reading into
curriculum to a great or very great degree as a result of Reading First.
State and local officials we talked to during some of our site visits
reinforced this opinion and in particular noted that Reading First
teachers had awareness of and were more focused on the five components. In
addition, the increased time devoted to reading activities under Reading
First may have contributed to improvement. Several district officials we
met with told us they were including a protected, uninterrupted block of
time for reading instruction of 90 minutes or more per day--which the
department's Guidance for the Reading First Program lists as a key element
of an effective reading program. Education's Reading First Implementation
Evaluation: Interim Report (The Interim Report)17 also found that Reading
First teachers reported allocating over 90 minutes per day, on average,
for a designated reading block.
States officials reported improvement in reading instruction resulting
from the use of assessments.18 In responding to our survey, one state
official said, "One of the strengths of the Reading First program has been
its strong adherence to SBRR and to the use of valid and reliable
assessments in guiding instruction and program evaluation." A number of
state and local officials we interviewed reported that the use of
assessments changed after Reading First, especially in the way that
teachers use data from these assessments to better inform reading
instruction. Specifically, district officials we talked to during our site
visits reported that teachers review students' assessment results to
determine the areas in which they need more targeted instruction. One
official also reported that assessment data can sometimes be used to
identify successful teachers from whom other teachers can learn teaching
techniques, with one official asserting that "Reading First has and is
making a great impact on teachers' instructional practices, techniques,
and strategies." Also, according to Education's Interim Report,
researchers estimated that 83 percent of Reading First teachers cited
assessment results essential to organizing instructional groups, 85
percent cited the results essential to determining progress on skills, and
75 percent cited the results essential to identifying students who need
reading intervention.
16 For the purposes of this report, the term "states" also includes the
District of Columbia.
17 Education's "Reading First Implementation Evaluation: Interim Report."
18 The types of assessments are screening, diagnostic, and progress
monitoring. See appendix II for more complete descriptions of each.
According to our survey, most states also reported that the assessments
they used differed greatly or very greatly from the ones they used prior
to Reading First. States reported a wide variety of reading assessments on
their state-approved lists, with over 40 different assessments listed. By
far, the most frequently approved assessment was Dynamic Indicators of
Basic Early Literacy Skills (DIBELS), approved by 45 states. Also, a few
states reported to us that they were moving toward a more uniform
systematic assessment system for the first time, whereas previously each
school could choose which assessment it would use. Some state and district
officials told us that having a more uniform and systematic assessment was
beneficial, because, for instance, it allowed the officials to track and
compare reading scores more easily.
Professional development is another area in which state officials noted
improvement. All states reported improvement in professional development
as a result of Reading First, with at least 41 states reporting that
professional development for reading teachers improved greatly or very
greatly in each of five key instructional areas. Further, a considerable
majority of states reported great or very great increases in the frequency
of professional development and the resources devoted to it, 45 and 39,
respectively. One state reported, "The provision of funding to be used to
support statewide professional development efforts for K-3 reading has
been an important aspect of the program." The Interim Report on the
Reading First program highlights that a vast majority of Reading First
teachers had received training on the five key components of reading. In
our site-visits, district officials confirmed that, for the most part,
teachers in their Reading First classrooms had received training. However,
in responding to our survey, 19 states did report some challenges in
training of 100 percent of Reading First teachers, with teacher turnover
cited by 12 states as the reason some Reading First teachers might not
have taken any type of Reading First training. Figure 2 summarizes
reported improvements in professional development for teachers.
Figure 2: Improvements in Professional Development for Reading First
Teachers
Professional development was provided by a variety of federal, state, and
private sources. Staff from the TACs and officials from at least one state
reported providing professional development to districts customized to the
individual district's needs and perceived future needs. Education's
Interim Report on Reading First implementation noted that state Reading
First coordinators in 33 states reported that state staff chose and
organized all statewide professional development efforts and played a key
role in selecting professional development topics for districts and
schools. In addition, publishers we spoke with told us they often provide
training to acclimate teachers to their products. Certain publishers of
major commercial reading programs and assessments told us that since the
implementation of Reading First, districts demand much more training.
Specifically, according to some of the publishers and TAC staff we spoke
with, districts have been interested in more in-depth workshops on
particular topics such as teaching techniques and using and interpreting
assessments.
Finally, another aspect of professional development pertinent to Reading
First is the presence of a Reading Coach. State and district officials
reported that Reading Coaches receive training that better enables them to
assist schools. Education's Interim Report found that each Reading Coach
worked with an average of 1.2 schools and with 21 teachers to help
implement activities aligned with SBRR.
Publishers Reported Making Few Changes to Reading Programs, and States Reported
Adopting a Variety of Approaches to Select Reading Programs
Three of the four major publishers of reading programs we spoke with
reported that they had not made significant changes to the content of
their reading programs as a result of Reading First. Two publishers stated
that they made minor changes to their reading materials to make more
explicit how the content of the existing programs align with the five
components emphasized in Reading First. Two of them reported that they
made changes to their programs based on the National Reading Panel's
findings, which was prior to the enactment of Reading First. For example,
representatives of one company stated that they launched a new reading
program based on the findings of the National Reading Panel that takes
into account the requirements of Reading First. Despite limited changes to
the actual instructional material, all the publishers noted a greater
emphasis on assessing the efficacy of their reading programs as a result
of Reading First. In an effort to measure the effectiveness of their
programs, the publishers reported devoting more effort to research and to
evaluate how effective their reading programs were at raising reading
assessment scores.
States followed two main approaches in selecting reading programs for
districts --22 identified a state-approved list of programs for districts
to select, while the other 29 did not have a state-approved list, thereby
requiring districts in those states to self-select reading programs and
determine, with some state oversight and subject to state approval,
whether they satisfy the requirements of SBRR. Of the 22 states with
approved lists, reading program publishers most frequently represented on
the lists were Houghton Mifflin, McGraw-Hill, and Harcourt (see table 1).
At the school level, Education found in its Interim Report that these
three reading program publishers were also the most frequently used,
estimating that between 11 and 23 percent of schools used programs from
one of them.
Table 1: Reading Program Publishers Most Frequently Represented on States'
Approved Lists
Publisher name Number of states
Houghton Mifflina 21
McGraw-Hill-Educationb 21
Harcourtc 19
Pearsond 15
Success for All Foundation "Success for All" 11
Source: GAO analysis.
aBecause some states listed publisher (parent company) names instead of
reading program names, our analysis will only reflect overall totals based
on publisher names. "The Nation's Choice," a program published by Houghton
Mifflin, would be included in this category.
bMcGraw-Hill Education includes SRA-"Open Court," SRA-"Reading Mastery,"
and "MacMillan McGraw-Hill Reading."
cHarcourt, Inc. publishing includes "Trophies."
dPearson Scott Foresman includes "Scott Foresman Reading," "Longman ESL,"
and "Scott Foresman Reading Street."
Additionally, of the 22 states that identified a list of approved core
reading programs19 for Reading First, 8 already had a list of approved
core reading programs for adoption by all schools in their state prior to
Reading First. Only two of these states reported removing reading
programs--a total of six--from their lists because they did not meet
Reading First requirements. According to Education's Interim Report, an
estimated 39 percent of Reading First schools reported adopting a new core
reading program at the beginning of the 2004-2005 school year in which
they received their Reading First grant, in contrast with an estimated 16
percent of non-Reading First Title I schools.
States used a variety of sources to help them identify and select reading
programs that met Reading First's criteria. For example, 15 of the 22
states with state-approved lists reported using the Consumer's Guide to
Evaluating A Core Reading Program Grades K-3: A Critical Elements Analysis
to make this decision.20 Other frequently used resources include criteria
in the state's application for Reading First, information obtained at
Reading First Leadership Academies21 provided by Education, and other
states' approved lists. Based on responses to our survey, the table below
summarizes approaches states used to develop their approved lists (see
table 2).
19 The Consumer's Guide to Evaluating A Core Reading Program Grades K-3: A
Critical Elements Analysis, or Consumer's Guide, describes the core
reading program concept in the following way: "A core reading program is
the primary instructional tool that teachers use to teach children to
learn to read and ensure they reach reading levels that meet or exceed
grade-level standards. The core program. . . should serve as the primary
reading program for the school and the expectation is that all teachers
within and between the primary grades will use the core reading program as
the base of reading instruction. Such programs may or may not be
commercial reading series."
Table 2: Approaches States Used to Develop Their Approved Lists
Approaches Number of states
Conducted review of reading programs using A Consumer's
Guide to Evaluating a Core Reading Program (Kame'enui &
Simmons) 15
Conducted review of reading programs using criteria
outlined in your state's Reading First application 12
Based on information obtained during Reading First
Leadership Academies 8
Adopted reading programs from another state's approved
list(s) 7
Used existing textbook adoption list 6
Requested a Reading First Technical Assistance Center
review of reading programs 3
Based on recommendation from U.S. Department of Education
officials 2
Source: GAO analysis.
Based on our survey results, 25 of the 29 states reporting that they did
not have a list of approved core reading programs said they provided
guidance for districts and schools to identify core reading programs.
Fifteen of these states reported directing districts and schools to
conduct a review of reading programs using A Consumer's Guide to
Evaluating a Core Reading Program. Other states reported providing a
variety of guidance to districts to help them select reading programs
supported by SBRR, including referring them to the approved lists of other
states and reviews conducted by academic experts.
20 Deborah C. Simmons and Edward Kame'enui. A Consumer's Guide to
Evaluating A Core Reading Program Grades K-3: A Critical Elements Analysis
(University of Oregon: March 2003).
http://reading.uoregon.edu/curricula/con_guide.php
21 Education officials sponsored three Reading Leadership Academies in the
early part of 2002. The Academies were forums for state education
officials to obtain information and build their capacity to implement key
aspects of the Reading First program, including reliance on SBRR and
professional development.
States Awarded Reading First Sub-Grants to School Districts Using a Variety of
Different Criteria, and Some States Reported Difficulties with Implementation
States varied in how they exercised their flexibility to set additional
eligibility and award criteria as allowed by the Reading First program,
and some states reported difficulty with implementing key aspects of the
Reading First program while other states did not. In the areas in which
they were given flexibility, states used a variety of criteria for
determining eligibility and in awarding sub-grants to eligible districts,
such as awarding grants to districts that had previously received federal
reading dollars. Education reported that over 3,400 school districts were
eligible to apply for Reading First sub-grants in the states' first school
year of funding.22,23 Of these districts, nearly 2,100 applied for and
nearly 1,200 received Reading First sub-grants in the states' first school
year of funding. In addition, 22 states reported that it was difficult or
very difficult to help districts with reading scores that had not improved
sufficiently. On the other hand, 28 states reported that it was easy or
very easy to determine whether districts' applications met criteria for
awarding sub-grants.
States Varied in How they Exercised Their Flexibility to Set Additional School
District Sub-grant Eligibility and Award Criteria
States varied in how they exercised their flexibility to set school
district eligibility criteria for sub-grants. The Reading First program
provides states with some flexibility to define eligibility criteria
within the statutory guidelines. For instance, while Reading First
requires that states target districts with students in kindergarten
through third grade reading below grade level, states have flexibility to
set eligibility criteria based on the percentage and/or number of these
students within districts. While 34 states reported electing to base
eligibility on a percentage of schools with students reading below grade
level, 18 states reported electing to base eligibility on a number of
students reading below grade level.24 After applying eligibility criteria,
Education reported that states determined that over 3,400 school districts
were eligible to apply for Reading First sub-grants for states' first
school year of funding, or about 20 percent of all school districts
nationwide. However, the percentage of eligible districts varied greatly
across the states, ranging from about 3 to 93 percent.
22 The first school year of funding varied across states. Twenty-five
states received their first year of funding in the 2002-2003 school year.
Twenty-five states received their first year of funding in the 2003-2004
school year.
23 Two states were excluded from our analysis because of data
inconsistencies. See appendix I for additional information about
methodology.
24 The total number of states does not add to 51 because 1 state reported
that its district eligibility included both a percentage figure as well as
the number of schools.
Of those districts eligible to apply, 62 percent, or nearly 2,100
districts, did so, as summarized in figure 3 below. States reported a
variety of reasons why eligible school districts did not apply such as the
prescriptive nature of the program, differences in educational philosophy,
and inadequate resources for the application process. For example,
officials from a few states reported that some districts did not have the
capacity to write the grant application. An official from one state
reported that some districts did not have the time and the staff to
complete the sub-grant process. Furthermore, an official from another
state reported that the application process was too lengthy and
time-consuming to complete.
Nineteen states reported in our survey that they exercised flexibility in
establishing priorities when awarding Reading First sub-grants. States set
a variety of additional priorities for awarding grants to school
districts. For instance, six states reported that they gave priority to
districts that already had other grants, such as Early Reading First
grants,25 or indicated that they could somehow use their Reading First
funds in combination with other resources to maximize the number of
students reading at grade level. In contrast, two states gave priority to
districts that had not received other grant funding. In addition, two
states gave priority to districts based on the population of Native
Americans or students with limited English proficiency. After applying
selection criteria, states awarded Reading First sub-grants to about 34
percent or nearly 1,200 school districts for states' first school year of
funding. This represented about 56 percent of the 2,100 eligible districts
that applied and nearly 7 percent of all school districts nationwide for
states' first school year of funding (see fig. 3).
25 The No Child Left Behind Act of 2001 added two new reading programs to
the Elementary and Secondary Education Act, Reading First and Early
Reading First. Early Reading First provides support to local efforts to
enhance the early language, literacy, and pre-reading development of
preschool-age children, particularly those from low-income families,
through strategies and professional development that are based on
scientifically based reading research.
Figure 3: All School Districts Nationwide That Were Eligible for, That
Applied for, and Were Awarded Reading First Sub-grants in the First School
Year of Funding
Some States Reported Difficulty Implementing Key Aspects of the Program
Some states reported difficulty in implementing key aspects of the Reading
First program. Twenty-two states reported that it was either difficult or
very difficult to help districts with reading scores that had not improved
sufficiently.26 Officials from one state said that this was difficult
because it requires close examination of students reading deficiencies and
the commitment of school leadership. Officials from another state reported
some difficulty in improving selected reading skills of students with
limited English proficiency, which are concentrated in pockets around the
state. Seventeen states reported that it was either difficult or very
difficult to assess how districts applied SBRR in choosing their reading
program. Finally, seven states reported difficulty implementing four or
more of six key program aspects listed in our survey and shown in figure
4. Officials from one of these states told us that the difficulty with
implementation was due to the newness of the program for which everything
had to be developed from scratch.
26 States develop their own performance thresholds for districts to meet
and develop policies for how to handle those districts with results that
are below these thresholds. When monitoring participating districts,
states must determine whether funding should continue or whether districts
should be dropped from the program. Sixteen states reported dropping at
least one district that was initially awarded a sub-grant. There were a
variety of reasons that a state dropped a district, including districts
not showing progress, non-compliance with state requirements, or a shift
in reading philosophy.
On the other hand, states reported ease implementing other key aspects. In
particular, 28 states reported that it was easy or very easy to determine
whether districts' applications met criteria for awarding sub-grants. For
example, states are required to determine whether districts will adhere to
the key components of the program, such as developing a professional
development program or using reading assessments to gauge performance.
Several states we interviewed suggested that it was easy to make this
determination because some of the Reading First requirements were already
in place in their states before Reading First was implemented. For
example, some state officials we interviewed mentioned using reading
assessments prior to Reading First. In addition, officials in one state
told us that they already had a professional development program in place
to train teachers on the state's reading program. Twenty-four states
reported that it was easy or very easy to identify reading programs based
on SBRR.
Figure 4: States Level of Ease or Difficulty Implementing Key Reading
First Program Aspects
Education Provided a Wide Range of Guidance, Assistance, and Oversight,
Generally Satisfying State Officials, but Education Lacked Controls to Guard
against Mandating or Endorsing Curricula and Did Not Provide Written Monitoring
Procedures
Education officials provided states a wide variety of guidance,
assistance, and oversight, but Education lacked written procedures to
guide its interactions with the states and provided limited information on
its monitoring procedures. Education's guidance and assistance included
written guidance, preparatory workshops, feedback during the application
process, and feedback from monitoring visits. Additionally, guidance and
assistance were provided by Education's contractors, including the
regional technical assistance centers. For the most part, state officials
characterized the guidance and assistance they received from Education
officials and contractors, especially the regional technical assistance
centers, as being helpful or very helpful, and many also reported relying
on the expertise of Reading First officials in other states. However,
Education lacked controls to ensure that its officials did not endorse or
otherwise mandate or direct states to adopt particular reading curricula.
For example, according to state officials, Education officials and
contractors made suggestions to some states to adopt or eliminate certain
reading programs, assessments, or professional development providers. In
addition, some state officials reported a lack of clarity about key
aspects of the annual monitoring process, including time frames and
expectations of states in responding to monitoring findings.
Education Provided a Wide Range of Guidance and Assistance
Education provided a variety of written and informal guidance and
assistance to states to help them prepare their applications. For example,
three months after the enactment of NCLBA in January 2002, Education
issued two key pieces of written guidance to states pertaining to the
Reading First program and grant application process: the Guidance for the
Reading First Program and Criteria for Review of State Applications.
Education officials also sponsored three Reading Leadership Academies in
the early part of 2002. The Academies were forums for state education
officials to obtain information and build their capacity to implement key
aspects of the Reading First program, including professional development
and the application of SBRR. Education contracted with RMC Research
Corporation to provide technical assistance to states related to the grant
application process. States reported seeking guidance from RMC on various
aspects of the Reading First application, in particular the use of
instructional assessments (17 states) and instructional strategies and
programs (14 states). Throughout the application process, both Education
and RMC officials were available to address states' questions. In
particular, Education officials provided feedback to states on the results
of expert review panel evaluations of their applications. Consequently, a
large number of states reported that Education required them to address
issues in their applications, most commonly related to the use of
instructional assessments (33 states) and instructional strategies and
programs (25 states). See figure 5 for issues raised about state
applications. Forty-eight states reported that they needed to modify their
application at least once, and 27 reported modifying them three or more
times.
Figure 5: Issues Education Raised About States' Reading First Applications
Once grants were awarded, Education continued to provide assistance and
contracted with RMC Research to oversee three regional TACs to help states
implement Reading First. RMC established three TACs affiliated with state
university centers in Florida, Texas, and Oregon, which RMC and TAC
officials told us were selected based on their expertise in one or more
areas central to the success of the Reading First program, such as
professional development or reading assessment. Each technical assistance
center was responsible for providing comprehensive support to each of the
states in its geographic region (see fig. 6). States reported that they
looked to these centers for guidance on a variety of issues, especially
creating professional development criteria, using reading assessments, and
helping districts with reading scores that had not improved sufficiently.
According to TAC staff, some of the most common requests they receive
pertain to the use and interpretation of assessment data and use of
Reading Coaches. TAC staff also told us that they catalog recurring issues
or problems.
Figure 6: Geographic Regions for Reading First Regional Technical
Assistance Centers
In addition, according to one RMC official and some state officials, the
TACs provided support to states during implementation to help them
supplement their capacity and expertise in evaluating whether or not
reading programs proposed by districts were based on SBRR. For instance,
staff from the TAC in Florida explained that some states in their region
had asked for assistance in evaluating reading programs that had been in
use prior to Reading First to gauge their compliance with the requirements
of Reading First. Staff from the TAC emphasized that in reviewing these
reading programs, they used the criteria in each state's approved state
plan as the criteria for determining compliance with Reading First
requirements. Officials in one state explained that while the staff at
their state educational agency (SEA) possessed the knowledge necessary to
conduct reviews of reading programs, scarce state staff resources would
have made it difficult to conclude the reviews in the short time frame
available. Though Education officials were aware of and initially condoned
the TAC review process, Education officials advised all TACs to
discontinue reviews of programs--to avoid the appearance of
impropriety--after allegations were raised about Reading First officials
expressing preference for specific reading programs. (Table 3 provides a
summary of the types of guidance and assistance provided by Education and
its contractors.)
Table 3: Guidance and Assistance Provided to States during Application and
Implementation Processes
Source Type Purpose Time frame
Final Guidance for Written Interpreting the provisions Application
the Reading First guidance of NCLBA authorizing the process
Program and Reading First program and
Criteria for providing guidance on the
Review of State criteria review panel members
Applications would be using in evaluating
states' applications.
Reading Leadership Workshops Helping states master key Application
Academies features of the Reading First process
program, in particular the
application of SBRR in
classroom and professional
development activities.
RMC Research Technical Providing technical Application
technical assistance assistance to states process
assistance including training in SBRR.
providers
Regional Reading Technical Providing guidance to states Implementation
First Technical assistance about conducting their process
Assistance Centers district sub-grant
competitions, delivering
training, assistance, and
related materials in response
to requests from state-level
Reading First officials, and
training including annual
workshops for states.
Source: GAO analysis.
During the application and implementation phases of the Reading First
program, many states came to rely on other unofficial sources of guidance,
including other states' Reading First officials, in addition to the
written guidance provided by Education. For example, as noted earlier,
among the 22 states that had an approved list of reading programs for
Reading First districts, 15 reported using A Consumer's Guide to
Evaluating a Core Reading Program to assist them in reviewing potential
reading programs. In addition, officials from 21 states reported that
other states' Reading First Coordinators provided great or very great help
during the Reading First state grant application process. Further, a
number of state officials reported using the information from other
states' websites, such as approved reading programs, to help inform their
own decisions pertaining to the selection of reading programs. One state
official explained, "With our limited infrastructure and dollars, we were
never able to muster the resources needed to run an in-house programs
review," and further that, "It worked well for us to use the programs and
materials review results from larger states that ran rigorous review
processes." Another state official reported that the state did not feel
equipped to apply the principles of SBRR in evaluating reading programs
and responded by comparing one state's review and subsequent list of
reading programs to those of a few other states to make judgments about
allowable programs.
States Were Generally Satisfied with the Guidance and Assistance Available to
Them
Most states reported making use of and being satisfied with the primary
sources of guidance available to them over the course of the Reading First
application and implementation processes. For example, 46 states reported
making use of the two key pieces of Education's written guidance in
preparing their Reading First applications. A majority of states also
reported that these pieces of guidance provided them with the information
needed to adequately address each of the key application components. For
example, over 40 states reported that the guidance related to the
definition of sub-grant eligibility and selection criteria for awarding
sub-grants helped them adequately address these areas in their
application. However, officials in eight states reported that the guidance
on the use of instructional assessments did not provide them with the
information needed to adequately address this area. (See fig. 7.)
Figure 7: States' Assessment of Whether Written Guidance Provided
Information Needed to Adequately Address Application Components
Overall, most state officials were also satisfied with the level of
assistance they received from Education staff and their contractors in
addressing issues related to the Reading First application and
implementation processes. For example, state officials in 39 states
reported that Education staff were of great or very great help during the
application or implementation process. Additionally, officials from 48
states reported that Education officials were helpful or very helpful in
addressing states' implementation-related questions, which frequently
dealt with using reading assessments and helping districts with reading
scores that had not improved sufficiently. A number of state officials
reported to us that they appreciated the guidance and attention they
received from Reading First officials at Education. For example, one state
Reading First Coordinator reported, "the U.S. Department of Education
personnel have been wonderful through the process of implementing Reading
First. I can't say enough about how accessible and supportive their
assistance has been." Another state official remarked that the state's
efforts to make reading improvements "would have been impossible without
their [Education officials and contractors] guidance and support." Even
officials from one state who had a disagreement with Education over its
suggestion to eliminate a certain reading program characterized most of
the guidance they received from Reading First officials as "excellent."
However, one state official reported feeling that the technical assistance
workshops have served as conduits for Education officials to send messages
about the specific reading programs and assessments they prefer. Another
state official reported that, "core programs and significant progress have
not been defined" and that "SBRR programs are not clearly designated."
According to responses obtained to our survey, the three TACs also
provided a resource for states seeking advice on issues pertaining to the
implementation of their Reading First programs. Specifically, 41 states
cited the Centers as helpful or very helpful in addressing states'
inquiries related to the implementation of Reading First. In addition, on
a variety of key implementation components, more state officials reported
seeking information from their regional TACs than they did from Education
officials (see table 4).
Table 4: States' Reported Sources of Guidance for Key Reading First
Implementation Components
Number of states seeking guidance or
assistance from Education and the TACs
Implementation component Education TACs
Creating professional development
criteria 6 41
Identifying reading programs based
on SBRR 9 24
Assessing how districts applied
principles of SBRR 10 29
Using reading assessments 20 40
Helping districts with reading
scores that have not improved
sufficiently 12 36
Other implementation issues related
to approving district applications 18 20
Source: GAO analysis.
Education Lacked Written Procedures to Guide Its Interactions with States
We found that Education developed no written guidance, policies, or
procedures to direct or train Education officials or contractors regarding
their interactions with the states. Federal agencies are required under
the Federal Managers' Financial Integrity Act of 198227 to establish and
maintain internal controls to provide a reasonable assurance that agencies
achieve objectives of effective and efficient operations, reliable
financial reporting, and compliance with applicable laws and
regulations.28 When executed effectively, internal controls work to ensure
compliance with applicable laws and regulations by putting in place an
effective set of policies, procedures, and related training. We found that
Education had not developed written guidance or training to guide managers
on how to implement and comply with statutory provisions prohibiting
Education officials from directing or endorsing state and local curricular
decisions. Department officials told us that it was their practice that
program managers should consult the Office of General Counsel if they had
questions regarding interactions with grantees. Reading First officials
told us that it was their approach to approve each state's method and
rationale for reviewing or selecting reading programs as outlined in each
state's plan and that state compliance with program requirements,
including adherence to the principles of SBRR, would then be assessed
using the provisions of these plans as the criteria. Similarly, officials
from Education's contractors responsible for conducting monitoring visits
told us that they were instructed by Education to use state plans as the
criteria for gauging states' compliance with Reading First reading program
requirements, but that they were provided no formal written guidance or
training. A senior Education attorney who is currently working with
Reading First program officials told us that he was not aware that they
had used this approach and that he felt that the statutory requirements
should also play an important role in the monitoring process. Following
the publication of the IG's report in September, Education's Office of
General Counsel has provided training to senior management on internal
control requirements and has begun working with the Reading First office
to develop procedures to guide the department's activities.
Despite the statutory prohibition against mandating or endorsing curricula
and the department's stated approach to rely on state plans, and the
processes articulated in them, to assess compliance, states reported to us
several instances in which Reading First officials or contractors appeared
to intervene to influence their selection of reading programs and
assessments. For example, officials from four states reported receiving
suggestions from Education or its contractors to adopt specific reading
programs or assessments. Specifically, two states reported that it was
suggested that they adopt a particular reading assessment. Similarly,
Education's first IG report also documented one instance in which Reading
First officials at Education worked in concert with state consultants to
ensure that a particular reading program was included on that state's list
of approved reading programs.29
27 31 U.S.C. S 3512 (c) and (d).
28 For more information, see GAO's Standards for Internal Control in the
Federal Government, GAO/AIMD-00-21.3.1 (Washington, D.C.: November 1999),
P.L. 97-255 and 31 USC 1105, 1113, and 3512.
In addition, states reported that Education officials or contractors
suggested that they eliminate specific reading programs or assessments
related to Reading First. Specifically, according to our survey results,
officials from 10 states reported receiving suggestions that they
eliminate specific programs or assessments. In some cases, the same
program was cited by officials from more than one state. In one instance,
state officials reported that Education officials alerted them that expert
reviewers objected to a reading program that was under consideration but
not named explicitly in the state's application. An official from a
different state reported receiving suggestions from Education officials to
eliminate a certain reading program, adding that Education's justification
was that it was not aligned with SBRR. In another instance, state
officials pointed out that they had adopted a program that was approved by
other states, according to the procedures in their approved state plan,
but were told by Education officials that it should be removed from their
list and that Education would subsequently take a similar course of action
with regard to those other states as well. Also, Education officials did
not always rely on the criteria found in state plans as the basis for
assessing compliance. We found, for example, one summary letter of
findings from a monitoring report in which Education officials wrote that
"Two of the monitored districts were implementing reading programs that
did not appear to be aligned with scientifically based reading research."
Officials we spoke to in that state told us that they did not feel that
they had been assessed on the basis of the procedures outlined in the
state's plan, but rather that the reading program itself was being called
into question. The IG also found that Reading First officials communicated
to several states against the use of certain reading programs or
assessments, including Rigby and Reading Recovery.
29 Department of Education, Office of the Inspector General, The Reading
First Program's Grant Application Process: Final Inspection Report
(Washington, D.C.: September 2006).
Officials from a few states also reported being contacted by Education
regarding district Reading First applications or reading programs. For
example, officials from four states reported being contacted by an
Education official about a district application under consideration and
one of those states also reported being approached by staff from one of
the regional technical assistance centers or another contractor for the
same reason. Officials from each of these states indicated that the reason
they were contacted stemmed from the reading programs being used by the
districts in question. In a few cases, state officials reported being
contacted by Education officials regarding the state's acceptance of a
reading program or assessment that was not in compliance with Reading
First. In one instance, state officials reported that Education contacted
them outside of the normal monitoring process after they had obtained
information from a national Reading First database maintained by a
non-profit research organization that districts in the state were using a
specific reading program.
Five states also reported receiving recommendations from Reading First
officials or contractors to change some of the professional development
providers proposed in their original grant applications. When asked about
the specific providers identified for elimination, three of the states
indicated that the providers identified for elimination were in-state
experts. In one case, a state was told that the review panel cited a lack
of detail about the qualifications of the state's proposed professional
development consultants.
We also found that while Education officials laid out an ambitious plan to
annually monitor every state, they failed to develop written procedures
guiding its monitoring visits. For example, Education did not establish
timelines for submitting final reports to states following monitoring
visits, specifically how and when state officials were expected to follow
up with Education officials regarding findings from the monitoring visits.
As a result, states did not always understand monitoring response
procedures, timelines, and expectations. While we found that most state
officials we spoke with understood that they were to be monitored with the
use of their state plans as the criteria, they did not always understand
what was required of them when responding to monitoring findings. For
example, one state official reported being unaware that the state was
supposed to respond to Education officials about findings from its
monitoring report. An official from another state maintained that he/she
was unclear about the process the state was to follow to respond to
findings, and that no timeline for responding was provided to him/her.
Furthermore, one state reported that findings were not delivered in a
timely manner, and another state reported that Education did not address
the state's responses to the monitoring findings. Key aspects of an
effective monitoring program include communicating to individuals
responsible for the function any deficiencies found during the monitoring.
Conclusions
The Reading First program, according to state coordinators, has brought
about changes and improvements to the way teachers, administrators, and
other education professionals approach reading instruction for children in
at-risk, low-performing schools during the critical years between
kindergarten and third grade. To assist states in implementing this large,
new federal reading initiative, Education has provided a wide range of
guidance, assistance, and oversight, that, for the most part, states have
found helpful. However, Education failed to develop comprehensive written
guidance and procedures to ensure that its interactions with states
complied with statutory provisions. Specifically, Education lacked an
adequate set of controls to ensure that Reading First's requirements were
followed, while at the same time ensuring that it did not intervene into
state and local curricular decisions. We concur with the Education IG's
recommendations that the Department develop a set of internal procedures
to ensure that federal statutes and regulations are followed and we feel
it is important for the Secretary to follow up on these recommendations to
ensure that they are properly implemented. Additionally, we feel it is
important for the department to have clear procedures in place to guide
departmental officials in their dealings with state and local officials.
While Education's stated approach was to rely on state plans as its
criteria for enforcing Reading First's requirements, states reported
several instances in which it appears that Education officials did attempt
to direct or endorse state and local curricular decisions. Such actions
would prevent states from exercising their full authority under the law
and would violate current statutory restrictions. Balancing Reading
First's requirements and the limits placed on the department requires
Education to have clear, explicit, and well-documented procedures to guide
its interactions with the states. Failure to do so places the department
at risk of violating the law and leaves it vulnerable to allegations of
favoritism.
Additionally, while Education's annual monitoring effort for Reading First
is ambitious, it did not provide clear guidelines and procedures to
states. As a result, states were not always aware of their roles and
responsibilities in responding to findings of non-compliance, and
Education was not always consistent in its procedures to follow up with
states to resolve findings and let states know if they had taken proper
actions. Key aspects of an effective monitoring program include
transparency and consistency. Letting all states know in a timely manner
whether or not their plans to address deficiencies are adequate is
important to ensure that findings are dealt with in an appropriate, timely
and clear manner.
Recommendations
In addition to addressing the IG's recommendations to develop internal (1)
policies and procedures to guide program managers on when to solicit
advice from General Counsel and (2) guidance on the prohibitions imposed
by section 103(b) of the DEOA, we recommend that, in order to ensure that
the department complies with statutory prohibitions against directing,
mandating, or endorsing state and local curricular decisions, the
Secretary of Education also establish control procedures to guide
departmental officials and contractors in their interactions with states,
districts, and schools.
In addition, to help the department conduct effective monitoring of the
Reading First program, we recommend that the Secretary of Education
establish and disseminate clear procedures governing the Reading First
monitoring process. In particular, Education should delineate states'
rights and responsibilities and establish timelines and procedures for
addressing findings.
Agency Comments
We provided a draft of this report to the Department of Education and
received written comments from the agency. In its comments, included as
appendix III of this report, Education agreed with our recommendations and
indicated that it will take actions to address them.
Specifically, Education said it will provide written guidance to all
departmental staff to remind them of the importance of impartiality in
carrying out their duties and not construing program statutes to authorize
the department to mandate, direct, or control curriculum and instruction,
except to the extent authorized by law. On February 7, 2007, the Secretary
of Education issued a memorandum to senior officers reminding them that it
is important to maintain objectivity, fairness, and professionalism when
carrying out their duties. The Secretary's memorandum also emphasizes the
importance of adhering to the statutory prohibitions against mandating,
directing, and controlling curriculum and instruction, and strongly
encourages managers to consult with Education's Office of General Counsel
early on to identify and resolve potential legal issues. Also, according
to Education's written comments on our draft report and the Secretary's
February 7, 2007, memorandum to senior officers, annual training will be
required on internal controls and this training will address statutory
prohibitions against mandating, directing or controlling local curriculum
and instruction decisions.
Regarding its monitoring process for Reading First, in its comments,
Education said that it will develop and disseminate guidelines to states
outlining the goals and purposes of its monitoring efforts, revise the
monitoring protocols, and develop timelines and procedures on states'
rights and responsibilities for addressing monitoring findings. Education
also included in its response a summary of its actions and planned actions
to address recommendations from the department's Office of Inspector
General's recent report on the implementation of the Reading First
program.
As agreed with your offices, unless you publicly announce its contents
earlier, we plan no further distribution of this report until 30 days
after its issue date. At that time, we will send copies of this report to
appropriate congressional committees, the Secretary of Education, and
other interested parties. Copies will also be made available upon request.
In addition, the report will be available at no charge on GAO's Web site
at http://www.gao.gov. If you or your staff have any questions about the
report, please contact me at (202) 512-7215. Contact points for our
Offices of Congressional Relations and Public Affairs may be found on the
last page of this report. GAO staff who made major contributions to this
report are listed in appendix IV.
Cornelia M. Ashby
Director, Education, Workforce, and
Income Security Issues
Appendix I: Objectives, Scope, and Methodology
Our objective was to answer the following questions: (1) What changes have
occurred to reading instruction since the inception of Reading First? (2)
What criteria have states used to award Reading First sub-grants to
districts, and what, if any, difficulty did states face in implementing
the program? (3) What guidance, assistance, and oversight did Education
provide states related to the Reading First program? To answer these
questions, we collected both qualitative and quantitative information
about the Reading First program from a variety of sources. We conducted a
Web-based survey of the Reading First Directors in all 50 states and the
District of Columbia. We also obtained and analyzed data from the
Department of Education for each state on Reading First districts'
eligibility, applications and awards for states' first school year of
funding. The first school year of funding varied across states.
Twenty-five states received their first year of funding in the 2002-2003
school year. Twenty-five states received their first year of funding in
the 2003-2004 school year. To assess the reliability of this data, we
talked to agency officials about data quality control procedures and
reviewed relevant documentation. We excluded two states because of
reporting inconsistencies, but determined that the data for the other
states were sufficiently reliable for the purposes of this report. We also
conducted semi-structured follow-up interviews with Reading First
Directors in 12 states, mostly over the telephone. We conducted site
visits to 4 of the 12 states. During the site visits, we met with state
officials, local program administrators, and state-level technical
assistance providers, as well as school officials from individual schools,
including teachers, principals, and Reading First coaches. In identifying
local sub-grant recipients to meet with in each state, we sought to
incorporate the perspectives of urban, rural, and suburban school
districts. We selected the 12 states to have diversity in a variety of
factors, including geographic distribution, grant size, poverty rate,
percentage of students reading at or below grade level, urban and rural
distinctions, the presence of a statewide list of approved reading
programs, and whether states had reported that they received guidance from
Education officials advocating for or against particular reading programs
or assessments. For both the survey and follow-up interviews, to encourage
candid responses, we promised to provide confidentiality. As a result,
state survey responses will be provided primarily in summary form or
credited to unnamed states, and the states selected for follow-up
interviews will not be specifically identified. Furthermore, in order to
adequately protect state identities, we are unable to provide the names of
particular reading programs or assessments Education officials or
contractors suggested a state use or not use. We did not attempt to verify
allegations made by state or local officials in their survey responses or
during interviews or otherwise make any factual findings about Education's
conduct.
We also visited or talked with administrators from each of the three
regional Reading First Technical Assistance Centers, located in Florida,
Texas and Oregon, as well as RMC Research, the federal contractor tasked
with administering the contract with the technical assistance centers. We
also interviewed several publishers and other providers of reading
curricula and assessments, to obtain their views about changes Reading
First has prompted in states, districts, and schools. We chose these
providers to reflect the perspectives of large, commercial reading
textbook programs that are widely represented nationwide on states' lists
of approved programs, as well as some other selected providers of reading
curricula, including some that have filed complaints related to Reading
First. We also interviewed Education officials about the implementation of
the Reading First program.
To obtain a better understanding of state program structure, as well as
the nature of interactions between Education officials and state grantees,
we reviewed state grant files, monitoring reports, and related
correspondence for the 12 states where we conducted follow-up interviews.
In addition, we reviewed NCLBA language authorizing Reading First, as well
as statements of work articulating the responsibilities of the regional
technical assistance centers and the contractor tasked with providing
assistance to states in conducting local sub-grant competitions. We
conducted our work from December 2005 through January 2007 in accordance
with generally accepted government auditing standards.
Survey of States
To better understand state implementation of the Reading First program, we
designed and administered a Web-based survey of the Reading First
Directors in all 50 states and the District of Columbia. The survey was
conducted between June and July 2006 with 100 percent of state Reading
First Directors responding. The survey included questions about
curriculum; professional development; and state Reading First grant
eligibility, application, award, and implementation processes. The survey
contained both closed- and open-ended questions. For the open-ended
questions, we used content analysis to classify and code the responses
from the states such as the publishers on states' approved lists. We had
two people independently code the material, then reconciled any
differences in coding. Because this was not a sample survey, there are no
sampling errors. However, the practical difficulties of conducting any
survey may introduce nonsampling errors, such as variations in how
respondents interpret questions and their willingness to offer accurate
responses. We took steps to minimize nonsampling errors, including
pre-testing draft instruments and using a Web-based administration system.
Specifically, during survey development, we pre-tested draft instruments
with one expert reviewer and Reading First Directors in four states during
April and May 2006. In the pre-tests, we were generally interested in the
clarity of the questions and the flow and layout of the survey. For
example, we wanted to ensure definitions used in the survey were clear and
known to the respondents, categories provided in closed-ended questions
were complete and exclusive, and the ordering of survey sections and the
questions within each section was appropriate. On the basis of the
pre-tests, the Web instrument underwent some slight revisions. A second
step we took to minimize nonsampling errors was using a Web-based survey.
By allowing respondents to enter their responses directly into an
electronic instrument, this method automatically created a record for each
respondent in a data file and eliminated the need for and the errors (and
costs) associated with a manual data entry process. To further minimize
errors, programs used to analyze the survey data were independently
verified to ensure the accuracy of this work.
Appendix II: Descriptions of Assessment Types
Reading First Assessment Requirements and Descriptions
o Reading programs under Reading First must include rigorous
assessments with proven validity and reliability.
o Assessments must measure progress in the five essential components of
reading instruction and identify students who may be at risk for
reading failure or who are already experiencing reading difficulty.
o Reading programs under Reading First must include screening
assessments, diagnostic assessments, and classroom-based instructional
assessments of progress.
Assessment type Description
Screening Determine which children are at risk for
reading difficulty and need additional
support
Diagnostic Provide more in-depth information on
students' skills and instructional needs;
forms the basis of intervention strategies
Classroom-based instructional Determine whether students are making
assessments of progress progress and/or need more support to
achieve grade-level reading outcomes
Source: GAO analysis based on Education guidance.
Appendix III: Comments from the Department of Education
Appendix IV: GAO Contacts and Staff Acknowledgments
GAO Contacts
Cornelia M. Ashby (202) 512-7215 [email protected]
Acknowledgments
Bryon Gordon, Assistant Director, and Tiffany Boiman, Analyst-in-Charge,
managed this engagement and made significant contributions to all aspects
of this report. Sonya Phillips, Sheranda Campbell, Janice Ceperich, and
Andrew Huddleston also made significant contributions. Jean McSween
provided methodological expertise and assistance. Sheila McCoy and Richard
Burkard delivered legal counsel and analysis. Susannah Compton, Charlie
Willson, and Scott Heacock assisted with message and report development.
(130541)
GAO's Mission
The Government Accountability Office, the audit, evaluation and
investigative arm of Congress, exists to support Congress in meeting its
constitutional responsibilities and to help improve the performance and
accountability of the federal government for the American people. GAO
examines the use of public funds; evaluates federal programs and policies;
and provides analyses, recommendations, and other assistance to help
Congress make informed oversight, policy, and funding decisions. GAO's
commitment to good government is reflected in its core values of
accountability, integrity, and reliability.
Obtaining Copies of GAO Reports and Testimony
The fastest and easiest way to obtain copies of GAO documents at no cost
is through GAO's Web site ( www.gao.gov ). Each weekday, GAO posts
newly released reports, testimony, and correspondence on its Web site. To
have GAO e-mail you a list of newly posted products every afternoon, go to
www.gao.gov and select "Subscribe to Updates."
Order by Mail or Phone
The first copy of each printed report is free. Additional copies are $2
each. A check or money order should be made out to the Superintendent of
Documents. GAO also accepts VISA and Mastercard. Orders for 100 or more
copies mailed to a single address are discounted 25 percent. Orders should
be sent to:
U.S. Government Accountability Office 441 G Street NW, Room LM Washington,
D.C. 20548
To order by Phone: Voice: (202) 512-6000 TDD: (202) 512-2537 Fax: (202)
512-6061
To Report Fraud, Waste, and Abuse in Federal Programs
Contact:
Web site: www.gao.gov/fraudnet/fraudnet.htm E-mail:
[email protected] Automated answering system: (800) 424-5454 or (202)
512-7470
Congressional Relations
Gloria Jarmon, Managing Director, [email protected] (202) 512-4400 U.S.
Government Accountability Office, 441 G Street NW, Room 7125 Washington,
D.C. 20548
Public Affairs
Paul Anderson, Managing Director, [email protected] (202) 512-4800
U.S. Government Accountability Office, 441 G Street NW, Room 7149
Washington, D.C. 20548
www.gao.gov/cgi-bin/getrpt?GAO-07-161 .
To view the full product, including the scope
and methodology, click on the link above.
For more information, contact Cornelia Ashby at (202) 512-8403 or
[email protected].
Highlights of [34]GAO-07-161 , a report to congressional requesters
February 2007
READING FIRST
States Report Improvements in Reading Instruction, but Additional
Procedures Would Clarify Education's Role in Ensuring Proper
Implementation by States
The Reading First program was designed to help students in kindergarten
through third grade develop stronger reading skills. This report examines
the implementation of the Reading First program, including (1) changes
that have occurred to reading instruction; (2) criteria states have used
to award sub-grants to districts, and the difficulties, if any, states
faced during implementation; and (3) the guidance, assistance, and
oversight the Department of Education (Education) provides states. GAO's
study is designed to complement several studies by Education's Inspector
General (IG) in order to provide a national perspective on some of the
specific issues being studied by the IG. For this report, GAO administered
a Web-based survey to 50 states and the District of Columbia, and
conducted site visits and interviews with federal, state, and local
education officials and providers of reading programs and assessments.
[35]What GAO Recommends
GAO recommends that Education establish control procedures to guide
departmental officials and contractors in their interactions with states,
districts, and schools to ensure compliance with statutory provisions. GAO
also recommends that Education establish and disseminate clear procedures
governing its monitoring process. Education, in its response to a draft of
this report, agreed with GAO's recommendations.
States reported that there have been a number of changes to, as well as
improvements in, reading instruction since the implementation of Reading
First. These included an increased emphasis on the five key components of
reading (awareness of individual sounds, phonics, vocabulary development,
reading fluency, and reading comprehension), assessments, and professional
development with more classroom time being devoted to reading activities.
However, according to publishers we interviewed, there have been limited
changes to instructional material. Similarly, states report that few
changes occurred with regard to their approved reading lists.
States awarded Reading First sub-grants using a variety of different
eligibility and award criteria, and some states reported difficulties with
implementing key aspects of the program. After applying federal and state
eligibility and award criteria, Education reported that over 3,400
districts were eligible to apply for sub-grants in the states' first
school year of funding. Of these districts, nearly 2,100 applied for and
nearly 1,200 districts received Reading First funding. (See figure for
percentages.)
Education officials made a variety of resources available to states during
the application and implementation processes, and states were generally
satisfied with the guidance and assistance they received. However,
Education developed no written policies and procedures to guide Education
officials and contractors in their interactions with state officials and
guard against officials mandating or directing states' decisions about
reading programs or assessments, which is prohibited by NCLBA and other
laws. Based on survey results, some state officials reported receiving
suggestions from Education officials or contractors to adopt or eliminate
certain reading programs or assessments. Similarly, the IG reported in
September 2006 that the Department intervened to influence a state's and
several school districts' selection of reading programs. In addition,
while Education officials laid out an ambitious plan for annual monitoring
of every state's implementation, they did not develop written procedures
guiding monitoring visits and, as a result, states did not always
understand monitoring procedures, timelines, and expectations for taking
corrective actions.
Percentages of Eligible, Applicant and Awarded Districts Nationwide During
States' First School Year of Funding
References
Visible links
34. http://www.gao.gov/cgi-bin/getrpt?GAO-07-161
*** End of document. ***