Public Schools: Insufficient Research to Determine Effectiveness
of Selected Private Education Companies (29-OCT-02, GAO-03-11).
In recent years, local school districts and traditional public
schools have taken various initiatives to improve failing
schools. School districts and charter schools are increasingly
contracting with private, for-profit companies to provide a range
of education and management services to schools. In the District
of Columbia, some public schools contract with three such
companies: Edison Schools, Mosaica Education, and Chancellor
Beacon Academies. These three companies have programs that
consist of both management services, such as personnel, and
educational services, which they offer to schools across the
nation; in the District, most of the schools managed by these
companies have either adopted selected elements of their
companies' programs or chosen other educational programs. Each
company provides services such as curriculum, assessments,
parental involvement opportunities, and student and family
support. Little is known about the effectiveness of these
companies' programs on student achievement, parental
satisfaction, parental involvement, or school climate because few
rigorous studies have been conducted. Although the companies
publish year-to-year comparisons of standardized test scores to
indicate that students in schools they manage are making academic
gains, they do not present data on comparable students who are
not in their programs, a necessary component of a program
effectiveness study.
-------------------------Indexing Terms-------------------------
REPORTNUM: GAO-03-11
ACCNO: A05401
TITLE: Public Schools: Insufficient Research to Determine
Effectiveness of Selected Private Education Companies
DATE: 10/29/2002
SUBJECT: Academic achievement
Charter schools
Education program evaluation
Educational research
Public schools
School districts
School management and organization
District of Columbia
******************************************************************
** This file contains an ASCII representation of the text of a **
** GAO Product. **
** **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced. Tables are included, but **
** may not resemble those in the printed version. **
** **
** Please see the PDF (Portable Document Format) file, when **
** available, for a complete electronic file of the printed **
** document's contents. **
** **
******************************************************************
GAO-03-11
Report to the Ranking Minority Member, Subcommittee on the District of
Columbia, Committee on Appropriations, House of Representatives
United States General Accounting Office
GAO
October 2002 PUBLIC SCHOOLS Insufficient Research to Determine
Effectiveness of Selected Private Education Companies
GAO- 03- 11
Page i GAO- 03- 11 Selected Private Education Companies Letter 1
Results in Brief 2 Background 3 Education Management Companies Have
Multifaceted Programs;
District Schools Varied In Their Implementation 5 Limited Research Exists
of the Effectiveness of These Companies*
Programs 11 Agency Comments 14
Appendix I Objectives, Scope, and Methodology 16 Objectives 16 Scope and
Methodology 16
Appendix II Studies and Other Information Sources Considered 18
Appendix III Comments from the Department of Education 21
Appendix IV GAO Contacts and Staff Acknowledgments 22 GAO Contacts 22
Acknowledgments 22
Related GAO Products 23
Table
Table 1: Profiles of Three Education Management Companies Operating in the
District of Columbia 7 Contents
Page 1 GAO- 03- 11 Selected Private Education Companies
October 29, 2002 The Honorable Chaka Fattah Ranking Minority Member
Subcommittee on the District of Columbia Committee on Appropriations House
of Representatives
Dear Mr. Fattah: In recent years, local school districts and traditional
public schools have taken various initiatives to improve failing schools.
For example, school districts and charter schools are increasingly
contracting with private, forprofit companies to provide a range of
education and management services to schools. To date, there has been
debate regarding the effectiveness of such companies in managing public
schools.
These companies generally offer schools services in areas such as school
organization, instruction, technology, and professional development. In
the District of Columbia, some public schools currently contract with
three such companies: Edison Schools, Mosaica Education, and Chancellor
Beacon Academies. As agreed with your office, we (1) identified the
characteristics of their programs and determined the extent to which
District schools managed by them have used their programs and (2)
determined what is known about the effectiveness of these companies*
educational programs, as measured primarily by student achievement.
To address these issues, we reviewed relevant research on charter and
traditional public schools managed by for- profit educational management
companies as well as documents and materials provided by the companies. In
addition, we observed an on- site review of one school*s program which was
conducted for District oversight authorities. We also interviewed
officials of the companies that manage District public schools, officials
of the District*s oversight authorities, and representatives of the
schools, as well as officials of the Department of Education, other
education experts and advocates. Finally, we reviewed evaluations
concerning the three companies operating in the District that met the
following criteria: included comparison groups and measurement of
performance over time, and focused on academic achievement, parental
satisfaction, parental involvement, or school climate. We assessed the
United States General Accounting Office Washington, DC 20548
Page 2 GAO- 03- 11 Selected Private Education Companies
quality of the evaluations* research designs, reviewed them for threats to
validity and determined whether we had confidence in their conclusions. We
conducted our work between January and September 2002 in accordance with
generally accepted government auditing standards.
Edison, Mosaica, and Chancellor Beacon have programs that consist of both
management services, such as personnel, and educational services, which
they offer to schools across the nation; in the District, most of the
schools managed by these companies have either adopted selected elements
of their companies* programs or chosen other educational programs. Each
company provides services such as curriculum, assessments, parental
involvement opportunities, and student and family support. They also offer
a variety of organizational options, including smaller class and school
sizes, as well as longer school days and school years. All of the
companies allow their schools some flexibility in adapting their programs
to local circumstances. The extent to which the District schools
implemented all the elements of these companies* educational programs
varied. For example, 6 of the 10 schools managed by these companies had
either partially implemented the company*s curriculum or had not
implemented that curriculum at all. Some schools have opted to customize
the company*s educational program; other schools have left in place the
educational program of a company that formerly managed them. In school
year 2001- 02, all 10 District schools managed by these companies were
charter schools with predominantly poor and minority student populations;
most enrolled elementary and middle school students.
Little is known about the effectiveness of these companies* programs on
student achievement, parental satisfaction, parental involvement, or
school climate because few rigorous studies have been conducted. While the
companies publish year- to- year comparisons of standardized test scores
to indicate that students in schools they manage are making academic
gains, they do not present data on comparable students who are not in
their programs, a necessary component of a program effectiveness study. An
effectiveness study attempts to isolate the effect a program has on
specific outcomes by, among other things, comparing outcomes for students
in company managed schools with those of a comparable group of traditional
public school students, tracking students over time, testing students
before and after exposure to the company*s program, and controlling for
differences between these groups. Of five studies we identified concerning
the three companies operating in the District, four focused only on one
company. Moreover, of the five studies, one* based on one school in
Florida* was rigorous enough to allow confidence in the Results in Brief
Page 3 GAO- 03- 11 Selected Private Education Companies
findings about the program*s effectiveness in that school. This particular
study, using two analytical techniques, found no difference between
students in the company*s program and other students. The remaining
studies had methodological limitations that precluded assessments about a
company*s effect on student achievement, parental satisfaction, parental
involvement, and school climate. Additional research on one company*s
program is planned by an organization with experience in conducting
educational evaluations.
During the last decade, a new kind of entity has emerged in public
education: the for- profit provider of education and management services.
Historically, school districts have contracted with private companies for
noninstructional services, such as transportation and food service, and
have also relied on contractors in some cases to provide limited
instructional services to specified populations. Until recently, public
schools have generally not contracted for the comprehensive programs of
educational and management services that these companies typically offer.
In recent years, the options available to public schools considering
contracting with private companies have steadily grown. Today,
approximately 20 major companies manage public schools. 1 Nationally, it
is estimated that these companies as well as other smaller companies serve
over 300 schools out of the nation*s approximately 92,000 public schools.
Although these companies manage public schools at all grade levels, most
such privately managed public schools are elementary and middle schools.
In these public schools, companies generally provide the same kinds of
educational and management services that school districts do for
traditional public schools. Educational services typically include a
curriculum as well as a range of services designed to enhance or support
student achievement, such as professional development opportunities for
teachers, opportunities for parental involvement and school environments
that aim to facilitate student support. Management services typically
include personnel, payroll, and facilities management. Although these are
the services that are typically offered to schools, companies also may
adapt their services to respond to the preferences or needs of individual
schools. For example, while some companies offer a particular curriculum
1 In this context, *major* refers to those companies that manage at least
3 schools and operate in multiple states. Background
Page 4 GAO- 03- 11 Selected Private Education Companies
or educational approach, others appear more willing to work with the
curriculum the school or school district has already adopted. 2
Typically, companies provide their services to public schools in one of
two ways. First, they can contract directly with school districts to
manage traditional public schools; such schools are known as *contract
schools.* Second, they can manage charter schools, which are public
schools that receive a degree of autonomy and freedom from certain school
district requirements in exchange for enhanced accountability. Generally,
charter schools are run by individual boards of trustees, which in most
states and the District of Columbia have the authority to decide whether
to contract with a private company. 3 Both contract schools and charter
schools remain public schools, however, and are generally subject to
federal and state requirements for public schools in areas such as the
application of standardized tests and special education.
While the reasons public schools turn to private companies vary, the
potential to increase student achievement appears to be one factor. In
particular, according to certain experts and company officials we spoke
to, school districts that seek a company*s help often do so with the
expectation of raising achievement in struggling or failing schools. While
management services appear to be especially important for charter schools
that contract with such companies, charter schools also consider the
potential to raise student achievement or a particular educational
approach consistent with the school*s mission, according to school
officials and experts we spoke with. Both types of schools that seek these
companies* assistance* struggling schools and charter schools* appear
concentrated in urban areas. Further, several of the major companies
reportedly serve a predominantly disadvantaged urban and minority student
population.
Recent changes in federal law have implications for the role played by
these companies in public schools. The No Child Left Behind Act of 2001 4
requires that schools that fail to meet state student achievement
standards for 5 consecutive years must be restructured by implementing one
or more
2 In fact, not all companies offer a particular curriculum or educational
approach, focusing instead on management services. For example, Arizona-
based ABS concentrates on management services like recruitment, personnel,
payroll, and facilities management.
3 However, in Arizona, private companies may run charter schools directly.
4 P. L. 107- 110, Jan. 8, 2002.
Page 5 GAO- 03- 11 Selected Private Education Companies
alternative governance actions. One of the alternatives available to
states and districts is to contract with an education management company.
5
Three companies currently operate in the District of Columbia: Edison
Schools, Mosaica Education, and Chancellor Beacon Academies. Edison began
operating its first District school in 1998, and Mosaica and Chancellor
Beacon first contracted with the District schools they manage in 2001.
Throughout this report, these companies will generally be discussed in
this order.
Mergers and acquisitions are common among such companies. In 2001, Edison
acquired nine schools nationwide through a merger with LearnNow. In the
same year, Mosaica acquired nine schools nationwide through its
acquisition of Advantage Schools. In addition, Chancellor and Beacon
merged into a single company. Such changes can have several outcomes: in
some cases, the company may operate schools that continue to use the
educational program of another company; in other cases, the school may
consider adopting the educational program of the new company or
terminating the contract.
The companies that operate public schools in the District of Columbia
offer management and educational services as part of their programs; the
extent to which District schools managed by these companies implemented
all of the components of the companies* programs varied. All of these
companies offer programs that include management and educational services,
such as curricula that integrate technology and professional development
opportunities for teachers. Of the 10 District schools managed by these
companies, 4 had completely implemented their company*s program. In school
year 2001- 02, all 10 District schools managed by these companies were
charter schools with predominantly poor and minority student populations;
most enrolled elementary and middle school students. Similar to
traditional public schools, the District schools managed by these
companies were required to be open to all students, up to their enrollment
limits, and to meet District standards in areas such as health, safety,
standardized testing, and compliance with federal special education
requirements.
5 Others include reconstituting by replacing teachers, principals, or
both; converting to public charter school status; and turning over school
operation to the state educational agency. Education
Management Companies Have Multifaceted Programs; District Schools Varied
in Their Implementation
Page 6 GAO- 03- 11 Selected Private Education Companies
The three for- profit companies that operate in the District of Columbia 6
*
Edison, Mosaica, and Chancellor Beacon* share common elements in terms of
the management and educational services they offer to schools nationwide
as well as those company officials described as distinctive. Each of the
three companies generally offers similar management services. For example,
all three offer management services such as personnel, payroll and
facilities management, services that can be important for charter schools.
In addition, the three companies employ some common approaches designed to
improve student achievement. All three companies offer an extended school
day and year. All three integrate technology in their educational
programs. For example, all three offer students access to classroom
computers. Similarly, all organize schools into smaller units to
facilitate their tracking of students* progress. All three provide summer
training to teachers as well as other forms of professional development.
Additionally, all have activities designed to involve and support parents
and students. For example, each company uses parent satisfaction surveys.
Experts we spoke to noted that these same approaches were being used in
some other public schools. Finally, officials of all three companies
stated that their companies contributed positively to school climate* a
sense of mission and an environment conducive to learning* and cited
aspects of school climate such as a safe and orderly school environment
and teacher motivation. In addition to the characteristics they had in
common, company officials identified others they believed were
distinctive. These include, for example, their programs* curriculum and
instruction as well as the ability to provide economies of scale, develop
community partnerships, and provide strong administrative support. As
Table 1 shows, all three companies provided their services to schools in
multiple states in 2001- 02.
6 In school year 2001- 02, a fourth for- profit company, Richard Milburn
Academies, continued to operate a District public charter school, pending
a final decision by District oversight authorities to revoke its charter.
The school closed in June 2002. All Three Companies Offer
Management and Educational Services to Schools Nationally
Page 7 GAO- 03- 11 Selected Private Education Companies
Table 1: Profiles of Three Education Management Companies Operating in the
District of Columbia
Edison Mosaica Chancellor Beacon
Year company was established 1992 1997 2001 a Number of states b where
company operated schools in school year 2001- 02 24 8 11 Number of schools
company operated nationwide in school year 2001- 02 136 21 82 Number of
schools company operated in the District of Columbia in school year 2001-
02 6 2 2 a Year of merger of Chancellor Academies (originally founded in
1999) and Beacon Education
Management (founded in 1993). b Including the District of Columbia.
Source: Edison, Mosaica, and Chancellor Beacon.
According to Edison officials, its program has a number of distinctive
characteristics. 7 The first of these is its curriculum, which emphasizes
basic skills, especially reading as the basis for future learning. It also
includes enrichment in areas such as world languages (e. g., Spanish) and
art. Edison*s basic skills curriculum includes components developed by
Edison, such as a remedial reading program, and other components that
Edison states are supported by research, such as Chicago Math and the
Success for All reading program. Instructional methods are a second
characteristic of Edison*s program. Edison schools use a variety of
instructional methods. One of these, direct instruction, relies on
repetition and drill. Other methods use projects, small groups, and
individualized lessons. A third characteristic of Edison schools is their
use of assessments. According to Edison officials, their program uses
frequent assessments and the results of these assessments are promptly
provided to teachers to assess student needs and provide appropriate
additional help.
*Systems and scale* is another key characteristic of Edison schools
according to company officials. The company views its schools as part of a
national system linked by a common purpose, and because of the system*s
size, the company says it is able to purchase supplies at lower costs.
7 Edison officials describe the educational approach as consisting of *10
fundamental* characteristics: school organization, extended school day and
year, curriculum, instructional methods, assessments, professional
teaching environment (including professional development), use of
technology, partnership with families, instructional programs that reflect
community interests, and the benefits of *systems and scale.*
Page 8 GAO- 03- 11 Selected Private Education Companies
Mosaica officials also identified certain distinctive characteristics of
their company*s program. 8 The first is the program*s curriculum, which
has two parts. According to Mosaica officials, its morning program
features instruction in traditional subjects such as reading and math. In
the afternoon, students use Paragon* Mosaica*s own curriculum. According
to company officials, Paragon stresses multidisciplinary learning, uses
projects to emphasize the humanities, and recognizes students* different
learning styles. For example, students may use their reading, math, and
social studies learning to build a pyramid or a Viking ship and thus study
a period of history. According to company officials, projects accommodate
a variety of learning styles* for example, some students learn visually,
others by performing. Community involvement is a second key characteristic
of Mosaica*s program. Company officials say that Mosaica brings community
support into the school by networking with various community
organizations. According to company officials, this provides its schools
with access to additional resources.
Chancellor Beacon officials also identified distinctive characteristics of
their program. One is their willingness to customize their educational
program to meet the needs and preferences of local schools. For example,
in response to community interest, some Chancellor Beacon schools feature
a cultural heritage element in the curriculum while one of its schools
emphasizes the environment. Chancellor Beacon*s own curriculum was
recently finalized in July 2002 and is based on an integration of the
curricula of Chancellor and Beacon before they merged. One component of
its curriculum is Core Knowledge* a program that expects students to
master specific content in language arts, history, geography, math,
science and fine arts. Other components emphasize ethics, morality and
community volunteerism. A second key characteristic of Chancellor Beacon*s
program is its operational support, according to company officials. These
officials told us that in focusing on operational support, Chancellor
Beacon allows schools to focus on academics.
While the Chancellor Beacon program emphasizes customization as a key
characteristic, the other two companies also allow schools to modify their
programs. For example, in its reading program, Edison allows schools some
flexibility regarding what books to read and in what order. In
8 Mosaica officials describe the educational approach as consisting of
*seven pillars* or characteristics: the Paragon curriculum, professional
development, use of technology, extended school day and year, parental
involvement, community involvement, and school climate.
Page 9 GAO- 03- 11 Selected Private Education Companies
addition, up to one- fourth of its curriculum can be determined by the
local school. Similarly, Mosaica allows its schools to use different
approaches or materials in their morning session.
While all of the 10 District schools managed by the companies during the
2001- 02 school year obtained management services from these companies,
the schools were more selective in implementing the companies* educational
programs. Of the 10 District schools, 4 have completely implemented the
companies* educational programs and 6 have adopted selected elements of
their companies* programs or chosen other programs, typically those of a
previous company. A key factor that helps explain the difference between
the programs the companies offer and what has been implemented by District
schools is that recent mergers and acquisitions have led to changes in
management companies in these 6 schools; these schools have generally left
in place the educational programs of the companies that formerly managed
them.
Four schools, all managed by Edison, implemented the company*s educational
program completely, according to company officials. These 4 schools all
opened in 1998 as the result of a partnership between Friendship House, a
nonprofit community organization serving District children and youth since
1904, and Edison. According to a Friendship House official, these schools
completely implemented Edison*s program because they saw it as
complementing their own goals. One of these schools* a high school* has
supplemented the Edison program by developing a program to expose certain
students to college through campus visits and workshops for parents.
Six District schools adopted selected elements of their companies*
educational programs or chose other educational programs. These 6 schools
include 2 schools managed by Edison, 2 by Mosaica, and 2 by Chancellor
Beacon. All 6 schools have had recent changes in management companies as a
result of mergers or acquisitions.
The 2 schools that received services from Edison have opted to retain the
curriculum already in place at the schools, rather than adopt the Edison
Of the 10 District Schools,
4 Completely Implemented the Company*s Educational Program While 6 Schools
Selected Elements of Their Companies* Programs or Chose Other Programs
Page 10 GAO- 03- 11 Selected Private Education Companies
program. 9 In 2001, Edison bought LearnNow, the company that formerly
provided services to the 2 schools. According to an Edison official
knowledgeable about the schools formerly managed by LearnNow, the primary
difference between the companies* curricula was in elementary language
arts, for which LearnNow preferred a different reading program than
Success for All, which the Edison program uses in its other schools.
The 2 schools managed by Mosaica have adopted some elements of the
company*s educational program, and have plans to adopt more by 2003. In
2001, Mosaica bought Advantage, the company that formerly managed these
schools. Both schools retained an instructional approach put in place by
the previous company. This approach* direct instruction* emphasizes drill
and repetition. By school year 2003, both schools expect to use direct
instruction during the morning session and Paragon in the afternoon.
The 2 schools managed by Chancellor Beacon both had distinct curricula in
place before being managed by this company; one has combined its existing
curriculum with elements of Chancellor Beacon*s, and the other has left
its existing curriculum in place. The school that has adopted elements of
Chancellor Beacon*s curriculum has done so by integrating the company*s
language arts and math curriculum with the school*s existing curriculum,
according to company officials. 10 This school, which serves at- risk
youth, had a curriculum called expeditionary learning, which focuses on
learning through field trips and experiences. The other Chancellor Beacon
school opted to retain its existing basic- skills curriculum, relying
instead on the company*s management services and selected educational
services, such as assessments. Chancellor Beacon officials support the
schools* choices regarding what company components to adopt.
Company and school officials identified several reasons why these 6
schools did not completely implement the current company*s educational
program, opting instead to continue with an existing
9 According to an Edison official, as of September 2002, Edison and one of
these 2 schools were planning a transition that would completely
discontinue Edison*s services to the school and return the school to
management by its own board of trustees. The services that Edison provided
to both schools in the 2001- 02 school year were limited to management
services, such as payroll.
10 This school*s existing educational approach is tied to a 3- year
federal comprehensive school reform demonstration grant. The grant expires
at the end of the 2004 school year.
Page 11 GAO- 03- 11 Selected Private Education Companies
curriculum. These included continuity for students, the company*s
flexibility with regard to local customization, and the right of charter
school boards to make broad curriculum decisions. 11
The 10 schools in the District managed by these companies shared certain
characteristics and served similar student populations in 2001- 02. All
were public charter schools governed by their own boards and accountable
to District oversight authorities. Most (9) were combined schools spanning
elementary and middle school grades. As public schools, they were required
to accept any student who applied, up to their enrollment limit. Their
student populations were substantially minority and poor: 92 to 100
percent African American and 48 to 95 percent receiving free or reduced
school lunch. 12 All served some students with special needs, such as
learning disabilities: in 9 of the schools, the percentage ranged from 5
to 13 percent, and in one school, 32 percent of the student population had
special needs. All but one served no or very few students with limited
English proficiency; at the remaining school, students with limited
English proficiency represented about 12 percent of all students enrolled.
Little rigorous research exists on the effectiveness of the three
educational management companies* Edison, Mosaica, and Chancellor Beacon*
in the schools they manage across the country; as a result, we cannot draw
conclusions about the effect that these companies* programs have on
student achievement, parental satisfaction, parental involvement, or
school climate. Students in company managed schools have demonstrated
academic progress, but more research is needed to determine if this
improvement is directly the result of the companies* programs and if this
progress is different from that of comparable students in traditional
public schools. We reviewed five studies that addressed student
achievement, but only one was conducted in a way that allows an assessment
of the effect the company*s program had on student achievement in one
school. The remaining studies had methodological limitations that
precluded such assessments. In an effort to learn more about
effectiveness, Edison has recently commissioned RAND, a nonprofit research
organization that has
11 In addition, according to officials of the agencies that oversee
District charter schools, substantial curriculum changes must be
officially approved. 12 Data were not available for all schools; these
ranges are based on 8 schools (percent African American) and 9 schools
(percent receiving subsidized lunch). Limited Research
Exists of the Effectiveness of These Companies* Programs
Page 12 GAO- 03- 11 Selected Private Education Companies
evaluated educational reforms, to complete a study to assess its program*s
impact.
Determining the effect of an educational company*s program can be
challenging for researchers. Ideally, evaluations of program effectiveness
should involve a comparison of outcomes for one group exposed to a
particular program with outcomes of a second group not exposed to the
program. Some evaluations assign participants randomly to one group or the
other to increase the likelihood that the two groups are roughly
equivalent on all characteristics that could affect outcomes. This
technique of random assignment is often problematic in educational
research because public school enrollment is generally based on residency
requirements. Therefore the most common way to compare student achievement
results from two different groups of students is to ensure the groups are
similar in a number of ways, including socioeconomic status, ethnicity,
and performance on prior academic assessments. In addition to controlling
for the effects of these background characteristics, it is critical to
follow the performance of students over time, preferably before any group
has been exposed to the program, and at least one point thereafter. 13 It
is also beneficial to analyze individual student data, rather than grade
or school- level averages, to account for individual differences and to
factor in the effects of missing data.
Within the context of rigorous educational program evaluations, various
measurements can be used to capture a student*s performance on
standardized tests. According to several experts, it is important to
examine both the percent of students in a particular grade or school
making yearly gains and the distribution of these gains across ability
levels to ensure that students of all achievement levels are demonstrating
academic growth. Another point of interest relates to the length of time
students participate in a particular program. Some experts claim that
students will exhibit greater gains the longer they participate in a
program. However, it is particularly challenging to design studies that
address this claim, because educational companies are still a relatively
new phenomenon.
We identified five studies concerning the three companies operating in the
District that met the criteria for our review: inclusion of comparison
13 Experts varied with regard to how many years of data are sufficient for
analysis. Generally, experts we spoke to indicated that 3 to 5 years of
data would be sufficient for analysis.
Page 13 GAO- 03- 11 Selected Private Education Companies
groups, measurement over time, and focus on academic achievement, parental
satisfaction, parental involvement, or school climate. 14 All of the
studies addressed the effectiveness of schools managed by Edison. One
study also addressed the effectiveness of schools managed by all three
private companies* Edison, Mosaica, and Chancellor Beacon. 15 We were
unable to identify any rigorous studies that included analysis of District
public schools managed by any of these three companies. 16 Of the studies
included in our review, four studies addressed only outcomes related to
student achievement, while one study addressed student achievement and
other outcomes such as parental satisfaction and school climate. 17
Only one of the studies, A Longitudinal Study of Achievement Outcomes in a
Privatized Public School: A Growth Curve Analysis, based on one Edison
school in Miami- Dade County, Florida, was conducted in a way that allows
an assessment of the program*s effect on student achievement. This study
followed individual student standardized test scores over a 3- year period
and found that Edison students progressed at similar rates to those in the
traditional Miami- Dade County Public Schools (MDCPS); this finding is not
generalizable to other schools managed by Edison or any other private
company. The study was designed to ensure that the Edison students were
similar to the random sample of students drawn from MDCPS in terms of
school grade, socioeconomic status, as indicated by the percent eligible
for free/ reduced price lunch, ethnicity, and achievement levels, as
indicated by comparability in test scores prior to students enrolling in
the Edison school. The study employed two different analytical techniques
and both resulted in the finding that the Edison students progressed at
similar rates to the traditional public school
14 While the study, Achievement Performance Report: Dallas- Edison
Partnership Schools 2001- 02 (Dallas Division of Evaluation and
Accountability, Dallas Independent School District, 2002), met the
criteria for inclusion, we were unable to review it because it was
published after completion of the review.
15 The evaluation included 18 schools managed by Beacon and was completed
prior to the merger of Chancellor and Beacon. 16 One study compares
academic achievement in District charter schools and traditional public
schools, but does not distinguish charter schools managed by private
companies. See Jeffrey Henig et al. Growing Pains: An Evaluation of
Charter Schools in the District of Columbia; 1999- 2000. (Washington, D.
C.: The Center for Washington Area Studies, The George Washington
University, 2001).
17 We did not review the student achievement portion of the Miami- Dade
County Public Schools Evaluation because the same data were analyzed, with
very similar results, by the same researcher in another study in the
review.
Page 14 GAO- 03- 11 Selected Private Education Companies
students. Several methodological techniques that would have strengthened
its overall findings could have been employed. These include controlling
more specifically for school- level differences between the participating
students as well as better ensuring the two groups of students remained
equivalent despite study dropouts (subsequently referred to as attrition).
Differences in the composition of these groups, after attrition, could
affect the test score results. This study did not examine the effect of
this company*s program on parental satisfaction, parental involvement, or
school climate.
Significant limitations in the other four studies preclude our making
assessments of the effectiveness of schools managed by Edison, Chancellor
Beacon, or Mosaica that were included in the studies. 18 These limitations
included use of comparison groups that did not adequately control for
differences between the students in the company*s schools and the students
in traditional public schools, instances where achievement data were not
available for all students, and lack of adjustment for high attrition
rates.
Company officials report that one way to determine if their programs are
effective is to assess whether students demonstrate academic growth as
evidenced by improvement on standardized tests. There is evidence to
support the assertion that students enrolled in schools managed by
Chancellor Beacon, Mosaica, and Edison have demonstrated academic
improvement from one point in time to another, but it is important to
determine if these gains are specifically the result of company programs.
Additional research is in progress. Edison commissioned RAND to evaluate
Edison schools across the country. Where possible, RAND plans to compare
the scores of individual Edison students to those of traditional public
schools students with similar characteristics. Since it is often difficult
to gather individual level student data, RAND will also compare Edison
data, either at the grade or school level, to publicly available state
data at that same level. RAND expects to publish its findings in 2004.
We received written comments on a draft report from the Department of
Education. These comments are presented in appendix III. Education
18 Please see appendix II for the studies and other information sources
considered for our review. Agency Comments
Page 15 GAO- 03- 11 Selected Private Education Companies
stated that there are insufficient data on the effectiveness of private
education companies. Education also stated that it encourages others*
evaluation efforts. We also received comments from an expert on private
education companies, the authors of the MDCPS study that we assessed, the
District of Columbia Board of Education, the District of Columbia Public
Charter School Board, as well as Edison Schools, Mosaica Education, and
Chancellor Beacon Academies. These comments were also incorporated where
appropriate.
We are sending a copy of this report to the Secretary of Education, the
District of Columbia Board of Education, the District of Columbia Public
Charter School Board, Edison Schools, Mosaica Education, and Chancellor
Beacon Academies. We will make copies available to others on request. In
addition, the report will be available at no charge on the GAO Web site at
http:// www. gao. gov.
If you or your staff have any questions about this report, please call me
at (202) 512- 7215. Other contacts and contributors to this report are
listed in appendix IV.
Sincerely yours, Marnie S. Shaul, Director Education, Workforce, and
Income Security Issues
Appendix I: Objectives, Scope, and Methodology
Page 16 GAO- 03- 11 Selected Private Education Companies
The objectives of our review were to (1) identify the characteristics of
the for- profit educational management companies operating in the District
and determine the extent to which District schools managed by these
companies have used their programs and (2) determine what is known about
the effectiveness of these programs, as measured primarily by student
achievement. We conducted our work between January and September 2002, in
accordance with generally accepted government auditing standards.
To identify the characteristics of the programs offered by for- profit
companies operating in the District, and determine the extent to which
District public schools managed by them have used their programs, we
interviewed company officials, representatives of the 10 schools, as well
as officials of the District*s chartering authorities. We collected
information on the companies from their Web sites and obtained technical
comments from the companies on the descriptions of their programs. We also
contacted education experts and advocates to obtain both their
recommendations on research regarding the three for- profit companies and
information on any research they might have conducted on the companies. We
also acquired information on the companies by reviewing relevant research
summaries. We also observed an on- site review of one school*s program
conducted for District oversight authorities.
To determine what is known about the effectiveness of these programs, we
collected, reviewed, and analyzed information from available published and
unpublished research on the effect on student achievement, parental
satisfaction, parental involvement, and school climate of the three
companies managing schools in the District. We also spoke with RAND
officials about the design and methods of their current evaluation of
Edison Schools. To identify relevant research, we followed three
procedures: (1) interviewed experts to find out what studies were
completed or in the process of being completed on the effectiveness of
company programs; (2) conducted library and Internet searches; and (3)
reviewed bibliographies of studies that focused on the effectiveness of
company programs.
We reviewed studies concerning the three companies operating in the
District that met the following criteria: included comparison groups and
measurement over time, and focused on academic achievement, parental
satisfaction, parental involvement, or school climate. Our final list of
studies for review consisted of five studies, as listed in appendix II. We
did Appendix I: Objectives, Scope, and
Methodology Objectives
Scope and Methodology
Appendix I: Objectives, Scope, and Methodology
Page 17 GAO- 03- 11 Selected Private Education Companies
not identify any studies that evaluated the effect of these three programs
in District schools.
Two GAO social scientists examined each study to assess the adequacy of
the samples and measures employed, the reasonableness and rigor of the
statistical techniques used to analyze them, and the validity of the
results and conclusions that were drawn from the analyses. For selected
studies, we contacted the researchers directly when we had questions about
their studies.
Appendix II: Studies and Other Information Sources Considered
Page 18 GAO- 03- 11 Selected Private Education Companies
In order to identify research that explicitly addresses the effect on
student achievement, parental satisfaction, parental involvement, or
school climate of the three companies managing schools in the District, we
interviewed experts to determine what studies were completed or in the
process of being completed, conducted library and Internet searches, and
reviewed bibliographies of studies that focused on the effect of these
companies* programs on student achievement. Although five studies met our
criteria for review (inclusion of comparison groups, measurement over
time, and focus on academic achievement, parental satisfaction, parental
involvement, or school climate), we cannot draw conclusions, due to
methodological weaknesses, from the four studies listed below. 1
Conclusions from A Longitudinal Study of Achievement Outcomes in a
Privatized Public School: A Growth Curve Analysis were presented in the
text.
Miron, Gary and Brooks Applegate. An Evaluation of Student Achievement
in Edison Schools Opened in 1995 and 1996. Kalamazoo, Michigan: The
Evaluation Center, Western Michigan University, December 2000.
Miron and Applegate analyzed both individual and aggregate level data
and compared improvements in the test scores of 10 Edison schools with
those of comparison schools, districts, states, and national norms, where
applicable. However, significant weaknesses prevented conclusive
statements on the effects of Edison schools. These weaknesses included
limitations in the available data, such as incompleteness and
inconsistency, high attrition rates, and the lack of corresponding
adjustments for attrition.
Horn, Jerry and Gary Miron. An Evaluation of the Michigan Charter School
Initiative: Performance, Accountability, and Impact. Kalamazoo, Michigan:
The Evaluation Center, Western Michigan University, July 2000.
Horn and Miron examined the percentage of students earning a passing
grade on achievement tests in individual charter schools in Michigan in
comparison with the percentage passing in the districts where these
1 While the study, Achievement Performance Report: Dallas- Edison
Partnership Schools 2001- 02 (Dallas Division of Evaluation and
Accountability, Dallas Independent School District, 2002), met the
criteria for inclusion, we were unable to review it because it was
published after completion of the review. Appendix II: Studies and Other
Information
Sources Considered
Appendix II: Studies and Other Information Sources Considered
Page 19 GAO- 03- 11 Selected Private Education Companies
schools were located. 2 The analysis included schools managed by Edison,
Mosaica, and Beacon. Weaknesses included inadequate controls for
differences between the students in charter schools and their host
districts, no consideration of attrition rates, and the likelihood that
analyses were often based on a small number of students.
American Federation of Teachers. Trends in Student Achievement for
Edison Schools, Inc.: The Emerging Track Record. Washington, D. C.:
October 2000.
Researchers examined school and grade- level achievement data from 40
Edison schools in eight states and compared it to data gathered from
school districts and other schools. Weaknesses included insufficient
information about the methodology employed by the states, including
construction of comparison groups and matching techniques, and a lack of
analysis of attrition rates.
Gomez. Ph. D., Joseph and Sally Shay, Ph. D. Evaluation of the Edison
Project School. Final Report, 1999- 00 (portions related to parental
satisfaction and involvement, and school climate). Office of Evaluation
and Research, Miami- Dade County Public Schools (MDCPS), April 2001. 3
Gomez and Shay examined responses from surveys MDCPS had administered to
parents and teachers from both the Edison school and the control group.
However, the outcomes related to parental satisfaction and involvement
were measured with single- item survey questions that do not seem to
capture the full context of the concepts. School climate was measured with
a single- item question on a teacher survey and with school archival data.
4 Shay and Gomez did not report whether any differences are statistically
significant, in part because
2 Analyses of individual schools were included in an appendix provided
upon request from the authors. We included this study in our review
because it explicitly addressed the results of schools managed by the
three companies in this report, unlike other studies that did not
disaggregate school results by company.
3 We only examined the portion of this study that relates to the effect of
the Edison school on parental satisfaction and involvement and school
climate. We examined the data and analysis that related to the academic
achievement piece in Dr. Shay*s dissertation, A Longitudinal Study of
Achievement Outcomes in a Privatized Public School: A Growth Curve
Analysis, reported in the text.
4 This includes data on students* attendance rates, student mobility,
indoor and outdoor suspensions, and the teacher- student ratio.
Appendix II: Studies and Other Information Sources Considered
Page 20 GAO- 03- 11 Selected Private Education Companies
they acknowledged it would be inappropriate to conduct tests of
significance on single- item questions. Therefore, there is no evidence to
determine whether Edison school parents were more satisfied or involved
than those in the control group, or whether the Edison school improved
school climate.
We are aware of other studies and reports that address the effect of
Chancellor Beacon Academies, Mosaica Education, and Edison Schools on
academic achievement, parental satisfaction, parental involvement, or
school climate; however, the following are examples that did not meet the
criteria for inclusion in our review.
District of Columbia Public Charter School Board. School Performance
Reports. Washington, D. C.: August 2001.
Department of Research, Evaluation, and Assessment, Minneapolis Public
Schools. Edison/ PPL School Information Report 2000- 2001.
Minneapolis, Minnesota: 2001.
Department of Administration, Counseling, Educational and School
Psychology, Wichita State University. An Independent Program Evaluation
for the Dodge- Edison Partnership School: First Year Interim Report.
Wichita, Kansas: 1996.
Missouri Department of Elementary and Secondary Education. Charter
School Performance Study: Kansas City Charter Schools. Jefferson City,
Missouri: 2001.
Company- provided information such as annual reports and school
performance reports.
Other sources of general information included school district websites and
other educational services, such as Standard and Poor*s School Evaluation
Services and the National Association of Charter School Authorizers*
Educational Service Provider Information Clearinghouse.
Appendix III: Comments from the Department of Education Page 21 GAO- 03-
11 Selected Private Education Companies
Appendix III: Comments from the Department of Education
Appendix IV: GAO Contacts and Staff Acknowledgments
Page 22 GAO- 03- 11 Selected Private Education Companies
Harriet Ganson, (202) 512- 7042 Chris Morehouse, (202) 512- 7214
In addition to those named above, Rebecca Ackley and N. Kim Scotten made
key contributions to this report. Jay Smale, Michele Fejfar, Kevin
Jackson, Sara Ann Moessbauer, and Shana Wallace provided important
methodological contributions to the review of research. Patrick Dibattista
and Jim Rebbe also provided key technical assistance. Appendix IV: GAO
Contacts and Staff
Acknowledgments GAO Contacts Acknowledgments
Related GAO Products Page 23 GAO- 03- 11 Selected Private Education
Companies
School Vouchers: Characteristics of Privately Funded Programs. GAO- 02752.
Washington, D. C.: September 26, 2002.
School Vouchers: Publicly Funded Programs in Cleveland and Milwaukee. GAO-
01- 914. Washington, D. C.: August 31, 2001.
Charter Schools: Limited Access to Facility Financing. GAO/ HEHS- 00163.
Washington, D. C.: September 12, 2000)
Charter Schools: Federal Funding Available but Barriers Exist. HEHS- 9884.
Washington, D. C.: April 30, 1998.
Charter Schools: Recent Experiences in Accessing Federal Funds.
T- HEHS- 98- 129. Washington, D. C.: March 31, 1998.
Charter Schools: Issues Affecting Access to Federal Funds. T- HEHS- 97216.
Washington, D. C.: September 16, 1997.
Private Management of Public Schools: Early Experiences in Four School
Districts. GAO/ HEHS- 96- 3. Washington, D. C.: April 19, 1996.
Charter Schools: New Model for Public Schools Provides Opportunities and
Challenges. GAO/ HEHS- 95- 42. Washington, D. C.: January 18, 1995.
School- Linked Human Services: A Comprehensive Strategy for Aiding
Students At Risk of School Failure. GAO/ HRD- 94- 21. Washington, D. C.:
December 30, 1993. Related GAO Products
(130121)
The General Accounting Office, the investigative arm of Congress, exists
to support Congress in meeting its constitutional responsibilities and to
help improve the performance and accountability of the federal government
for the American people. GAO examines the use of public funds; evaluates
federal programs and policies; and provides analyses, recommendations, and
other assistance to help Congress make informed oversight, policy, and
funding decisions. GAO*s commitment to good government is reflected in its
core values of accountability, integrity, and reliability.
The fastest and easiest way to obtain copies of GAO documents at no cost
is through the Internet. GAO*s Web site (www. gao. gov) contains abstracts
and fulltext files of current reports and testimony and an expanding
archive of older products. The Web site features a search engine to help
you locate documents using key words and phrases. You can print these
documents in their entirety, including charts and other graphics.
Each day, GAO issues a list of newly released reports, testimony, and
correspondence. GAO posts this list, known as *Today*s Reports,* on its
Web site daily. The list contains links to the full- text document files.
To have GAO e- mail this list to you every afternoon, go to www. gao. gov
and select *Subscribe to daily E- mail alert for newly released products*
under the GAO Reports heading.
The first copy of each printed report is free. Additional copies are $2
each. A check or money order should be made out to the Superintendent of
Documents. GAO also accepts VISA and Mastercard. Orders for 100 or more
copies mailed to a single address are discounted 25 percent. Orders should
be sent to:
U. S. General Accounting Office 441 G Street NW, Room LM Washington, D. C.
20548
To order by Phone: Voice: (202) 512- 6000 TDD: (202) 512- 2537 Fax: (202)
512- 6061
Contact: Web site: www. gao. gov/ fraudnet/ fraudnet. htm E- mail:
fraudnet@ gao. gov Automated answering system: (800) 424- 5454 or (202)
512- 7470
Jeff Nelligan, managing director, NelliganJ@ gao. gov (202) 512- 4800 U.
S. General Accounting Office, 441 G Street NW, Room 7149 Washington, D. C.
20548 GAO*s Mission
Obtaining Copies of GAO Reports and Testimony
Order by Mail or Phone To Report Fraud, Waste, and Abuse in Federal
Programs
Public Affairs
*** End of document. ***