Adult Education: Measuring Program Results Has Been Challenging (Chapter
Report, 09/08/95, GAO/HEHS-95-153).
--------------------------- Indexing Terms -----------------------------
REPORTNUM: HEHS-95-153
TITLE: Adult Education: Measuring Program Results Has Been
Challenging
DATE: 09/08/95
SUBJECT: Accountability
Adult education
Education program evaluation
Secondary education
Educational grants
State-administered programs
Disadvantaged persons
Educational programs
IDENTIFIER: JOBS Program
Connecticut
JTPA
Iowa
California JOBS Program
Comprehensive Adult Student Assessment System
California
National Evaluation of Adult Education Programs
Dept. of Education Adult Education Act State Program
Job Training Consolidation Act of 1995
Consolidated and Reformed Education, Employment, and
Rehabilitation Systems Act
**************************************************************************
* This file contains an ASCII representation of the text of a GAO *
* report. Delineations within the text indicating chapter titles, *
* headings, and bullets are preserved. Major divisions and subdivisions *
* of the text, such as Chapters, Sections, and Appendixes, are *
* identified by double and single lines. The numbers on the right end *
* of these lines indicate the position of each of the subsections in the *
* document outline. These numbers do NOT correspond with the page *
* numbers of the printed product. *
* *
* No attempt has been made to display graphic images, although figure *
* captions are reproduced. Tables are included, but may not resemble *
* those in the printed version. *
* *
* A printed copy of this report may be obtained from the GAO Document *
* Distribution Facility by calling (202) 512-6000, by faxing your *
* request to (301) 258-4066, or by writing to P.O. Box 6015, *
* Gaithersburg, MD 20884-6015. We are unable to accept electronic orders *
* for printed documents at this time. *
**************************************************************************
Cover
================================================================ COVER
Report to Congressional Requesters
September 1995
ADULT EDUCATION - MEASURING
PROGRAM RESULTS HAS BEEN
CHALLENGING
GAO/HEHS-95-153
Adult Education Act
(104801)
Abbreviations
=============================================================== ABBREV
ABE - Adult Basic Education
AEA - Adult Education Act
ASE - Adult Secondary Education
CASAS - Comprehensive Adult Student Assessment System
ESL - English as a Second Language
GED - General Educational Development
JOBS - Job Opportunities and Basic Skills
JTPA - Job Training Partnership Act
VOC ED - Perkins Vocational Education
Letter
=============================================================== LETTER
B-259608
September 8, 1995
The Honorable William F. Goodling
Chairman
The Honorable William L. Clay
Ranking Minority Member
Committee on Economic and
Educational Opportunities
House of Representatives
The Honorable Randy (Duke) Cunningham
Chairman
The Honorable Dale E. Kildee
Ranking Minority Member
Subcommittee on Early Childhood,
Youth and Families
Committee on Economic and
Educational Opportunities
House of Representatives
The Adult Education Act (AEA) represents the primary federal effort
to improve the literacy skills of educationally disadvantaged adults.
Programs funded under the AEA are also important in providing the
basic skills needed by many clients of federal employment training
programs.
This report responds to a request from the former House Committee on
Education and Labor and its Subcommittee on Elementary, Secondary,
and Vocational Education that we review several issues in
anticipation of the AEA's reauthorization. In subsequent discussions
with your staff, we agreed to provide information on the AEA's
largest program (the State Grant Program), its coordination with
federal employment training programs, and the extent to which it
ensures accountability for results.
We are sending copies of this report to the Secretaries of Education,
Health and Human Services, and Labor, as well as interested
congressional committees. Copies will be made available to others
upon request. Major contributors to this report are listed in
appendix III. If you have any questions, please call me at (202)
512-7014.
Linda G. Morra
Director, Education and
Employment Issues
EXECUTIVE SUMMARY
============================================================ Chapter 0
PURPOSE
---------------------------------------------------------- Chapter 0:1
Nearly 90 million adults in the United States have deficient literacy
skills, according to a recent national survey. These adults may not
be able to write a letter explaining an error on a credit card bill,
use a bus schedule to determine which bus to take, or calculate the
difference between the regular and sale price of an item. Deficient
literacy skills, however, are not just an individual concern. Adult
literacy problems also threaten the nation's economy, which depends
on increasingly high levels of workplace skills to remain competitive
in a global market.
The Congress passed the Adult Education Act (AEA) to, among other
things, help states fund programs for adults to acquire the basic
skills needed for literate functioning, benefit from job training,
and continue their education through at least high school. The AEA
was last reauthorized in 1991. In anticipation of its next
reauthorization, the former House Committee on Education and Labor
and its Subcommittee on Elementary, Secondary, and Vocational
Education requested that GAO provide information on (1) the AEA's
largest program--the State Grant Program--and its coordination with
federal employment training programs and (2) the extent to which the
program ensures accountability for program quality and results. GAO
relied on national data in conducting its review and performed more
detailed work in three states--California, Connecticut, and Iowa.
BACKGROUND
---------------------------------------------------------- Chapter 0:2
Programs funded under the AEA are administered by the Department of
Education. For the State Grant Program, the Department makes grants
to states on the basis of the number of people in each state who are
at least 16 years old, not required to be in school, and lack a high
school degree. Local adult education providers apply to the states
for funds. The Department of Education reported that, in fiscal year
1995, about 4 million adults were enrolled in classes funded by the
State Grant Program; federal funding was $252 million, and state and
local sources provided an additional $890 million. Although total
current state and local contributions far exceed federal
expenditures, federal dollars still total more than half the adult
education funds in almost half of the states.
Programs funded under the AEA are important in providing the basic
literacy skills needed by clients of federal employment training
programs administered by the Departments of Education, Health and
Human Services, and Labor. Thus, the AEA and employment training
legislation require coordination among these programs to avoid
duplication and enhance service delivery.
The federal role in administering programs funded under the AEA may
soon change. The 104th Congress is considering legislation that
would consolidate adult education and other programs and provide one
or more block grants to states. The Senate bill would repeal most
existing federal employment training programs, as well as the State
Grant Program, and replace them with a single block grant. The House
bill would also repeal most employment training programs but would
replace them with four block grants, including a separate grant for
adult education and literacy programs.
RESULTS IN BRIEF
---------------------------------------------------------- Chapter 0:3
The goals of the AEA, which encompasses the State Grant Program, are
broad to enable people with diverse needs to receive varying types of
instruction. The most common types of instruction funded under the
State Grant Program are basic education (for adults functioning below
the eighth grade level), secondary education, and English as a Second
Language. Because many clients of federal employment training
programs need instruction provided by the State Grant Program,
coordination among these programs is essential.
Although the State Grant Program funds programs that address the
educational needs of millions of adults, it has had difficulty
ensuring accountability for results because of a lack of clearly
defined program objectives, questionable validity of adult student
assessments, and poor student data. Amendments to the AEA required
the Department of Education to improve accountability by developing
model indicators of program quality that states could adopt and use
to evaluate local programs. However, experts disagree about whether
developing indicators would help states to define measurable program
objectives or evaluate local programs and collect more accurate data.
Other federal efforts may help states achieve better accountability
systems, but it is too soon to evaluate their effectiveness.
PRINCIPAL FINDINGS
---------------------------------------------------------- Chapter 0:4
LARGEST PROGRAM STRIVES TO
MEET MANY GOALS AND NEEDS
-------------------------------------------------------- Chapter 0:4.1
The AEA's and the State Grant Program's goals include assisting
adults who lack the basic literacy skills needed for effective
citizenship, productive employment, and high school completion. Many
clients of federal employment training programs need the basic skills
taught in adult education classes. For example, almost 30 percent of
the participants in the Department of Labor's Job Training
Partnership Act Program are school dropouts; as many as 50 percent
may lack basic skills.
In accordance with their federally approved plans, states fund local
organizations to provide varying services to a wide range of adults.
For example, in one community we visited, a 19-year-old mother with a
tenth grade education was taking basic skills classes so that she
could complete high school and become a cosmetologist, a 28-year-old
stock clerk was taking high school equivalency classes with the hope
of going to college and becoming a police officer, and a 62-year-old
immigrant who had been an accountant in Russia was enrolled in
English classes so that she could become a U.S. citizen.
In further keeping with state plans, which call for coordinating
adult education with employment training programs, a variety of
coordination activities were taking place in the states and
communities we visited. These activities included pooling funds,
establishing one-stop centers, and developing uniform assessment
systems. For example, one state pooled funds from many sources for
coordinated grants that, among having other advantages, enabled
service providers to respond to a single request for proposal.
ENSURING PROGRAM
ACCOUNTABILITY HAS PROVEN
DIFFICULT
-------------------------------------------------------- Chapter 0:4.2
Evaluating program results depends on having clearly defined
objectives, valid assessment instruments, and accurate program data.
Some program officials and experts have raised concerns that, because
the State Grant Program lacks clearly defined objectives, the types
of skills and knowledge adults need to be considered literate are not
clear and, thus, states do not have sufficient direction for
measuring results. In addition, some research has questioned the
validity and appropriateness of the student assessments used in adult
education programs and, therefore, the usefulness of the data
generated from these assessments. In the states we visited, local
program officials had mixed views of the assessment instruments the
states required them to use. Finally, missing and inaccurate data
may compromise any attempt to improve program accountability.
Federal and state officials acknowledged serious problems with the
data that states report to the Department of Education. Recent
studies have attributed difficulties in obtaining accurate data to
the sporadic attendance patterns of adult students and the limited
time and expertise of local adult education program staff.
Federal efforts to improve accountability have focused on developing
indicators of program quality, providing technical assistance to
states, and requiring states to set aside funds for demonstration
projects and training. The Department of Education has developed
model indicators in eight areas, including student outcomes, and
provided examples of measures that could be used to quantify
performance in these areas. And, as required, states have adopted
the indicators or developed their own to use to evaluate local
programs. Experts and some program officials, however, have had
mixed views about whether the indicators would help states evaluate
local programs or collect better quality data. Because the 1993-94
program year was the first year the indicators were required to be
used for evaluation, it is too soon to tell whether the indicators
will lead to improvements. It is also too early to assess whether
the Department's technical assistance efforts or the state set-asides
for demonstration projects and training will help states develop
better accountability systems.
RECOMMENDATIONS
---------------------------------------------------------- Chapter 0:5
GAO is making no recommendations in this report.
AGENCY COMMENTS
---------------------------------------------------------- Chapter 0:6
The Department of Education provided written comments on a draft of
this report. The Department recognized that GAO identified the three
areas that are critically important to improving accountability in
adult education: clear purpose and expectations, good assessment
instruments, and high-quality data. The Department also stated its
commitment to improving program accountability through several
current initiatives, such as developing an individualized student
record keeping system and training programs for adult education staff
in collecting, analyzing, and reporting student program data. (See
app. II for the Department's letter.)
INTRODUCTION
============================================================ Chapter 1
The National Adult Literacy Survey\1 estimated that approximately 90
million American adults have deficient literacy skills. Of those,
between 40 and 44 million adults--about 22 percent of the country's
adult population--have severe problems with literacy, defined as the
ability to read, write, and speak English and compute and solve
problems proficiently. An additional 50 million adults are likely to
encounter some problems functioning in society and need improved
literacy skills.
The Adult Education Act\2 (AEA) is administered by the Department of
Education. The act represents the primary federal effort to
alleviate problems in adult literacy and provides the basic
legislative authority and largest source of federal funds for
programs that benefit educationally disadvantaged adults.\3 The act's
largest program is the Adult Education State-Administered Basic Grant
Program (State Grant Program). In fiscal year 1995, federal funding
for this program was $252 million, while state and local sources
provided $890 million, or 78 percent of the program's total budget.
For the first 15 years of the State Grant Program (1966 to 1980),
federal expenditures exceeded total state and local contributions;
however, total state and local contributions have since surpassed
federal expenditures. (See table I.2 in app. I for total annual
expenditures.) Figure 1.1 compares federal expenditures with state
and local expenditures since the AEA's passage in 1966.
Figure 1.1: Comparison of
Federal Expenditures With State
and Local Expenditures, Fiscal
Years 1966-1995
(See figure in printed
edition.)
Note: This figure reflects federal appropriations data because
federal expenditure data were not readily available. According to
Department of Education officials, appropriations and expenditure
data for the program tend to be virtually the same. For fiscal years
1992 to 1995, state and local expenditures were estimated.
Source: U.S. Department of Education.
Although total state and local contributions currently far exceed
federal expenditures, federal dollars still total more than half the
funds for adult education in almost half of the states. The
contribution of each state relative to the federal contribution
varies widely, depending on each state's commitment to providing
adult education services. For example, in fiscal year 1991,\4
state and local contributions ranged from a low of 21 percent to a
high of 96 percent; conversely, federal expenditures ranged from 4 to
79 percent. Since 1992, the AEA has restricted the federal share of
each state's expenditure to no more than 75 percent. (See table I.4
in app. I for further information on expenditures by state.)
The AEA makes grants to states and requires that they be used in
accordance with federally approved state plans. In developing their
plans, states must assess the needs of adults, including
educationally disadvantaged adults, and the capability of programs
and institutions to meet those needs. The Department of Education
annually makes its grants to states\5 on the basis of the number of
individuals in each state who are at least 16 years old, not enrolled
in school, and lack a high school degree or General Educational
Development (GED) credential.\6 Local adult education providers then
apply to the states for funds.
Following are the three most common types of instruction offered
under the State Grant Program:
Adult Basic Education (ABE), which is instruction designed for
adults functioning below the eighth grade level;
Adult Secondary Education (ASE), which is instruction designed for
adults functioning at the secondary level that may culminate in
a high school diploma or may serve as preparation for the GED\7
examination; and
English as a Second Language (ESL), which is instruction designed
to teach English to non-English speakers.
(See table I.3 in app. I for further information on enrollment by
instructional area.)
Programs funded under the AEA are important in providing basic
literacy skills needed by clients of federal employment training
programs such as Perkins Vocational Education (VOC ED), administered
by the Department of Education; Job Opportunities and Basic Skills
(JOBS), administered by the Department of Health and Human Services;
and the Job Training Partnership Act (JTPA), administered by the
Department of Labor. Consequently, the AEA and employment training
legislation require coordination among these programs to avoid
duplication and enhance service delivery.
The National Literacy Act of 1991\8 amended the AEA and authorized
several new programs. Major provisions included the creation of the
National Institute for Literacy, the establishment of state and
regional literacy resource centers, and a requirement for the
Department of Education to develop model indicators of program
quality to guide states in developing their own indicators for
improved program evaluation.
The 104th Congress is considering legislation that would consolidate
adult education and other programs and provide one or more block
grants to states. The Senate bill\9 would repeal most existing
federal employment training programs, including the State Grant
Program, and replace them with a single block grant. The House
bill\10 would also repeal most employment training programs but
replace them with four block grants, including a separate grant for
adult education and literacy programs.
--------------------
\1 This 1992 survey profiled the literacy of U.S. adults on the
basis of their performance in a wide array of tasks that reflect the
types of materials and demands they encounter in their daily lives.
The Department of Education contracted with the Educational Testing
Service to conduct this survey.
\2 The AEA was originally passed as part of the Elementary and
Secondary Education Amendments of 1966 (P.L. 89-750) and was
rewritten as part of the Augustus F. Hawkins-Robert T. Stafford
Elementary and Secondary School Improvement Amendments of 1988 (P.L.
100-297).
\3 These are adults who demonstrate basic skills equivalent to or
below that of students at the fifth grade level.
\4 According to the Department of Education, this is the most recent
year for which expenditure data are available (obligation authority
was from July 1, 1991, to June/Sept. 30, 1993).
\5 Each state receives $250,000 before the formula is applied.
\6 More than 44 million people were in this category, according to
the 1990 Census.
\7 The GED is equivalent to a high school diploma and is awarded upon
successful completion of a battery of nationally normed and scored
tests.
\8 Public Law 102-73.
\9 The Job Training Consolidation Act of 1995 (S. 143, 104th Cong.)
was introduced on January 4, 1995.
\10 The Consolidated and Reformed Education, Employment, and
Rehabilitation Systems Act or CAREERS Act (H.R. 1617, 104th Cong.)
was introduced on May 11, 1995.
OBJECTIVES, SCOPE, AND
METHODOLOGY
---------------------------------------------------------- Chapter 1:1
At the request of the Chairman and Ranking Minority Member of the
former House Committee on Education and Labor and the Chairman and
Ranking Minority Member of the former Subcommittee on Elementary,
Secondary, and Vocational Education, we reviewed several issues
related to the AEA. Specifically, we examined
the goals of the AEA and its largest program (the State Grant
Program), the population served by the program, program
services, and its coordination with federal employment training
programs and
the extent to which the State Grant Program ensures accountability
for program quality and results, including how states have
implemented quality indicators.
We focused our review primarily on the State Grant Program because it
is the largest of the AEA's funded programs. In fiscal year 1995, 83
percent of AEA funds were allocated to this program.
To obtain nationwide information on the State Grant Program, we
interviewed federal officials from the Department of Education. We
also reviewed Department of Education data and recent national
studies, including the National Evaluation of Adult Education
Programs and the National Adult Literacy Survey.
We selected three states for closer review: California, Connecticut,
and Iowa. We selected these states because they provided some
geographic dispersion and represented a range of (1) state and local
financial commitments (as demonstrated by the percentage of matching
funds each state contributes), (2) program size (as demonstrated by
dollars and enrollment), and (3) ESL enrollment levels. Within each
state, we visited at least two communities that we selected with the
help of state adult education officials. We selected communities
that represented different types of locales (urban, suburban, rural)
and were involved in a variety of local coordination activities.
To identify the goals of the AEA, including the State Grant Program,
we reviewed federal legislation. To determine the populations served
and services provided by the program, we reviewed Department of
Education data and national studies. We also interviewed local adult
education providers.
To provide information on the coordination of AEA programs with
employment training programs, we interviewed federal officials at the
Departments of Education, Labor, and Health and Human Services, and
held discussions with national experts, including representatives of
the National Institute for Literacy. We also reviewed studies on
coordination. In the states we visited, we met with state officials
from adult education and JOBS, JTPA, and VOC ED programs. In Iowa
and parts of California, where the community college system is the
major adult education provider, we also met with state community
college representatives. At local levels in the three states, we met
with adult education providers as well as representatives of local
employment training programs.
To provide information on program accountability and quality, we
interviewed Department of Education officials and held discussions
with national adult education experts. We also reviewed the
Department's model indicators of program quality and studies on
program accountability and quality issues. In addition, we
interviewed state and local officials in California, Connecticut, and
Iowa and reviewed program documents, including the quality indicators
developed by these states.
We conducted our work between November 1994 and August 1995 in
accordance with generally accepted government auditing standards.
LARGEST AEA PROGRAM STRIVES TO
MEET MANY GOALS AND NEEDS
============================================================ Chapter 2
The AEA is a broad and flexible act and its largest program, the
State Grant Program, reflects this. The State Grant Program, the
federal government's primary adult education program, has many goals
and enables people with a wide range of needs to receive instruction
from a variety of service providers. Many clients of employment
training programs are among those needing the basic skills taught by
adult education. Although the program has some restrictions, it
allows states considerable flexibility in the types of instruction
they fund with their federal grants as long as they fund programs in
accordance with federally approved state plans. In keeping with
their state plans, which call for coordinating adult education with
employment training programs, a variety of coordination activities
were taking place in the states and communities we visited.
GOALS ARE BROAD
---------------------------------------------------------- Chapter 2:1
Recognizing the wide range of adult literacy needs in this country,
the Congress passed the AEA with broadly stated goals. Although
adult education programs are commonly viewed as the means to obtain a
high school diploma or its equivalent, the AEA established goals that
are far broader and include citizenship and employment as well as the
overall improvement of the adult education system. Specifically, the
purpose of the AEA is to
improve educational opportunities for adults who lack literacy
skills necessary for effective citizenship and productive
employment;
expand and improve the current adult education delivery system; and
encourage the establishment of adult education programs for adults
to (1) acquire basic skills needed for literate functioning, (2)
acquire basic education needed to benefit from job training and
obtain and keep productive employment, and (3) continue their
education to at least the secondary school level.
ADULT EDUCATION STUDENTS ARE
DIVERSE
---------------------------------------------------------- Chapter 2:2
Adult education students have diverse needs, circumstances, and
personal characteristics. A student might be a high school dropout,
a client in a job training program, an immigrant or refugee, a
displaced worker or homemaker, an adult in the workplace, a welfare
recipient, or a retiree. Students who enroll in adult education
classes vary in age, race and ethnicity, and employment status.
In one program we visited in a rural California town, a 23-year-old
refugee was enrolled in an Adult Secondary Education (ASE) class. He
was a machine operator with an eleventh grade education who planned
to earn his GED and become a bilingual teacher. A 45-year-old
unemployed mother of four with a third grade education was enrolled
in an English as a Second Language (ESL) class. Having done seasonal
work in the past, her goal was to obtain a GED and find work in the
nursing field.
In an urban Connecticut program, a 19-year-old mother on welfare
wanted to complete high school and become a cosmetologist. Although
she had a tenth grade education, she needed the basic skills taught
in an Adult Basic Education (ABE) class. In the same program, a
62-year-old immigrant who had been an accountant in Russia was
attending ESL classes. Her goal was to become a U.S. citizen. In a
suburban program in Connecticut, a 28-year-old part-time stock clerk
was enrolled in an ASE class. He had a ninth grade education and
lived with his parents. He hoped to earn his GED, attend college,
and become a police officer.
In a rural town in Iowa, a 48-year-old father of four from Laos spoke
no English and was enrolled in a beginning ESL class. He worked
part-time as an upholstery worker but hoped to learn English well
enough to get a full-time job. In a city in Iowa, a married,
35-year-old mother of three was enrolled in an ABE class. A former
welfare recipient, she had a job as a child care aide that was
contingent upon her earning a GED. Her goal was to earn the GED and
keep her job.
National statistics also suggest that adult education students are
fairly diverse. Nationwide, 38 percent of students enrolled in adult
education classes in 1993 were between the ages of 16 and 24, 46
percent were between the ages of 25 and 44, and the remaining
students were 45 years old or older, according to the Department of
Education. Also, 36 percent of the students were white, 31 percent
Hispanic, 18 percent black, 14 percent Asian or Pacific Islander, and
1 percent American Indian or Alaskan Native.
The National Evaluation of Adult Education Programs\11 conducted a
survey of students who entered the adult education system between
April 1991 and April 1992. It found that 42 percent of these
students were employed, and 58 percent were either unemployed or not
in the workforce when they enrolled. During the year before
enrollment, 43 percent of ABE students, 31 percent of ASE students,
and 14 percent of ESL students received public assistance or welfare
payments.
--------------------
\11 National Evaluation of Adult Education Programs, Second Interim
Report: Profiles of Client Characteristics, Development Associates,
Inc. (Arlington, Va.: 1993).
MANY EMPLOYMENT TRAINING
CLIENTS NEED ADULT EDUCATION
SERVICES
---------------------------------------------------------- Chapter 2:3
Many clients of federal employment training programs rely on the
State Grant Program for the basic skills they lack. According to the
Department of Labor, unless an attempt is made to upgrade the
literacy skills of clients in federal employment training programs,
clients' success may be limited and access to the job market may be
denied. Nationwide, almost 30 percent of JTPA clients are school
dropouts,\12 and as many as half may lack basic skills.\13 One-fourth
of JOBS clients in fiscal year 1992 were enrolled in a high school
completion program.\14
Adult education enrollment has risen almost every year since 1966,
and the Congress is considering welfare reform proposals that may
place even greater demands on adult education providers. These
proposals may also make the coordination among adult education,
welfare, and employment training programs even more critical. Some
states are already implementing their own welfare reform efforts that
require certain welfare clients to obtain adult education or
employment training services to receive assistance. For example,
California's JOBS program requires that welfare recipients have
opportunities to remedy basic skill deficiencies and earn a high
school diploma or GED credential. The state is currently required to
provide adult education to its JOBS clients with low assessment
scores and to continue to provide education until clients attain a
specified level of proficiency.
Connecticut has piloted a welfare reform program that targets certain
welfare recipients. Individuals in the pilot can receive needed
remedial services, such as adult education or vocational training,
for 2 years before being required to find jobs. According to a state
official, the pilot was limited to two communities because of
concerns about the state's ability to provide remedial services to
all needy individuals, particularly adult education services. If a
client has only 2 years to seek remedial services and faces a waiting
list for adult education services, both the client and the entire
program are at risk, explained the official.
Similarly, Iowa's JOBS program has a goal of moving people off
welfare within 2 years by providing remedial education and employment
training. This welfare reform effort has increased the percentage of
welfare recipients required to participate in the JOBS program from
24 to 88 percent. This increase is achieved, in part, by exempting
fewer welfare recipients from participating in the JOBS program. For
example, only parents with children under 6 months of age are exempt;
previously, parents with children under the age of 3 were exempt.
--------------------
\12 Workplace Literacy and the Nation's Unemployed Workers, U.S.
Department of Labor (Washington, D.C.: 1993).
\13 I. Kirch, A. Jungeblut, and A. Campbell, Beyond the School
Doors: The Literacy Needs of Job Seekers Served by the U.S.
Department of Labor, U.S. Department of Labor (Washington, D.C.:
1992).
\14 Overview of Entitlement Programs: 1994 Green Book, Committee on
Ways and Means, U.S. House of Representatives (Washington, D.C.:
1994).
WIDE RANGE OF ADULT EDUCATION
PROVIDERS USE A VARIETY OF
APPROACHES
---------------------------------------------------------- Chapter 2:4
Under the State Grant Program, states may fund local educational
agencies\15 and a variety of public or private nonprofit agencies,\16
organizations, and institutions to provide adult education classes.
Most programs are administered by local educational agencies. Figure
2.1 shows the extent to which different organizations provide adult
education.
Figure 2.1: Adult Education
Providers, Fiscal Year 1992
(See figure in printed
edition.)
Note: Other entities include public/private nonprofit organizations
and correctional institutions.
Source: U.S. Department of Education.
Many adult education providers use flexible and, in some cases, less
traditional approaches to education that may better suit the
responsibilities and needs of adult students. For example, to make
classes more accessible to adults, providers may offer both day and
night classes. Unemployed adults may prefer daytime classes; adults
who work or have child care responsibilities may only be able to
attend night classes. We also found that some programs offer on-site
child care, which can make it easier for parents to attend adult
education classes.
The flexible "open-entry/open-exit" feature of most adult education
providers may also better suit their students' lives than the
traditional September-to-June school year. The National Evaluation
of Adult Education Programs found that 66 percent of adult education
programs allowed students to enroll and begin instruction at any
time.\17
Service providers use a variety of instructional methods to meet
students' needs. For example, the principal of an adult school in a
rural California town explained that her program provides
"individualized instruction," which means that teachers assess the
individual goals and abilities of the students and take these into
consideration in planning classroom instruction. Several methods or
a combination of methods may then be employed: large group lectures
or presentations; small-group instruction, including role play or
practice in conversation or writing skills; or one-on-one tutoring if
the ratio of aides to students permits it.
Some programs encourage adults who need both basic skills and
employment training to enroll in both concurrently; others recommend
basic skills training first so that students have the necessary
foundation for employment training. Concurrent enrollment, some
state and local officials argue, may enhance learning and move adults
into the workforce faster.
--------------------
\15 In most cases, the local education agency is a school district.
\16 For-profit agencies may be included under certain circumstances.
\17 M.B. Young, M. Morgan, N. Fitzgerald, and H. Fleischman,
National Evaluation of Adult Education Programs: Draft Final Report,
Development Associates, Inc. (Arlington, Va.: 1994).
WITHIN CONFINES OF AEA, STATES
HAVE FLEXIBILITY IN FUNDING
INSTRUCTION
---------------------------------------------------------- Chapter 2:5
The AEA limits states' flexibility in determining how to spend their
State Grant Program funds by specifying how a significant portion of
the funds are to be spent. However, with the remaining unrestricted
funds, the combinations and types of instruction states fund vary
greatly.
The AEA specifies that states must spend at least 15 percent of their
grants on teacher training and program innovation and at least 10
percent on programs serving incarcerated or institutionalized adults.
No more than 20 percent can be spent on programs for certificates of
high school equivalency, and no more than 5 percent can be spent on
state administration.
States can decide on the types and combinations of instruction they
wish to fund as long as they meet the AEA's set-asides and fund
programs in accordance with their state plans. Most providers offer
the three most common types of instruction--ABE, ASE, and ESL.
Enrollment in each varies greatly by state and community. Table 2.1
shows how enrollment levels vary by instructional area nationally as
well as in the three states we visited.
Table 2.1
State Grant Program Enrollment, July 1,
1992, to June 30, 1993
ABE ASE ESL
(perce (perce (perce
State nt) nt) nt)
---------------------------------------------- ------ ------ ------
U.S. (all states) 34 26 40
California 10 6 84
Connecticut 21 42 37
Iowa 62 27 11
----------------------------------------------------------------------
Source: U.S. Department of Education.
STATES AND COMMUNITIES
COORDINATE WITH EMPLOYMENT
TRAINING PROGRAMS IN A VARIETY
OF WAYS
---------------------------------------------------------- Chapter 2:6
In keeping with state plans, which call for coordinating adult
education with employment training programs, a variety of
coordination activities were taking place in the states and
communities we visited. These activities, however, were not easy to
establish. They took time to develop and often depended on the
perseverance of agency staff and local service providers. State and
local coordination efforts included pooling funds, establishing
one-stop centers, and developing uniform assessment systems.
POOLING FUNDS
-------------------------------------------------------- Chapter 2:6.1
Connecticut has pooled funds from many\19 sources for Coordinated
Education and Training Opportunities grants. When a service provider
receives a grant, it may contain funds from one or more funding
sources. These coordinated grants are implemented through regional
workforce development boards responsible for a range of tasks,
including identifying local needs, evaluating grant proposals, and
overseeing operations. These grants have the advantage of allowing
service providers to deal with a single planning process and a single
request for proposal. However, according to officials of one
regional workforce development board, although these grants may make
things "seamless" for the client, the service provider still must
meet all the federal reporting requirements of their many funding
sources.
In Iowa, the community colleges coordinate funds from the State
Grant, JTPA, and VOC ED programs. Each of the state's 15 community
colleges administers the State Grant Program and offers adult
education classes. In addition, half of the colleges administer JTPA
programs. Services from these many programs are often administered
by staff who are both located at the college and operate within the
same department. This arrangement facilitates coordinated program
planning, service delivery, and referral of clients to multiple
programs. At one college, administrators from the adult education,
JTPA, and VOC ED programs told us that their close proximity enabled
them to review a client's total needs and provide the maximum
allowable services. For example, a welfare recipient might receive
adult education instruction from the State Grant Program, a clothing
allowance from the JTPA program, and a transportation subsidy from
the JOBS program.
Some of the coordination between California's JOBS program and State
Grant Program takes the form of financial support from many agencies.
Adult education programs that serve the state's JOBS clients receive
adult education and JOBS funds and may also draw funds from an
8-percent set-aside of JTPA funds for education programs that are
matched by the state. In some counties, the state's JOBS program
pays for adult education programs to meet JOBS' data collection and
reporting requirements, which include administering the same
competency-based assessment to all state JOBS clients. Adult
education does not cover these costs. One adult education principal
told us that, if not for the JOBS program's covering these costs,
program staff would not be able to do as much record keeping or
assessment as they do.
--------------------
\19 These sources are the State Grant Program; JTPA's title IIA and
IIC State Education programs; Perkins Single Parents, Displaced
Homemakers, and Single Pregnant Women programs; and a state match of
federal JOBS funds.
ESTABLISHING ONE-STOP
CENTERS
-------------------------------------------------------- Chapter 2:6.2
All three states we visited had begun efforts to establish one-stop
centers. These centers are intended to help clients who need
services from many programs find all of the services they need in one
location or go to a single location to access information about the
services they need. Officials in one community spoke of the
administrative burden imposed by the multiple federal program
requirements of establishing a one-stop center.
All three states recently received grants from the Department of
Labor to pilot one-stop centers, which are being established around
the country even in places that have not received Department of Labor
grants. The one-stop center we visited in California was established
without a grant from Labor. The Department of Labor's one-stop
grants support voluntary state coordination. All three states
considered adult education important and, thus, included adult
education officials in planning their efforts.
With the help of the Department of Labor's one-stop grant, Iowa
recently opened its first one-stop center to provide services to
clients of employment training programs. Adult education staff were
on site to perform client intake and assessment. Adult education
instruction has been offered on site since February 1995.
Connecticut was using its Department of Labor grant to develop
one-stop centers, where client intake and evaluation would take place
and where clients could be referred to multiple agencies for services
they need. Local officials in one community said their goal was to
develop three centers and install computers in libraries, bus
terminals, and shopping malls so the public could access information
on local services, such as adult education classes.
One community we visited in California set up a one-stop center
without a Department of Labor grant. To prevent unnecessary
duplication of services and facilitate successful completion of
training and the transition to employment, this center established
linkages to more than 100 agencies and businesses. Features of the
center included a central information line, career library,
computerized career assessment, and on-site employment interviews.
DEVELOPING UNIFORM
ASSESSMENT SYSTEMS
-------------------------------------------------------- Chapter 2:6.3
A single assessment system used across state agencies can facilitate
coordination and make access to services easier for clients. Using
one system allows clients to move easily among education and training
programs, provides a common assessment vocabulary so that all
agencies can determine initial client proficiency levels as well as
ongoing progress, and minimizes duplicative or unnecessary testing of
clients. However, not all adult education and employment training
officials agree that a single assessment system can appropriately
measure adults' skills.
To varying degrees, Connecticut and California were using common
assessments. Connecticut required that adult education, JTPA, and
JOBS programs all use the same assessment system. California's JOBS
program uses the Comprehensive Adult Student Assessment System
(CASAS) for assessing its clients, but the state's adult education
program uses CASAS only on a sample of programs. Iowa was piloting
CASAS but only for use by adult education providers.
MEASURING PROGRAM RESULTS HAS
PROVEN DIFFICULT
============================================================ Chapter 3
Measuring results in the State Grant Program has proven difficult
because program objectives have not been clearly defined and
questions exist about the validity and appropriateness of student
assessments and the usefulness of nationally reported data on
results. Although the Department of Education has focused on
developing model program indicators that states could use to evaluate
local programs, experts and program officials disagree about whether
the indicators alone will enhance accountability. Efforts to enhance
the evaluation capabilities of state agency staff and improve data
collection continue, but it is too early to assess their impact.
DIFFICULTIES IN ESTABLISHING
PROGRAM OBJECTIVES
---------------------------------------------------------- Chapter 3:1
Evaluating program results depends on clear program objectives as
well as criteria for measuring the achievement of those objectives.
The broad objectives of the State Grant Program give the states the
flexibility to set their own priorities but, some argue, they do not
provide states with sufficient direction for measuring results.
Moreover, reaching a consensus on measurable objectives for adult
education is difficult.
Because the State Grant Program's objectives are so broadly defined,
state officials have developed a variety of views on measuring
program results. For example, some officials told us that they might
measure program success by whether adults gained the skill to read to
their children and, thus, contribute to their children's literacy.
Others might focus on whether adults can read street signs or the
newspaper. And, in one state we visited, an official contended that
completing high school and finding productive work should be the
objectives of the states' adult education programs because completing
a basic skills program and becoming a citizen are no longer
sufficient to succeed in society.
Several experts and program officials told us that the State Grant
Program lacks a coherent vision of the skills and knowledge adults
need to be considered literate. Similarly, some state officials said
that they would like the federal government to further specify the
types of results expected from state adult education programs.
Reaching consensus on measurable objectives, however, may be
difficult since research findings are often inconclusive about the
long-term benefits to adults of achieving various program results.
For example, many adult education programs focus on preparing adults
to take the GED examination as a means of high school completion.
Yet research findings are mixed about whether GED attainment reflects
increased literacy skills and whether GED recipients are economically
better off than high school dropouts.\19
--------------------
\19 See, for example, J. Baldwin, NALS, SALS, and GED: Related
Studies and Their Implications, presentation at the Annual
Conference, State Directors of Adult Education (Louisville, Ky.:
July 1994); H. Beder, What Has Happened to Iowa's GED Graduates?
Iowa State Department of Education (Des Moines, Ia.: 1992); S.
Cameron and J. Heckman, The Nonequivalence of High School
Equivalents, National Bureau of Economic Research (Cambridge, Mass.:
1991); D. Kaplan and R. Venezky, What Can Employers Assume About
the Literacy Skills of GED Graduates? National Center on Adult
Literacy (Philadelphia: 1993).
CONCERNS RAISED ABOUT ADULT
STUDENT ASSESSMENTS
---------------------------------------------------------- Chapter 3:2
Ensuring accountability has also been hampered by limitations in the
assessment instruments used to measure student outcomes in adult
education programs. The research literature raises questions about
the validity of standardized tests used to measure adult literacy,
and local program staff have questioned the appropriateness of using
these assessments to measure program results.
The AEA requires states to gather and analyze standardized test data
as one way of evaluating local programs.\20 These assessments tend to
focus on either academic skills or functional literacy. Academic
tests, such as the Tests of Adult Basic Education (known as "TABE"),
focus on measuring such basic skills as reading comprehension,
vocabulary, language expression, and mathematical proficiency.
Functional literacy or competency-based tests, such as the
Comprehensive Adult Student Assessment System (CASAS), focus on the
ability to perform literacy-related tasks in situations faced by
adults in everyday life at home, at work, or in the community.
Experts have questioned the validity of both the academic and
functional literacy tests used in adult education programs. For
example, two recent reviews point to a lack of normative data for the
age ranges of participants in most adult education programs.\21
Functional literacy tests may lack validity because they are not
derived from theoretical models of ability but from everyday literacy
tasks. According to a recent review, without further analyses, the
instructional implications of test performance are unclear.\22 Thus,
these assessments may not provide useful information about the skills
and needs of adult students.
A more serious problem affecting the validity of assessments is the
lack of research examining the long-term retention of learning gains
in adult education programs. According to one researcher, a
comprehensive search did not uncover a single published study on the
effectiveness of adult education programs in helping adults retain
the skills they may have acquired during instruction.\23 This being
the case, improved test scores may not necessarily mean that adults
will be better equipped for high- skilled jobs, function better as
parents, or participate more fully as citizens.
However, officials in the three states we visited felt that
competency-based assessment systems could be useful in measuring
progress in local adult education programs and, thus, strongly
advocated these systems. California had developed CASAS and required
its use in a sample of one-third of its adult education programs.
Connecticut had designed its own competency-based testing system
(adapted from CASAS) and required its use in all adult education
programs. Iowa had recently decided to move toward a
competency-based system and was piloting CASAS in 9 of its 15
community college districts.
Local adult education and employment training staff had mixed views
about their states' competency-based assessments. Some local program
staff saw the CASAS assessment system as a valuable and flexible
tool. However, some English as a Second Language (ESL) teachers were
dissatisfied with the CASAS test as a measure of how well adult
education students learned to communicate in English. And some
employment training staff said that the CASAS test did not give them
sufficiently specific information about their clients or focused too
much on life skills.
Finally, several local staff questioned the appropriateness of CASAS
as the sole assessment tool and, therefore, used CASAS in conjunction
with other tests. Administrators and experts also told us that they
thought no single test could measure all relevant aspects of student
performance.
--------------------
\20 The act does not specify which tests must be used or how states
must gather test data from local programs.
\21 D. Wagner and R. Venezky, "Adult Literacy: The Next
Generation," NCAL Connections, National Center on Adult Literacy
(Philadelphia: 1995); W. Merz and J. Kruckenberg, Quality
Standards and Performance Measures: Preliminary Feasibility Study on
the Uses of Basic Skills Assessment Tests, Adult Education Institute
for Research and Planning (Sacramento, Cal.: Dec. 1991).
\22 R. Venezky, Matching Literacy Testing With Social Policy: What
Are the Alternatives? National Center on Adult Literacy
(Philadelphia: 1992).
\23 D. Wagner, Use It or Lose It? The Problem of Adult Literacy
Skill Retention, National Center on Adult Literacy (Philadelphia:
1994).
ADULT STUDENT DATA MISSING OR
INACCURATE
---------------------------------------------------------- Chapter 3:3
The poor quality of the data on adult education students collected at
state and local levels also hampers accountability. Federal and
state officials as well as recent studies have cited problems with
these data. The studies have attributed difficulties in obtaining
accurate data to the sporadic attendance patterns of adult students
and the limited time and expertise of local adult education program
staff.
State officials are required to submit to the Department of Education
annual statistical performance reports that include information on
students served by local programs. State-submitted reports include
(1) the number of students served and their demographic
characteristics, (2) the skill levels of students when they start
adult education programs, (3) student progress over the program year,
(4) eight types of student achievements,\24 and (5) the number of
students who do not complete their objectives and their reasons for
separation. The reports also include information on program staff
and the types of instructional settings in which students are served.
Department of Education officials acknowledged serious problems with
the quality of the statistical reports, some of which are based on
double counting or undercounting of students in adult education
programs. Another Department official charged that many of the data
are questionable and that very few local programs have record systems
that allow them to report the data the Department requires.
Comments of officials in one state confirmed these data problems.
They said that they did not have all the information the Department
requires for their statistical reports because too many resources are
required to collect the data. As a result, they simply do not report
some of the data elements and provide estimates of the other
information. They noted that the data they submit need not be
certified and that the Department has never audited their statistical
reports. Furthermore, they asserted that these data have nothing to
do with receiving federal funds. The only thing that really counts,
they said, is the number of adults in the state who do not have
diplomas because that is what drives the funding formula.
Also, some local staff failed to see the utility in collecting the
data that states require for reporting to the federal government.
Some said they thought that the information they are required to
report does not accurately reflect the accomplishments of their adult
education students.
Difficulties in obtaining accurate data can also be attributed to
attendance patterns of adult students and the limited capacity and
expertise of local program staff. The open-entry/open-exit feature
of many programs adds to the difficulty of tracking adult students.
Because students may not stay in the program long or may attend on a
sporadic basis, program staff do not always have sufficient
information to report on student progress or results. Because local
programs have difficulty following up on students, program officials
may rely on information reported by teachers or the students
themselves.
In addition, many local programs lack sufficient staff to handle data
collection and reporting responsibilities, according to a survey of
adult education programs in nine states.\25 Programs are typically
staffed by part-time personnel, and these responsibilities become an
extra burden. Also, local program staff may lack expertise in
collecting assessment data that can help track program effectiveness.
For example, when the National Evaluation of Adult Education Programs
asked local adult education program staff to provide certain
assessment data, it found that about one-third of the information was
invalid because (1) the wrong test forms were used, (2) data were
inaccurately recorded, or (3) tests were administered at the wrong
times.\26
Similarly, as Connecticut began to implement a new assessment system
statewide, administrators discovered that they needed to clarify
program guidance because some local programs were mistakenly
measuring literacy gains using a test designed solely for student
placement.
--------------------
\24 The categories in the federal reporting forms are (1) obtained an
adult high school diploma, (2) passed the GED test, (3) entered other
education or training program, (4) received U.S. citizenship, (5)
registered to vote or voted for the first time, (6) gained
employment, (7) secured employment retention or obtained job
advancement, and (8) removed from public assistance.
\25 L. Condelli, Implementing the Enhanced Evaluation Model:
Lessons Learned from the Field Test, Pelavin Associates, Inc.
(Washington, D.C.: 1994).
\26 M.B. Young, N. Fitzgerald, and M. Morgan, National Evaluation
of Adult Education Programs: Executive Summary, Development
Associates, Inc. (Arlington, Va.: 1994). One component of the
evaluation was a longitudinal study of a nationally representative
sample of local programs that provided information on the
characteristics of a national sample of 22,000 adults who enrolled in
local programs between April 1991 and April 1992.
EFFORTS TO IMPROVE PROGRAM
QUALITY AND ACCOUNTABILITY
---------------------------------------------------------- Chapter 3:4
Federal efforts to improve quality and accountability have focused on
(1) developing model indicators; (2) providing technical assistance
to states and local programs on data collection, assessment, and
developing performance standards and measures; and (3) requiring
states to set aside funds for training and demonstration projects.
FEDERAL AND STATE PROGRAM
QUALITY INDICATORS
-------------------------------------------------------- Chapter 3:4.1
Provisions of the National Literacy Act focus on improving quality in
adult education programs by requiring the Secretary of Education to
develop indicators of program quality. The indicators were to be
used as models for judging state and local programs receiving federal
funding. States were also required to develop and implement their
own indicators, which might or might not correspond to the federal
model, and use them to evaluate state and local programs.
The Department of Education developed model indicators by (1)
reviewing adult education indicators already being developed by
various states and indicators used by other federal programs, (2)
meeting with experts and adult educators, (3) commissioning
background papers by experts in the field, and (4) conducting
workshops for state directors who would be responsible for developing
and implementing the state indicators. The resulting eight model
indicators of program quality are listed in table 3.1. The
indicators cover student outcomes, that is, learner progress toward
attainment of basic skills and competencies and learner advancement
in the program. They also focus on recruiting and retaining adult
education students and other indicators of program quality--planning,
curriculum and instruction, staff development, and provision of
support services.
Table 3.1
Model Indicators of Program Quality for
Adult Education Programs
Topic Indicator
---------------------------------- ----------------------------------
Educational gains 1.Learners demonstrate progress
toward attainment of basic skills
and competencies that support
their educational needs.
2.Learners advance in the
instructional program or complete
program educational requirements
that allow them to continue their
education or training.
Program planning 3.Program has a planning process
that is ongoing and participatory,
guided by evaluation, and based on
a written plan that considers
community demographics, needs,
resources, and economic and
technological trends, and is
implemented to the fullest extent.
Curriculum and instruction 4.Program has curriculum and
instruction geared to individual
student learning styles and levels
of student needs.
Staff development 5.Program has an ongoing staff
development process that considers
the specific needs of its staff,
offers training in the skills
necessary to provide quality
instruction, and includes
opportunities for practice and
systematic follow-up.
Support services 6.Program identifies students'
needs for support services
available to students directly or
through referral to other
educational and service agencies
with which the program
coordinates.
Recruitment 7.Program successfully recruits
the population in the community
identified in the AEA as needing
literacy services.
Retention 8.Students remain in the program
long enough to meet their
educational needs.
----------------------------------------------------------------------
Source: Model Indicators of Program Quality for Adult Education
Programs, Office of Vocational and Adult Education, U.S. Department
of Education (Washington, D.C.: July 1992).
The Department did not attempt to set performance standards for adult
education programs but limited its work to developing indicators and
providing some sample measures for each indicator. The Department
defined an indicator as a variable that reflects effective and
efficient program performance. It is to be distinguished from a
specific measure used to determine the quantitative level of
performance for the indicator. For example, to measure learner
progress, states could use standardized test score gains, teacher
reports of gains in communication competencies, or alternative
assessment methods (such as portfolio assessments, student reports of
attainment, or improvements in specific employability or life
skills). An indicator is also to be distinguished from a performance
standard, which defines acceptable performance in terms of a specific
numeric criterion.
The National Literacy Act also required states to adopt indicators by
July 1993 and use them to evaluate local programs. States were
required to adopt, at a minimum, indicators for recruitment,
retention, and student learning outcomes. However, decisions about
whether to adopt the Department's model indicators, what measures to
use, and whether to develop performance standards were left to the
states. A review of amendments to state adult education plans
submitted in July 1993 showed that for the most part states had
adopted indicators similar to the Department's model, especially in
the areas of student outcomes, recruitment, and retention.\27
However, states were less consistent in how they measured indicators.
The review found that states were using different standardized tests
to measure learner progress and had defined learner advancement in
different ways. A 1995 survey of state adult education directors
showed that 16 states had implemented standards and 8 states had
developed but not yet implemented standards.\28
Each of the three states we visited had developed standards for
student outcomes, but not all of these standards were readily
quantifiable. California had developed standards for seven levels of
language proficiency for ESL students (the majority of adult
education students in the state) but had not yet quantified
performance on specific assessment measures. Standards for other
kinds of adult education students in California had not yet been
completed. Connecticut had set standards for educational gains
expected over a specific time period and measured their achievement
using test scores and the number of course credits or competencies
attained. Iowa had set standards for grade level increases on
standardized tests and for the performance of GED graduates on the
GED exam. Since Iowa had not yet determined a specific strategy for
competency-based education, the state had not yet established
standards for competency-based tests.
Experts as well as federal and state officials with whom we spoke
disagreed about whether developing indicators would improve
accountability and program quality. Some were concerned that the
indicators do not move the field forward because they do not specify
the types of results the federal government expects from state and
local programs. One federal official doubted whether the indicators
alone would help state and local programs collect higher quality
data. However, other experts told us that they thought the
indicators were a good first step. Still others said that the
federal government should not be setting standards for states because
states' literacy problems and clientele differ.
--------------------
\27 B.G. Elliott and B.J. Hayward, Adult Education Performance
Data: Improving Federal Management and Program Performance, Research
Triangle Institute (Research Triangle Park, N.C.: 1994).
\28 Status of Development of Measures and Standards for the Quality
Indicators for the Adult Education Program, U.S. Department of
Education (Washington, D.C.: May 1995).
TOO SOON TO ASSESS EFFORTS
TO ENHANCE ACCOUNTABILITY
AND PROGRAM QUALITY
-------------------------------------------------------- Chapter 3:4.2
It is too soon to tell whether state-developed indicators, measures,
and performance standards will result in the collection of more
useful data or help states evaluate local programs since the 1993-94
program year was the first year in which indicators were to be used
for evaluation. One state we visited planned to use information
collected during the 1993-94 program year as baseline data and begin
to hold local programs accountable for performance on the state's
indicators in subsequent years.
Other federal efforts have been initiated to help states develop
better accountability systems. Two of these efforts are designed to
help build the capacity and expertise of state adult education staff
to evaluate local programs. In 1993, the Department hired a
contractor for a 3-year technical assistance effort designed to
assist state education agencies with assessment, evaluation, and the
development of performance standards and measures. And, in 1993, the
National Institute for Literacy awarded grants to five states to
develop performance measurement systems for literacy, with a specific
focus on integrating systems used by different agencies that provide
literacy services.
Department officials also told us that they were acting to improve
the quality of data collected on adult education programs. In
concert with state adult education directors, the Department has been
examining whether to modify the existing federal reporting
requirements. They have held several meetings but have not yet
issued any recommendations. In addition, the Department has
developed and tested an automated management information system that
would allow programs to collect data on individual students and a
computer program that would help states more easily convert data they
collected to the statistical reports required by the Department. A
field test of the management information system in selected local
programs in five states revealed that local staff appreciated the
system's report-writing capabilities but remained highly resistant to
performing data collection and entry.
In addition to these efforts, the requirement that states set aside a
portion of their federal funds for demonstration projects and
training may also help states move toward better accountability
systems.\29 Although the Department has not completed an ongoing
national evaluation of the use of these funds, state officials
asserted that the set-aside was critical to their efforts to improve
program quality. All three states had used these federal funds, in
part, to develop competency-based instruction and assessment systems;
they had also used the funds to address state-specific issues.
California had developed a training institute for ESL teachers,
Connecticut had used some of the funds to help implement a new
statewide management information system, and Iowa had held a state
literacy conference to examine how to better measure adult student
progress through qualitative assessments.
--------------------
\29 The 1988 AEA amendments required that at least 10 percent of the
state grant be used for special demonstration projects and for
training adult education teachers and other staff. The National
Literacy Act raised the requirement to 15 percent and specified that
at least two-thirds of this set-aside (10 percent of the total grant)
be used for training.
CONCLUSIONS AND AGENCY COMMENTS
============================================================ Chapter 4
CONCLUSIONS
---------------------------------------------------------- Chapter 4:1
The broad goals and flexibility of the AEA and its State Grant
Program have resulted in a federal program that is serving many
different populations, yet has difficulty determining its target
populations, objectives, or a means to measure program results.
Although the broad goals and corresponding flexibility give state and
local officials the latitude to design programs and quality
indicators tailored to their particular needs and priorities, some
state officials and experts have voiced concerns that the federal
government has not provided sufficient vision and guidance. This
poses a challenge for developing accountability measures.
The program has had difficulty ensuring accountability for
results--that is, being able to clearly or accurately say what
program funds have accomplished. Although the Department of
Education relies on federal reporting requirements and program
quality indicators to provide this information, the data the
Department receives are of questionable value. Because state and
local client data are missing or inaccurate, attempts to make the
program accountable may be compromised. Until further guidance is
developed on measurable objectives and ensuring the quality of client
data, state-developed indicators and standards are unlikely to
improve accountability.
AGENCY COMMENTS
---------------------------------------------------------- Chapter 4:2
In its written comments on a draft of the report, the Department of
Education recognized that we identified the three areas that are
critically important to improving accountability in adult education:
clear purpose and expectations, good assessment instruments, and
high-quality data.
The Department also stated its commitment to improving program
accountability through several current initiatives. These
initiatives include developing an individualized student record
keeping system; moving toward an outcomes-based national data
collection system; conducting evaluations of delivery systems,
effective practice, assessment, and performance measurement;
providing technical assistance in designing and using performance
measures and standards; and developing training programs for adult
education staff in collecting, analyzing, and reporting student and
program data. (The Department's letter appears in app. II.)
STATE GRANT PROGRAM BUDGET AND
ENROLLMENT DATA
=========================================================== Appendix I
Table I.1
AEA Appropriations, Fiscal Year 1995
(Dollars in millions)
Approp
riatio
Programs ns
-------------------------------------------------------------- ------
Funded
----------------------------------------------------------------------
State Grant Program $252.3
National Demonstration Workplace Literacy Partnership Program 18.7
Literacy Training for Homeless Adults Program 9.5
State Literacy Resource Centers Program 7.8
Functional Literacy for State and Local Prisoners Program and 5.1
Life Skills for State and Local Prisoners Program
Adult Education, National Programs: Evaluation/Technical 3.9
Assistance
Adult Education, National Programs: National Institute for 4.9
Literacy
======================================================================
Total $302.2
Unfunded
----------------------------------------------------------------------
Adult Literacy Volunteer Training
Adult Migrant Farmworker/Immigrant Education
Education Programs for Commercial Drivers
English Literacy Grants
National Workforce Literacy Strategies Program
State-Administered Workplace Literacy Programs
----------------------------------------------------------------------
Source: U.S. Department of Education.
Table I.2
State Grant Program Enrollment and
Funding History, Fiscal Years 1966-1995
Federal,
Enrollment State and state, and
Year \a Federal\b local local
---- ---------- ------------ ------------ ------------
1966 377,600 $19,879,912 $9,919,000 $29,798,912
1967 388,900 26,280,000 8,334,000 34,614,000
1968 455,700 30,590,000 9,574,000 40,164,000
1969 484,600 36,000,000 11,686,000 47,686,000
1970 535,600 40,000,000 12,461,000 52,461,000
1971 620,900 44,866,102 15,322,000 60,188,102
1972 820,500 51,134,000 17,371,000 68,505,000
1973 822,500 74,834,000 20,127,000 94,961,000
1974 965,100 53,286,000 27,296,000 80,582,000
1975 1,221,200 67,500,000 43,230,000 110,730,000
1976 1,651,100 67,500,000 41,125,000 108,625,000
1977 1,686,300 71,500,000 41,992,000 113,492,000
1978 1,811,100 80,500,000 51,477,000 131,977,000
1979 1,806,300 90,750,000 63,064,000 153,814,000
1980 2,058,000 100,000,000 74,288,475 174,288,475
1981 2,261,300 100,000,000 104,212,622 204,212,622
1982 2,176,900 100,000,000 128,654,773 228,654,773
1983 2,576,300 86,400,000 172,691,213 259,091,213
1984 2,596,500 100,000,000 196,691,149 296,691,149
1985 2,879,100 101,963,000 240,410,289 342,373,289
1986 2,797,500 97,579,000 319,942,176 417,521,176
1987 2,945,600 105,981,000 400,383,790 506,364,790
1988 3,039,400 115,367,000 491,329,659 606,696,659
1989 3,257,000 136,344,000 566,656,213 703,000,213
1990 3,567,200 157,811,000 622,069,755 779,880,755
1991 3,722,600 201,032,000 809,540,972 1,010,572,97
2
1992 3,838,000 235,750,000 832,208,120\ 1,067,958,12
d 0
1993 3,880,400 254,624,000 855,509,950\ 1,110,133,95
d 0
1994 3,978,000\ 254,624,000 872,620,150\ 1,127,244,15
c d 0
1995 4,078,000\ 252,345,000 890,072,550\ 1,142,417,55
c d 0
----------------------------------------------------------
\a Enrollment is the number of participants who complete 12 or more
program hours, with each participant reported only once, regardless
of the number of classes or programs attended during the program year
reported.
\b This column reflects federal appropriations data because federal
expenditure data were not readily available. According to Department
of Education officials, appropriations and expenditure data for the
program tend to be virtually the same.
\c Projected.
\d Estimated.
Source: U.S. Department of Education.
Table I.3
State Grant Program Enrollment by
Instructional Area, Fiscal Years 1970-
1995
(Enrollment in thousands)
Fisc Adult Adult English as a
al Basic Secondary Second
year Education Education Language Total
---- ---------- ------------ ------------ ------------
1970 536 \b \b 536
1971 621 \b \b 621
1972 599 222 \c 821
1973 633 189 \c 822
1974 772 193 \c 965
1975 \c \c \c 1,221
1976 1,073 578 \c 1,651
1977 822 658 206\d 1,686
1978 831 583 397\d 1,811
1979 830 587 389\d 1,806
1980 938 543 577\d 2,058
1981 1,607 654 \c 2,261
1982 1,607 570 \c 2,177
1983 1,569 600 407\d 2,576
1984 1,939 657 \c 2,596
1985 1,295 733 851\e 2,879
1986 923 1,009 866 2,798
1987 1,006 1,028 912 2,946
1988 1,066 1,067 906 3,039
1989 1,139 997 1,122 3,258
1990 1,273 1,102 1,193 3,568
1991 1,358 1,182 1,183 3,723
1992 1,410 1,248 1,180 3,838
1993 1,321 1,005 1,554 3,880
1994 \c \c \c 3,978\f
1995 \c \c \c 4,078\f
----------------------------------------------------------
\a Enrollment is the number of participants who complete 12 or more
program hours, with each participant reported only once, regardless
of the number of classes or programs attended during the program year
reported.
\b All enrollments were in the Adult Basic Education category.
\c Data not available.
\d According to Department of Education officials, these data are
incomplete for fiscal years 1977-1984 and represent only a portion of
ESL enrollment. No federal requirement for states to report
enrollment data existed during this time.
\e Fiscal year 1985 was the first year ESL was reported as a separate
instructional category.
\f Projected.
Source: U.S. Department of Education.
Table I.4
State Grant Program, State Expenditure
and Enrollment Data, Fiscal Year 1991
(Dollars in thousands)
Enrollment
Federal, (program
State and state, and State match year 1991/
State Federal local\a local (percent)\b 92)
------------------- ------------ ------------ ------------ ------------ ------------
Alabama $4,107 $3,026 $7,133 42 49,510
Alaska 446 1,761 2,207 80 6,046
Arizona 2,138 3,040 5,178 59 33,805
Arkansas 2,589 12,337 14,926 83 38,135
California 16,631 282,767 299,398 95 1,023,899
Colorado 1,919 812 2,731 30 15,890
Connecticut 2,568 16,689 19,257 87 54,588
Delaware 704 239 943 25 3,126
District of 791 4,221 5,012 84 20,732
Columbia
Florida 8,463 58,079 66,542 87 436,870
Georgia 5,658 2,601 8,259 31 85,794
Hawaii 854 1,934 2,788 69 56,873
Idaho 858 234 1,092 21 9,611
Illinois 9,470 7,046 16,516 43 88,815
Indiana 4,638 23,726 28,364 84 51,134
Iowa 2,293 3,741 6,034 62 40,371
Kansas 1,836 500 2,336 21 12,936
Kentucky 4,122 2,537 6,659 38 34,255
Louisiana 4,201 6,918 11,119 62 45,857
Maine 1,112 4,999 6,111 82 17,339
Maryland 3,621 3,977 7,598 52 33,829
Massachusetts 4,260 9,622 13,882 69 21,085
Michigan 7,364 183,605 190,969 96 219,306
Minnesota 2,960 14,694 17,654 83 45,348
Mississippi 2,773 845 3,618 23 21,244
Missouri 4,533 2,535 7,068 36 38,742
Montana 759 616 1,375 45 6,333
Nebraska 1,279 345 1,624 21 7,314
Nevada 772 474 1,246 38 19,230
New Hampshire 886 717 1,603 45 6,949
New Jersey 5,790 20,701 26,491 78 61,364
New Mexico 1,221 1,577 2,798 56 30,514
New York 14,703 31,143 45,846 68 197,865
North Carolina 6,309 21,112 27,421 77 126,698
North Dakota 745 425 1,170 36 3,642
Ohio 8,817 6,987 15,804 44 120,529
Oklahoma 2,595 702 3,297 21 30,501
Oregon 1,935 11,756 13,691 86 38,409
Pennsylvania 9,673 9,283 18,956 49 50,797
Rhode Island 1,122 1,568 2,690 58 8,926
South Carolina 3,457 10,354 13,811 75 103,041
South Dakota 769 291 1,060 27 3,849
Tennessee 4,893 1,369 6,262 22 58,896
Texas 12,744 8,653 21,397 40 224,037
Utah 972 4,110 5,082 81 26,609
Vermont 607 2,495 3,102 80 5,977
Virginia 5,049 5,018 10,067 50 31,397
Washington 2,738 7,646 10,384 74 37,331
West Virginia 2,207 1,292 3,499 37 26,113
Wisconsin 3,705 7,089 10,794 66 80,455
Wyoming 516 308 824 37 4,203
-----------------------------------------------------------------------------------------
Note: According to the Department of Education, this is the most
recent year for which complete expenditure data are available.
\a Obligation authority was from July 1, 1991, to June/September 30,
1993.
\b State match reflects state and local expenditures as a percentage
of total expenditures.
Source: U.S. Department of Education.
Table I.5
State Grant Program Enrollment by
Instructional Area, Fiscal Year 1993
Adult English as a
Adult Basic Secondary Second
Education Education Language
State Enrollment (percent) (percent) (percent)
------------------- ------------- ------------- ------------- -------------
Alabama 52,132 72 24 4
Alaska 5,822 57 3 40
Arizona 44,828 31 26 43
Arkansas 45,045 58 39 3
California 1,126,731 10 6 84
Colorado 12,732 36 31 33
Connecticut 49,334 21 42 37
Delaware 3,323 62 12 26
District of 16,505 39 26 35
Columbia
Florida 425,852 33 40 27
Georgia 84,516 55 28 17
Hawaii 59,034 33 35 32
Idaho 9,566 62 23 15
Illinois 83,153 27 20 53
Indiana 51,884 54 39 7
Iowa 38,072 62 27 11
Kansas 14,910 63 20 17
Kentucky 33,485 65 26 9
Louisiana 39,485 51 44 5
Maine 18,468 39 54 7
Maryland 33,004 26 60 14
Massachusetts 22,718 39 26 35
Michigan 193,027 25 65 10
Minnesota 42,232 46 25 29
Mississippi 21,752 79 17 4
Missouri 38,845 72 16 12
Montana 6,453 60 37 3
Nebraska 7,178 67 10 24
Nevada 16,853 5 79 16
New Hampshire 7,144 41 41 18
New Jersey 62,132 28 32 40
New Mexico 30,273 42 36 22
New York 191,349 44 20 36
North Carolina 128,147 54 35 11
North Dakota 3,582 52 31 17
Ohio 116,627 73 16 11
Oklahoma 30,072 65 22 13
Oregon 39,365 29 39 32
Pennsylvania 52,963 53 25 22
Rhode Island 7,089 47 24 29
South Carolina 104,009 38 60 2
South Dakota 4,263 65 26 9
Tennessee 57,310 67 26 7
Texas 209,871 36 27 37
Utah 27,770 16 72 12
Vermont 5,095 90 5 5
Virginia 32,106 41 28 31
Washington 39,409 43 16 41
West Virginia 25,866 68 30 2
Wisconsin 75,542 63 26 11
Wyoming 3,742 46 39 15
-------------------------------------------------------------------------------
Source: U.S. Department of Education.
(See figure in printed edition.)Appendix II
COMMENTS FROM THE DEPARTMENT OF
EDUCATION
=========================================================== Appendix I
(See figure in printed edition.)
(See figure in printed edition.)
GAO CONTACTS AND STAFF
ACKNOWLEDGMENTS
========================================================= Appendix III
GAO CONTACTS
Clarence C. Crawford, Associate Director, (202) 512-7017
Lori A. Weiss, Evaluator-in-Charge, (213) 346-8066
STAFF ACKNOWLEDGMENTS
In addition to those named above, the following individuals made
important contributions to this report: Gary P. Galazin, Senior
Evaluator; Tom Jessor, Senior Evaluator; Nancy Kawahara, Senior
Evaluator; David K. Porter, Evaluator; Revae E. Steinman, Senior
Evaluator; and Yelena K. Thompson, Evaluator.