Adult Education: Measuring Program Results Has Been Challenging (Chapter
Report, 09/08/95, GAO/HEHS-95-153).

--------------------------- Indexing Terms -----------------------------

 REPORTNUM:  HEHS-95-153
     TITLE:  Adult Education: Measuring Program Results Has Been 
             Challenging
      DATE:  09/08/95
   SUBJECT:  Accountability
             Adult education
             Education program evaluation
             Secondary education
             Educational grants
             State-administered programs
             Disadvantaged persons
             Educational programs
IDENTIFIER:  JOBS Program
             Connecticut
             JTPA
             Iowa
             California JOBS Program
             Comprehensive Adult Student Assessment System
             California
             National Evaluation of Adult Education Programs
             Dept. of Education Adult Education Act State Program
             Job Training Consolidation Act of 1995
             Consolidated and Reformed Education, Employment, and 
             Rehabilitation Systems Act
             
**************************************************************************
* This file contains an ASCII representation of the text of a GAO        *
* report.  Delineations within the text indicating chapter titles,       *
* headings, and bullets are preserved.  Major divisions and subdivisions *
* of the text, such as Chapters, Sections, and Appendixes, are           *
* identified by double and single lines.  The numbers on the right end   *
* of these lines indicate the position of each of the subsections in the *
* document outline.  These numbers do NOT correspond with the page       *
* numbers of the printed product.                                        *
*                                                                        *
* No attempt has been made to display graphic images, although figure    *
* captions are reproduced. Tables are included, but may not resemble     *
* those in the printed version.                                          *
*                                                                        *
* A printed copy of this report may be obtained from the GAO Document    *
* Distribution Facility by calling (202) 512-6000, by faxing your        *
* request to (301) 258-4066, or by writing to P.O. Box 6015,             *
* Gaithersburg, MD 20884-6015. We are unable to accept electronic orders *
* for printed documents at this time.                                    *
**************************************************************************


Cover
================================================================ COVER


Report to Congressional Requesters

September 1995

ADULT EDUCATION - MEASURING
PROGRAM RESULTS HAS BEEN
CHALLENGING

GAO/HEHS-95-153

Adult Education Act

(104801)


Abbreviations
=============================================================== ABBREV

  ABE - Adult Basic Education
  AEA - Adult Education Act
  ASE - Adult Secondary Education
  CASAS - Comprehensive Adult Student Assessment System
  ESL - English as a Second Language
  GED - General Educational Development
  JOBS - Job Opportunities and Basic Skills
  JTPA - Job Training Partnership Act
  VOC ED - Perkins Vocational Education

Letter
=============================================================== LETTER


B-259608

September 8, 1995

The Honorable William F.  Goodling
Chairman
The Honorable William L.  Clay
Ranking Minority Member
Committee on Economic and
 Educational Opportunities
House of Representatives

The Honorable Randy (Duke) Cunningham
Chairman
The Honorable Dale E.  Kildee
Ranking Minority Member
Subcommittee on Early Childhood,
 Youth and Families
Committee on Economic and
 Educational Opportunities
House of Representatives

The Adult Education Act (AEA) represents the primary federal effort
to improve the literacy skills of educationally disadvantaged adults. 
Programs funded under the AEA are also important in providing the
basic skills needed by many clients of federal employment training
programs. 

This report responds to a request from the former House Committee on
Education and Labor and its Subcommittee on Elementary, Secondary,
and Vocational Education that we review several issues in
anticipation of the AEA's reauthorization.  In subsequent discussions
with your staff, we agreed to provide information on the AEA's
largest program (the State Grant Program), its coordination with
federal employment training programs, and the extent to which it
ensures accountability for results. 

We are sending copies of this report to the Secretaries of Education,
Health and Human Services, and Labor, as well as interested
congressional committees.  Copies will be made available to others
upon request.  Major contributors to this report are listed in
appendix III.  If you have any questions, please call me at (202)
512-7014. 

Linda G.  Morra
Director, Education and
 Employment Issues


EXECUTIVE SUMMARY
============================================================ Chapter 0


   PURPOSE
---------------------------------------------------------- Chapter 0:1

Nearly 90 million adults in the United States have deficient literacy
skills, according to a recent national survey.  These adults may not
be able to write a letter explaining an error on a credit card bill,
use a bus schedule to determine which bus to take, or calculate the
difference between the regular and sale price of an item.  Deficient
literacy skills, however, are not just an individual concern.  Adult
literacy problems also threaten the nation's economy, which depends
on increasingly high levels of workplace skills to remain competitive
in a global market. 

The Congress passed the Adult Education Act (AEA) to, among other
things, help states fund programs for adults to acquire the basic
skills needed for literate functioning, benefit from job training,
and continue their education through at least high school.  The AEA
was last reauthorized in 1991.  In anticipation of its next
reauthorization, the former House Committee on Education and Labor
and its Subcommittee on Elementary, Secondary, and Vocational
Education requested that GAO provide information on (1) the AEA's
largest program--the State Grant Program--and its coordination with
federal employment training programs and (2) the extent to which the
program ensures accountability for program quality and results.  GAO
relied on national data in conducting its review and performed more
detailed work in three states--California, Connecticut, and Iowa. 


   BACKGROUND
---------------------------------------------------------- Chapter 0:2

Programs funded under the AEA are administered by the Department of
Education.  For the State Grant Program, the Department makes grants
to states on the basis of the number of people in each state who are
at least 16 years old, not required to be in school, and lack a high
school degree.  Local adult education providers apply to the states
for funds.  The Department of Education reported that, in fiscal year
1995, about 4 million adults were enrolled in classes funded by the
State Grant Program; federal funding was $252 million, and state and
local sources provided an additional $890 million.  Although total
current state and local contributions far exceed federal
expenditures, federal dollars still total more than half the adult
education funds in almost half of the states. 

Programs funded under the AEA are important in providing the basic
literacy skills needed by clients of federal employment training
programs administered by the Departments of Education, Health and
Human Services, and Labor.  Thus, the AEA and employment training
legislation require coordination among these programs to avoid
duplication and enhance service delivery. 

The federal role in administering programs funded under the AEA may
soon change.  The 104th Congress is considering legislation that
would consolidate adult education and other programs and provide one
or more block grants to states.  The Senate bill would repeal most
existing federal employment training programs, as well as the State
Grant Program, and replace them with a single block grant.  The House
bill would also repeal most employment training programs but would
replace them with four block grants, including a separate grant for
adult education and literacy programs. 


   RESULTS IN BRIEF
---------------------------------------------------------- Chapter 0:3

The goals of the AEA, which encompasses the State Grant Program, are
broad to enable people with diverse needs to receive varying types of
instruction.  The most common types of instruction funded under the
State Grant Program are basic education (for adults functioning below
the eighth grade level), secondary education, and English as a Second
Language.  Because many clients of federal employment training
programs need instruction provided by the State Grant Program,
coordination among these programs is essential. 

Although the State Grant Program funds programs that address the
educational needs of millions of adults, it has had difficulty
ensuring accountability for results because of a lack of clearly
defined program objectives, questionable validity of adult student
assessments, and poor student data.  Amendments to the AEA required
the Department of Education to improve accountability by developing
model indicators of program quality that states could adopt and use
to evaluate local programs.  However, experts disagree about whether
developing indicators would help states to define measurable program
objectives or evaluate local programs and collect more accurate data. 
Other federal efforts may help states achieve better accountability
systems, but it is too soon to evaluate their effectiveness. 


   PRINCIPAL FINDINGS
---------------------------------------------------------- Chapter 0:4


      LARGEST PROGRAM STRIVES TO
      MEET MANY GOALS AND NEEDS
-------------------------------------------------------- Chapter 0:4.1

The AEA's and the State Grant Program's goals include assisting
adults who lack the basic literacy skills needed for effective
citizenship, productive employment, and high school completion.  Many
clients of federal employment training programs need the basic skills
taught in adult education classes.  For example, almost 30 percent of
the participants in the Department of Labor's Job Training
Partnership Act Program are school dropouts; as many as 50 percent
may lack basic skills. 

In accordance with their federally approved plans, states fund local
organizations to provide varying services to a wide range of adults. 
For example, in one community we visited, a 19-year-old mother with a
tenth grade education was taking basic skills classes so that she
could complete high school and become a cosmetologist, a 28-year-old
stock clerk was taking high school equivalency classes with the hope
of going to college and becoming a police officer, and a 62-year-old
immigrant who had been an accountant in Russia was enrolled in
English classes so that she could become a U.S.  citizen. 

In further keeping with state plans, which call for coordinating
adult education with employment training programs, a variety of
coordination activities were taking place in the states and
communities we visited.  These activities included pooling funds,
establishing one-stop centers, and developing uniform assessment
systems.  For example, one state pooled funds from many sources for
coordinated grants that, among having other advantages, enabled
service providers to respond to a single request for proposal. 


      ENSURING PROGRAM
      ACCOUNTABILITY HAS PROVEN
      DIFFICULT
-------------------------------------------------------- Chapter 0:4.2

Evaluating program results depends on having clearly defined
objectives, valid assessment instruments, and accurate program data. 
Some program officials and experts have raised concerns that, because
the State Grant Program lacks clearly defined objectives, the types
of skills and knowledge adults need to be considered literate are not
clear and, thus, states do not have sufficient direction for
measuring results.  In addition, some research has questioned the
validity and appropriateness of the student assessments used in adult
education programs and, therefore, the usefulness of the data
generated from these assessments.  In the states we visited, local
program officials had mixed views of the assessment instruments the
states required them to use.  Finally, missing and inaccurate data
may compromise any attempt to improve program accountability. 
Federal and state officials acknowledged serious problems with the
data that states report to the Department of Education.  Recent
studies have attributed difficulties in obtaining accurate data to
the sporadic attendance patterns of adult students and the limited
time and expertise of local adult education program staff. 

Federal efforts to improve accountability have focused on developing
indicators of program quality, providing technical assistance to
states, and requiring states to set aside funds for demonstration
projects and training.  The Department of Education has developed
model indicators in eight areas, including student outcomes, and
provided examples of measures that could be used to quantify
performance in these areas.  And, as required, states have adopted
the indicators or developed their own to use to evaluate local
programs.  Experts and some program officials, however, have had
mixed views about whether the indicators would help states evaluate
local programs or collect better quality data.  Because the 1993-94
program year was the first year the indicators were required to be
used for evaluation, it is too soon to tell whether the indicators
will lead to improvements.  It is also too early to assess whether
the Department's technical assistance efforts or the state set-asides
for demonstration projects and training will help states develop
better accountability systems. 


   RECOMMENDATIONS
---------------------------------------------------------- Chapter 0:5

GAO is making no recommendations in this report. 


   AGENCY COMMENTS
---------------------------------------------------------- Chapter 0:6

The Department of Education provided written comments on a draft of
this report.  The Department recognized that GAO identified the three
areas that are critically important to improving accountability in
adult education:  clear purpose and expectations, good assessment
instruments, and high-quality data.  The Department also stated its
commitment to improving program accountability through several
current initiatives, such as developing an individualized student
record keeping system and training programs for adult education staff
in collecting, analyzing, and reporting student program data.  (See
app.  II for the Department's letter.)


INTRODUCTION
============================================================ Chapter 1

The National Adult Literacy Survey\1 estimated that approximately 90
million American adults have deficient literacy skills.  Of those,
between 40 and 44 million adults--about 22 percent of the country's
adult population--have severe problems with literacy, defined as the
ability to read, write, and speak English and compute and solve
problems proficiently.  An additional 50 million adults are likely to
encounter some problems functioning in society and need improved
literacy skills. 

The Adult Education Act\2 (AEA) is administered by the Department of
Education.  The act represents the primary federal effort to
alleviate problems in adult literacy and provides the basic
legislative authority and largest source of federal funds for
programs that benefit educationally disadvantaged adults.\3 The act's
largest program is the Adult Education State-Administered Basic Grant
Program (State Grant Program).  In fiscal year 1995, federal funding
for this program was $252 million, while state and local sources
provided $890 million, or 78 percent of the program's total budget. 
For the first 15 years of the State Grant Program (1966 to 1980),
federal expenditures exceeded total state and local contributions;
however, total state and local contributions have since surpassed
federal expenditures.  (See table I.2 in app.  I for total annual
expenditures.) Figure 1.1 compares federal expenditures with state
and local expenditures since the AEA's passage in 1966. 

   Figure 1.1:  Comparison of
   Federal Expenditures With State
   and Local Expenditures, Fiscal
   Years 1966-1995

   (See figure in printed
   edition.)

Note:  This figure reflects federal appropriations data because
federal expenditure data were not readily available.  According to
Department of Education officials, appropriations and expenditure
data for the program tend to be virtually the same.  For fiscal years
1992 to 1995, state and local expenditures were estimated. 

Source:  U.S.  Department of Education. 

Although total state and local contributions currently far exceed
federal expenditures, federal dollars still total more than half the
funds for adult education in almost half of the states.  The
contribution of each state relative to the federal contribution
varies widely, depending on each state's commitment to providing
adult education services.  For example, in fiscal year 1991,\4

state and local contributions ranged from a low of 21 percent to a
high of 96 percent; conversely, federal expenditures ranged from 4 to
79 percent.  Since 1992, the AEA has restricted the federal share of
each state's expenditure to no more than 75 percent.  (See table I.4
in app.  I for further information on expenditures by state.)

The AEA makes grants to states and requires that they be used in
accordance with federally approved state plans.  In developing their
plans, states must assess the needs of adults, including
educationally disadvantaged adults, and the capability of programs
and institutions to meet those needs.  The Department of Education
annually makes its grants to states\5 on the basis of the number of
individuals in each state who are at least 16 years old, not enrolled
in school, and lack a high school degree or General Educational
Development (GED) credential.\6 Local adult education providers then
apply to the states for funds. 

Following are the three most common types of instruction offered
under the State Grant Program: 

  Adult Basic Education (ABE), which is instruction designed for
     adults functioning below the eighth grade level;

  Adult Secondary Education (ASE), which is instruction designed for
     adults functioning at the secondary level that may culminate in
     a high school diploma or may serve as preparation for the GED\7

examination; and

  English as a Second Language (ESL), which is instruction designed
     to teach English to non-English speakers. 

(See table I.3 in app.  I for further information on enrollment by
instructional area.)

Programs funded under the AEA are important in providing basic
literacy skills needed by clients of federal employment training
programs such as Perkins Vocational Education (VOC ED), administered
by the Department of Education; Job Opportunities and Basic Skills
(JOBS), administered by the Department of Health and Human Services;
and the Job Training Partnership Act (JTPA), administered by the
Department of Labor.  Consequently, the AEA and employment training
legislation require coordination among these programs to avoid
duplication and enhance service delivery. 

The National Literacy Act of 1991\8 amended the AEA and authorized
several new programs.  Major provisions included the creation of the
National Institute for Literacy, the establishment of state and
regional literacy resource centers, and a requirement for the
Department of Education to develop model indicators of program
quality to guide states in developing their own indicators for
improved program evaluation. 

The 104th Congress is considering legislation that would consolidate
adult education and other programs and provide one or more block
grants to states.  The Senate bill\9 would repeal most existing
federal employment training programs, including the State Grant
Program, and replace them with a single block grant.  The House
bill\10 would also repeal most employment training programs but
replace them with four block grants, including a separate grant for
adult education and literacy programs. 


--------------------
\1 This 1992 survey profiled the literacy of U.S.  adults on the
basis of their performance in a wide array of tasks that reflect the
types of materials and demands they encounter in their daily lives. 
The Department of Education contracted with the Educational Testing
Service to conduct this survey. 

\2 The AEA was originally passed as part of the Elementary and
Secondary Education Amendments of 1966 (P.L.  89-750) and was
rewritten as part of the Augustus F.  Hawkins-Robert T.  Stafford
Elementary and Secondary School Improvement Amendments of 1988 (P.L. 
100-297). 

\3 These are adults who demonstrate basic skills equivalent to or
below that of students at the fifth grade level. 

\4 According to the Department of Education, this is the most recent
year for which expenditure data are available (obligation authority
was from July 1, 1991, to June/Sept.  30, 1993). 

\5 Each state receives $250,000 before the formula is applied. 

\6 More than 44 million people were in this category, according to
the 1990 Census. 

\7 The GED is equivalent to a high school diploma and is awarded upon
successful completion of a battery of nationally normed and scored
tests. 

\8 Public Law 102-73. 

\9 The Job Training Consolidation Act of 1995 (S.  143, 104th Cong.)
was introduced on January 4, 1995. 

\10 The Consolidated and Reformed Education, Employment, and
Rehabilitation Systems Act or CAREERS Act (H.R.  1617, 104th Cong.)
was introduced on May 11, 1995. 


   OBJECTIVES, SCOPE, AND
   METHODOLOGY
---------------------------------------------------------- Chapter 1:1

At the request of the Chairman and Ranking Minority Member of the
former House Committee on Education and Labor and the Chairman and
Ranking Minority Member of the former Subcommittee on Elementary,
Secondary, and Vocational Education, we reviewed several issues
related to the AEA.  Specifically, we examined

  the goals of the AEA and its largest program (the State Grant
     Program), the population served by the program, program
     services, and its coordination with federal employment training
     programs and

  the extent to which the State Grant Program ensures accountability
     for program quality and results, including how states have
     implemented quality indicators. 

We focused our review primarily on the State Grant Program because it
is the largest of the AEA's funded programs.  In fiscal year 1995, 83
percent of AEA funds were allocated to this program. 

To obtain nationwide information on the State Grant Program, we
interviewed federal officials from the Department of Education.  We
also reviewed Department of Education data and recent national
studies, including the National Evaluation of Adult Education
Programs and the National Adult Literacy Survey. 

We selected three states for closer review:  California, Connecticut,
and Iowa.  We selected these states because they provided some
geographic dispersion and represented a range of (1) state and local
financial commitments (as demonstrated by the percentage of matching
funds each state contributes), (2) program size (as demonstrated by
dollars and enrollment), and (3) ESL enrollment levels.  Within each
state, we visited at least two communities that we selected with the
help of state adult education officials.  We selected communities
that represented different types of locales (urban, suburban, rural)
and were involved in a variety of local coordination activities. 

To identify the goals of the AEA, including the State Grant Program,
we reviewed federal legislation.  To determine the populations served
and services provided by the program, we reviewed Department of
Education data and national studies.  We also interviewed local adult
education providers. 

To provide information on the coordination of AEA programs with
employment training programs, we interviewed federal officials at the
Departments of Education, Labor, and Health and Human Services, and
held discussions with national experts, including representatives of
the National Institute for Literacy.  We also reviewed studies on
coordination.  In the states we visited, we met with state officials
from adult education and JOBS, JTPA, and VOC ED programs.  In Iowa
and parts of California, where the community college system is the
major adult education provider, we also met with state community
college representatives.  At local levels in the three states, we met
with adult education providers as well as representatives of local
employment training programs. 

To provide information on program accountability and quality, we
interviewed Department of Education officials and held discussions
with national adult education experts.  We also reviewed the
Department's model indicators of program quality and studies on
program accountability and quality issues.  In addition, we
interviewed state and local officials in California, Connecticut, and
Iowa and reviewed program documents, including the quality indicators
developed by these states. 

We conducted our work between November 1994 and August 1995 in
accordance with generally accepted government auditing standards. 


LARGEST AEA PROGRAM STRIVES TO
MEET MANY GOALS AND NEEDS
============================================================ Chapter 2

The AEA is a broad and flexible act and its largest program, the
State Grant Program, reflects this.  The State Grant Program, the
federal government's primary adult education program, has many goals
and enables people with a wide range of needs to receive instruction
from a variety of service providers.  Many clients of employment
training programs are among those needing the basic skills taught by
adult education.  Although the program has some restrictions, it
allows states considerable flexibility in the types of instruction
they fund with their federal grants as long as they fund programs in
accordance with federally approved state plans.  In keeping with
their state plans, which call for coordinating adult education with
employment training programs, a variety of coordination activities
were taking place in the states and communities we visited. 


   GOALS ARE BROAD
---------------------------------------------------------- Chapter 2:1

Recognizing the wide range of adult literacy needs in this country,
the Congress passed the AEA with broadly stated goals.  Although
adult education programs are commonly viewed as the means to obtain a
high school diploma or its equivalent, the AEA established goals that
are far broader and include citizenship and employment as well as the
overall improvement of the adult education system.  Specifically, the
purpose of the AEA is to

  improve educational opportunities for adults who lack literacy
     skills necessary for effective citizenship and productive
     employment;

  expand and improve the current adult education delivery system; and

  encourage the establishment of adult education programs for adults
     to (1) acquire basic skills needed for literate functioning, (2)
     acquire basic education needed to benefit from job training and
     obtain and keep productive employment, and (3) continue their
     education to at least the secondary school level. 


   ADULT EDUCATION STUDENTS ARE
   DIVERSE
---------------------------------------------------------- Chapter 2:2

Adult education students have diverse needs, circumstances, and
personal characteristics.  A student might be a high school dropout,
a client in a job training program, an immigrant or refugee, a
displaced worker or homemaker, an adult in the workplace, a welfare
recipient, or a retiree.  Students who enroll in adult education
classes vary in age, race and ethnicity, and employment status. 

In one program we visited in a rural California town, a 23-year-old
refugee was enrolled in an Adult Secondary Education (ASE) class.  He
was a machine operator with an eleventh grade education who planned
to earn his GED and become a bilingual teacher.  A 45-year-old
unemployed mother of four with a third grade education was enrolled
in an English as a Second Language (ESL) class.  Having done seasonal
work in the past, her goal was to obtain a GED and find work in the
nursing field. 

In an urban Connecticut program, a 19-year-old mother on welfare
wanted to complete high school and become a cosmetologist.  Although
she had a tenth grade education, she needed the basic skills taught
in an Adult Basic Education (ABE) class.  In the same program, a
62-year-old immigrant who had been an accountant in Russia was
attending ESL classes.  Her goal was to become a U.S.  citizen.  In a
suburban program in Connecticut, a 28-year-old part-time stock clerk
was enrolled in an ASE class.  He had a ninth grade education and
lived with his parents.  He hoped to earn his GED, attend college,
and become a police officer. 

In a rural town in Iowa, a 48-year-old father of four from Laos spoke
no English and was enrolled in a beginning ESL class.  He worked
part-time as an upholstery worker but hoped to learn English well
enough to get a full-time job.  In a city in Iowa, a married,
35-year-old mother of three was enrolled in an ABE class.  A former
welfare recipient, she had a job as a child care aide that was
contingent upon her earning a GED.  Her goal was to earn the GED and
keep her job. 

National statistics also suggest that adult education students are
fairly diverse.  Nationwide, 38 percent of students enrolled in adult
education classes in 1993 were between the ages of 16 and 24, 46
percent were between the ages of 25 and 44, and the remaining
students were 45 years old or older, according to the Department of
Education.  Also, 36 percent of the students were white, 31 percent
Hispanic, 18 percent black, 14 percent Asian or Pacific Islander, and
1 percent American Indian or Alaskan Native. 

The National Evaluation of Adult Education Programs\11 conducted a
survey of students who entered the adult education system between
April 1991 and April 1992.  It found that 42 percent of these
students were employed, and 58 percent were either unemployed or not
in the workforce when they enrolled.  During the year before
enrollment, 43 percent of ABE students, 31 percent of ASE students,
and 14 percent of ESL students received public assistance or welfare
payments. 


--------------------
\11 National Evaluation of Adult Education Programs, Second Interim
Report:  Profiles of Client Characteristics, Development Associates,
Inc.  (Arlington, Va.:  1993). 


   MANY EMPLOYMENT TRAINING
   CLIENTS NEED ADULT EDUCATION
   SERVICES
---------------------------------------------------------- Chapter 2:3

Many clients of federal employment training programs rely on the
State Grant Program for the basic skills they lack.  According to the
Department of Labor, unless an attempt is made to upgrade the
literacy skills of clients in federal employment training programs,
clients' success may be limited and access to the job market may be
denied.  Nationwide, almost 30 percent of JTPA clients are school
dropouts,\12 and as many as half may lack basic skills.\13 One-fourth
of JOBS clients in fiscal year 1992 were enrolled in a high school
completion program.\14

Adult education enrollment has risen almost every year since 1966,
and the Congress is considering welfare reform proposals that may
place even greater demands on adult education providers.  These
proposals may also make the coordination among adult education,
welfare, and employment training programs even more critical.  Some
states are already implementing their own welfare reform efforts that
require certain welfare clients to obtain adult education or
employment training services to receive assistance.  For example,
California's JOBS program requires that welfare recipients have
opportunities to remedy basic skill deficiencies and earn a high
school diploma or GED credential.  The state is currently required to
provide adult education to its JOBS clients with low assessment
scores and to continue to provide education until clients attain a
specified level of proficiency. 

Connecticut has piloted a welfare reform program that targets certain
welfare recipients.  Individuals in the pilot can receive needed
remedial services, such as adult education or vocational training,
for 2 years before being required to find jobs.  According to a state
official, the pilot was limited to two communities because of
concerns about the state's ability to provide remedial services to
all needy individuals, particularly adult education services.  If a
client has only 2 years to seek remedial services and faces a waiting
list for adult education services, both the client and the entire
program are at risk, explained the official. 

Similarly, Iowa's JOBS program has a goal of moving people off
welfare within 2 years by providing remedial education and employment
training.  This welfare reform effort has increased the percentage of
welfare recipients required to participate in the JOBS program from
24 to 88 percent.  This increase is achieved, in part, by exempting
fewer welfare recipients from participating in the JOBS program.  For
example, only parents with children under 6 months of age are exempt;
previously, parents with children under the age of 3 were exempt. 


--------------------
\12 Workplace Literacy and the Nation's Unemployed Workers, U.S. 
Department of Labor (Washington, D.C.:  1993). 

\13 I.  Kirch, A.  Jungeblut, and A.  Campbell, Beyond the School
Doors:  The Literacy Needs of Job Seekers Served by the U.S. 
Department of Labor, U.S.  Department of Labor (Washington, D.C.: 
1992). 

\14 Overview of Entitlement Programs:  1994 Green Book, Committee on
Ways and Means, U.S.  House of Representatives (Washington, D.C.: 
1994). 


   WIDE RANGE OF ADULT EDUCATION
   PROVIDERS USE A VARIETY OF
   APPROACHES
---------------------------------------------------------- Chapter 2:4

Under the State Grant Program, states may fund local educational
agencies\15 and a variety of public or private nonprofit agencies,\16
organizations, and institutions to provide adult education classes. 
Most programs are administered by local educational agencies.  Figure
2.1 shows the extent to which different organizations provide adult
education. 

   Figure 2.1:  Adult Education
   Providers, Fiscal Year 1992

   (See figure in printed
   edition.)

Note:  Other entities include public/private nonprofit organizations
and correctional institutions. 

Source:  U.S.  Department of Education. 

Many adult education providers use flexible and, in some cases, less
traditional approaches to education that may better suit the
responsibilities and needs of adult students.  For example, to make
classes more accessible to adults, providers may offer both day and
night classes.  Unemployed adults may prefer daytime classes; adults
who work or have child care responsibilities may only be able to
attend night classes.  We also found that some programs offer on-site
child care, which can make it easier for parents to attend adult
education classes. 

The flexible "open-entry/open-exit" feature of most adult education
providers may also better suit their students' lives than the
traditional September-to-June school year.  The National Evaluation
of Adult Education Programs found that 66 percent of adult education
programs allowed students to enroll and begin instruction at any
time.\17

Service providers use a variety of instructional methods to meet
students' needs.  For example, the principal of an adult school in a
rural California town explained that her program provides
"individualized instruction," which means that teachers assess the
individual goals and abilities of the students and take these into
consideration in planning classroom instruction.  Several methods or
a combination of methods may then be employed:  large group lectures
or presentations; small-group instruction, including role play or
practice in conversation or writing skills; or one-on-one tutoring if
the ratio of aides to students permits it. 

Some programs encourage adults who need both basic skills and
employment training to enroll in both concurrently; others recommend
basic skills training first so that students have the necessary
foundation for employment training.  Concurrent enrollment, some
state and local officials argue, may enhance learning and move adults
into the workforce faster. 


--------------------
\15 In most cases, the local education agency is a school district. 

\16 For-profit agencies may be included under certain circumstances. 

\17 M.B.  Young, M.  Morgan, N.  Fitzgerald, and H.  Fleischman,
National Evaluation of Adult Education Programs:  Draft Final Report,
Development Associates, Inc.  (Arlington, Va.:  1994). 


   WITHIN CONFINES OF AEA, STATES
   HAVE FLEXIBILITY IN FUNDING
   INSTRUCTION
---------------------------------------------------------- Chapter 2:5

The AEA limits states' flexibility in determining how to spend their
State Grant Program funds by specifying how a significant portion of
the funds are to be spent.  However, with the remaining unrestricted
funds, the combinations and types of instruction states fund vary
greatly. 

The AEA specifies that states must spend at least 15 percent of their
grants on teacher training and program innovation and at least 10
percent on programs serving incarcerated or institutionalized adults. 
No more than 20 percent can be spent on programs for certificates of
high school equivalency, and no more than 5 percent can be spent on
state administration. 

States can decide on the types and combinations of instruction they
wish to fund as long as they meet the AEA's set-asides and fund
programs in accordance with their state plans.  Most providers offer
the three most common types of instruction--ABE, ASE, and ESL. 
Enrollment in each varies greatly by state and community.  Table 2.1
shows how enrollment levels vary by instructional area nationally as
well as in the three states we visited. 



                               Table 2.1
                
                State Grant Program Enrollment, July 1,
                         1992, to June 30, 1993

                                                   ABE     ASE     ESL
                                                (perce  (perce  (perce
State                                              nt)     nt)     nt)
----------------------------------------------  ------  ------  ------
U.S. (all states)                                   34      26      40
California                                          10       6      84
Connecticut                                         21      42      37
Iowa                                                62      27      11
----------------------------------------------------------------------
Source:  U.S.  Department of Education. 


   STATES AND COMMUNITIES
   COORDINATE WITH EMPLOYMENT
   TRAINING PROGRAMS IN A VARIETY
   OF WAYS
---------------------------------------------------------- Chapter 2:6

In keeping with state plans, which call for coordinating adult
education with employment training programs, a variety of
coordination activities were taking place in the states and
communities we visited.  These activities, however, were not easy to
establish.  They took time to develop and often depended on the
perseverance of agency staff and local service providers.  State and
local coordination efforts included pooling funds, establishing
one-stop centers, and developing uniform assessment systems. 


      POOLING FUNDS
-------------------------------------------------------- Chapter 2:6.1

Connecticut has pooled funds from many\19 sources for Coordinated
Education and Training Opportunities grants.  When a service provider
receives a grant, it may contain funds from one or more funding
sources.  These coordinated grants are implemented through regional
workforce development boards responsible for a range of tasks,
including identifying local needs, evaluating grant proposals, and
overseeing operations.  These grants have the advantage of allowing
service providers to deal with a single planning process and a single
request for proposal.  However, according to officials of one
regional workforce development board, although these grants may make
things "seamless" for the client, the service provider still must
meet all the federal reporting requirements of their many funding
sources. 

In Iowa, the community colleges coordinate funds from the State
Grant, JTPA, and VOC ED programs.  Each of the state's 15 community
colleges administers the State Grant Program and offers adult
education classes.  In addition, half of the colleges administer JTPA
programs.  Services from these many programs are often administered
by staff who are both located at the college and operate within the
same department.  This arrangement facilitates coordinated program
planning, service delivery, and referral of clients to multiple
programs.  At one college, administrators from the adult education,
JTPA, and VOC ED programs told us that their close proximity enabled
them to review a client's total needs and provide the maximum
allowable services.  For example, a welfare recipient might receive
adult education instruction from the State Grant Program, a clothing
allowance from the JTPA program, and a transportation subsidy from
the JOBS program. 

Some of the coordination between California's JOBS program and State
Grant Program takes the form of financial support from many agencies. 
Adult education programs that serve the state's JOBS clients receive
adult education and JOBS funds and may also draw funds from an
8-percent set-aside of JTPA funds for education programs that are
matched by the state.  In some counties, the state's JOBS program
pays for adult education programs to meet JOBS' data collection and
reporting requirements, which include administering the same
competency-based assessment to all state JOBS clients.  Adult
education does not cover these costs.  One adult education principal
told us that, if not for the JOBS program's covering these costs,
program staff would not be able to do as much record keeping or
assessment as they do. 


--------------------
\19 These sources are the State Grant Program; JTPA's title IIA and
IIC State Education programs; Perkins Single Parents, Displaced
Homemakers, and Single Pregnant Women programs; and a state match of
federal JOBS funds. 


      ESTABLISHING ONE-STOP
      CENTERS
-------------------------------------------------------- Chapter 2:6.2

All three states we visited had begun efforts to establish one-stop
centers.  These centers are intended to help clients who need
services from many programs find all of the services they need in one
location or go to a single location to access information about the
services they need.  Officials in one community spoke of the
administrative burden imposed by the multiple federal program
requirements of establishing a one-stop center. 

All three states recently received grants from the Department of
Labor to pilot one-stop centers, which are being established around
the country even in places that have not received Department of Labor
grants.  The one-stop center we visited in California was established
without a grant from Labor.  The Department of Labor's one-stop
grants support voluntary state coordination.  All three states
considered adult education important and, thus, included adult
education officials in planning their efforts. 

With the help of the Department of Labor's one-stop grant, Iowa
recently opened its first one-stop center to provide services to
clients of employment training programs.  Adult education staff were
on site to perform client intake and assessment.  Adult education
instruction has been offered on site since February 1995. 

Connecticut was using its Department of Labor grant to develop
one-stop centers, where client intake and evaluation would take place
and where clients could be referred to multiple agencies for services
they need.  Local officials in one community said their goal was to
develop three centers and install computers in libraries, bus
terminals, and shopping malls so the public could access information
on local services, such as adult education classes. 

One community we visited in California set up a one-stop center
without a Department of Labor grant.  To prevent unnecessary
duplication of services and facilitate successful completion of
training and the transition to employment, this center established
linkages to more than 100 agencies and businesses.  Features of the
center included a central information line, career library,
computerized career assessment, and on-site employment interviews. 


      DEVELOPING UNIFORM
      ASSESSMENT SYSTEMS
-------------------------------------------------------- Chapter 2:6.3

A single assessment system used across state agencies can facilitate
coordination and make access to services easier for clients.  Using
one system allows clients to move easily among education and training
programs, provides a common assessment vocabulary so that all
agencies can determine initial client proficiency levels as well as
ongoing progress, and minimizes duplicative or unnecessary testing of
clients.  However, not all adult education and employment training
officials agree that a single assessment system can appropriately
measure adults' skills. 

To varying degrees, Connecticut and California were using common
assessments.  Connecticut required that adult education, JTPA, and
JOBS programs all use the same assessment system.  California's JOBS
program uses the Comprehensive Adult Student Assessment System
(CASAS) for assessing its clients, but the state's adult education
program uses CASAS only on a sample of programs.  Iowa was piloting
CASAS but only for use by adult education providers. 


MEASURING PROGRAM RESULTS HAS
PROVEN DIFFICULT
============================================================ Chapter 3

Measuring results in the State Grant Program has proven difficult
because program objectives have not been clearly defined and
questions exist about the validity and appropriateness of student
assessments and the usefulness of nationally reported data on
results.  Although the Department of Education has focused on
developing model program indicators that states could use to evaluate
local programs, experts and program officials disagree about whether
the indicators alone will enhance accountability.  Efforts to enhance
the evaluation capabilities of state agency staff and improve data
collection continue, but it is too early to assess their impact. 


   DIFFICULTIES IN ESTABLISHING
   PROGRAM OBJECTIVES
---------------------------------------------------------- Chapter 3:1

Evaluating program results depends on clear program objectives as
well as criteria for measuring the achievement of those objectives. 
The broad objectives of the State Grant Program give the states the
flexibility to set their own priorities but, some argue, they do not
provide states with sufficient direction for measuring results. 
Moreover, reaching a consensus on measurable objectives for adult
education is difficult. 

Because the State Grant Program's objectives are so broadly defined,
state officials have developed a variety of views on measuring
program results.  For example, some officials told us that they might
measure program success by whether adults gained the skill to read to
their children and, thus, contribute to their children's literacy. 
Others might focus on whether adults can read street signs or the
newspaper.  And, in one state we visited, an official contended that
completing high school and finding productive work should be the
objectives of the states' adult education programs because completing
a basic skills program and becoming a citizen are no longer
sufficient to succeed in society. 

Several experts and program officials told us that the State Grant
Program lacks a coherent vision of the skills and knowledge adults
need to be considered literate.  Similarly, some state officials said
that they would like the federal government to further specify the
types of results expected from state adult education programs. 

Reaching consensus on measurable objectives, however, may be
difficult since research findings are often inconclusive about the
long-term benefits to adults of achieving various program results. 
For example, many adult education programs focus on preparing adults
to take the GED examination as a means of high school completion. 
Yet research findings are mixed about whether GED attainment reflects
increased literacy skills and whether GED recipients are economically
better off than high school dropouts.\19


--------------------
\19 See, for example, J.  Baldwin, NALS, SALS, and GED:  Related
Studies and Their Implications, presentation at the Annual
Conference, State Directors of Adult Education (Louisville, Ky.: 
July 1994); H.  Beder, What Has Happened to Iowa's GED Graduates? 
Iowa State Department of Education (Des Moines, Ia.:  1992); S. 
Cameron and J.  Heckman, The Nonequivalence of High School
Equivalents, National Bureau of Economic Research (Cambridge, Mass.: 
1991); D.  Kaplan and R.  Venezky, What Can Employers Assume About
the Literacy Skills of GED Graduates?  National Center on Adult
Literacy (Philadelphia:  1993). 


   CONCERNS RAISED ABOUT ADULT
   STUDENT ASSESSMENTS
---------------------------------------------------------- Chapter 3:2

Ensuring accountability has also been hampered by limitations in the
assessment instruments used to measure student outcomes in adult
education programs.  The research literature raises questions about
the validity of standardized tests used to measure adult literacy,
and local program staff have questioned the appropriateness of using
these assessments to measure program results. 

The AEA requires states to gather and analyze standardized test data
as one way of evaluating local programs.\20 These assessments tend to
focus on either academic skills or functional literacy.  Academic
tests, such as the Tests of Adult Basic Education (known as "TABE"),
focus on measuring such basic skills as reading comprehension,
vocabulary, language expression, and mathematical proficiency. 
Functional literacy or competency-based tests, such as the
Comprehensive Adult Student Assessment System (CASAS), focus on the
ability to perform literacy-related tasks in situations faced by
adults in everyday life at home, at work, or in the community. 

Experts have questioned the validity of both the academic and
functional literacy tests used in adult education programs.  For
example, two recent reviews point to a lack of normative data for the
age ranges of participants in most adult education programs.\21
Functional literacy tests may lack validity because they are not
derived from theoretical models of ability but from everyday literacy
tasks.  According to a recent review, without further analyses, the
instructional implications of test performance are unclear.\22 Thus,
these assessments may not provide useful information about the skills
and needs of adult students. 

A more serious problem affecting the validity of assessments is the
lack of research examining the long-term retention of learning gains
in adult education programs.  According to one researcher, a
comprehensive search did not uncover a single published study on the
effectiveness of adult education programs in helping adults retain
the skills they may have acquired during instruction.\23 This being
the case, improved test scores may not necessarily mean that adults
will be better equipped for high- skilled jobs, function better as
parents, or participate more fully as citizens. 

However, officials in the three states we visited felt that
competency-based assessment systems could be useful in measuring
progress in local adult education programs and, thus, strongly
advocated these systems.  California had developed CASAS and required
its use in a sample of one-third of its adult education programs. 
Connecticut had designed its own competency-based testing system
(adapted from CASAS) and required its use in all adult education
programs.  Iowa had recently decided to move toward a
competency-based system and was piloting CASAS in 9 of its 15
community college districts. 

Local adult education and employment training staff had mixed views
about their states' competency-based assessments.  Some local program
staff saw the CASAS assessment system as a valuable and flexible
tool.  However, some English as a Second Language (ESL) teachers were
dissatisfied with the CASAS test as a measure of how well adult
education students learned to communicate in English.  And some
employment training staff said that the CASAS test did not give them
sufficiently specific information about their clients or focused too
much on life skills. 

Finally, several local staff questioned the appropriateness of CASAS
as the sole assessment tool and, therefore, used CASAS in conjunction
with other tests.  Administrators and experts also told us that they
thought no single test could measure all relevant aspects of student
performance. 


--------------------
\20 The act does not specify which tests must be used or how states
must gather test data from local programs. 

\21 D.  Wagner and R.  Venezky, "Adult Literacy:  The Next
Generation," NCAL Connections, National Center on Adult Literacy
(Philadelphia:  1995); W.  Merz and J.  Kruckenberg, Quality
Standards and Performance Measures:  Preliminary Feasibility Study on
the Uses of Basic Skills Assessment Tests, Adult Education Institute
for Research and Planning (Sacramento, Cal.:  Dec.  1991). 

\22 R.  Venezky, Matching Literacy Testing With Social Policy:  What
Are the Alternatives?  National Center on Adult Literacy
(Philadelphia:  1992). 

\23 D.  Wagner, Use It or Lose It?  The Problem of Adult Literacy
Skill Retention, National Center on Adult Literacy (Philadelphia: 
1994). 


   ADULT STUDENT DATA MISSING OR
   INACCURATE
---------------------------------------------------------- Chapter 3:3

The poor quality of the data on adult education students collected at
state and local levels also hampers accountability.  Federal and
state officials as well as recent studies have cited problems with
these data.  The studies have attributed difficulties in obtaining
accurate data to the sporadic attendance patterns of adult students
and the limited time and expertise of local adult education program
staff. 

State officials are required to submit to the Department of Education
annual statistical performance reports that include information on
students served by local programs.  State-submitted reports include
(1) the number of students served and their demographic
characteristics, (2) the skill levels of students when they start
adult education programs, (3) student progress over the program year,
(4) eight types of student achievements,\24 and (5) the number of
students who do not complete their objectives and their reasons for
separation.  The reports also include information on program staff
and the types of instructional settings in which students are served. 

Department of Education officials acknowledged serious problems with
the quality of the statistical reports, some of which are based on
double counting or undercounting of students in adult education
programs.  Another Department official charged that many of the data
are questionable and that very few local programs have record systems
that allow them to report the data the Department requires. 

Comments of officials in one state confirmed these data problems. 
They said that they did not have all the information the Department
requires for their statistical reports because too many resources are
required to collect the data.  As a result, they simply do not report
some of the data elements and provide estimates of the other
information.  They noted that the data they submit need not be
certified and that the Department has never audited their statistical
reports.  Furthermore, they asserted that these data have nothing to
do with receiving federal funds.  The only thing that really counts,
they said, is the number of adults in the state who do not have
diplomas because that is what drives the funding formula. 

Also, some local staff failed to see the utility in collecting the
data that states require for reporting to the federal government. 
Some said they thought that the information they are required to
report does not accurately reflect the accomplishments of their adult
education students. 

Difficulties in obtaining accurate data can also be attributed to
attendance patterns of adult students and the limited capacity and
expertise of local program staff.  The open-entry/open-exit feature
of many programs adds to the difficulty of tracking adult students. 
Because students may not stay in the program long or may attend on a
sporadic basis, program staff do not always have sufficient
information to report on student progress or results.  Because local
programs have difficulty following up on students, program officials
may rely on information reported by teachers or the students
themselves. 

In addition, many local programs lack sufficient staff to handle data
collection and reporting responsibilities, according to a survey of
adult education programs in nine states.\25 Programs are typically
staffed by part-time personnel, and these responsibilities become an
extra burden.  Also, local program staff may lack expertise in
collecting assessment data that can help track program effectiveness. 
For example, when the National Evaluation of Adult Education Programs
asked local adult education program staff to provide certain
assessment data, it found that about one-third of the information was
invalid because (1) the wrong test forms were used, (2) data were
inaccurately recorded, or (3) tests were administered at the wrong
times.\26

Similarly, as Connecticut began to implement a new assessment system
statewide, administrators discovered that they needed to clarify
program guidance because some local programs were mistakenly
measuring literacy gains using a test designed solely for student
placement. 


--------------------
\24 The categories in the federal reporting forms are (1) obtained an
adult high school diploma, (2) passed the GED test, (3) entered other
education or training program, (4) received U.S.  citizenship, (5)
registered to vote or voted for the first time, (6) gained
employment, (7) secured employment retention or obtained job
advancement, and (8) removed from public assistance. 

\25 L.  Condelli, Implementing the Enhanced Evaluation Model: 
Lessons Learned from the Field Test, Pelavin Associates, Inc. 
(Washington, D.C.:  1994). 

\26 M.B.  Young, N.  Fitzgerald, and M.  Morgan, National Evaluation
of Adult Education Programs:  Executive Summary, Development
Associates, Inc.  (Arlington, Va.:  1994).  One component of the
evaluation was a longitudinal study of a nationally representative
sample of local programs that provided information on the
characteristics of a national sample of 22,000 adults who enrolled in
local programs between April 1991 and April 1992. 


   EFFORTS TO IMPROVE PROGRAM
   QUALITY AND ACCOUNTABILITY
---------------------------------------------------------- Chapter 3:4

Federal efforts to improve quality and accountability have focused on
(1) developing model indicators; (2) providing technical assistance
to states and local programs on data collection, assessment, and
developing performance standards and measures; and (3) requiring
states to set aside funds for training and demonstration projects. 


      FEDERAL AND STATE PROGRAM
      QUALITY INDICATORS
-------------------------------------------------------- Chapter 3:4.1

Provisions of the National Literacy Act focus on improving quality in
adult education programs by requiring the Secretary of Education to
develop indicators of program quality.  The indicators were to be
used as models for judging state and local programs receiving federal
funding.  States were also required to develop and implement their
own indicators, which might or might not correspond to the federal
model, and use them to evaluate state and local programs. 

The Department of Education developed model indicators by (1)
reviewing adult education indicators already being developed by
various states and indicators used by other federal programs, (2)
meeting with experts and adult educators, (3) commissioning
background papers by experts in the field, and (4) conducting
workshops for state directors who would be responsible for developing
and implementing the state indicators.  The resulting eight model
indicators of program quality are listed in table 3.1.  The
indicators cover student outcomes, that is, learner progress toward
attainment of basic skills and competencies and learner advancement
in the program.  They also focus on recruiting and retaining adult
education students and other indicators of program quality--planning,
curriculum and instruction, staff development, and provision of
support services. 



                               Table 3.1
                
                Model Indicators of Program Quality for
                        Adult Education Programs

Topic                               Indicator
----------------------------------  ----------------------------------
Educational gains                   1.Learners demonstrate progress
                                    toward attainment of basic skills
                                    and competencies that support
                                    their educational needs.

                                    2.Learners advance in the
                                    instructional program or complete
                                    program educational requirements
                                    that allow them to continue their
                                    education or training.

Program planning                    3.Program has a planning process
                                    that is ongoing and participatory,
                                    guided by evaluation, and based on
                                    a written plan that considers
                                    community demographics, needs,
                                    resources, and economic and
                                    technological trends, and is
                                    implemented to the fullest extent.

Curriculum and instruction          4.Program has curriculum and
                                    instruction geared to individual
                                    student learning styles and levels
                                    of student needs.

Staff development                   5.Program has an ongoing staff
                                    development process that considers
                                    the specific needs of its staff,
                                    offers training in the skills
                                    necessary to provide quality
                                    instruction, and includes
                                    opportunities for practice and
                                    systematic follow-up.

Support services                    6.Program identifies students'
                                    needs for support services
                                    available to students directly or
                                    through referral to other
                                    educational and service agencies
                                    with which the program
                                    coordinates.

Recruitment                         7.Program successfully recruits
                                    the population in the community
                                    identified in the AEA as needing
                                    literacy services.

Retention                           8.Students remain in the program
                                    long enough to meet their
                                    educational needs.
----------------------------------------------------------------------
Source:  Model Indicators of Program Quality for Adult Education
Programs, Office of Vocational and Adult Education, U.S.  Department
of Education (Washington, D.C.:  July 1992). 

The Department did not attempt to set performance standards for adult
education programs but limited its work to developing indicators and
providing some sample measures for each indicator.  The Department
defined an indicator as a variable that reflects effective and
efficient program performance.  It is to be distinguished from a
specific measure used to determine the quantitative level of
performance for the indicator.  For example, to measure learner
progress, states could use standardized test score gains, teacher
reports of gains in communication competencies, or alternative
assessment methods (such as portfolio assessments, student reports of
attainment, or improvements in specific employability or life
skills).  An indicator is also to be distinguished from a performance
standard, which defines acceptable performance in terms of a specific
numeric criterion. 

The National Literacy Act also required states to adopt indicators by
July 1993 and use them to evaluate local programs.  States were
required to adopt, at a minimum, indicators for recruitment,
retention, and student learning outcomes.  However, decisions about
whether to adopt the Department's model indicators, what measures to
use, and whether to develop performance standards were left to the
states.  A review of amendments to state adult education plans
submitted in July 1993 showed that for the most part states had
adopted indicators similar to the Department's model, especially in
the areas of student outcomes, recruitment, and retention.\27
However, states were less consistent in how they measured indicators. 
The review found that states were using different standardized tests
to measure learner progress and had defined learner advancement in
different ways.  A 1995 survey of state adult education directors
showed that 16 states had implemented standards and 8 states had
developed but not yet implemented standards.\28

Each of the three states we visited had developed standards for
student outcomes, but not all of these standards were readily
quantifiable.  California had developed standards for seven levels of
language proficiency for ESL students (the majority of adult
education students in the state) but had not yet quantified
performance on specific assessment measures.  Standards for other
kinds of adult education students in California had not yet been
completed.  Connecticut had set standards for educational gains
expected over a specific time period and measured their achievement
using test scores and the number of course credits or competencies
attained.  Iowa had set standards for grade level increases on
standardized tests and for the performance of GED graduates on the
GED exam.  Since Iowa had not yet determined a specific strategy for
competency-based education, the state had not yet established
standards for competency-based tests. 

Experts as well as federal and state officials with whom we spoke
disagreed about whether developing indicators would improve
accountability and program quality.  Some were concerned that the
indicators do not move the field forward because they do not specify
the types of results the federal government expects from state and
local programs.  One federal official doubted whether the indicators
alone would help state and local programs collect higher quality
data.  However, other experts told us that they thought the
indicators were a good first step.  Still others said that the
federal government should not be setting standards for states because
states' literacy problems and clientele differ. 


--------------------
\27 B.G.  Elliott and B.J.  Hayward, Adult Education Performance
Data:  Improving Federal Management and Program Performance, Research
Triangle Institute (Research Triangle Park, N.C.:  1994). 

\28 Status of Development of Measures and Standards for the Quality
Indicators for the Adult Education Program, U.S.  Department of
Education (Washington, D.C.:  May 1995). 


      TOO SOON TO ASSESS EFFORTS
      TO ENHANCE ACCOUNTABILITY
      AND PROGRAM QUALITY
-------------------------------------------------------- Chapter 3:4.2

It is too soon to tell whether state-developed indicators, measures,
and performance standards will result in the collection of more
useful data or help states evaluate local programs since the 1993-94
program year was the first year in which indicators were to be used
for evaluation.  One state we visited planned to use information
collected during the 1993-94 program year as baseline data and begin
to hold local programs accountable for performance on the state's
indicators in subsequent years. 

Other federal efforts have been initiated to help states develop
better accountability systems.  Two of these efforts are designed to
help build the capacity and expertise of state adult education staff
to evaluate local programs.  In 1993, the Department hired a
contractor for a 3-year technical assistance effort designed to
assist state education agencies with assessment, evaluation, and the
development of performance standards and measures.  And, in 1993, the
National Institute for Literacy awarded grants to five states to
develop performance measurement systems for literacy, with a specific
focus on integrating systems used by different agencies that provide
literacy services. 

Department officials also told us that they were acting to improve
the quality of data collected on adult education programs.  In
concert with state adult education directors, the Department has been
examining whether to modify the existing federal reporting
requirements.  They have held several meetings but have not yet
issued any recommendations.  In addition, the Department has
developed and tested an automated management information system that
would allow programs to collect data on individual students and a
computer program that would help states more easily convert data they
collected to the statistical reports required by the Department.  A
field test of the management information system in selected local
programs in five states revealed that local staff appreciated the
system's report-writing capabilities but remained highly resistant to
performing data collection and entry. 

In addition to these efforts, the requirement that states set aside a
portion of their federal funds for demonstration projects and
training may also help states move toward better accountability
systems.\29 Although the Department has not completed an ongoing
national evaluation of the use of these funds, state officials
asserted that the set-aside was critical to their efforts to improve
program quality.  All three states had used these federal funds, in
part, to develop competency-based instruction and assessment systems;
they had also used the funds to address state-specific issues. 
California had developed a training institute for ESL teachers,
Connecticut had used some of the funds to help implement a new
statewide management information system, and Iowa had held a state
literacy conference to examine how to better measure adult student
progress through qualitative assessments. 


--------------------
\29 The 1988 AEA amendments required that at least 10 percent of the
state grant be used for special demonstration projects and for
training adult education teachers and other staff.  The National
Literacy Act raised the requirement to 15 percent and specified that
at least two-thirds of this set-aside (10 percent of the total grant)
be used for training. 


CONCLUSIONS AND AGENCY COMMENTS
============================================================ Chapter 4


   CONCLUSIONS
---------------------------------------------------------- Chapter 4:1

The broad goals and flexibility of the AEA and its State Grant
Program have resulted in a federal program that is serving many
different populations, yet has difficulty determining its target
populations, objectives, or a means to measure program results. 
Although the broad goals and corresponding flexibility give state and
local officials the latitude to design programs and quality
indicators tailored to their particular needs and priorities, some
state officials and experts have voiced concerns that the federal
government has not provided sufficient vision and guidance.  This
poses a challenge for developing accountability measures. 

The program has had difficulty ensuring accountability for
results--that is, being able to clearly or accurately say what
program funds have accomplished.  Although the Department of
Education relies on federal reporting requirements and program
quality indicators to provide this information, the data the
Department receives are of questionable value.  Because state and
local client data are missing or inaccurate, attempts to make the
program accountable may be compromised.  Until further guidance is
developed on measurable objectives and ensuring the quality of client
data, state-developed indicators and standards are unlikely to
improve accountability. 


   AGENCY COMMENTS
---------------------------------------------------------- Chapter 4:2

In its written comments on a draft of the report, the Department of
Education recognized that we identified the three areas that are
critically important to improving accountability in adult education: 
clear purpose and expectations, good assessment instruments, and
high-quality data. 

The Department also stated its commitment to improving program
accountability through several current initiatives.  These
initiatives include developing an individualized student record
keeping system; moving toward an outcomes-based national data
collection system; conducting evaluations of delivery systems,
effective practice, assessment, and performance measurement;
providing technical assistance in designing and using performance
measures and standards; and developing training programs for adult
education staff in collecting, analyzing, and reporting student and
program data.  (The Department's letter appears in app.  II.)


STATE GRANT PROGRAM BUDGET AND
ENROLLMENT DATA
=========================================================== Appendix I



                               Table I.1
                
                  AEA Appropriations, Fiscal Year 1995

                         (Dollars in millions)

                                                                Approp
                                                                riatio
Programs                                                            ns
--------------------------------------------------------------  ------
Funded
----------------------------------------------------------------------
State Grant Program                                             $252.3
National Demonstration Workplace Literacy Partnership Program     18.7
Literacy Training for Homeless Adults Program                      9.5
State Literacy Resource Centers Program                            7.8
Functional Literacy for State and Local Prisoners Program and      5.1
 Life Skills for State and Local Prisoners Program
Adult Education, National Programs: Evaluation/Technical           3.9
 Assistance
Adult Education, National Programs: National Institute for         4.9
 Literacy
======================================================================
Total                                                           $302.2

Unfunded
----------------------------------------------------------------------
Adult Literacy Volunteer Training
Adult Migrant Farmworker/Immigrant Education
Education Programs for Commercial Drivers
English Literacy Grants
National Workforce Literacy Strategies Program
State-Administered Workplace Literacy Programs
----------------------------------------------------------------------
Source:  U.S.  Department of Education. 



                         Table I.2
          
             State Grant Program Enrollment and
          Funding History, Fiscal Years 1966-1995


                                                  Federal,
      Enrollment                   State and    state, and
Year          \a     Federal\b         local         local
----  ----------  ------------  ------------  ------------
1966     377,600   $19,879,912    $9,919,000   $29,798,912
1967     388,900    26,280,000     8,334,000    34,614,000
1968     455,700    30,590,000     9,574,000    40,164,000
1969     484,600    36,000,000    11,686,000    47,686,000
1970     535,600    40,000,000    12,461,000    52,461,000
1971     620,900    44,866,102    15,322,000    60,188,102
1972     820,500    51,134,000    17,371,000    68,505,000
1973     822,500    74,834,000    20,127,000    94,961,000
1974     965,100    53,286,000    27,296,000    80,582,000
1975   1,221,200    67,500,000    43,230,000   110,730,000
1976   1,651,100    67,500,000    41,125,000   108,625,000
1977   1,686,300    71,500,000    41,992,000   113,492,000
1978   1,811,100    80,500,000    51,477,000   131,977,000
1979   1,806,300    90,750,000    63,064,000   153,814,000
1980   2,058,000   100,000,000    74,288,475   174,288,475
1981   2,261,300   100,000,000   104,212,622   204,212,622
1982   2,176,900   100,000,000   128,654,773   228,654,773
1983   2,576,300    86,400,000   172,691,213   259,091,213
1984   2,596,500   100,000,000   196,691,149   296,691,149
1985   2,879,100   101,963,000   240,410,289   342,373,289
1986   2,797,500    97,579,000   319,942,176   417,521,176
1987   2,945,600   105,981,000   400,383,790   506,364,790
1988   3,039,400   115,367,000   491,329,659   606,696,659
1989   3,257,000   136,344,000   566,656,213   703,000,213
1990   3,567,200   157,811,000   622,069,755   779,880,755
1991   3,722,600   201,032,000   809,540,972  1,010,572,97
                                                         2
1992   3,838,000   235,750,000  832,208,120\  1,067,958,12
                                           d             0
1993   3,880,400   254,624,000  855,509,950\  1,110,133,95
                                           d             0
1994  3,978,000\   254,624,000  872,620,150\  1,127,244,15
               c                           d             0
1995  4,078,000\   252,345,000  890,072,550\  1,142,417,55
               c                           d             0
----------------------------------------------------------
\a Enrollment is the number of participants who complete 12 or more
program hours, with each participant reported only once, regardless
of the number of classes or programs attended during the program year
reported. 

\b This column reflects federal appropriations data because federal
expenditure data were not readily available.  According to Department
of Education officials, appropriations and expenditure data for the
program tend to be virtually the same. 

\c Projected. 

\d Estimated. 

Source:  U.S.  Department of Education. 



                         Table I.3
          
             State Grant Program Enrollment by
           Instructional Area, Fiscal Years 1970-
                            1995

                 (Enrollment in thousands)


Fisc       Adult         Adult  English as a
al         Basic     Secondary        Second
year   Education     Education      Language         Total
----  ----------  ------------  ------------  ------------
1970         536            \b            \b           536
1971         621            \b            \b           621
1972         599           222            \c           821
1973         633           189            \c           822
1974         772           193            \c           965
1975          \c            \c            \c         1,221
1976       1,073           578            \c         1,651
1977         822           658         206\d         1,686
1978         831           583         397\d         1,811
1979         830           587         389\d         1,806
1980         938           543         577\d         2,058
1981       1,607           654            \c         2,261
1982       1,607           570            \c         2,177
1983       1,569           600         407\d         2,576
1984       1,939           657            \c         2,596
1985       1,295           733         851\e         2,879
1986         923         1,009           866         2,798
1987       1,006         1,028           912         2,946
1988       1,066         1,067           906         3,039
1989       1,139           997         1,122         3,258
1990       1,273         1,102         1,193         3,568
1991       1,358         1,182         1,183         3,723
1992       1,410         1,248         1,180         3,838
1993       1,321         1,005         1,554         3,880
1994          \c            \c            \c       3,978\f
1995          \c            \c            \c       4,078\f
----------------------------------------------------------
\a Enrollment is the number of participants who complete 12 or more
program hours, with each participant reported only once, regardless
of the number of classes or programs attended during the program year
reported. 

\b All enrollments were in the Adult Basic Education category. 

\c Data not available. 

\d According to Department of Education officials, these data are
incomplete for fiscal years 1977-1984 and represent only a portion of
ESL enrollment.  No federal requirement for states to report
enrollment data existed during this time. 

\e Fiscal year 1985 was the first year ESL was reported as a separate
instructional category. 

\f Projected. 

Source:  U.S.  Department of Education. 



                                        Table I.4
                         
                          State Grant Program, State Expenditure
                          and Enrollment Data, Fiscal Year 1991

                                  (Dollars in thousands)


                                                                               Enrollment
                                                     Federal,                    (program
                                      State and    state, and   State match    year 1991/
State                     Federal       local\a         local   (percent)\b           92)
-------------------  ------------  ------------  ------------  ------------  ------------
Alabama                    $4,107        $3,026        $7,133            42        49,510
Alaska                        446         1,761         2,207            80         6,046
Arizona                     2,138         3,040         5,178            59        33,805
Arkansas                    2,589        12,337        14,926            83        38,135
California                 16,631       282,767       299,398            95     1,023,899
Colorado                    1,919           812         2,731            30        15,890
Connecticut                 2,568        16,689        19,257            87        54,588
Delaware                      704           239           943            25         3,126
District of                   791         4,221         5,012            84        20,732
 Columbia
Florida                     8,463        58,079        66,542            87       436,870
Georgia                     5,658         2,601         8,259            31        85,794
Hawaii                        854         1,934         2,788            69        56,873
Idaho                         858           234         1,092            21         9,611
Illinois                    9,470         7,046        16,516            43        88,815
Indiana                     4,638        23,726        28,364            84        51,134
Iowa                        2,293         3,741         6,034            62        40,371
Kansas                      1,836           500         2,336            21        12,936
Kentucky                    4,122         2,537         6,659            38        34,255
Louisiana                   4,201         6,918        11,119            62        45,857
Maine                       1,112         4,999         6,111            82        17,339
Maryland                    3,621         3,977         7,598            52        33,829
Massachusetts               4,260         9,622        13,882            69        21,085
Michigan                    7,364       183,605       190,969            96       219,306
Minnesota                   2,960        14,694        17,654            83        45,348
Mississippi                 2,773           845         3,618            23        21,244
Missouri                    4,533         2,535         7,068            36        38,742
Montana                       759           616         1,375            45         6,333
Nebraska                    1,279           345         1,624            21         7,314
Nevada                        772           474         1,246            38        19,230
New Hampshire                 886           717         1,603            45         6,949
New Jersey                  5,790        20,701        26,491            78        61,364
New Mexico                  1,221         1,577         2,798            56        30,514
New York                   14,703        31,143        45,846            68       197,865
North Carolina              6,309        21,112        27,421            77       126,698
North Dakota                  745           425         1,170            36         3,642
Ohio                        8,817         6,987        15,804            44       120,529
Oklahoma                    2,595           702         3,297            21        30,501
Oregon                      1,935        11,756        13,691            86        38,409
Pennsylvania                9,673         9,283        18,956            49        50,797
Rhode Island                1,122         1,568         2,690            58         8,926
South Carolina              3,457        10,354        13,811            75       103,041
South Dakota                  769           291         1,060            27         3,849
Tennessee                   4,893         1,369         6,262            22        58,896
Texas                      12,744         8,653        21,397            40       224,037
Utah                          972         4,110         5,082            81        26,609
Vermont                       607         2,495         3,102            80         5,977
Virginia                    5,049         5,018        10,067            50        31,397
Washington                  2,738         7,646        10,384            74        37,331
West Virginia               2,207         1,292         3,499            37        26,113
Wisconsin                   3,705         7,089        10,794            66        80,455
Wyoming                       516           308           824            37         4,203
-----------------------------------------------------------------------------------------
Note:  According to the Department of Education, this is the most
recent year for which complete expenditure data are available. 

\a Obligation authority was from July 1, 1991, to June/September 30,
1993. 

\b State match reflects state and local expenditures as a percentage
of total expenditures. 

Source:  U.S.  Department of Education. 



                                   Table I.5
                    
                       State Grant Program Enrollment by
                      Instructional Area, Fiscal Year 1993

                                                           Adult   English as a
                                      Adult Basic      Secondary         Second
                                        Education      Education       Language
State                   Enrollment      (percent)      (percent)      (percent)
-------------------  -------------  -------------  -------------  -------------
Alabama                     52,132             72             24              4
Alaska                       5,822             57              3             40
Arizona                     44,828             31             26             43
Arkansas                    45,045             58             39              3
California               1,126,731             10              6             84
Colorado                    12,732             36             31             33
Connecticut                 49,334             21             42             37
Delaware                     3,323             62             12             26
District of                 16,505             39             26             35
 Columbia
Florida                    425,852             33             40             27
Georgia                     84,516             55             28             17
Hawaii                      59,034             33             35             32
Idaho                        9,566             62             23             15
Illinois                    83,153             27             20             53
Indiana                     51,884             54             39              7
Iowa                        38,072             62             27             11
Kansas                      14,910             63             20             17
Kentucky                    33,485             65             26              9
Louisiana                   39,485             51             44              5
Maine                       18,468             39             54              7
Maryland                    33,004             26             60             14
Massachusetts               22,718             39             26             35
Michigan                   193,027             25             65             10
Minnesota                   42,232             46             25             29
Mississippi                 21,752             79             17              4
Missouri                    38,845             72             16             12
Montana                      6,453             60             37              3
Nebraska                     7,178             67             10             24
Nevada                      16,853              5             79             16
New Hampshire                7,144             41             41             18
New Jersey                  62,132             28             32             40
New Mexico                  30,273             42             36             22
New York                   191,349             44             20             36
North Carolina             128,147             54             35             11
North Dakota                 3,582             52             31             17
Ohio                       116,627             73             16             11
Oklahoma                    30,072             65             22             13
Oregon                      39,365             29             39             32
Pennsylvania                52,963             53             25             22
Rhode Island                 7,089             47             24             29
South Carolina             104,009             38             60              2
South Dakota                 4,263             65             26              9
Tennessee                   57,310             67             26              7
Texas                      209,871             36             27             37
Utah                        27,770             16             72             12
Vermont                      5,095             90              5              5
Virginia                    32,106             41             28             31
Washington                  39,409             43             16             41
West Virginia               25,866             68             30              2
Wisconsin                   75,542             63             26             11
Wyoming                      3,742             46             39             15
-------------------------------------------------------------------------------
Source:  U.S.  Department of Education. 




(See figure in printed edition.)Appendix II
COMMENTS FROM THE DEPARTMENT OF
EDUCATION
=========================================================== Appendix I



(See figure in printed edition.)



(See figure in printed edition.)


GAO CONTACTS AND STAFF
ACKNOWLEDGMENTS
========================================================= Appendix III

GAO CONTACTS

Clarence C.  Crawford, Associate Director, (202) 512-7017
Lori A.  Weiss, Evaluator-in-Charge, (213) 346-8066

STAFF ACKNOWLEDGMENTS

In addition to those named above, the following individuals made
important contributions to this report:  Gary P.  Galazin, Senior
Evaluator; Tom Jessor, Senior Evaluator; Nancy Kawahara, Senior
Evaluator; David K.  Porter, Evaluator; Revae E.  Steinman, Senior
Evaluator; and Yelena K.  Thompson, Evaluator.