Drug Abuse Treatment: Data Limitations Affect the Accuracy of National
and State Estimates of Need (Letter Report, 09/15/1998, GAO/HEHS-98-229).

The federal government provides about $3 billion annually for drug abuse
prevention and treatment activities. Reliable assessments of treatment
need--at the national, state, and local levels--are essential to
accurately targeting treatment services. Although the Substance Abuse
and Mental Health Services Administration (SAMHSA) is trying to improve
its national estimates through the expansion of the National Household
Survey on Drug Abuse, the survey is still likely to underestimate
treatment need. For example, the survey excludes certain groups at high
risk of drug use, such as the homeless and prisoners, and does not
identify a large enough sample of certain subpopulations, such as
pregnant women, to adequately estimate treatment need among these
groups. At the same time, the survey's reliance on self-reported data
likely results in underreported drug use. Also, the goals of the State
Treatment Needs Assessment Program--to help states develop estimates of
treatment need and improve state reporting of need data--have not been
fully accomplished. Even though states are required to provide estimates
of treatment need as part of their block grant applications, not all
states report this information and some of the data reported are
inaccurate. SAMHSA recognizes the need to increase state reporting and
has set a target for increasing the number of states that provide the
information. It also recognizes that the overall quality of the data
reported is problematic. SAMHSA has not indicated, however, how it will
increase state reporting or improve the quality of the data reported by
states in block grant applications.

--------------------------- Indexing Terms -----------------------------

 REPORTNUM:  HEHS-98-229
     TITLE:  Drug Abuse Treatment: Data Limitations Affect the Accuracy
	     of National and State Estimates of Need
      DATE:  09/15/1998
   SUBJECT:  Drug treatment
	     Drug abuse
	     Alcohol abuse
	     Statistical data
	     Surveys
	     Block grants
	     Alcoholics treatment
	     Data collection
	     Federal/state relations
	     Projections
IDENTIFIER:  SAMHSA National Household Survey on Drug Abuse
	     SAMHSA State Treatment Needs Assessment Program

******************************************************************
** This file contains an ASCII representation of the text of a  **
** GAO report.  Delineations within the text indicating chapter **
** titles, headings, and bullets are preserved.  Major          **
** divisions and subdivisions of the text, such as Chapters,    **
** Sections, and Appendixes, are identified by double and       **
** single lines.  The numbers on the right end of these lines   **
** indicate the position of each of the subsections in the      **
** document outline.  These numbers do NOT correspond with the  **
** page numbers of the printed product.                         **
**                                                              **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced.  Tables are included, but    **
** may not resemble those in the printed version.               **
**                                                              **
** Please see the PDF (Portable Document Format) file, when     **
** available, for a complete electronic file of the printed     **
** document's contents.                                         **
**                                                              **
** A printed copy of this report may be obtained from the GAO   **
** Document Distribution Center.  For further details, please   **
** send an e-mail message to:                                   **
**                                                              **
**                                            **
**                                                              **
** with the message 'info' in the body.                         **
******************************************************************
GAO/HEHS-98-229

Cover
================================================================ COVER

Report to Congressional Requesters

September 1998

DRUG ABUSE TREATMENT - DATA
LIMITATIONS AFFECT THE ACCURACY OF
NATIONAL AND STATE ESTIMATES OF
NEED

GAO/HEHS-98-229

Estimating Drug Abuse Treatment Need

(108363)

Abbreviations
=============================================================== ABBREV

  CSAT - Center for Substance Abuse Treatment
  DSM - Diagnostic and Statistical Manual of Mental Disorders
  HHS - Department of Health and Human Services
  NDATUS - National Drug Abuse Treatment Unit Survey
  NHSDA - National Household Survey on Drug Abuse
  NTC - National Technical Center
  OAS - Office of Applied Studies
  ONDCP - Office of National Drug Control Policy
  SAMHSA - Substance Abuse and Mental Health Services Administration
  STNAP - State Treatment Needs Assessment Program
  UCR - Uniform Crime Reports

Letter
=============================================================== LETTER

B-279746

September 15, 1998

The Honorable Christopher J.  Dodd
Ranking Minority Member
Subcommittee on Children and Families
Committee on Labor and Human Resources
United States Senate

The Honorable Bill Frist
Chairman
Subcommittee on Public Health and Safety
Committee on Labor and Human Resources
United States Senate

In 1996, an estimated 13 million Americans had used an illicit drug
in the past month.\1

Each year, the federal government provides about $3 billion to fund
drug abuse prevention and treatment activities.  However, determining
the need for treatment services--for the general population as well
as for specific subpopulations, such as women and women with
children--may be problematic due to limitations in national and state
data on treatment need.  Therefore, you asked us to (1) describe the
Substance Abuse and Mental Health Services Administration's (SAMHSA)
efforts to estimate drug abuse treatment need on a national basis,
including estimates of subpopulations, and possible limitations of
these efforts and (2) obtain state estimates of drug abuse treatment
need.

To conduct our work, we interviewed and obtained documents from
officials in SAMHSA's Center for Substance Abuse Treatment (CSAT),
the Office of Applied Studies (OAS), and the Office of the
Administrator.  We also held discussions with officials at the
National Association of State Alcohol and Drug Abuse Directors and
the Office of National Drug Control Policy (ONDCP) and with experts
in the substance abuse research community.  In describing states'
efforts to estimate need, we reviewed needs assessment information
submitted by states as part of their 1997 Substance Abuse Prevention
and Treatment block grant applications; we also examined selected
studies and reports provided to CSAT from 10 states under its State
Treatment Needs Assessment Program (STNAP).  In addition, we attended
a CSAT-sponsored workshop that included all states with current STNAP
contracts in which states reported on their needs assessment studies.
We did not independently verify the accuracy of the information
provided by CSAT nor did we evaluate the effectiveness of SAMHSA's
efforts to estimate treatment need.  We conducted our review between
March and September 1998 in accordance with generally accepted
government auditing standards.

--------------------
\1 Illicit drugs are defined as marijuana and hashish, nonmedical use
of psychotherapeutics, cocaine, heroin, hallucinogens, and inhalants.

   RESULTS IN BRIEF
------------------------------------------------------------ Letter :1

SAMHSA's national estimates of drug abuse treatment need are
primarily derived from the agency's National Household Survey on Drug
Abuse (NHSDA).  While NHSDA is the principal measure of the
prevalence of illicit drug use in the United States, SAMHSA and
others have recognized that certain survey limitations affect the
accuracy of need estimates, which may result in an underestimate of
treatment need.  For example, NHSDA excludes certain groups at high
risk of drug use, such as persons who are homeless or in prisons, and
does not identify a large enough sample of certain subpopulations,
such as pregnant women, to adequately estimate treatment need among
these populations.  Moreover, NHSDA's reliance on self-reported data
likely results in underreported drug use.  To compensate for these
limitations, in 1996, SAMHSA developed a method for assessing
treatment need that adjusts NHSDA prevalence data with other data
sources, including crime reports and treatment facility data.  Using
this method, SAMHSA estimated that in 1995, about 8.9 million people
in the United States needed treatment for an illicit drug, compared
with its estimate of 6.9 million derived solely from NHSDA data.
Beginning in 1999, SAMHSA will expand NHSDA to provide better
national drug use estimates of subpopulations, such as adolescents
and pregnant women, and to provide state estimates of prevalence and
treatment need.  This expansion is expected to cost about $34
million.  Some experts believe that methodologies, such as modeling
techniques that use data from current drug use surveys (including
NHSDA), could provide better state estimates at a lower cost.  SAMHSA
officials contend, however, that the approach used in the expanded
NHSDA will result in more precise estimates.  In any case, these
adjustments will only partially correct NHSDA's limitations and are
likely to still result in an underestimate of treatment need.

States use various methods to develop estimates of treatment need,
which are used to help make planning and resource allocation
decisions.  States are required to report these estimates in
applications for federal block grant funds for substance abuse
prevention and treatment.  However, our review of fiscal year 1997
block grant applications show that not all states submitted such
data, and of those that did, some submitted incomplete or inaccurate
data.  According to SAMHSA, the incomplete and inaccurate data are
due, in part, to states' lack of sufficient data and resources to
complete block grant applications.  In response to prior concerns
about the lack of state and substate estimates of treatment need,
STNAP, administered by CSAT, was initiated in 1992.  Under 3-year
contracts with CSAT, states are provided financial and technical
assistance for conducting needs assessments and developing estimates
of treatment need to include in their block grant applications.  Even
though some states reported using data developed through STNAP
contracts to target resources, for the most part, STNAP
objectives--which also include developing states' in-house capacity
to assess need and improving block grant reporting--have not been
fully accomplished.  Of the 50 states and 3 territories, 19 have
completed their contracts under STNAP; only 8 states have done so
within the original 3-year time frame.

SAMHSA has established the improvement of state STNAP needs
assessment reporting as a goal in its fiscal year 1999 performance
plan, required by the Government Performance and Results Act.
However, the plan does not include a strategy for how the goal will
be achieved.  Therefore, we have made a recommendation that SAMHSA
include such a strategy in HHS' year 2000 performance plan.

   BACKGROUND
------------------------------------------------------------ Letter :2

In fiscal year 1998, authorized federal funding for drug treatment
programs totaled approximately $3.2 billion,\2 with the Department of
Health and Human Services (HHS) receiving $1.7 billion.  SAMHSA
received more than half ($944 million) of HHS' drug treatment budget.
Approximately 80 percent of SAMHSA's total budget, which includes
funding for both drug prevention and treatment, is distributed to
states through block grants and formula grant programs.  SAMHSA also
supports activities that include the administration of NHSDA and
STNAP.

Since 1972, NHSDA has provided national estimates of the prevalence
of drug use in the U.S.  civilian noninstitutionalized population
aged 12 years and older.  NHSDA, administered by OAS, is an ongoing
survey of members of households in the United States on their use of
illicit drugs, their nonmedical use of prescription drugs, and their
use of alcohol and tobacco products.  NHSDA is currently the nation's
most comprehensive survey of drug use.  It provides annual
information on national trends in the use of substances and data that
can be used to analyze patterns of substance use, the size and
characteristics of substance use among various special populations,
and the populations needing treatment.

To determine the need for treatment, SAMHSA combines various measures
of symptoms, problems, and patterns of use included in the NHSDA
questionnaire.  This information is intended to approximate clinical
criteria for drug dependence and to supplement it with other data
that indicate treatment need.  SAMHSA calculates the number of
persons in need of treatment as those who met at least one of the
following criteria in the past year:  dependence on any illicit drug;
heavy drug use (that is, used heroin at least once, used marijuana
daily, or frequent use of some other drug); injection drug use of
heroin, cocaine, or stimulants; or received drug abuse treatment.

States are also expected to develop estimates of treatment need on a
statewide and local basis and report them to CSAT in their block
grant applications and through STNAP.  Under the 1992 Alcohol, Drug
Abuse, and Mental Health Administration Reorganization Act (P.L.
102-321), states are required to use needs assessment data in
developing and implementing the plans submitted as part of their
block grant applications.  Specifically, states are required to
develop and report in their block grant applications estimates of
treatment need by age, sex, and race or ethnicity for the state as a
whole and for each substate planning area.  Through STNAP, CSAT
provides states with funding and technical assistance to conduct
studies to determine the need and demand for substance abuse
treatment in relation to the states' resource availability.

The Government Performance and Results Act was enacted in 1993 in
part to improve performance measurement by federal agencies.  It
requires agencies to set goals, measure performance, and report on
their accomplishments.  The legislation was enacted to increase
program effectiveness and public accountability by having federal
agencies focus on results and service quality.  SAMHSA developed
several performance goals as part of HHS' 1999 Results Act
performance plan.  These goals include providing estimates of the
prevalence of substance abuse in each of the 50 states and the
District of Columbia and increasing to 80 percent the proportion of
block grant applications that include needs assessment data developed
from STNAP.

--------------------
\2 These programs were in the Departments of Health and Human
Services, Veterans Affairs, Defense, Education, Housing and Urban
Development, and Justice; the Federal Judiciary; and ONDCP.

   NHSDA LIMITATIONS AFFECT THE
   ACCURACY OF NATIONAL ESTIMATES
   OF DRUG ABUSE TREATMENT NEED
------------------------------------------------------------ Letter :3

Although OAS relies primarily on NHSDA to make national estimates of
drug abuse treatment need for the general population and certain
subpopulations, the survey has limitations that can lead to
underestimates of treatment need.  These limitations include the
survey's use of self-reported data; the exclusion of certain
high-risk populations; and a sample for some subpopulations, such as
pregnant women, that is too small to produce valid estimates.  To
improve the accuracy of its estimates, OAS adjusted the NHSDA data
with data from other sources that are presumed to be more reliable.
For example, with this adjustment, OAS estimates of treatment need in
1995 increased by nearly a third.  This adjusted estimate, however,
is still considered conservative and does not provide subpopulation
estimates of treatment need.  OAS plans to expand NHSDA (effective in
1999) to further improve the accuracy of drug use and treatment need
estimates.

      NHSDA DATA CAN RESULT IN
      UNDERESTIMATES OF TREATMENT
      NEED
---------------------------------------------------------- Letter :3.1

Several limitations of NHSDA can result in underestimates of
treatment need for the general population and subpopulations, such as
pregnant women.  HHS and the Institute of Medicine have reported on a
number of these limitations.\3 For example, NHSDA data are based on
self-reports, which rely on respondents' truth and memory.  Although
NHSDA procedures were designed to encourage honesty and improve
recall, SAMHSA and others assume some degree of underreporting;
however, SAMHSA has not adjusted NHSDA data to account for this
limitation.

NHSDA also excludes certain populations at high risk for drug use.
NHSDA was initially designed as a survey to determine the rate of
drug use within U.S.  households and as such has excluded drug use by
individuals in institutional settings, such as prisons and
residential treatment centers, and by those with no permanent
residence, including homeless and transient people.  As a result, the
survey does not include population groups known to have high rates of
drug use who are often not in a household environment.

In addition, NHSDA's sample size for some subpopulations is too small
to produce valid estimates.  For example, for the 1994-95 survey, OAS
reported that the number of women who were pregnant at the time of
the interview--770--and reported using illicit drugs--28--was too
small to make certain estimates.

--------------------
\3 National Institutes of Health, The Validity of Self-Reported Drug
Use:  Improving the Accuracy of Survey Estimates, National Institute
on Drug Abuse Research Monograph Series 167 (Washington, D.C.:  HHS,
1997); Institute of Medicine, Treating Drug Problems (Washington,
D.C.:  Institute of Medicine, 1990).

      OAS' ADJUSTED NHSDA DATA
      PROVIDE A HIGHER ESTIMATE OF
      TREATMENT NEED BY
      COMPENSATING FOR SOME
      UNDERCOUNTING
---------------------------------------------------------- Letter :3.2

To partially account for NHSDA's undercoverage of hard-to-reach
populations and underreporting of drug use by survey respondents, OAS
developed a methodology that substitutes data from sources presumed
to be more reliable.  Using this methodology, OAS estimated that in
1995, about 8.9 million people in the United States needed drug abuse
treatment compared with the 6.9 million estimate--including 2.6
million women--derived solely from NHSDA.  While this adjustment
results in a treatment need estimate that is about 29 percent higher
than the estimate based on only NHSDA data, it still results in
conservative estimates of treatment need.

In addition, while OAS' ratio adjustment was designed to improve the
national estimate of treatment need for the general population, it
does not estimate treatment need for women and other subpopulations.
The ratio adjustment replaces some NHSDA data with information from
Uniform Crime Reports (UCR) and the National Drug and Alcohol
Treatment Unit Survey (NDATUS), now known as the Uniform Facility
Data Set, to estimate treatment need.  These data sources provide
information on the number of persons arrested, treated for drugs, or
both and are presumed to be more reliable.\4 UCR, compiled by the
Federal Bureau of Investigation from administrative records of police
departments nationally, contains information on arrests and is
adjusted for nonresponse and underreporting.  NDATUS is a 1-day
annual census of all specialty drug abuse and alcohol treatment units
nationally.  To obtain data on persons treated for drug abuse,
approximately 11,800 specialty providers are surveyed on the number
and type of patients treated and services received.

This adjustment categorizes NHSDA responses into one of four arrest
and treatment groups:  arrested and treated, treated but not
arrested, arrested but not treated, and not arrested and not treated.
According to OAS, the NHSDA estimates appear to significantly
underestimate the number in each of the first three categories; to
compensate, numbers from UCR and NDATUS are substituted for NHSDA
data.  The methodology provides only a partial adjustment because any
underreporting in the not arrested and not treated category is not
affected by the adjustment.  Also, the adjustment is still subject to
NHSDA limitations.  Therefore, according to OAS, the ratio-adjusted
estimates represent improved, but conservative estimates of treatment
need.

--------------------
\4 Although OAS considers data from UCR and NDATUS to be more
reliable, these sources also have certain limitations.  UCR is not
complete for some counties and cities and is not comparable because
of varying policing practices in different areas.  NDATUS collects
data for only 1 day out of the year.

      EXPANDED NHSDA EXPECTED TO
      PRODUCE MORE ACCURATE
      NATIONAL AND STATE ESTIMATES
      OF TREATMENT NEED
---------------------------------------------------------- Letter :3.3

OAS is expanding NHSDA's sample from 18,000 to 70,000 respondents
each year and modifying its methodology to obtain state-level data
and better national and subpopulation prevalence estimates.  The
expanded NHSDA will capture larger samples of youth, racial and
ethnic minorities, pregnant women, and hard core drug users, which
are expected to result in more accurate subpopulation estimates.  The
expanded NHSDA is expected to produce comparable state estimates of
need annually; however, the sample sizes are not large enough to
produce annual substate estimates.  According to SAMHSA officials, it
will be possible to generate substate estimates by combining multiple
years of NHSDA data.  While the additional data are expected to
result in more precise estimates, treatment need will likely still be
underestimated due to the survey's continued exclusion of certain
high-risk populations and reliance on self-reported data.

A major component of the expansion is to allow for estimates for each
of the 50 states and the District of Columbia.  A regression model
OAS developed in 1996 uses NHSDA sample data and local area
indicators to estimate state-level drug prevalence and treatment
need.  However, because of sample size requirements, this methodology
only generated estimates for 26 states and 25 metropolitan areas.
(See the appendix for a description of this methodology and
individual state and metropolitan area estimates.) The expanded
sample uses a similar methodology but has been designed to produce
direct estimates for the 8 most populous states with smaller samples
drawn for the other 42 states and the District of Columbia.  The
smaller samples will support model-based estimates that use
information from the national sample, local indicators derived from
the Census Bureau and other sources, and state samples.

The method for collecting information and the content of the NHSDA
questionnaire will also be modified under the expansion.
Specifically, NHSDA will employ computer-assisted interviewing in
1999, which is expected to minimize respondent errors and partially
increase the reliability of self-reporting by building in greater
privacy for the respondent.  The content of the questionnaire will
also be augmented to obtain income and insurance data, national
mental health statistics, data on treatment and prevention, and
information on crime and other deviant behaviors.  The projected
annual cost for the expansion is $34 million.

According to SAMHSA officials, the expanded NHSDA will help them
identify states with serious drug abuse problems and help target
technical assistance and discretionary funds.  SAMHSA expects the
expanded NHSDA to improve its prevalence estimates of drug abuse in
the 50 states and the District of Columbia--one of the goals included
in its 1999 performance plan.  SAMHSA officials also said that the
expanded NHSDA will provide data to monitor the performance of
various federal and state agencies engaged in efforts to reduce the
supply and demand of illicit drugs.  For example, the expanded NHSDA
is expected to allow for measurement of the national goal of reducing
past month use of illicit drugs among 12- to 17-year-olds by 35
percent by the end of year 2002.

Some experts question the additional cost associated with expanding
NHSDA's sample size to provide state-level estimates.  They state
that less costly alternatives using modeling techniques that rely on
currently available estimates, such as synthetic estimation, could
achieve similar goals at a significantly reduced cost.  However,
SAMHSA officials believe that the approach used in the expanded NHSDA
will result in more accurate estimates than those produced using a
purely synthetic estimation methodology.  They also pointed out that
the methodology used for the expanded NHSDA has been tested and
validated.  However, SAMHSA officials and other experts believe that
more validation is needed overall in the methods used to estimate
drug abuse treatment need.

   STATE-REPORTED DATA ARE
   INCOMPLETE AND OF VARIABLE
   QUALITY
------------------------------------------------------------ Letter :4

SAMHSA collects state and local treatment need data through state
block grant applications and state reports required under STNAP.
Through STNAP, CSAT has provided financial and technical assistance
to states to conduct needs assessments.  However, while SAMHSA is
overseeing state efforts to develop and report estimates of treatment
need, not all states have produced such estimates.  In addition,
CSAT's monitoring and review of states' block grant reporting does
not ensure the data are complete, accurate, and consistently
reported.  Our review of needs assessment information in states'
fiscal year 1997 block grant applications found the data to be
incomplete and of questionable quality.

While data developed under STNAP have been used as a state resource
and planning tool, the program has been limited in developing state
in-house capacity and improving states' reporting in block grant
applications, as intended.  One of SAMHSA's goals is to increase the
proportion of state block grant applications that include needs
assessment data developed under STNAP.  However, HHS' performance
plan did not include any information on how SAMHSA will accomplish
its goal of increasing state reporting or how it would improve the
accuracy of the data reported by states.  Further, SAMHSA's oversight
of STNAP does not encourage coordination among CSAT staff providing
oversight and technical assistance or strict monitoring of states'
compliance with program requirements.

      STATE BLOCK GRANT REPORTING
      OF TREATMENT NEED DATA IS
      INCOMPLETE AND IN SOME CASES
      INACCURATE
---------------------------------------------------------- Letter :4.1

More than $1 billion in block grant funds are distributed to states
for planning, carrying out, and evaluating activities to prevent and
treat substance abuse.\5 States report, as part of their annual block
grant applications, information on intended use of federal funds for
drug treatment.  They are asked to report information on populations,
areas, and localities with the greatest need for treatment services
and information on the state's capability to provide treatment.  This
information is collected to provide SAMHSA with information on how
states are using block grant funds and assist states in identifying
gaps in services and targeting resources.  Although states are
required by federal law to report needs assessment information in
block grant applications, the data reported does not affect their
block grant awards.\6

Our review of needs assessment information in fiscal year 1997 block
grant applications found the data to be incomplete, inaccurate, and
inconsistently reported.  According to SAMHSA, this is due, in part,
to states' lack of sufficient data and resources to complete the
extensive amount of data required in block grant applications.  While
some states have reported complete information, our review showed
that about 25 percent (14 states) did not report on the total
population needing treatment and about a third (18 states) did not
report information on the total population seeking treatment.  In
addition, a number of states did not provide information on
subpopulations.  For example, about 25 percent of states did not
report information on women needing treatment, and almost 60 percent
did not report information on children and adolescents aged 17 and
under needing treatment.\7 We also found inaccuracies in the data
reported by states.  For example, the number of males and females
under age 11 reported needing treatment in one state was greater than
the state's entire population.

Our review of 1997 applications also revealed inconsistencies in
states' reporting of needs assessments, both within a state and
across states.  For example, some states' reporting of total women
needing treatment on one of the forms in the application was
inconsistent with the reporting of that same
information--disagreggated by age, sex, and race or ethnicity--on
another form in the application.  States' reporting of information is
also not consistent across states.  States define need differently
and employ different methods and databases to estimate need.

Due to the lack of quality of needs assessment data reported in block
grant applications, the data have limited use in determining gaps
between needs and services available and assuring federal officials
that federal funds are being used for the purposes intended.  Under
block grant regulations, states are required to submit the best
available needs assessment data.  According to agency officials, the
phrase "best available" leaves the agency little basis on which to
challenge the data submitted by states in block grant applications.
While SAMHSA has not taken the initiative to ensure that accurate,
complete, and consistent information is reported in the
applications--nor has it validated state estimates or reviewed the
methodologies used to develop them--SAMHSA officials expressed
concerned about the quality of the data and are in the process of
addressing these concerns.

--------------------
\5 Fiscal year 1997 block grant funding was about $1.36 billion,
which included a 5-percent SAMHSA set-aside (approximately $65.5
million) for training, technical assistance, and administrative
activities.  States are required to set aside not less than 20
percent of block grant funds for prevention.

\6 The state block grant allocations, awarded annually, are
statutorily determined by a formula that takes into account estimates
of population in need from the decennial census, cost of service, and
state funding capacity.

\7 Some states reported that they intended to provide estimates of
treatment need for the total population and subpopulations once the
data became available.

      STNAP HAS HELPED SOME STATES
      IMPROVE RESOURCE ALLOCATION
      BUT HAS BEEN LESS SUCCESSFUL
      IN MEETING ITS OTHER GOALS
---------------------------------------------------------- Letter :4.2

In 1992, CSAT developed STNAP to help states produce better estimates
of treatment need and develop plans for use of treatment resources.
Between 1992 and 1996, CSAT awarded STNAP contracts to 53 states and
territories totaling $59 million.  As of June 1998, 23 states and
territories had been awarded new contracts, totaling approximately
$24 million, to continue activities under a second round of contract
awards.

STNAP was designed to develop and maintain a data collection and
analysis infrastructure to assist states in surveillance, planning,
budgeting, and policy development.  STNAP has three primary
objectives:  (1) assist states in better allocating treatment funds,
(2) enhance and sustain states' in-house capabilities to assess need,
and (3) improve states' reporting in block grant applications.  The
program has had limited success in meeting its objectives.

According to some state officials, STNAP has been useful in helping
states target resources and enhance service delivery.  For example,
New Jersey reported using prevalence estimates, developed from an
STNAP contract, in its allocation formula for distributing alcohol
treatment money to better reflect the distribution of need at the
county level.  Iowa reported using its results to allocate funds
based on objective estimates of need, which helped them target
outreach efforts that offer the most potential for success.  Iowa
officials also reported that they used STNAP data to redesign the
state's approach for providing tailored outreach and treatment
services for women.  Data generated in New Mexico were reportedly
used to initiate substance abuse recognition and counseling training
in public health offices and create specialized counseling for health
care providers to create smoking and alcohol cessation programs for
pregnant women.

However, states have been slow in developing in-house capacity to
assess need--one of STNAP's objectives.  According to CSAT, most
states have been unable to develop sufficient capacity due to
inadequate state-level resources and expertise and, as a result, have
relied on outside consulting firms, local universities, or both.
CSAT officials characterize these relationships as mixed and said
that effective contracts with consultants and universities is
dependent on the quality of state oversight.  While contracts with
consultants and universities can limit the development of in-house
expertise and result in a lack of continuity and a sustained data
infrastructure, they have allowed some states to establish and
maintain a knowledge base and network.  For example, while Texas and
South Carolina used universities for data collection, they used
in-house staff expertise for analyses and reporting.  To further
assist states in developing their in-house capacity to assess need,
CSAT contracted with the National Technical Center (NTC) at Harvard
Medical School to provide technical assistance.\8

According to CSAT officials, states' reporting of results developed
under STNAP in block grant applications--the third objective of the
program--has not yet been fully realized because most states have not
completed their planned data collection and analyses.  As of February
1998, 19 states\9 have contracts that have been completed or allowed
to expire, with some work remaining on final reports.  Although
states were initially awarded 3-year contracts, most states received
unfunded contract extensions and are taking, on average, 5 years to
finish.  SAMHSA requires states to incorporate needs assessments
developed under STNAP in block grant applications, but SAMHSA has not
enforced this requirement for those states that have completed their
contracts.  Although one of SAMHSA's performance goals is to increase
to 80 percent the proportion of state applications that include STNAP
needs assessment data, SAMHSA did not provide any information in the
performance plan on how it will increase state reporting or verify
the data reported by states in block grant applications.

Individual state estimates developed under STNAP were also originally
intended to be used as a basis for developing national estimates of
need.  However, this goal has been dropped by SAMHSA because of data
incomparability across states.  Specifically, while states are
required to assess need for a core set of abused drugs using clinical
definitions of dependence,\10 states have overall flexibility in
designing their studies.  As a result, states employed different
survey instruments and sample sizes that affect the resulting
estimates' comparability.

--------------------
\8 NTC had a 5-year contract, which ended in December 1997, that was
funded at $1 million each year and staffed with approximately 12 to
14 full-time equivalents.  It assisted states in developing
appropriate methodologies and integrating various data sets.
However, some outside researchers questioned the quality of the
technical assistance provided by NTC to states.  For example, while
NTC staff were very knowledgeable about telephone household surveys,
they lacked knowledge and experience in other methods of data
collection, such as using social indicator models that would produce
targeted studies to fill in gaps from the states' household surveys.

\9 Includes a special supplemental contract awarded to Missouri to
examine needs caused by flooding in 1993.

\10 The Diagnostic and Statistical Manual of Mental Disorders (DSM)
is used by clinicians and researchers for diagnosing psychiatric
disorders, including substance abuse and dependence.  The most
current DSM criteria for dependence are used to define the need for
admission to treatment under STNAP.

      CSAT'S MANAGEMENT OF STNAP
      DOES NOT ENSURE STATE
      COMPLIANCE WITH SOME PROGRAM
      REQUIREMENTS
---------------------------------------------------------- Letter :4.3

CSAT's oversight of STNAP has not ensured timely completion of the
contract or compliance with some program reporting requirements.  Of
the 19 states that completed the contract, only 8 (42 percent) did so
within the original 3-year time frame.  According to a former state
official, the complex data collection and analysis procedures and
unrealistic expectations about response rates developed under CSAT's
contract attributed to delays in contract completion.  CSAT officials
stated that the extended time necessary for states to complete the
contract is an indication of a need for more program direction.
States are also required to report findings to CSAT--through monthly,
annual, and final reports--as part of their contracts and to report
STNAP-collected data in block grant applications.  Our review found
that only 11 states have submitted final reports, and CSAT could only
locate 6 of the 11 reports.  Further, some states completed the
project but did not report data in their block grant applications.

CSAT has not consistently communicated STNAP objectives and
requirements to states.  Also, CSAT project officers acknowledge that
their review of state contracts has been inconsistent and there is
little coordination among them.  Specifically, CSAT project officers
responsible for STNAP oversight and state technical assistance have
not coordinated their efforts or taken advantage of experiences and
lessons learned from their involvement with different states.  CSAT
officials acknowledge that stricter monitoring of states' compliance
with program requirements is needed.  According to SAMHSA, some
changes have been instituted to improve monitoring; however, specific
plans of action to achieve these goals have not yet been fully
developed.

   CONCLUSIONS
------------------------------------------------------------ Letter :5

Reliable assessments of treatment need--at national, state, and local
levels and for specific population groups--are an essential component
to accurately target treatment services.  While SAMHSA has efforts
under way to improve its national estimates through the expansion of
NHSDA, the survey is still likely to result in an underestimate of
treatment need.  Also, STNAP's goals to help states develop estimates
of treatment need and improve state reporting of need data have not
been fully accomplished.  Even though states are required to provide
estimates of treatment need as part of their block grant
applications, not all states report this information and some of the
data reported are inaccurate.  SAMHSA recognizes the need to increase
state reporting and has set a target for increasing the number of
states that provide the information.  It also recognizes that the
overall quality of the data reported is problematic.  However, SAMHSA
has not indicated how it will increase state reporting or improve the
quality of the data reported by states in block grant applications.

   RECOMMENDATION
------------------------------------------------------------ Letter :6

In keeping with its goal of improving state reporting, we recommend
that the Administrator of SAMHSA develop an action plan for how the
agency will increase states' reporting of accurate, complete, and
consistent treatment need data in block grant applications and
include a summary of these actions in HHS' year 2000 performance
plan.

   AGENCY AND OTHER COMMENTS
------------------------------------------------------------ Letter :7

We provided copies of a draft of this report to SAMHSA and others for
review.  SAMHSA generally agreed with the report's findings and with
the need for an action plan aimed at improving state reporting of
treatment need data as we recommended.  While SAMHSA recognized the
need for an action plan, it stated that it would be inappropriate to
include in a performance plan the level of detail required for an
action plan.  We did not intend to imply that the performance plan
should include extensive detail; however, it should include a
discussion of strategies the agency will use to achieve its goals.
Accordingly, we modified our recommendation to clarify how action
plan information should be reflected in the performance plan.  SAMHSA
also provided a number of technical comments, which we incorporated
where appropriate.  We also obtained comments from researchers and
experts in the field who were knowledgeable about these issues and
incorporated their comments where appropriate.

---------------------------------------------------------- Letter :7.1

We are sending copies of this report to the Secretary of HHS, the
Administrator of SAMHSA, officials of state substance abuse agencies,
appropriate congressional committees, and other interested parties.
We will make copies available to others upon request.

Please contact me at (202) 512-7119 or James O.  McClyde, Assistant
Director, at (202) 512-7152, if you or your staff have any questions.
Other major contributors to this report were Ann Calvaresi Barr and
Janina Johnson.

Marsha Lillie-Blanton
Associate Director
Health Services Quality and
 Public Health Issues

OAS' REGRESSION ANALYSES AND
ESTIMATES OF TREATMENT NEED FOR 26
STATES AND 25 METROPOLITAN AREAS
==================================================== Appendix Appendix

In 1996, OAS developed models for estimating state-level treatment
need that use regression analyses combining NHSDA data with local
area indicators--such as drug-related arrests, alcohol-related death
rates, and Census Bureau data--that were found to be associated with
substance abuse.  The models produce estimates that are a weighted
average of an indirect synthetic regression estimate and a direct
survey estimate.  Therefore, the models require at least some NHSDA
sample data for each area under consideration.  A total of 26 states
and 25 metropolitan areas met the sample size criteria (at least 300
interviews) required for estimation using these models.

According to OAS, the analysis applies a consistent methodology
across states; however, the estimates produced are subject to many of
the limitations of NHSDA national estimates.  OAS has developed state
and selected metropolitan area estimates using this regression
analysis for 1991 through 1993.  (See tables 1 and 2.) According to
an OAS official, OAS is developing state estimates using 1994 through
1996 NHSDA data.

                                Table 1

                 Estimated Number of People Aged 12 and
                Older Needing Treatment for Illicit Drug
                      Use, by State, 1991 to 1993

                                                                Number
                                                                   (in
                                                                thousa
Region/state                                                      nds)
--------------------------------------------------------------  ------
Northeast region
----------------------------------------------------------------------
New Jersey                                                         131
New York                                                           367
Pennsylvania                                                       217

South region
----------------------------------------------------------------------
Florida                                                            277
Georgia                                                            206
Kentucky                                                            69
Louisiana                                                           94
North Carolina                                                     136
Oklahoma                                                            97
South Carolina                                                      63
Tennessee                                                           86
Texas                                                              421
Virginia                                                           152
West Virginia                                                       32

North Central region
----------------------------------------------------------------------
Illinois                                                           218
Indiana                                                             91
Kansas                                                              55
Michigan                                                           232
Minnesota                                                           78
Missouri                                                           138
Ohio                                                               229
Wisconsin                                                           97

West region
----------------------------------------------------------------------
California                                                       1,029
New Mexico                                                          40
Oregon                                                              53
Washington                                                         145
----------------------------------------------------------------------
Source:  SAMHSA, Substance Abuse in States and Metropolitan Areas:
Model-Based Estimates From the 1991-1993 National Household Surveys
on Drug Abuse Summary Report (Washington, D.C.:  HHS, 1996).

                                Table 2

                 Estimated Number of People Aged 12 and
                Older Needing Treatment for Illicit Drug
                 Use, by Metropolitan Statistical Area,
                              1991 to 1993

                                                                Number
                                                                   (in
                                                                thousa
Metropolitan statistical area                                     nds)
--------------------------------------------------------------  ------
Anaheim-Santa Ana, Calif.                                           89
Atlanta, Ga.                                                       117
Baltimore, Md.                                                      45
Boston, Mass.                                                      122
Chicago, Ill.                                                      137
Dallas, Tex.                                                        52
Denver, Colo.                                                       52
Detroit, Mich.                                                     129
El Paso, Tex.                                                       11
Houston, Tex.                                                      103
Los Angeles, Calif.                                                288
Miami-Hialeah, Fla.                                                 35
Minneapolis-St. Paul, Minn.                                         49
Nassau-Suffolk, N.Y.                                                41
New York, N.Y.                                                     177
Newark, N.J.                                                        28
Oakland, Calif.                                                     90
Philadelphia, Pa.\a                                                121
Phoenix, Ariz.                                                      56
San Antonio, Tex.                                                   30
San Bernardino, Calif.                                              81
San Diego, Calif.                                                   67
St. Louis, Mo.\b                                                    55
Tampa-St. Petersburg, Fla.                                          38
Washington, D.C.                                                    89
----------------------------------------------------------------------
\a Includes areas in New Jersey.

\b Includes areas in Illinois.

Source:  SAMHSA, Substance Abuse in States and Metropolitan Areas:
Model-Based Estimates From the 1991-1993 National Household Surveys
on Drug Abuse Summary Report.

*** End of document. ***