Child Welfare: Most States Are Developing Statewide Information
Systems, but the Reliability of Child Welfare Data Could Be
Improved (31-JUL-03, GAO-03-809).
To better monitor children and families served by state child
welfare agencies, Congress authorized matching funds for the
development of statewide automated child welfare information
systems (SACWIS) and required that the Department of Health and
Human Services (HHS) compile information on the children served
by state agencies. This report reviews (1) states' experiences in
developing child welfare information systems and HHS's role in
assisting in their development, (2) factors that affect the
reliability of data that states collect and report on children
served by their child welfare agencies and HHS's role in ensuring
the reliability of those data, and (3) practices that child
welfare agencies use to overcome challenges associated with
SACWIS development and data reliability.
-------------------------Indexing Terms-------------------------
REPORTNUM: GAO-03-809
ACCNO: A07799
TITLE: Child Welfare: Most States Are Developing Statewide
Information Systems, but the Reliability of Child Welfare Data
Could Be Improved
DATE: 07/31/2003
SUBJECT: Data collection
Data integrity
Federal aid to states
Federal funds
Federal/state relations
Internal controls
Strategic planning
Information systems
Children
Reporting requirements
Child welfare
Colorado
Iowa
New York
North Carolina
Oklahoma
Statewide Automated Child Welfare
Information System
NCCAN National Child Abuse and Neglect
Data System
HHS Adoption and Foster Care Analysis
and Reporting System
******************************************************************
** This file contains an ASCII representation of the text of a **
** GAO Product. **
** **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced. Tables are included, but **
** may not resemble those in the printed version. **
** **
** Please see the PDF (Portable Document Format) file, when **
** available, for a complete electronic file of the printed **
** document's contents. **
** **
******************************************************************
GAO-03-809
Report to Congressional Requesters
United States General Accounting Office
GAO
July 2003 CHILD WELFARE Most States Are Developing Statewide Information
Systems, but the Reliability of Child Welfare Data Could Be Improved
GAO- 03- 809
HHS reported that 47 states are developing or operating a SACWIS, but many
continue to face challenges developing their systems. Most state officials
said they recognize the benefit their state will achieve by developing
SACWIS, such as contributing to the timeliness of child abuse and neglect
investigations; however, despite the availability of federal funds since
1994, states reported a median delay of 2- 1/2 years beyond the timeframes
they set for completion. States reported that they encountered some
difficulties during SACWIS development, such as challenges receiving state
funding and creating a system that reflected their work processes. In
response to some of these challenges, HHS has provided technical
assistance to help states develop their systems and conducted on- site
reviews of SACWIS to verify that the systems meet federal requirements.
Despite efforts to implement comprehensive information systems, several
factors affect the states* ability to collect and report reliable
adoption, foster care, and child abuse and neglect data. States responding
to GAO*s survey and officials in the 5 states GAO visited reported that
insufficient caseworker training and inaccurate and incomplete data entry
affect the quality of the data reported to HHS. In addition, states
reported technical challenges reporting data. Despite HHS*s assistance,
many states report ongoing challenges, such as the lack of clear and
documented guidance on how to report child welfare data. In addition,
although states were mandated to begin reporting data to the Adoption and
Foster Care Analysis and Reporting System (AFCARS) in 1995, few reviews of
states* AFCARS reporting capabilities have been conducted to assist states
in resolving some of their reporting challenges. Some states are using a
variety of practices to address the challenges associated with developing
SACWIS and improving data reliability. For
example, 44 states included caseworkers and other system users in the
design and testing of SACWIS, and 28 states reported using approaches to
help caseworkers identify and better understand the data elements that are
required for federal reporting.
Factors That Affect the Reliability of Data Reported to HHS Which may lead
to inaccurate measures of:
State performance on federal outcomes Children's experiences, such as time
spent in foster care
Information
Factors affecting data quality: Inaccurate and incomplete data entry by
caseworkers Insufficient caseworker training Differences between state and
federal data definitions Lack of clear, documented guidance from HHS
Difficulty accessing technical assistance from HHS
Source: GAO analysis. Resulting in
potentially unreliable information on abused and neglected children
available in federal data systems
To better monitor children and families served by state child welfare
agencies, Congress authorized matching funds for the development of
statewide automated child welfare information systems (SACWIS) and
required that the Department of Health and Human Services (HHS) compile
information on the
children served by state agencies. This report reviews (1) states*
experiences in developing child welfare information systems and
HHS*s role in assisting in their development, (2) factors that affect the
reliability of data that states collect and report on children
served by their child welfare agencies and HHS*s role in ensuring the
reliability of those data, and (3) practices that child welfare agencies
use to overcome challenges associated with SACWIS development and data
reliability. In order to improve the reliability
of state- reported child welfare data, GAO recommends that the Secretary
of HHS consider ways to enhance the guidance and assistance offered to
states to help them overcome the key challenges in collecting and
reporting child
welfare data. In commenting on this report, HHS generally agreed with
GAO*s findings and commented that the report provides a useful perspective
of the problems states face in collecting data and of HHS*s effort to
provide
ongoing technical assistance to improve child welfare data. www. gao. gov/
cgi- bin/ getrpt? GAO- 03- 809. To view the full product, including the
scope and methodology, click on the link above. For more information,
contact Cornelia M. Ashby at (202) 512- 8403 or AshbyC@ gao. gov.
Highlights of GAO- 03- 809, a report to
Congressional Requesters
July 2003
CHILD WELFARE
Most States Are Developing Statewide Information Systems, but the
Reliability of Child Welfare Data Could Be Improved
Page i GAO- 03- 809 Child Welfare Letter 1 Results in Brief 2 Background 5
Most States Are Developing SACWIS, but Challenges Remain
Despite HHS*s Oversight and Technical Assistance 10 Several Factors Affect
the States* Ability to Ensure Reliable Data on Children*s Experiences, and
Some of HHS*s Oversight and Assistance Is Problematic 24 States Are Using
Various Practices to Overcome System Development Challenges and Improve
Data on Children*s
Experiences 38 Conclusion 42 Recommendation to the Secretary of Health and
Human Services 43 Agency Comments 43 Appendix I Scope and Methodology 47
Appendix II State SACWIS Stages of Development 49
Appendix III Comments from the Department of Health and Human Services 50
Appendix IV GAO Contacts and Acknowledgments 62 GAO Contacts 62 Staff
Acknowledgments 62 Bibliography 63
Related GAO Products 64
Tables
Table 1: Significant Child Welfare Information and Data Systems 6 Contents
Page ii GAO- 03- 809 Child Welfare
Table 2: Number of States in Various Stages of SACWIS Development 12 Table
3: Selected SACWIS Child Welfare and Administrative
Services 14 Table 4: Commonly Used SACWIS Development Participants and
Their Level of Helpfulness 15 Table 5: State- Reported Benefits of SACWIS
Development 17 Table 6: Number of Months States Delayed in SACWIS
Development 18 Table 7: Common SACWIS Development Challenges 19 Table 8:
Selected AFCARS Elements and Six States* Levels of Compliance in Meeting
Reporting Requirements 32 Figures
Figure 1: Most Common Caseworker Issues That Affect Data Quality 26 Figure
2: Federal Practices That Affect Data Quality 35 Abbreviations
ACF Administration for Children and Families ADP advance planning document
AFCARS Adoption and Foster Care Analysis and Reporting System CAPTA Child
Abuse Prevention and Treatment Act CRSR Child and Family Services Reviews
FFP federal financial participation HHS Department of Health and Human
Services IT information technology NCANDS National Child Abuse and Neglect
Data System NYTD National Youth in Transition Database SACWIS Statewide
Automated Child Welfare Information System VCIS Voluntary Cooperative
Information System
This is a work of the U. S. Government and is not subject to copyright
protection in the United States. It may be reproduced and distributed in
its entirety without further permission from GAO. It may contain
copyrighted graphics, images or other materials. Permission from the
copyright holder may be necessary should you wish to reproduce copyrighted
materials separately from GAO*s product.
Page 1 GAO- 03- 809 Child Welfare
July 31, 2003 The Honorable Charles E. Grassley Chairman, Committee on
Finance United States Senate
The Honorable Tom DeLay Majority Leader House of Representatives
Recent news reports of tragedies involving child welfare agencies across
the country highlight the long- standing problems states have had
monitoring children in their care. Given that hundreds of thousands of
children are found to be victims of abuse and neglect and are estimated to
spend some time in foster care each year, the Congress required that the
Department of Health and Human Services (HHS) compile information on the
children served by state agencies and authorized federal funds to match
those of states for use in the development of state child welfare
information systems. Since 1994, designated federal matching funds have
been available to states to develop and implement comprehensive case
management systems* Statewide Automated Child Welfare Information Systems
(SACWIS)* to manage their child welfare cases as well as to report child
abuse and neglect, foster care, and adoption information to the federal
government. States have the option to implement a SACWIS or develop
different information systems without using SACWIS funds to support their
child welfare agencies and collect information on their child welfare
cases. Regardless of the type of system a state develops, child welfare
caseworkers at the county or local level are the key personnel who collect
and document information on children and families served by child welfare
agencies, in addition to performing a wide range of services to protect
children* such as investigating child abuse or neglect reports or
providing support services to maintain the children in their homes.
Currently, HHS compiles state- reported child welfare data in two
databases: the Adoption and Foster Care Analysis and Reporting System
(AFCARS) and the National Child Abuse and Neglect Data System (NCANDS).
HHS relies on the information available in its databases to analyze and
track children*s experiences in the child welfare system, to determine
states* performance on federal child welfare outcome measures, and to
report to Congress on children*s well being and child welfare experiences.
United States General Accounting Office Washington, DC 20548
Page 2 GAO- 03- 809 Child Welfare
Because of your interest in how states have developed systems to collect
and report information on the children they serve and the reliability of
that information, you asked us to determine (1) states* experiences in
developing child welfare information systems and HHS*s role in assisting
in their development; (2) factors that affect the reliability of data that
states collect and report on children served by their child welfare
agencies, and HHS*s role in ensuring the reliability of those data; and
(3) practices that child welfare agencies use to overcome challenges
associated with SACWIS development and data reliability. To conduct our
work, we surveyed all 50 states and the District of Columbia regarding
their experiences in developing and using information systems and their
ability to report data to HHS. We received responses from 49 states and
the District of Columbia, 1 although all states did not respond to every
question. We also reviewed a variety of HHS documents, including the
protocol and reports for its reviews of SACWIS systems and states* AFCARS
reporting capabilities. In addition, we visited 5 states* Colorado, Iowa,
New York, North Carolina, and Oklahoma* to obtain firsthand information on
their experiences developing SACWIS and reporting data to HHS. We selected
these states to represent geographic diversity and different stages of
SACWIS implementation. Finally, we interviewed HHS officials and child
welfare and data experts and reviewed relevant literature. We conducted
our work between June 2002 and June 2003 in
accordance with generally accepted government auditing standards. A more
detailed discussion of our scope and methodology appears in appendix I.
HHS reported that 47 states are developing or operating a SACWIS, but many
continue to face challenges despite HHS*s oversight and technical
assistance. Most states are using federal SACWIS funds and are in various
stages of development. States reported in our survey that they have spent
approximately $2.4 billion in federal, state, and local funding on SACWIS.
Most state officials said they recognize the benefit their state will
achieve by developing SACWIS, such as contributing to the timeliness of
child abuse and neglect investigations; however, despite the availability
of federal funds since 1994, many child welfare agencies lag behind the
1 Throughout this report, references to state survey responses include the
District of Columbia. Forty- six of these states reported that they are
developing or operating a SACWIS. Nevada, which HHS reported has an
operational SACWIS, did not respond to our survey. Results in Brief
Page 3 GAO- 03- 809 Child Welfare
timeframes they set for completion, with delays ranging from 2 months to 8
years. Forty- two of the 46 states responding to our survey that they are
developing SACWIS reported at least some challenge obtaining state
funding. In Iowa, for example, state officials reported that insufficient
state funds delayed them in making the necessary modifications to meet
federal requirements for system completion. Some states had difficulties
developing a system that met the state child welfare agency*s needs
statewide. For example, state officials in New York* a state where the
counties are responsible for administering child welfare services* said
that building a uniform system was stalled when significant frustration
with the system*s design led some county officials to request that the
state stop SACWIS development. In addition, 32 states reported at least
some challenge securing information technology contractors with knowledge
of child welfare practice to develop their SACWIS. In response to some of
these challenges, HHS has provided technical assistance to help states
develop their systems and conducted on- site reviews of SACWIS to verify
that the systems meet all federal requirements. For example, at the time
of our review, HHS had conducted on- site reviews in 26 states with
operational SACWIS to ensure that the systems met all federal requirements
and to offer assistance to states that faced challenges
completing the development of their SACWIS. Several factors affect states*
ability to collect and report reliable data on children served by state
child welfare agencies, and some problems exist, such as a lack of clear
and documented guidance, with HHS*s oversight and technical assistance.
Almost all of the states responding to our survey reported that
insufficient caseworker training and inaccurate and incomplete data entry
into their information system affect the quality of the data reported to
HHS. Although most states reported these as separate factors, HHS and the
states we visited found that insufficient training and inaccurate and
incomplete data entry are often linked. In addition, 36 of the 50 states
that responded to our survey reported that technical
challenges, such as matching their state data element definitions to HHS*s
data categories, affected the quality of the data that they report to the
federal government. Similarly, during assessments of 6 states* compliance
with AFCARS reporting standards, HHS found that these issues affect data
reliability. Despite HHS*s assistance in helping states improve their
data,
such as testing state data quality and providing the results to the states
to aid them in resubmitting data, states report ongoing challenges. For
example, 41 of the 50 states responding to our survey reported that a lack
of clear and documented guidance from HHS affects the quality of the data
they report to AFCARS, and 25 states said the lack of clear, documented
guidance also affected the data reported to NCANDS. In addition, although
Page 4 GAO- 03- 809 Child Welfare
states were mandated to begin reporting AFCARS in 1995, few reviews of
states AFCARS reporting capabilities have been conducted to assist states
in resolving some of their challenges.
Some states are using a variety of practices to address the challenges
associated with developing SACWIS and improving data reliability, although
no formal evaluations are available on their effectiveness. To overcome
the challenge of developing a system to meet statewide needs, many states
relied on caseworkers and other system users for input on design and
testing of SACWIS. Few states reported that they devised strategies to
overcome the other challenges, such as limited funding and finding
contractors with knowledge of child welfare. However, Oklahoma child
welfare officials* in order to maximize the limited state funding for
maintaining their SACWIS* reported saving $1 million each year by hiring
some of the contractors who developed their SACWIS as permanent staff. To
improve data reliability, the 5 states we visited routinely review their
data to identify data entry errors so that managers can ensure that the
missing data are entered appropriately. In addition, some states reported
that frequent use of the data, such as publishing periodic management
reports detailing local offices* performance on outcome measures, helps
caseworkers understand the importance of entering timely information.
In order to improve the reliability of state- reported child welfare data,
we are recommending that the Secretary of HHS consider ways to enhance the
guidance and assistance offered to states to help them overcome the key
challenges in collecting and reporting child welfare data. In commenting
on this report, HHS*s Administration for Children and
Families (ACF) generally agreed with our findings and commented that the
report provides a useful perspective of the problems states face in
collecting data and of ACF*s effort to provide ongoing technical
assistance to improve the quality of child welfare data. In response to
our
recommendation, ACF said that we did not recognize the long- term efforts
it has taken to provide AFCARS and NCANDS guidance. ACF also noted that
the data definitions need to be updated and revised and said it is
currently in the process of revising the AFCARS regulations to further
standardize the information states are to report* which we acknowledge in
our report. Further, ACF added that although staff turnover in state child
welfare agencies is a significant contributor to data quality issues, we
did not focus on this as a significant factor. However, because we
recently issued a detailed report on a variety of caseworker issues, we
primarily focused in this report on the key data entry challenges
caseworkers face and refer readers to our previous work for additional
information on challenges related to caseworker recruitment and retention
and their
Page 5 GAO- 03- 809 Child Welfare
affect on child welfare agencies. ACF commented that it is firmly
committed to continue to support the states and to provide technical
assistance and other guidance as its resources will permit. We believe
that the recent activities to formally obtain, document, and incorporate
feedback from the states with regard to collection and reporting adoption
and foster care data represent are a step towards improving states* data.
Our recommendation encourages HHS to consider ways to enhance the
guidance and assistance already offered to states as a step to helping
them better comply with the reporting requirements. ACF*s Children*s
Bureau is responsible for the administration and oversight of federal
funding to states for child welfare services under Titles IV- B and IV- E
of the Social Security Act. However, the monitoring of children served by
state child welfare agencies is the responsibility of the state agencies
that provide the services to these children and their families. Child
welfare caseworkers at the county or local level are the key personnel
responsible for documenting the wide range of services offered to children
and families, such as investigations of abuse and neglect; treatment
services offered to families to keep them intact and prevent the need for
foster care; and arrangements made for permanent or adoptive placements
when children must be removed from their homes. Caseworkers are supported
by supervisors who typically assign new cases to workers and monitor
caseworkers* progress in achieving desired outcomes, analyzing and
addressing problems, and making decisions about cases. A number of efforts
at the national level have been taken to implement
comprehensive data systems that capture, report, and analyze the child
welfare information collected by the states (see table 1 for information
on national data systems as well as information on state systems).
Background
Page 6 GAO- 03- 809 Child Welfare
Table 1: Significant Child Welfare Information and Data Systems System
History Use State information systems
Statewide Automated Child Welfare Information System (SACWIS) The Omnibus
Budget Reconciliation Act
(OBRA) of 1993 authorized the use of an enhanced federal financial
participation (FFP) rate of 75 percent to assist states develop uniform
automated information systems* SACWIS* that support the
administration of services offered under their child welfare programs. a
The enhanced FFP was available initially from federal fiscal years 1994
through 1996 and subsequently extended through federal fiscal year 1997.
After 1997, states receive a 50 percent match for SACWIS- related
activities. Funding approval for SACWIS is based on states* estimated
costs for development and operation, and no time limits are placed on
the receipt of federal funding. A SACWIS is designed and developed for use
by states*
caseworkers and other personnel for the purposes of establishing an
electronic case file for children and families
served by the state child welfare agency. Some of the data captured in
SACWIS are reported to HHS.
National data systems
Voluntary Cooperative Information System (VCIS) With funding from HHS, the
American Public Welfare Association* now known as the American Public
Human Services Association* established VCIS in an effort to compile
state- specific child welfare data. States voluntarily reported aggregate-
level data on the characteristics of children in foster care and those
adopted from state
child welfare agencies. VCIS compiled data between 1982 and 1994.
The data available in VCIS was used to produce reports at the national
level on the
characteristics of children in foster care and those adopted from state
child welfare
agencies. However, the data reported to VCIS were inconsistent. b
Following the implementation of AFCARS, states no longer reported to VCIS.
Page 7 GAO- 03- 809 Child Welfare
System History Use Adoption and Foster Care Analysis and Reporting System
(AFCARS) In 1986, Congress added Section 479 to
Title IV- E of the Social Security Act, which required HHS to establish
and implement a mandatory foster care and adoption data collection system.
Despite the legislative requirement that HHS complete final regulations
for the system by the end of calendar year 1988, regulations were not
issued until December 1993. States began reporting to AFCARS in 1995 on
the demographic characteristics of adoptive and foster children and their
parents as well as foster children*s type of placement and permanency
goals. States are required to report 66 foster care and 37 adoption data
elements for each child in care during 6- month periods. Between 1998 and
2001, states not meeting certain reporting standards were subject to
penalties. The
penalties were rescinded in January 2002 following an appeal filed by 12
states.
Some of the key uses of AFCARS data include:
Calculating the national standard for five federal outcome measures*
incidence of child abuse and/ or neglect in foster care, foster care re-
entries, stability of foster care placements, length of time to achieve
reunification, and length of time to achieve adoption.
The publication of the annual Child Welfare Outcomes Report.
Determining the allocation of funds in the adoption incentive program and
Chafee Foster Care Independence Program. c National Child Abuse and
Neglect Data System
(NCANDS) In 1988, Congress amended the Child Abuse Prevention and
Treatment Act
(CAPTA) by directing HHS to establish a national data collection and
analysis system consisting of state child abuse and neglect information.
Walter R. McDonald & Associates was awarded the contract to compile and
analyze the state- reported data. States began voluntarily reporting
annually in 1990; however, the 1996 CAPTA amendments directed states to
report certain data to receive CAPTA grant funding. States submit either
child- specific records or aggregate- level data. In 2001* the most
recently available data from HHS* 39 states and the District of Columbia
submitted child- specific data, which includes the demographics of the
children and their perpetrators, the types of maltreatment, investigation
or assessment findings, risk factors, and services provided as a result of
the investigation or assessment. The remaining 11 states submitted
aggregate data.
Some of the key uses of NCANDS data include:
Calculating the national standard for two federal outcome measures on
child safety* recurrence of maltreatment and incidence of child abuse and/
or neglect in foster care.
The publication of the annual Child Maltreatment Report.
The publication of the annual Child Welfare Outcomes Report.
Page 8 GAO- 03- 809 Child Welfare
System History Use National Youth in Transition Database (NYTD) In
response to requirements included in the Foster Care Independence Act of
1999, HHS has proposed to develop the NYTD to capture more detailed data
than is currently reported to AFCARS on older foster children who receive
independent living services.
Such services include life- skills training in financial management or
career planning. HHS reported to Congress in September 2001 that
nationwide implementation of NYTD was planned for October 2003, but recent
HHS information indicates that the actual implementation date will likely
be 2005 or 2006. Once implemented, states will
be required to report to NYTD. In response to the legislation,
data available in the NYTD will be used, in part, to track (1) the number
and characteristics of older youth receiving independent living services,
(2) the type and quantity of services states provided, and (3) state
performance on the
outcome measures required by the legislation. Source: GAO analysis based
on federal legislation, regulations, and other documents. a The allowable
costs under the 75 percent FFP included planning, design, development, and
installation of a SACWIS. Other costs, such as operation of the SACWIS,
were covered at 50 percent. b See www. acf. hhs. gov/ programs/ cb/ dis/
vcis/ maintoc. htm for more complete details on the
shortcomings of the data available in VCIS. c The total number of
finalized adoptions reported to AFCARS are used to determine the amount,
if
any, of the adoption incentive payments awarded to states. In addition,
AFCARS data are used to determine the ratio of the number of children in
foster care in each state to the total number of children in foster care
in all states to calculate state allotments under the Chafee Foster Care
Independence Program.
To qualify for federal funding for SACWIS, states must prepare and submit
an advance planning document (APD) to ACF*s Children*s Bureau, in which
they describe the state*s plan for managing the design, development,
implementation, and operation of a SACWIS that meets federal requirements
and state needs in an efficient, comprehensive, and costeffective manner.
2 In addition, the state must establish SACWIS and program performance
goals in terms of projected costs and benefits in the APD. States are
required to submit separate APDs for the planning and development phases,
in addition to periodic updates.
2 When states choose to develop information systems that include other
human services, such as food stamps, child support enforcement, or
Medicaid, states must submit APDs to each cognizant federal agency. In a
hearing held last year before the Subcommittee on
Technology and Procurement Policy, House Committee on Government Reform,
we testified that the federal agencies do not have systems to monitor
states requests for federal approval and funding through the life cycle of
a state request. (See U. S. General Accounting Office, Human Services:
Federal Approval and Funding Processes for States* Information System,
GAO- 02- 347T (Washington, D. C: July 9, 2002.))
Page 9 GAO- 03- 809 Child Welfare
Since the administration and structure of state child welfare agencies
vary across the nation, states can design their SACWIS to meet their state
needs, as long as states meet certain federal requirements. Federal
funding is available to states for SACWIS that
meet the requirements for reporting AFCARS data to HHS; to the extent
practicable, are capable of linking with the state data
collection system that collects information on child abuse and neglect;
to the extent practicable, are capable of linking with, and retrieving
information from, the state data collection system that collects
information on the eligibility of individuals under Title IV- A* Temporary
Assistance for Needy Families; and provides for more efficient,
economical, and effective administration of
the programs carried out under a state*s plans approved under Titles IV- B
and IV- E of the Social Security Act.
A SACWIS must operate uniformly as a single system in each state and must
encompass all entities that administer programs provided under Titles IV-
B and IV- E. In some cases, HHS will allow the statewide system to link to
another state system to perform required functions, such as linking to
financial systems to issue and reconcile payments to child welfare service
providers. The state*s APD must describe how its SACWIS will link to other
systems to meet the requirements in the SACWIS regulations. In addition to
monitoring the APDs of the states that are developing
SACWIS, HHS reviews state information systems through formal SACWIS
assessment reviews and the Child and Family Services Reviews (CFSR)* a
federal review process to monitor states* compliance with child welfare
laws and federal outcome measures. The formal SACWIS reviews are conducted
by ACF*s Children*s Bureau to determine if a state has developed and
implemented all components detailed in the state*s APD and if the system
adheres to federal requirements. The CFSR assesses statewide information
systems, along with other systemic factors, to determine if the state is
operating a system that can readily identify the status, demographic
characteristics, location, and goals for placement of every child who is
in foster care. This systemic factor is reviewed in all states, regardless
of whether the state is developing a SACWIS or the stage of system
development. According to results from the fiscal years 2001 and 2002
CFSRs, 4 of the 32 states in which HHS reviewed were not in substantial
conformity on the statewide information system indicator. These 4 states
must address how they will come into conformity with this
Page 10 GAO- 03- 809 Child Welfare
factor in a program improvement plan. HHS has also conducted SACWIS
reviews in 2 of these states.
While 47 states are developing or operating a SACWIS, many challenges
remain despite HHS*s oversight and technical assistance. Since 1994,
states reported that they have spent approximately $2.4 billion in
federal, state, and local funding on SACWIS. While most state officials we
interviewed and those responding to our survey said that they recognize
the benefits their state will achieve by developing a statewide system,
many states reported that the development of their SACWIS is delayed
between 2 months and 8 years beyond the time frames the states set for
completion, with a median delay of 2- 1/2 years. Most states responding to
our survey faced challenges, such as obtaining state funding and
developing a system that met the child welfare agency*s needs statewide.
In response to some of these challenges, HHS has provided technical
assistance to help states develop their systems and conducted on- site
SACWIS reviews to verify that the systems meet all federal requirements.
Currently, 47 states are developing or operating a SACWIS and are in
various stages of development* ranging from planning to complete. The
states responding to our survey reported using approximately $1.3 billion
in federal funds 3 and approximately $1.1 billion in state and local funds
4 for
their SACWIS. However, HHS estimated that it allocated approximately $821
million between fiscal years 1994 and 2001 in SACWIS developmental funds 5
and $173 million between fiscal years 1999 and 2001 in SACWIS
3 Forty- four states provided information on the total amount of federal
funds they received to develop and operate SACWIS. Alaska, Hawaii,
Missouri, North Carolina, Texas, and Vermont did not report federal
funding information. Nevada did not respond to our survey. State- reported
figures may include some funding allocated in fiscal year 2003 since the
survey was issued in October 2002 and completed as late as December 2002.
4 Forty- four states provided information on the total amount of state
funds used to develop and operate SACWIS. Arkansas, Hawaii, Missouri,
North Carolina, Texas, and Vermont did not report state funding
information. Nevada did not respond to our survey. State- reported
figures may include some funding allocated in fiscal year 2003 since the
survey was issued in October 2002 and completed as late as December 2002.
5 This figure includes developmental funds allocated by HHS to 49 states
and the District of Columbia. Hawaii did not take any federal money for
SACWIS development. Most States Are
Developing SACWIS, but Challenges Remain Despite HHS*s Oversight and
Technical Assistance
States Are Using Federal and State Funds and Various Participants to
Develop MultiComponent SACWIS
Page 11 GAO- 03- 809 Child Welfare
operational funds. 6 The total amount of federal funding provided to
states for SACWIS is unknown because states claimed operational costs as a
part of their Title IV- E administrative expenses prior to 1999. 7
Although the federal government matched state funding at an enhanced rate
of 75 percent beginning in 1994, many states did not apply for federal
funding or begin SACWIS development until 1996 and 1997 when more than
$467 million* the bulk of federal funds* were allocated. Most states were
still developing their SACWIS by the time enhanced funding expired in
1997, after which states could receive a 50 percent FFP for SACWIS
development and operation. Although 47 states are currently developing or
operating a SACWIS, all states except Hawaii received some federal SACWIS
funds. For example, according to figures provided by HHS, North Carolina
received approximately $9.6 million in developmental funds and North
Dakota received approximately $2.4 million in developmental funds and
$245,000 in operational funds for SACWIS, but both states encountered
difficulties that prevented them from completing their systems. In these
situations, HHS entered into negotiations with the states about the amount
of money that the states must return to the federal government. In order
to track states* SACWIS development, HHS places them in six
categories that identify their stage of development (see table 2). States
are required to submit APD updates periodically, which inform HHS of their
progress in developing SACWIS. See appendix II for a complete list of
states* phases of development. Although most states continue to advance in
the development of their systems, some encounter problems that cause HHS
to recategorize them into a lower stage of development. In Pennsylvania,
for example, the child welfare agency encountered difficulties, such as
inadequate computer software to support a comprehensive SACWIS, after
attempting to implement its SACWIS in
6 This figure includes operational funds allocated to 35 states. States
begin claiming operational costs when some or all components of their
SACWIS are operating in local offices. Operational activities include
routine maintenance, minor enhancements, and other changes that do not
significantly increase or modify the functionality of the system.
7 According to HHS officials, prior to fiscal year 2000, states reported
SACWIS operational expenses as part of their Title IV- E administrative
expenses because the claims sheet states used for reporting did not have a
separate column for SACWIS operational expenditures. In fiscal year 2000,
states were required to use a claims sheet that was reformatted to provide
space for SACWIS operational expenditures. In addition, an HHS official
explained that the difference between the state- reported figures and the
federal figures may be due to states claiming some SACWIS expenses under
different programs, such as Title IV- E administrative funds, rather than
separately as SACWIS expenses.
Page 12 GAO- 03- 809 Child Welfare
2000. Due to these problems, the state is in the process of shutting down
the system and has resubmitted an APD for a new system to HHS for review
and approval for further federal funding. According to figures provided by
HHS, Pennsylvania has received approximately $9.7 million in federal
funding thus far. In addition, while HHS may classify a state system
as complete following an assessment of their SACWIS, a state may make
additional changes to the system since SACWIS, like other computer
systems, continually evolve as technology and child welfare practices
change. States can claim federal funding for these changes as operational
expenses. For example, Oklahoma*s SACWIS was the first system to be
determined complete, but it has made enhancements to its system since HHS
found the system in compliance with federal requirements in 1998. In
addition, Oklahoma is currently considering moving to a Web- based system.
An HHS official reported that such changes do not need prior approval
unless they are in excess of $5 million.
Table 2: Number of States in Various Stages of SACWIS Development Stage
Number of states
Complete a 5 Operational b 24 Partially operational c 9 Implementation d 2
Planning e 7 No SACWIS f 4 Source: HHS. Note: Status is as of May 22,
2003. a SACWIS assessment process is completed, and all functional
requirements and specifications set
forth in the APD are either included in the system or in an accepted
corrective action plan. b All functional requirements and specifications
in APD are included in system, and system is
functional statewide, but state has not completed SACWIS assessment or is
working on other issues. c State is still rolling out system to field
sites or still adding functions to systems that are operational
statewide. d In active design and development, even if delayed while
waiting to resolve problems such as funding. e Working through options for
a SACWIS.
f Have never pursued SACWIS funding or have abandoned plans to develop a
system.
In developing a system, states have considerable flexibility in the design
of their SACWIS. According to HHS officials, a state should be using its
SACWIS as a case management tool that uses automation to support the
various aspects of state child welfare programs, such as recording child
Page 13 GAO- 03- 809 Child Welfare
protection, out of home care, and foster care and adoption services. To
further assist child welfare practice, states have designed their systems
to follow the natural flow of child welfare practice in their state and
have added design features to help track key events during a case. For
example,
in Iowa child welfare work is divided between child abuse and neglect
investigations and ongoing case management for children brought into the
care of the child welfare agency. As a result, Iowa designed a SACWIS to
reflect this work process by linking two databases* one to record child
abuse and neglect information and one to record ongoing case records* that
share information with one another. 8 In Rhode Island, the SACWIS was
designed to alert caseworkers if an alleged perpetrator has been the
subject of three reports of abuse or neglect within 1 year. Regardless of
the findings of each report, this alert notifies the caseworker to
initiate an investigation when a third report is received. Since many
states are in different phases of SACWIS development, their systems
currently support to varying degrees a variety of child welfare and
administrative components (see table 3). According to HHS, while the
components represented in table 3 are required for a state*s SACWIS to be
considered compliant with federal guidance* either through an interface or
built within the system* some of the subcomponents, such as a function
that helps caseworkers manage their caseloads, are optional. HHS has
encouraged states to automate as many functions as possible in the SACWIS
in an effort to cut down on the additional paperwork or duplicative steps
inherent in manual data collection. One of these services, tracking
independent living, is becoming more important for states as HHS decides
how to implement the Foster Care Independence Act of 1999 and considers
the development of the NYTD. 9 Some states have already started collecting
data on older youth and the services they receive. Currently,
27 states reported in our survey that they are at some stage of using
their SACWIS to track independent living services, and an additional 14
states plan to include this component in their system in preparation for
the
8 Although the Iowa state officials described their SACWIS as including
the child abuse and neglect system, HHS commented on a draft of this
report that it does not view the child abuse and neglect system as part of
the state*s SACWIS. However, HHS said that the state has met the SACWIS
requirement in this area by building an interface between the two systems.
9 The Foster Care Independence Act of 1999 increased federal support to
states for
independent living programs* programs designed to assist youth who are
identified as likely to remain in foster care until age 18. Independent
living services can include education or training necessary for the youth
to obtain employment.
Page 14 GAO- 03- 809 Child Welfare
requirements. However, 21 of the 46 states reporting to our survey that
they are developing or operating a SACWIS reported that they would have to
make substantial changes to their SACWIS in order to capture this
information.
Table 3: Selected SACWIS Child Welfare and Administrative Services Service
Fully or partially operational in SACWIS Planned
for SACWIS Child Welfare Services Child protection a 38 5
Out of home care b 35 8 Adoption 34 9 Independent living 27 14 Intensive
home- based services c 27 13
Administrative Services
Workload management 32 8 IV- E eligibility d 29 14 Foster care maintenance
payments 28 14
Adoption assistance payments 25 17 Contract provider payment 24 15 Source:
GAO survey. Note: Based on responses from 46 states developing or
operating a SACWIS. The rows for the columns *fully or partially
operational* and *planned* do not add to 46 because the respondents may
have answered *not supported,* *don*t know,* or *no answer.* a Child
protection includes services such as intake and screening, investigation,
and disposition.
b Out of home care includes things such as foster care, group homes, and
residential placement. c Intensive home- based services include efforts to
avoid placing a child in foster care. d IV- E funding is available for
foster care, adoption, and independent living services. To assist with the
design of their SACWIS, states relied on a number of
different participants including internal users, such as caseworkers and
managers, information technology (IT) staff, and contractors. Most states
found these participants to be extremely or very helpful in the process
(see table 4). In Oklahoma, for example, 150 child welfare staff from the
field worked closely with the contractor in intensive work group sessions
to design and test the system. To complement the caseworkers* knowledge of
child welfare practice, 43 states relied on IT staff. In Colorado, for
example, IT staff said that during SACWIS design and development, they
shared office space with program staff that had been assigned to help with
Page 15 GAO- 03- 809 Child Welfare
SACWIS development. This co- location of staff aided in the exchange of
information pertaining to the development of the system. Finally, 42
states reported that they hired private contractors to conduct a large
part of SACWIS design and development. The contractors helped states meet
federal requirements, designed the system with state specific options,
wrote the necessary software, tested and implemented the system, and
trained users.
Table 4: Commonly Used SACWIS Development Participants and Their Level of
Helpfulness
SACWIS development participant Number of states
using participant Number of states rating participant as extremely or very
helpful
Internal users 44 41 IT staff 43 31 Private contractors 42 37 Source: GAO
survey. Note: Based on responses from 46 states developing or operating a
SACWIS.
At the time of our review, HHS reported that 4 states were not pursuing
SACWIS development and most of these states reported various reasons in
our survey for not developing a system. In Hawaii, the child welfare
agency chose not to pursue SACWIS because it already had a statewide
system in place that it believed was adequately meeting its needs and
which was collecting and reporting federal child welfare data. After an
attempt to develop a system, North Carolina cancelled its efforts because
it could not build consensus across its 100 counties on the design of a
uniform system. On our site visit to North Carolina, child welfare
officials reported that they are currently working on a statewide
information system that will encompass a number of social services, such
as food stamps and mental health services, but an HHS official reported
that North Carolina is not seeking federal SACWIS funding to support the
development of this system. Vermont officials reported that they did not
pursue SACWIS because the legislature declined to provide the matching
state funds. In retrospect, they believe that the choice not to develop
SACWIS was best for the state because they found the SACWIS requirements
too restrictive to enable the state to design a system to meet its needs.
Officials said that the state would not use a number of the required
SACWIS components, such as developing all the required electronic links to
other agencies* systems, especially since the state has a small child
welfare population. Another state* North Dakota* did not
report in our survey the reason for stopping SACWIS development;
Page 16 GAO- 03- 809 Child Welfare
however, HHS officials reported that the state had attempted to develop a
SACWIS, but faced a variety of problems, such as receiving state funding.
While most state child welfare agency officials said they recognize the
benefits the state will achieve by developing SACWIS, such as enhancing
their ability to track the whereabouts of foster children, 31 state
agencies lag behind the time frames they set for completion, with 26
states reporting delays ranging from 2 months to 8 years. State officials
reported in our survey and during site visits that SACWIS has contributed
to more efficient and effective agency functioning, which can improve
states* capabilities to manage their child welfare cases, including
keeping track of where the children are living and the services they are
receiving. Child welfare officials in Colorado reported that automation
has improved agency functioning by making child welfare case information
available statewide, which is especially helpful when families move from
one county to another. In Oklahoma, caseworkers and state officials noted
that they believe their children are safer since the implementation of
SACWIS simply because the information on the children is easily accessible
to the caseworkers and their supervisors. According to survey results,
automated systems provided easier access to data and allowed caseworkers
to better monitor children in their care, which may contribute to
additional child welfare and administrative benefits, such as decreased
incidences of child abuse and neglect, shortened length of time to achieve
adoption,
timeliness of payments to foster families, and timeliness of payments to
foster facilities (see table 5). New Jersey reported in our survey that
its goal in developing a SACWIS is to integrate the more than 40 stand-
alone systems that currently capture information on the children served by
their child welfare agency. By pulling all of these systems together into
a uniform SACWIS, the state hopes to improve the recording of casework
activities in a timely manner and to develop a tool to better target
resources and services. Effectively integrating these systems will require
the state to use a disciplined IT management approach that includes (1)
detailed analyses of users* needs and requirements, (2) a clearly defined
strategy for addressing information needs, and (3) sufficient technical
expertise and resources to support the effort. States Accrue Benefits
from Using SACWIS, but Several Issues Create Delays in Completing States*
Systems
Page 17 GAO- 03- 809 Child Welfare
Table 5: State- Reported Benefits of SACWIS Development Child welfare
benefits
Number of states identifying SACWIS as extremely or very effective in
achieving measure
Decreased incidence of child abuse and neglect 25
Shortened length of time to achieve adoption 24 Decreased recurrence of
child maltreatment 23 Shortened length of time to achieve reunification 22
Management benefits
Timeliness of payment to foster families 36 Timeliness of payment to
foster facilities 34 Overall case management for children and families 33
Timeliness of child abuse and neglect investigations 33
Source: GAO survey. Note: Based on responses from 46 states developing or
operating SACWIS. States not included answered *moderately effective,*
*somewhat effective,* *not at all effective,* *not a state goal,* *system
doesn*t support,* *don*t know,* or *no answer.*
Despite the benefits that many states have accrued with SACWIS, 31 states
reported in our survey that they have been delayed in system completion
beyond their initial deadline and identified a number of challenges that
have led to the delay (see table 6). 10 Some of the common difficulties
states reported in developing SACWIS included receiving state funding
approval, reaching internal agreement on system development, and creating
a system that reflects child welfare work processes and is user friendly
(see table 7).
10 Twelve of the 46 states reporting that they are developing or operating
a SACWIS reported that they have not experienced delays in developing
their systems. In response to the length of the delays reported by 26
states in our survey, ACF commented on a draft of this report that these
states may be using different definitions in defining their delays.
However, ACF did not provide further information on how the delays
represented in this
report differ from its perception of states* experiences. In our survey,
we asked states to report on the delays that exceeded the time line
outlined in their initial APD.
Page 18 GAO- 03- 809 Child Welfare
Table 6: Number of Months States Delayed in SACWIS Development State
Length of delay in months a
Alabama 36 Arkansas 6 California 36 Colorado 26 Connecticut 96 District of
Columbia 36 Georgia 25 Idaho 21 Illinois 79 Indiana 6 Kansas 72 Louisiana
12 Maryland 12 Michigan 26 Minnesota 12 Mississippi 12 New Jersey 42 New
Mexico 3 Ohio 36 Oregon 70 Rhode Island 14 South Carolina 47 Tennessee 36
Utah 48 Virginia 2 Washington 36 Source: GAO survey. Note: While 31 states
reported in the survey that they have experienced a delay in SACWIS
development, only 26 states reported the length of their delay. The survey
was issued in October 2002 and completed by states as late as December
2002. a States were asked to report the number of months the delays
exceeded the time line outlined in their
APD.
Page 19 GAO- 03- 809 Child Welfare
Table 7: Common SACWIS Development Challenges SACWIS development
challenges Number of states experiencing at
least some challenge a Receiving state funding approval 42 Reaching
internal agreement on system development 41 Creating a system that
reflects work
processes 40 Creating a user friendly system 39 Insufficient state funding
allocation 32 Securing contractors with knowledge of child welfare 32
Source: GAO survey. Note: Based on responses from 46 states developing or
operating a SACWIS. a States were asked the extent to which certain
factors were a challenge in developing SACWIS using
the following scale: very great, great, moderate, some, or no challenge.
The number of states reported in our analysis of challenges represents the
total number of states reporting any extent the factor posed a challenge.
States not included answered *no challenge,* *have not yet reached this
stage,* *don*t know,* or *no answer.*
Forty- two states reported challenges receiving funding approval, and 32
states reported that insufficient state funding allocations for SACWIS
development were a challenge in developing a comprehensive system. For
example, Vermont officials reported that the state legislature declined to
provide the matching state funds needed to secure federal funding for
SACWIS. As a result, the state could not pursue development. In addition
to the development challenges reported in our survey, 2 of the 5 states we
visited reported that insufficient funding affected ongoing SACWIS
maintenance. In Colorado, state agencies have received a series of budget
cuts, which child welfare officials report have impacted their ability to
respond to child welfare caseworkers* needs for system improvements. In
Iowa, child welfare officials reported that the state legislature
appropriated $17,000 for state fiscal year 2002 for all child welfare
automated systems activity, which they noted was an insufficient amount to
maintain and upgrade systems as well as to pay staff. They reported that,
as a result, the child welfare agency provided the information systems
department with approximately $1 million from other parts of the agency*s
budget.
Despite user involvement in system design, some states still faced
challenges trying to reach internal agreement among agency officials and
caseworkers on the design of a system, resulting in a delay in
development. In New York* a state where the counties are responsible for
Page 20 GAO- 03- 809 Child Welfare
administering child welfare services* the development of SACWIS was
stalled when significant frustration with the system*s design led
Commissioners from five large counties and New York City to request that
the state stop SACWIS development until a reassessment of the design and
plans for the implementation of the system was completed. After a thorough
evaluation of the project, the state made changes to the project plan and
developed statewide work groups to ensure all counties were in agreement
with the system design. In addition, they hired a contractor to monitor
system development and ensure that all users* requirements are
seriously considered. Similarly, despite states* heavy reliance on
contractors, many reported that securing contractors with knowledge of
child welfare practice was a challenge for timely SACWIS development.
Contractors are hired by the state for their system development knowledge
but often are unfamiliar with child welfare policies and practices,
especially since they vary from state to state. Officials in Colorado, for
example, said they encountered
difficulties with their contractors because of high turnover among the
contractor staff and their lack of knowledge of child welfare policies. A
contractor who has worked with 7 states to develop their SACWIS reported
that contractors are asked to learn the child welfare business practices
of a state in a short amount of time and that states cannot devote many
resources, such as caseworkers, to help in the design process
because caseworkers need to devote their time to providing services to
children and families. Therefore, contractors often have to acquire
knowledge on their own.
Many states reported that creating a system that reflects child welfare
work processes and is user friendly was a challenge in developing SACWIS.
These issues were also identified in the federal reviews of states*
SACWIS. For example, one state explained in the SACWIS review that it had
designed a system to meet the caseworkers* needs and reflect the nature of
the child welfare work processes by developing a system that required
events to be documented as they occurred. However, this design limited the
SACWIS*s functionality because it did not allow the caseworkers to go back
and enter information after an event happened. The state explained that
caseworkers do not use the system in real time,
but provide services to the children and families and then record the
information in the system. The state had to redesign the system to correct
for this design flaw. In addition, the 14 states reporting that they have
adapted a system from another state have experienced some challenge
modifying the systems to reflect their work processes. While HHS advises
states to consider adapting another state*s system if it requires few
Page 21 GAO- 03- 809 Child Welfare
changes, states report that they and their contractors were not always
aware of the changes that would need to be made to adapt a system.
Although Colorado and New York decided to modify another state*s SACWIS
instead of designing a new system, child welfare officials in these states
reported that adapting a SACWIS from another state has created more
problems than anticipated. Colorado and New York adapted systems from
state- administered child welfare agencies, which required extensive
modifications to meet their needs as county- administered states. For
example, Colorado needed a system that supported its administrative
structure and could handle a larger number of cases.
HHS has assisted states in a variety of ways in developing and completing
their SACWIS. 11 As a part of its regulatory responsibilities, HHS must
review, assess, and inspect the planning, design, development,
installation, and operation of SACWIS. In addition to reviewing and
monitoring states* APDs, HHS conducts on- site SACWIS reviews to comply
with these responsibilities. HHS officials told us that these reviews are
a detailed and thorough assessment of state systems to ensure the systems*
compliance
with SACWIS requirements. In addition, officials reported that they
provide verbal technical assistance during the on- site review to help
states that do not fully conform with the applicable regulations and
policies. At the time of our review, HHS had conducted 26 SACWIS reviews*
5 of
which were determined as meeting all the requirements and classified as
complete. HHS officials told us that since states have the flexibility to
build a SACWIS that meets their needs, a large portion of the formal
reviews concentrate on ensuring that the systems conform to state business
practices. For example, while SACWIS regulations require that a state
report all AFCARS data from their SACWIS, one state HHS reviewed
relied on a separate state system to report data on the children served by
the juvenile justice agency who are eligible for IV- E foster care funds.
The state proved it had developed an automated process to merge data from
both systems to compile a single AFCARS report that included children
captured in both their SACWIS and juvenile justice systems. Therefore,
11 With regard to the budget difficulties that states reported facing,
since 1994 the federal government has made a commitment to help states
develop and maintain their SACWIS by matching 75 percent of states*
development funds through 1997 and providing an ongoing match of 50
percent of state funding for the development and maintenance of their
systems. However, since the states* legislatures must make the initial
commitment to fund SACWIS, the federal government cannot assist state
child welfare agencies with this challenge. HHS Provides Some
Assistance to Help States Meet SACWIS Requirements
Page 22 GAO- 03- 809 Child Welfare
HHS recognized that this process best met the state*s needs and determined
the SACWIS to be complete and meeting all requirements.
Few systems have been determined complete after an on- site review because
of unresolved issues, such as not being able to build links to other state
information systems or not implementing certain eligibility determination
functions. To help states address some of these development challenges,
the SACWIS review team provides the state with recommendations for
complying with SACWIS requirements. For example, HHS observed during a
review in one state that the SACWIS was available statewide, but
information collected in one county was not available to caseworkers in
other counties. The federal officials offered recommendations to the state
to meet the SACWIS requirement that all information be available
statewide. In addition, HHS officials reported that once the draft report
with the results of the SACWIS review are completed, federal staff
schedule a conference call with the state officials to walk through the
system*s deficiencies and offer guidance on how the state can move
forward.
HHS facilitates the sharing of information between states developing
SACWIS through an automated system users group that allows state and
federal officials to exchange information, ideas, and concerns. According
to some state and HHS officials, the trust level at these meetings is very
high, which promotes open discussions and also creates an atmosphere for
informal dialogue with HHS. The systems users group developed out of
another active group* the child welfare users group* when HHS solicited
state representatives to help HHS define a model child welfare information
system, which was later used as the basis for the SACWIS functional
requirements after the passage of the 1993 legislation authorizing
enhanced federal funding. State officials in Iowa and New York reported
that the systems users group continues to play an important role in
providing a forum for the honest exchange of information on SACWIS
development. For example, child welfare and technical officials in New
York said that the systems users group has been very beneficial because
they have learned from other states* positive and negative experiences in
developing SACWIS, as well as the experiences unique to states with
county- administered agencies. In addition to the users group, HHS
officials also sponsor a listserv* an electronic mailing list* that allows
state officials to exchange information, and a monthly conference call
with state information technology directors. Iowa child welfare
information
Page 23 GAO- 03- 809 Child Welfare
technology officials said that they find the monthly SACWIS telephone
conference call helpful because project managers discuss issues such as
promising practices and new regulations. 12 Technical assistance for
SACWIS development is also available to states
through the National Resource Center for Information Technology in Child
Welfare (Resource Center). According to survey results, 9 states said they
used the Resource Center for assistance in developing SACWIS and
14 states reported using it for help with SACWIS maintenance and
improvements. According to Resource Center officials, they assist states
with SACWIS development by helping states understand the technology that
is available for use, providing information on the automation of child
welfare work and converting data, and reviewing the APD documentation. For
example, the Resource Center offered technical assistance to Pennsylvania
to help the state decide if it should continue development of its current
SACWIS, abandon the SACWIS project and allow the counties
to operate individual systems, or design a different SACWIS. The Resource
Center evaluated the current SACWIS to determine if it could capture
information based on the SACWIS regulations and if it was user friendly
for the caseworker. Following the Resource Center*s analysis, Pennsylvania
decided to discontinue the existing SACWIS and develop a new SACWIS. When
the Resource Center opened in 1999* 5 years after many states started
developing SACWIS* staff were not very familiar with many of the efforts
states made during development. In an attempt to remedy this lack of
knowledge on states* issues developing SACWIS, Resource Center staff
participated in some of the on- site SACWIS reviews conducted by HHS. Both
HHS and Resource Center officials believe this exposure to the SACWIS
systems enhanced the availability of technical assistance resources and
knowledge available to the states. 12 In commenting on a draft of this
report, HHS indicated that a Web resource is available to
states interested in learning about other states* efforts to develop human
services* child welfare, food stamps, Temporary Assistance to Needy
Families, child care, and child support enforcement* information systems
at http:// www. acf. hhs. gov/ nhsitrc.
Page 24 GAO- 03- 809 Child Welfare
Several factors affect states* ability to collect and report reliable 13
data on children served by state child welfare agencies, and some problems
exist, such as a lack of clear and documented guidance, with HHS*s
oversight and technical assistance. Almost all of the states responding to
our survey reported that insufficient caseworker training and inaccurate
and incomplete data entry affect the quality of the data reported to HHS.
14 In addition, 36 of the 50 15 states that responded to our survey
reported that technical challenges, such as matching their state data
element definitions to HHS*s data categories, affected the quality of the
data that they report to the federal government. For example, North
Carolina officials told us that
while state policy mandates that they count every location in which a
child resides, including hospital stays, AFCARS regulations say that
hospital stays and other short- term placements should not be included in
the count of foster care placements. In cases where state policy differs
from federal policy, state officials must carefully re- format their data
in order to meet federal reporting requirements. Similarly, during
assessments of 6 states* compliance with AFCARS reporting standards, HHS
found that these issues affect data reliability. Despite the assistance
that HHS offers to states, such as testing state data quality and
providing the results to states to aid them in resubmitting data, states
report ongoing challenges receiving clear and documented guidance and
accessing technical assistance.
13 Data are reliable when they are complete and accurate. A subcategory of
accuracy is consistency. Consistency refers to the need to obtain and use
data that are clear and welldefined enough to yield similar results in
similar analysis. See U. S. General Accounting Office, Assessing the
Reliability of Computer- Processed Data, GAO- 02- 15G (Washington,
D. C.: Sept. 2002). 14 States were asked the extent to which certain
problems may decrease the quality of the data submitted to AFCARS and
NCANDS using the following scale: very great, great,
moderate, some, and no affect. 15 The analysis of survey responses about
reporting data to HHS is based on responses from 49 states and the
District of Columbia. All states, regardless of SACWIS development, were
asked to complete these questions. Several Factors Affect
the States* Ability to Ensure Reliable Data on Children*s Experiences, and
Some of HHS*s Oversight and Assistance Is Problematic
Page 25 GAO- 03- 809 Child Welfare
Almost every state responding to our survey and all the states we visited
reported that insufficient training for caseworkers and inaccurate and
incomplete data entry affect the quality of the data reported to AFCARS
and NCANDS (see fig. 1). Although most states reported these as separate
factors, HHS and the states we visited found that insufficient training
and inaccurate and incomplete data entry are often linked. For example, in
official reviews of states* information systems capability to capture data
and report them to AFCARS, HHS advised states to offer additional training
to caseworkers on several AFCARS data elements, such as recording the
reasons for a child leaving foster care, to improve the accuracy of the
data submitted. Similarly, Oklahoma reported that the state found that
caseworkers were misinterpreting reports of policy violations by foster
parents and inaccurately recording them as abuse or neglect allegations.
However, state officials told us that training is typically one of the
first programs cut when states face tight budget restrictions. For
example, Iowa officials told us that training has been significantly
reduced in recent years because of budget cuts and new workers may wait 2
to 3 months before being trained how to enter data appropriately into
their SACWIS. Insufficient Caseworker
Training and Inaccurate and Incomplete Data Entry Are the Most Common
Factors That Affect Data Reliability
Page 26 GAO- 03- 809 Child Welfare
Figure 1: Most Common Caseworker Issues That Affect Data Quality
Notes: Based on responses from 50 states. The results reported in the
figure are a sum of the states that reported the issue had a very great
affect, great affect, moderate affect, or some affect on the quality of
state data submitted to HHS. Very great and great affect responses are
represented in the top section of each bar. Moderate and some affect
responses are represented in the bottom section of each bar. States not
included answered *no affect,* *don*t know,* or *no answer.*
Inaccurate and incomplete data entry can also result from a number of
other factors, such as caseworkers* hesitation to ask families for
sensitive information. For example, caseworkers in Oklahoma reported that
they did not feel comfortable asking if a child*s mother was married at
the time of birth or if a child is of Hispanic origin* both of which are
required AFCARS data elements. In commenting on a draft of this report,
Oklahoma added that caseworkers did not understand why the data elements
were required and how the federal government used the information. In
addition, Iowa state officials said that caseworkers may guess the racial
backgrounds of children in their care or record them as unknown,
especially when children come from mixed racial backgrounds, rather than
asking the family for the information. HHS noted similar issues in
States reporting at least some affect in data quality 0 10
20 30
40 50
Inaccurate data entry by caseworkers
Insufficient training for caseworkers
Incomplete data entry by caseworkers
AFCARS NCANDS Source: GAO survey.
49 45
49 42
46 43
18 31
12 33
15 34
11 31
17 29
12 31
Page 27 GAO- 03- 809 Child Welfare
5 states that have had an AFCARS review. 16 Caseworkers were inaccurately
recording a child*s race as *unable to determine* even though this option
should be selected only if the child*s parents or relatives
cannot provide the information, such as when a child is abandoned. 17
Caseworkers, supervisors, and managers in the 5 states we visited reported
that additional factors, such as difficulties balancing data entry with
the time that they spend with the families and children, contributed to
inaccurate or incomplete data entry. In addition, our recent work on
caseworker recruitment and retention found that caseworkers struggle to
balance the time they spend with children and data entry, and reportedly
spend at least 50 percent of their time documenting case records. 18
Supervisors in Iowa explained that since caseworkers are responsible for
ensuring that children and their families receive the services they need,
the caseworkers tend to initially limit data entry to the information that
is necessary to ensure timely payment to foster care providers, and
complete all other data elements when the caseworkers have time. In
addition,
caseworkers in Colorado said that they are between 30 and 60 days behind
in their data entry, so the information in the automated system may not
accurately reflect the current circumstances of children in care. The
caseworkers reported that they tend to concentrate only on entering data
that will allow them to open a case in their SACWIS. HHS*s Inspector
General recently issued a report in which more than two- thirds of the
states reported that caseworkers* workloads, turnover, a lack of training,
and untimely and incomplete data entry affected the reporting of AFCARS
data. 19 16 We reviewed AFCARS reports from 6 of the 8 states assessed by
HHS* Arkansas,
Connecticut, New Mexico, Texas, Vermont, and Wyoming. HHS conducted
reviews in Delaware and West Virginia after we completed our analysis.
17 In commenting on a draft of this report, ACF said that the finding from
the AFCARS reviews indicates that information is often defaulted to the
response *unable to determine* in order for the element not to fail the
missing data standard, not that workers are
recording *unknown*; however, the report findings we used in this analysis
instruct states to fix the defaults and address caseworker practice by
enhancing training on the correct use of *unable to determine* when noting
a child*s race.
18 See U. S. General Accounting Office, Child Welfare: HHS Could Play a
Greater Role in Helping Child Welfare Agencies Recruit and Retain Staff,
GAO- 03- 357 (Washington, D. C.: Mar. 31, 2003).
19 Department of Health and Human Services, Office of Inspector General,
Adoption and Foster Care Analysis and Reporting System (AFCARS):
Challenges and Limitations, OEI- 07- 01- 00660 (Washington, D. C.: Mar.
2003).
Page 28 GAO- 03- 809 Child Welfare
In addition to data quality being affected by caseworker issues, many
states experienced technical challenges reporting their data to HHS. The
problems reported by states are typically a result of challenges
associated with data *mapping** matching state data elements to the
federal data elements. For example, 36 states reported in our survey that
matching their state- defined data to HHS*s definitions affected the
quality of the data reported to NCANDS and AFCARS. Similarly, 24 states
reported that
matching the more detailed data options available in their states*
information systems to the federal data elements affected the quality of
the data reported to NCANDS. Twenty- nine states reported that this issue
created challenges in reporting data to AFCARS. For example, following an
AFCARS assessment, HHS instructed a state that collects detailed
information on children*s disabilities, such as Downs Syndrome, Attention
Deficit Disorder, and eating disorders, to map the information to the more
limited options in AFCARS, such as mental retardation and emotionally
disturbed. The Inspector General*s report found that states faced similar
challenges mapping their data to meet the AFCARS reporting requirements.
In many cases, states have to balance state policy with federal
requirements to ensure that they are reporting accurate data to AFCARS and
NCANDS, but are not contradicting their state policies. For example, Texas
officials reported that although the findings of their AFCARS review
instructed them to modify their SACWIS to collect, map, and extract data
on guardianship placements, the state does not support guardianship
arrangements. 20 In addition, a recent report from the Child Welfare
League of America (CWLA) found that when reporting the number of times
children move from one foster care placement to another, states varied in
the type of placements included in that count. 21 For example, 29 percent
of the states responding to CWLA*s survey included respite, 22 25 percent
included runaways, and 16 percent included trial home visits when
reporting the number of placements a child had during the AFCARS report
period. According to federal guidance, the *number of placements* element
is meant to gather information on the number of times the child
20 Guardianship arrangements occur when permanent legal custody of a child
is awarded to an individual, such as a relative, but the child is not
legally adopted. 21 Child Welfare League of America. National Working
Group Highlights, *Placement Stability Measure and Diverse Out- of- Home
Care Populations* (Washington, D. C., Apr. 2002).
22 Respite care provides temporary childcare for children away from their
caretakers. Technical Challenges, such
as Matching State Definitions to Federal Definitions, Affect Data
Reliability
Page 29 GAO- 03- 809 Child Welfare
welfare agency found it necessary to move a child while in foster care and
that by including runaways or trial home visits, a state is inflating the
number of moves a child experienced. However, North Carolina officials
told us that although the federal definition for placements instructs
states
not to include such stays when counting the number of children*s foster
care placements, the state instructs them to count each time a child is
sleeping in a different place as a new placement. The Inspector General
reported that the placement definitions were the most commonly cited
source of confusion among the states surveyed. 23 In addition to the
challenges reported in our survey, HHS reported that
transferring data from older data systems into SACWIS affects the quality
of the data reported to AFCARS and NCANDS. HHS officials reported that
they have observed that states experience the biggest change in data
quality when they begin reporting from their SACWIS. In general, the first
data submissions are of low quality because of the time it takes states to
transfer data or the system re- sets the information for data elements.
For example, in 1 state, 65 percent of the records reviewed by HHS during
an AFCARS assessment recorded the date the children were removed from
their homes as July 28, 1997* the date the SACWIS came on- line; however,
the actual dates of removal for these children ranged from 1988 to 1997.
HHS provides technical assistance for AFCARS and NCANDS reporting through
a number of resources. HHS officials in the central office and NCANDS
contractor staff serve as the points of contact for states to ask
questions and seek guidance on reporting child welfare data. HHS officials
reported that assistance is offered in a number of ways, including
telephone and e- mail communication. The officials in 3 of the 5 states
that we visited said that the one- on- one focused technical assistance
was useful when provided in a timely fashion. Most state officials found
the NCANDS data easier to report, in part because more people were
available for consultation and they were more accessible and responsive.
For example, states have access to four NCANDS specialists and staff in
the contractor*s central office when they need assistance reporting child
abuse and neglect information. However, some of the states we visited
23 Although the findings from the Inspector General*s report and our study
are more recent, ACF commented on a draft of this report that it issued
policy clarifications regarding placement information on July 5, 2002.
Although HHS Has Taken
Steps to Help States Improve Their Data, Some Problems with Its Efforts
Exist
Page 30 GAO- 03- 809 Child Welfare
reported that only one or two staff in HHS*s central office are available
to assist with AFCARS reporting.
In addition, the Resource Center offers states assistance with improving
data quality; however, Resource Center staff reported that the assistance
is geared more towards improving the limited data used in the federal
review process to monitor states* compliance with child welfare laws and
federal
outcome measures* CFSR* rather than all the data reported to HHS. The
Resource Center also sponsors an annual information technology conference
during which sessions covering all data- related issues are held,
including practices for ensuring data quality and outcome evaluation in
child welfare.
In conjunction with the national data conference, the HHS officials and
the contractors that operate NCANDS hold an annual technical assistance
meeting for states to share ideas with one another, discuss data elements
that pose difficulties, and explore ways to address these problems. For
example, at a recent technical assistance meeting, approximately 43 state
representatives attended sessions on preparing the calendar year 2002
NCANDS data submissions and received a detailed explanation of how the
NCANDS staff test states* data submissions for quality. In addition, an
NCANDS state advisory group meets annually to talk with HHS officials
about NCANDS data and their experiences reporting data. From these
meetings, the state advisory group proposes changes or improvements to
NCANDS. HHS and state officials reported that this partnership has helped
ease some of the challenges in reporting child abuse and neglect data.
In addition to the direct assistance through consultation with HHS
officials and the Resource Center, HHS has made available to states the
software it uses to examine states* AFCARS and NCANDS submissions for
inconsistencies and invalid data. Officials in all the states we visited
said that they regularly use this software, and an HHS official said that
nearly every state has used the software at least once. When the data are
submitted to HHS, they are run through the same software, and HHS notifies
the states of areas where data are missing or inconsistent and allows the
states to resubmit the data after errors are corrected. For example, HHS
officials said that they worked with one state that was trying to
determine the source of data errors in reporting to AFCARS the race or
ethnicity of children in their care. The state was not able to
determine the source of the problem, so an HHS official examined the
state*s submissions and helped correct the data errors. The officials
reported that these tests help them to identify some data quality errors,
such as missing data, and said that they believe that, in general, data
have
Page 31 GAO- 03- 809 Child Welfare
improved in recent years. However, they indicated that the tests cannot
pinpoint the underlying problems contributing to these errors.
Furthermore, one official reported that no specific efforts have been
conducted to track the individual data elements and, therefore, HHS cannot
report on how data quality has changed over time. The results of these
quality tests had been the basis for penalties levied against states that
submitted low quality AFCARS data before the penalties were rescinded. HHS
officials reported that the penalties served as an effective motivation to
states to correct their data. Although HHS was not able to report how the
lack of penalties might be affecting recent data quality, an official
reported that the agency plans to conduct this analysis in the
future. In an attempt to help states comply with the reporting standards
and address some of the factors that contribute to data quality problems,
HHS performs comprehensive reviews of state information systems* ability
to capture AFCARS data to identify problems associated with data
collection and reporting, and to ensure that the information in the
automated system correctly reflects children*s experiences in care. The
assessments include a technical review of the states* computer code, a
comparison of the data from selected cases available in the information
system to the case files,
and an improvement plan to resolve any errors. In addition, HHS officials
offer guidance to the states on improvements that can be made to the
information system and changes to program code used to report the AFCARS
data. HHS conducted pilot reviews in eight states between 1996 and 2000.
By March 2003, HHS had conducted eight official reviews* even though
states began reporting to AFCARS in 1995. According to results from six of
the eight official AFCARS assessments we reviewed, no state met the
reporting requirements for all AFCARS data elements. Table 8 shows a
selection of the data elements and the states* ratings. The problems noted
in the reviews are similar to those we heard from states responding to our
survey and those we visited. For example, most states
Page 32 GAO- 03- 809 Child Welfare
received ratings of 2 or 3, indicating technical and/ or data entry errors
that affect the AFCARS data quality. 24 Table 8: Selected AFCARS Elements
and Six States* Levels of Compliance in Meeting Reporting Requirements
Data Elements
Rating factor a (# of states) Foster Care Child race
(American Indian or Alaska native, Asian, Black or African American,
Native Hawaiian or other Pacific Islander, White, unable to determine) 2
(4 states)
3 (1 state) 4 (1 state)
Has the child been diagnosed with a disability? (yes, no, not yet
determined) 2 (6 states)
Mental retardation, visually/ hearing impaired, physically disabled,
emotionally disturbed, other diagnosed condition b (does not apply,
applies)
2 (5 states) 3 (1 state)
Has child ever been adopted? (yes, no, unable to determine) 1 (1 state) 2
(3 states) 3 (2 states)
Total number of removals from home c 2 (3 states) 3 (2 states) 4 (1 state)
Date of discharge from previous episode c 2 (3 states) 3 (2 states) 4 (1
state)
Date of latest removal c 2 (3 states) 3 (3 states)
Date of placement in current setting 2 (5 states) 4 (1 state)
Number of previous placement settings in this episode c 2 (6 states)
Current placement setting (pre- adoptive home, foster family home-
relative, foster family home- nonrelative, group home, institution,
supervised independent living, runaway, trial home visit)
2 (4 states) 3 (1 state) 4 (1 state)
Out of state placement (yes, no) 2 (4 states) 3 (1 state) 4 (1 state)
24 A rating of 2 indicates that the state*s information system does not
fully meet the requirements for AFCARS reporting, whereas if a state
receives a 3, the information system requirements for AFCARS reporting are
in place, but there are data entry problems affecting the quality of data.
According to an HHS official, data elements that have a combination of
technical and data entry problems are rated as 2 until the technical
issues
are resolved. HHS will then rate the element as a 3 until the data entry
practices are changed. A state receives a rating of 4 if in compliance
with the AFCARS requirements or a 1 if the requirement is not addressed in
the state*s information system.
Page 33 GAO- 03- 809 Child Welfare
Data Elements Rating factor a
(# of states)
Most recent case plan goal (reunify with parents or principal care takers,
live with other relative( s), adoption, long term foster care,
emancipation, guardianship, case plan goal not yet established) 1 (1
state)
2 (5 states) Date of discharge c 2 (2 states)
3 (2 states) 4 (2 states)
Reason for discharge c (not applicable, reunification with parent( s) or
primary caretaker( s), living with other relative( s), adoption,
emancipation, guardianship, transfer to another agency, runaway, death of
child) 2 (4 states)
3 (1 state) 4 (1 state)
Adoption
Child race (American Indian or Alaska native, Asian, Black or African
American, Native Hawaiian or other Pacific Islander, White, unable to
determine)
2 (4 states) 3 (1 state) 4 (1 state)
Primary basis for determining special needs (not applicable; racial/
original background; age; membership in a sibling group; medical
conditions or mental, physical, or emotional disabilities; other)
2 (6 states) Mental retardation, physically disabled, emotionally
disturbed b (does not apply, applies) 2 (5 states)
3 (1 state) Visually/ hearing impaired (does not apply, applies) 2 (4
states)
3 (2 states) Other diagnosed condition (does not apply, applies) 2 (6
states)
Source: HHS. Note: Analysis based on results of AFCARS reviews in 6
states. States are rated on 66 foster care elements and 37 adoption
elements. a HHS rates each data element using a four- point scale: (1) the
AFCARS requirement( s) has not been
implemented in the information system; (2) the technical system
requirements for AFCARS reporting do not fully meet the standards; (3) the
technical system requirements for AFCARS reporting are in place, but there
are data entry problems affecting the quality of the data; (4) all of the
AFCARS requirements have been met. b States are rated on each category.
The data elements were combined for purposes of this analysis.
c AFCARS data element used in the CFSR.
State officials in these 6 states reported that they found the reviews
useful for improving their AFCARS data submissions. In particular, they
valued the thorough review by HHS officials of the computer code states
use to report the data. Some of these officials reported that if all
states were reviewed, the quality of data available in AFCARS would
improve tremendously. However, HHS officials reported that they are not
mandated to conduct the AFCARS reviews and that priority is placed on
other reviews, such as the CFSR and SACWIS reviews. In addition, officials
explained that the AFCARS reviews are not conducted in states developing
SACWIS until the systems are operational. HHS expects to
Page 34 GAO- 03- 809 Child Welfare
complete approximately four reviews each year depending on available
resources and has scheduled states through 2006. Similar to the SACWIS
reviews, HHS officials offer recommendations and technical assistance to
states during the review on how they can improve the quality of the data
reported to AFCARS.
Although the states we visited appreciated some of HHS*s efforts to assist
with improving state data quality, they and most states responding to our
survey agreed that the assistance is not always consistent or easily
accessible (see fig. 2). States reported similar information to the
Inspector General* AFCARS data elements were not clearly and consistently
defined and technical assistance is effective but difficult to access.
Page 35 GAO- 03- 809 Child Welfare
Figure 2: Federal Practices That Affect Data Quality
Notes: Based on responses from 50 states. The results reported in the
figure are a sum of the states that reported the issue had a very great
affect, great affect, moderate affect, or some affect on the quality of
state data submitted to HHS. Very great and great affect responses are
represented in the top section of each bar. Moderate and some affect
responses are represented in the bottom section of each bar. States not
included answered *no affect,* *don*t know,* or *no answer.*
The primary concerns reported by the states we visited were delays in
receiving clear, written guidance on defining and reporting certain data
elements and the lack of state input in suggesting changes to AFCARS.
Despite the written guidance available to states in the form of
regulations and an on- line policy manual, states reported that the
variation in state
policies and practices make it difficult to interpret how to apply the
general guidance. As a result, states consult with HHS to ensure they are
applying the regulations appropriately. However, in commenting on a draft
of this report, officials in Oklahoma told us that a common concern among
the states is the lack of timely response from HHS when seeking guidance
on how to report data. In addition, officials in New York explained they
0 10
20 30
40 50
AFCARS NCANDS
States reporting at least some affect in data quality
Source: GAO survey.
Imprecise definitions for required fields
Lack of clear, documented
guidance Difficulty
accessing technical assistance and guidance
45 35
41 25
35 17 17
28 9
26 11
30 5
20 6
29 4 13
Page 36 GAO- 03- 809 Child Welfare
have made it a practice to check the HHS Web site on a regular basis for
current guidance, but have not found it a useful tool, and may turn to
other states for guidance on AFCARS reporting. In commenting on a draft of
this report, HHS explained that it first refers states to its Web site for
information and believes that the available guidance addresses states*
concerns in most instances. In addition, the states that have had an
AFCARS review experienced delays in obtaining guidance on how to proceed
following the on- site review. Although they found the review to be very
helpful, some states reported that HHS officials are delayed in responding
to their questions. For example, Texas officials reported that the state
sought clarification on its improvement plan and submitted additional
questions to HHS following the review; however, when we spoke with the
state officials, they said that they had been waiting 3 months for a
response on how to proceed. An HHS official told us that since the review
process is relatively new, the agency is still developing a process to
respond to the states and recognizes that it has not been responsive to
the states already reviewed. In addition, HHS is taking steps to gather
feedback from states and other users of AFCARS data to determine how to
improve the system to make the data more accurate and useable. As a part
of these efforts, HHS has published a Federal Register notice soliciting
comments and held focus group meetings at national
conferences. The difficulties states face in receiving federal guidance
and assistance, as well as the other challenges they face in reporting
data, may negatively impact the reliability of the data available in
AFCARS and NCANDS. As a result, states are concerned that the national
standards used in the CFSR are based on unreliable data and should not be
used as a basis of comparison and potential financial penalty. The
variation in states* reporting practices may affect the validity of the
measures and may place some states at a disadvantage. For example, the
CWLA and Inspector
General studies found that approximately half the states include the
juvenile justice population in their AFCARS reports, while the other
states
Page 37 GAO- 03- 809 Child Welfare
do not. 25 Child welfare experts and some state officials believe that the
states that include children served by the juvenile justice agency in
their AFCARS report may report a higher number of re- entries into the
child welfare system or a higher number of moves within the system when
compared to states that do not have IV- E agreements 26 with their
juvenile justice systems. 27 As a result, a state that includes such
children in their AFCARS report are likely to fare less favorably when
compared to the national standard than other states on two outcome
measures* foster care re- entries and stability of foster care placements*
and may face financial penalties associated with the CFSR.
25 Federal guidance states that children who are (1) removed from their
home and initially placed in a juvenile justice facility are not to be
included in AFCARS reporting; (2) in a foster care setting who are moved
to a juvenile justice facility and who are expected to be
returned to a foster care setting should continue to be included in the
AFCARS reporting population; and (3) in a foster care setting and are
moved to a juvenile justice facility and who become the responsibility of
another agency should not be included in the AFCARS reporting population.
However, these studies show that the arrangements between child welfare
and juvenile justice agencies vary, which affect the population of
children reported to AFCARS. For example, some state child welfare
agencies have responsibility for all
children in the juvenile justice system and include these children in
their AFCARS reports, while other states only report children who are in
the custody of the juvenile justice system, but receiving Title IV- E
funding. 26 According to HHS, state child welfare agencies can claim Title
IV- E foster care maintenance funds for eligible children for costs
incurred by juvenile justice agencies that have entered into an agreement
with the child welfare agencies. Delinquent children served by these
agencies who meet the Title IV- E eligibility criteria are eligible for
Title IV- E foster
care maintenance funds. States must meet all Titles IV- B and IV- E
program and/ or eligibility requirements with respect to the children who
are adjudicated delinquent and are receiving Title IV- E funding or for
children in foster care under the placement and care responsibility of the
child welfare agency.
27 Children in the juvenile justice system may enter care because of
behavior problems and return home when the behavior is controlled.
However, the problem may arise again and the child could re- enter state
custody. Also, the children may experience a greater number of placements
while in foster care because of their behavior problems or a *step- down*
approach that some states use to gradually decrease the level of security
required to care for the children.
Page 38 GAO- 03- 809 Child Welfare
Some states are using a variety of practices to address the challenges
associated with developing SACWIS and improving data reliability, although
no formal evaluations are available on their effectiveness. To address the
challenge of developing a system to meet statewide needs, states relied on
caseworkers and supervisors from local offices to assist in the design and
testing of the system. Few states reported in our survey strategies to
overcome the other key challenges, such as limited funding and securing
knowledgeable contractors, but some states we visited have devised some
useful approaches. For example, Oklahoma child welfare officials* in order
to maximize the limited state funding for maintaining their SACWIS*
reported saving $1 million each year by hiring some of the contractors who
developed their SACWIS as permanent staff. To improve data reliability,
the 5 states we visited routinely review their data to identify data entry
errors so that managers can ensure that the missing data are entered
appropriately. In addition, some states reported that frequent use of the
data, such as publishing periodic management reports detailing local
offices* performance on outcome measures, helps caseworkers understand the
importance of entering timely information.
To overcome development challenges, survey respondents emphasized the
importance of including system users in the various phases of completing
SACWIS* planning, design, development, testing, and implementation. Past
GAO work and other research efforts have determined similar approaches as
best practices in building information systems. 28 Forty- four of the 46
states responding to our survey that they are developing or operating a
SACWIS indicated that they relied on internal users, such as caseworkers
and supervisors, in the development of their systems and 34 of these
states said that they were extremely helpful participants. The extent to
which the users were involved in development differed across the states.
For example, in Texas, caseworkers from all of their child welfare regions
were recruited to provide input on design and development, as well as
during initial testing, pilot testing, and implementation of the system.
Arkansas reported establishing a committee made up of users to review the
work plan and sign off on recommended changes. In addition, states
reported that their system users served a
28 See U. S. General Accounting Office, Executive Guide: Improving Mission
Performance Through Strategic Information Management and Technology, GAO/
AIMD- 94- 115 (Washington, D. C.: May 1, 1994); Center for Technology in
Government, University of
Albany, SUNY. Tying a Sensible Knot: A Practical Guide to State- Local
Information Systems. Albany, N. Y., June 1997. States Are Using
Various Practices to Overcome System Development Challenges and Improve
Data on Children*s Experiences
States Are Primarily Relying on SACWIS Users to Overcome Some of the
Challenges to Completing Their Systems
Page 39 GAO- 03- 809 Child Welfare
number of purposes, including serving as experts on the different
specialties within child welfare, such as child abuse, foster care, or
adoption, and as representatives from local or county offices to assist in
identifying the diverse approaches to capturing information across the
state. For example, Indiana reported that caseworkers involved in SACWIS
development represented the unique needs of the different geographical
areas of the state and helped design a uniform statewide system to meet
the diverse needs of large, intermediate, and small local offices.
Ten states noted that user input should not be limited to frontline
workers, such as caseworkers, but should include representatives from
other areas of the agency, such as the financial staff, and other agencies
that serve children, such as child support enforcement. 29 Since many
SACWIS link with other state information systems, states advised that
developing a collaborative relationship with other state agencies will
help the development of the system. While not one of the most common
challenges reported in our survey, New Hampshire reported that one of its
challenges
with meeting its SACWIS timeframe was not working collaboratively with
other agencies, such as Temporary Assistance for Needy Families (TANF) 30
and child support enforcement, to develop the payment component of SACWIS.
Similarly, we previously reported that the difficulty developing linkages
between social services agencies limits the effectiveness of all the
programs to serve families. 31 To attempt to overcome this challenge, 26
of the 46 states responding to our survey that they are developing or
operating a SACWIS indicated that they included external public agency
users and 23 reported using representatives from other state agencies that
serve children in developing their SACWIS.
Indiana said that a task force made up of representatives from the TANF
and child support enforcement agencies was developed to design the
29 The Child Support Enforcement Program is a joint federal, state, and
local partnership that was established in 1975 under Title IV- D of the
Social Security Act. Each state runs a child support program, which
provides four major services: locating non- custodial parents,
establishing paternity, establishing child support obligations, and
collecting child support for families. 30 In 1996, the Congress created
the block grant Temporary Assistance for Needy Families
program replacing the Aid to Families with Dependent Children (AFDC) and
related welfare programs. States were given increased flexibility in
designing the eligibility criteria and benefit rules, which require work
in exchange for time- limited benefits.
31 See U. S. General Accounting Office, Welfare Reform: Improving State
Automated Systems Requires Coordinated Federal Effort, GAO/ HEHS- 00- 48
(Washington, D. C.: Apr. 27, 2000).
Page 40 GAO- 03- 809 Child Welfare
linkages between the systems. In addition, Colorado officials reported
that they are working with the Department of Youth Corrections* an agency
that shares the SACWIS with child welfare* to ensure that the shared
screens use the same definitions. In addition to seeking input from
caseworkers and other system users while developing SACWIS, many states
continue to include users as a part of the implementation teams, to serve
as contacts in the field and provide ongoing assistance, and to provide
input on system enhancements. Alabama responded in our survey that the
state had *mentors* in each county to help caseworkers adjust to the new
system. These mentors continue to provide ongoing support now that the
system is implemented. Similarly, Oklahoma developed Field Implementation
Teams consisting of one contractor and one child welfare staff person.
During system
implementation, the teams went to field offices to provide on- site
assistance with using SACWIS and becoming accustomed with the new method
of recording child welfare information. Furthermore, Oklahoma recruits
experienced child welfare field staff for its SACWIS help desk because of
their knowledge of the system and child welfare policy and practice.
Although states faced other challenges in completing their SACWIS, few
reported implementing approaches to overcome the barriers. According to
survey results, a common problem states faced in developing SACWIS was
receiving insufficient state funding for development. However, in our
previous work on managing information technology, we found that the IT
products can become obsolete in a matter of months rather than years,
calling for more frequent investments in upgrades and enhancements. 32 In
addition, officials in Iowa told us that maintaining systems takes just as
much money as building them. States did not report in our survey
approaches for obtaining more funding for developing SACWIS, and few
states reported developing strategies in an attempt to overcome the
challenges associated with tight budgets for maintaining their systems.
For example, Iowa officials engaged in careful planning with system users
to ensure that they addressed the highest priorities when enhancing the
system. In particular, the officials reported that maintaining tight
control over the development and maintenance processes helps them avoid
investing inordinate amounts of resources to make corrections to the 32
See U. S. General Accounting Office, Managing Technology: Best Practices
Can Improve Performance and Produce Results, GAO/ T- AIMD- 97- 38
(Washington, D. C.: Jan. 31, 1997).
Page 41 GAO- 03- 809 Child Welfare
system. In Oklahoma, child welfare officials reported that they relied on
the contractors who developed their SACWIS to conduct ongoing maintenance
activities until the contract expired in 2001. At that time, the agency
hired some of the contract staff as full- time state employees to
continue with the maintenance activities. State officials explained that
this approach ensured continuity of service, in addition to saving the
agency approximately $1 million each year. Similarly, few states reported
on approaches to overcome the challenge of finding contractors with
knowledge of child welfare practice. However, Iowa officials explained
that once the contract staff are hired, they are required to attend the
same training as new caseworkers to ensure that they are familiar with the
state*s child welfare policies and to familiarize themselves with casework
practices.
Twenty- eight states reported using approaches to help caseworkers
identify the data elements that are required for federal reporting and to
help them better understand the importance of entering timely and accurate
data. Ten states responding to our survey reported reviewing the federal
reporting requirements in training sessions as a promising approach they
use to improve data quality or as a lesson learned. For example, Tennessee
reported that the state added a component about AFCARS to the initial and
ongoing training workers receive about using SACWIS. The curriculum
addresses the AFCARS report in general and the individual data elements to
help the caseworkers better understand the purpose of collecting the
information. In Nebraska, a *desk aid* that explains the data elements and
where and why to enter them in the system is available on the caseworkers*
computer desktops. In addition, New York
has developed a step- by- step guide explaining to workers how NCANDS data
should be entered, with references to the policy or statute requiring the
information.
To improve data reliability, some states have designed their information
systems with special features to encourage caseworkers to enter the
information. Four states responding to our survey and 3 states we visited
designed their SACWIS with color- coded fields to draw attention to the
data elements that caseworker are required to enter. For example, the
AFCARS data fields in Oklahoma*s system are coded red until the data are
entered, after which the fields change to blue. In addition, workers can
look at a single screen in the Oklahoma system to see what AFCARS data
elements need to be completed without having to scroll through the entire
case record. Colorado, Iowa, New York, and Oklahoma have built into their
systems alerts* also known as *ticklers** to remind caseworkers States Use
Strategies, such
as Producing Reports That Identify Missing Data, in an Attempt to Improve
the Reliability of the Data Reported to HHS
Page 42 GAO- 03- 809 Child Welfare
and supervisors of tasks that they need to complete. For example, in Iowa,
alerts are sent to supervisors if a caseworker fails to enter the data
necessary to complete a payment to a foster care provider. Whereas, in
Oklahoma, a stoplight icon on the caseworker*s computer desktop reminds
the worker when tasks are due. A green light indicates that nothing is due
within 5 days; a yellow light means that something is due within 5 days;
and a red light means that something is overdue. Caseworkers and
supervisors in the states we visited had mixed responses about the
usefulness and effectiveness of the alerts. Some caseworkers found them to
be a nuisance, while other caseworkers and supervisors found them to be
useful tools in managing workloads and prioritizing daily tasks.
Six states reported that the best way to improve data quality was to use
the data in published reports and hold the caseworkers and supervisors
accountable for the outcomes of the children in their care. In addition, 6
states responding to our survey reported using the data available in their
information systems to measure state outcomes similar to the CFSR. State
officials reported that this approach is an effective way to get local
offices invested in the quality of the data. For example, North Carolina
publishes monthly reports for each county comparing their performance on
state data indicators, such as the length of time children spend in care,
to counties of similar size and the state as a whole. County officials
reported that these reports encourage workers to improve the quality of
the data collected and entered into the state system since their
performance is
being widely published and compared to other counties. In addition, all
the states we visited reported that frequent review of their data, such as
using software from HHS to test their AFCARS and NCANDS data to pin- point
data entry errors prior to submitting them to HHS, has
helped improve data quality. When the states identify poor data, they
alert the caseworkers and supervisors of needed corrections and data entry
improvements. For example, Colorado runs these reports about 4 to 5 times
a year, with one run occurring approximately 6 weeks before each AFCARS
submission. When the data specialists find errors, they notify the
caseworker to clean up the data. New York officials told us that they
incorporate the results from these tests in training if a consistent
pattern of errors is identified.
While most states are developing statewide information systems, challenges
with data reliability remain. Although SACWIS development is delayed in
many states, state officials recognize the benefits of having a Conclusion
Page 43 GAO- 03- 809 Child Welfare
uniform system that enhances the states* ability to monitor the services
provided and the outcomes for children in their care. Although states
began reporting to NCANDS in1990 and were mandated to begin reporting to
AFCARS in 1995, most states continue to face challenges providing
complete, accurate, and consistent data to HHS. In addition, the results
of more recent HHS efforts, such as conducting AFCARS- related focus
groups, are unknown. Reliable data are essential to the federal
government*s development of policies that address the needs of the
children served by state child welfare agencies and its ability to assist
states in improving child welfare system deficiencies. Without
welldocumented, clearer guidance and the completion of more comprehensive
reviews of states* AFCARS reporting capabilities, states are limited in
overcoming challenges that affect data reliability. Because these
challenges still remain, HHS may be using some questionable data as the
foundation for national reports and national standards for the CFSR and
may not have a clear picture of how states meet the needs of children in
their care.
To improve the reliability of state- reported child welfare data, we are
recommending that the Secretary of HHS consider, in addition to HHS*s
recent efforts to improve AFCARS data, ways to enhance the guidance and
assistance offered to states to help them overcome the key challenges in
collecting and reporting child welfare data. These efforts could include a
stronger emphasis placed on conducting AFCARS reviews and more timely
follow- up to help states implement their improvement plans or identifying
a useful method to provide clear and consistent guidance on AFCARS and
NCANDS reporting.
We obtained comments on a draft of this report from the Department of
Health and Human Services* Administration for Children and Families (ACF).
These comments are reproduced in appendix III. ACF also provided technical
clarifications, which we incorporated when appropriate. ACF generally
agreed with our findings and commented that the report
provides a useful perspective of the problems states face in collecting
data and of ACF*s effort to provide ongoing technical assistance to
improve the quality of child welfare data. In response to our
recommendation, ACF said that we categorized its efforts as *recent* and
did not recognize the long- term efforts to provide AFCARS and NCANDS
related guidance to the states. Although we did not discuss each effort in
depth, we do mention Recommendation to
the Secretary of
Health and Human Services
Agency Comments
Page 44 GAO- 03- 809 Child Welfare
the agency*s ongoing efforts in our report. However, we refer to the
recent efforts in the recommendation in recognition of the agency*s
current activities to formally obtain, document, and incorporate feedback
from the states with regard to collecting and reporting adoption and
foster care data. ACF also noted in its comments that the data definitions
need to be updated and revised and said it is currently in the process of
revising the
AFCARS regulations to further standardize the information states are to
report* which we acknowledge in our report. In addition to the steps HHS
is taking to further improve the AFCARS data, our recommendation
encourages HHS to consider ways to enhance the ongoing guidance and
assistance offered to states to help them overcome the key challenges in
collecting and reporting child welfare data. ACF requested specific
recommendations on approaches to overcome the difficulty of collecting and
merging information from multiple state and county programs into a single
national database. While there may be additional methodologies that the
agency could use to overcome such challenges, our recommendation focuses
on improving the guidance already offered to the states as a step to
helping them better comply with the reporting requirements.
In addition, ACF added that although staff turnover in state child welfare
agencies is a significant contributor to data quality issues, we did not
focus on this as a significant factor. ACF also commented that it is
firmly committed to continue to support the states and to provide
technical assistance and other guidance as its resources will permit.
However, because we recently issued a detailed report on a variety of
caseworker issues, 33 we primarily focused in this report on the key data
entry challenges caseworkers face and refer readers to our previous work
for additional information on challenges related to caseworker recruitment
and retention and their affect on child welfare agencies. In commenting on
our previous work prior to its release, HHS indicated that it does not
have the authority to require states to address factors that contribute to
staff turnover, such as high caseloads and said that it has limited
resources to assist the states in the area of staff recruitment and
retention. ACF commented that it provided increased funding to the
National
Resource Centers in fiscal year 2003, which they believe will improve 33
See U. S. General Accounting Office, Child Welfare: HHS Could Play a
Greater Role in Helping Child Welfare Agencies Recruit and Retain Staff.
GAO- 03- 357. (Washington, D. C.: Mar. 31, 2003).
Page 45 GAO- 03- 809 Child Welfare
ACF*s ability to provide assistance to the states. After receiving the
draft report for comment, HHS separately provided information on an
additional service the National Resource Center for Information Technology
in Child Welfare provides to states. In an effort to assist states with
improving the quality of their AFCARS data, the Resource Center will
review states* programming code used for AFCARS data. As of June 2003, HHS
reported that the Resource Center provided this assistance to Arkansas,
Louisiana, Mississippi, North Carolina, Nevada, New Jersey, and Rhode
Island, and 3 states* Maryland, Michigan, and Wisconsin* and the District
of Columbia have requested the assistance.
In response to our survey methodology, ACF requested that we explain why
the territory of Puerto Rico was not included in the state survey.
Although Puerto Rico receives federal child welfare funds, we
traditionally focus on the states and therefore do not include the U. S.
territories, including American Samoa, the Commonwealth of the Northern
Mariana Islands, Guam, Puerto Rico, and the Virgin Islands, in the scope
of our reviews.
Finally, in response to our discussion of the AFCARS review process, ACF
provided a few clarifications. During the course of our review, an HHS
official characterized the AFCARS review process as relatively new and
explained that the agency is still developing a process to respond to the
states following the completion of the on- site review. When responding to
a draft of this report, ACF disagreed with this characterization. ACF
commented that the review process has been in place since 1996, pointing
to the pilot reviews as evidence that the agency has a defined process.
However, when we requested AFCARS reports for review, HHS explained that
the states undergoing pilot reviews would be re- reviewed and that the
official process was formalized in 2001 with the release of an AFCARS
review guide and the start of the official reviews. In addition, ACF
commented that SACWIS reviews do not take priority over AFCARS reviews.
However, officials had previously explained that although SACWIS and
AFCARS reviews can happen at the same time, in practice, the AFCARS
reviews are scheduled to occur in the states that are developing SACWIS
after they have participated in a SACWIS review. Furthermore, ACF
explained that states do not develop their improvement plan following the
conclusion of the AFCARS review. Instead, ACF officials draft the plan for
the state. Although state representatives had described a challenge in
receiving timely feedback on their improvement plan, we have changed the
language in the report to reflect ACF*s comment.
Page 46 GAO- 03- 809 Child Welfare
We also provided a copy of our draft to child welfare officials in the 5
states we visited* Colorado, Iowa, North Carolina, New York, and Oklahoma.
Iowa and New York had technical clarifications, which we in incorporated
when appropriate. Oklahoma provided additional information, which was
incorporated. Colorado had no suggested corrections or edits. North
Carolina did not provide any comments.
As agreed with your offices, unless you publicly announce its contents
earlier, we plan no further distribution of this report until 30 days
after its issue date. At that time, we will send copies of this report to
the Secretary of Health and Human Services, state child welfare directors,
and other interested parties. We will make copies available to others on
request. In addition, the report will be available at no charge on GAO*s
Web site at http:// www. gao. gov. If you or your staff have any questions
or wish to discuss this material further, please call me at (202) 512-
8403 or Diana Pietrowiak at (202) 512- 6239. Key contributors to this
report are listed in appendix IV. Cornelia M. Ashby
Director, Education, Workforce, and Income Security Issues
Appendix I: Scope and Methodology Page 47 GAO- 03- 809 Child Welfare
To determine the progress states have made in developing Statewide
Automated Child Welfare Information Systems (SACWIS), we surveyed all 50
states and the District of Columbia through a Web- based survey. We
pretested the survey instrument in Maryland and the District of Columbia.
We received responses from 49 states and the District of Columbia. The
state of Nevada did not respond to the survey. We discarded a question
that asked states to report the date their advance planning document (APD)
was approved by the Department of Health and Human Services (HHS). Due to
a technical error, the date was truncated and a valid answer was not
stored in the responses. Of the 50 survey responses, 46 were from states
that are pursuing SACWIS development. The 4 states not developing SACWIS
were asked to skip sections of the survey that asked about
SACWIS development, system modifications, and supported services and
links. We did not independently verify the survey responses. In addition,
we visited 5 states to obtain more detailed and qualitative information
regarding states* experiences developing SACWIS. We conducted site visits
in Colorado, Iowa, New York, North Carolina, and Oklahoma. We selected
these states to represent a range of SACWIS development stages, sizes of
foster care populations, and geographic locations. During our site visits,
we interviewed state and local child welfare staff, state and local staff
that regularly exchange information with the child welfare agency, and
private contractors. We also spoke with HHS staff in the central and
regional offices, National Resource Center officials, contractors involved
in SACWIS development, and child welfare experts from the Child Welfare
League of America and the American Public Human Services Association.
To determine how states and HHS ensure reliable data exist on children
served by child welfare agencies we surveyed states using the
abovementioned survey instrument. In addition, we interviewed state and
HHS officials on their efforts to analyze and compile data and HHS*s role
in providing technical assistance to states. We spoke with state officials
during our site visits and HHS officials in the central and regional
offices and attended the 6th National Child Welfare Data Conference. We
obtained and reviewed available SACWIS and Adoption and Foster Care
Analysis and Reporting System (AFCARS) reports. At the time of our review,
HHS had conducted 26 SACWIS reviews. We obtained and reviewed 23 reports.
The remaining reports were not available for review because HHS has not
yet completed the report or shared the results with the state. Most of the
SACWIS reports were considered drafts since many states are in the process
of resolving issues with completing their systems. We reviewed AFCARS
assessment reports from 6 of the 8 states assessed by HHS* Arkansas,
Connecticut, New Mexico, Texas, Vermont, and Wyoming. HHS conducted
reviews in Delaware and West Virginia after we Appendix I: Scope and
Methodology
Appendix I: Scope and Methodology Page 48 GAO- 03- 809 Child Welfare
completed our analysis. We did not review any of the eight pilot review
reports since these were not final reports and HHS plans to conduct
official reviews in these states. These AFCARS assessment reports were
analyzed to understand the breadth of on- site assistance HHS provides to
states during the review and to identify common data collection and
reporting difficulties among states. Finally, we talked with officials in
6 of the 8 states that had an AFCARS review about their experiences during
the
review and child welfare experts. To identify practices state and local
child welfare agencies are using to help ensure the accuracy, timeliness,
and completeness of child welfare data we interviewed state and local
child welfare officials on our site visits and inquired about the
practices they are employing. We also included questions on practices and
lessons learned in our survey. In addition, we spoke with numerous child
welfare experts, including individuals from the National Resource Center
for Information Technology in Child Welfare, the Child Welfare League of
America, and the American Public Human Services Association. 1 1 The Child
Welfare League of America is an association of almost 1,200 public and
private
nonprofit agencies that assist abused and neglected children and their
families with a wide range of services. The American Public Human Services
Association is an organization of individuals and agencies concerned with
human services, including state and local human service agencies and
individuals who work in or otherwise have an interest in human service
programs.
Appendix II: State SACWIS Stages of Development
Page 49 GAO- 03- 809 Child Welfare
SACWIS stages of development Completed Partially operational
Arizona Alabama Massachusetts Florida Oklahoma Illinois Rhode Island
Mississippi West Virginia New York
Oregon
Operational South Carolina Arkansas Tennessee California Wisconsin
Colorado Connecticut Implementation
Delaware Alaska District of Columbia Maryland Idaho
Indiana Planning Iowa Georgia Kentucky Kansas Maine Louisiana Michigan
Missouri Minnesota New Jersey Montana Ohio Nebraska Pennsylvania Nevada
New Hampshire No SACWIS
New Mexico Hawaii South Dakota North Carolina Texas North Dakota
Utah Vermont Virginia Washington Wyoming Source: HHS. Note: Status is as
of May 22, 2003.
Appendix II: State SACWIS Stages of Development
Appendix III: Comments from the Department of Health and Human Services
Page 50 GAO- 03- 809 Child Welfare
Appendix III: Comments from the Department of Health and Human Services
Appendix III: Comments from the Department of Health and Human Services
Page 51 GAO- 03- 809 Child Welfare
Appendix III: Comments from the Department of Health and Human Services
Page 52 GAO- 03- 809 Child Welfare
Appendix III: Comments from the Department of Health and Human Services
Page 53 GAO- 03- 809 Child Welfare
Appendix III: Comments from the Department of Health and Human Services
Page 54 GAO- 03- 809 Child Welfare
Appendix III: Comments from the Department of Health and Human Services
Page 55 GAO- 03- 809 Child Welfare
Appendix III: Comments from the Department of Health and Human Services
Page 56 GAO- 03- 809 Child Welfare
Appendix III: Comments from the Department of Health and Human Services
Page 57 GAO- 03- 809 Child Welfare
Appendix III: Comments from the Department of Health and Human Services
Page 58 GAO- 03- 809 Child Welfare
Appendix III: Comments from the Department of Health and Human Services
Page 59 GAO- 03- 809 Child Welfare
Appendix III: Comments from the Department of Health and Human Services
Page 60 GAO- 03- 809 Child Welfare
Appendix III: Comments from the Department of Health and Human Services
Page 61 GAO- 03- 809 Child Welfare
Appendix IV: GAO Contacts and Acknowledgments
Page 62 GAO- 03- 809 Child Welfare
Diana Pietrowiak, (202) 512- 6239 Sara L. Schibanoff, (202) 512- 4176
In addition to those named above, Leah DeWolf and Rachel Seid made key
contributions to this report. Avrum Ashery, Patrick DiBattista, Barbara
Johnson, Valerie Melvin, and Rebecca Shea also provided key technical
assistance. Appendix IV: GAO Contacts and
Acknowledgments GAO Contacts Staff Acknowledgments
Bibliography Page 63 GAO- 03- 809 Child Welfare
The American Public Welfare Association. Statewide Automated Child Welfare
Information Systems: Survey of State Progress. Washington, D. C., July
1997.
The American Public Welfare Association. Child Welfare Information
Systems: Some Concepts and Their Implications. Washington, D. C., July
1994.
The American Public Welfare Association. Survey of State Child Welfare
Information Systems: Status of AFCARS and SACWIS. Washington, D. C., April
1995.
Caliber Associates, Analysis of State Child Welfare Data: VCIS Survey Data
from 1990 through 1994, May 1998, Department of Health and Human Services.
Center for Technology in Government, University of Albany, SUNY. Tying a
Sensible Knot: A Practical Guide to State- Local Information Systems.
Albany, N. Y., June 1997. Child Welfare League of America. National
Working Group Highlights,
*Child Maltreatment in Foster Care: Understanding the Data.* Washington,
D. C., October 2002.
Child Welfare League of America. National Working Group Highlights,
*Placement Stability Measure and Diverse Out- of- Home Care Populations.*
Washington, D. C., April 2002.
U. S. Department of Health and Human Services, Administration for Children
and Families, Administration on Children, Youth and Families, Children*s
Bureau. Child Maltreatment 2001. Washington, D. C., 2003.
U. S. Department of Health and Human Services, Administration for Children
and Families, Administration on Children, Youth and Families, Children*s
Bureau. Child Welfare Outcomes 1999: Annual Report.
Washington, D. C., n. d.. U. S. Department of Health and Human Services,
Office of Inspector General. Adoption and Foster Care Analysis and
Reporting System (AFCARS): Challenges and Limitations. Washington, D. C.,
March 2003. Bibliography
Related GAO Products Page 64 GAO- 03- 809 Child Welfare
Child Welfare and Juvenile Justice: Federal Agencies Could Play a Stronger
Role in Helping States Reduce the Number of Children Placed Solely to
Obtain Mental Health Services. GAO- 03- 397. Washington, D. C.: April 21,
2003.
Child Welfare: HHS Could Play a Greater Role in Helping Child Welfare
Agencies Recruit and Retain Staff. GAO- 03- 357. Washington, D. C.: March
31, 2003.
Human Services: Federal Approval and Funding Processes for States*
Information Systems. GAO- 02- 347T. Washington, D. C.: July 9, 2002.
Foster Care: Recent Legislation Helps States Focus on Finding Permanent
Homes for Children, but Long- Standing Barriers Remain.
GAO- 02- 585. Washington, D. C.: June 28, 2002.
Human Services Integration: Results of a GAO Cosponsored Conference on
Modernizing Information Systems. GAO- 02- 121. Washington, D. C.: January
31, 2002.
District of Columbia Child Welfare: Long- Term Challenges to Ensuring
Children*s Well- Being. GAO- 01- 191. Washington, D. C.: December 29,
2000.
Child Welfare: New Financing and Service Strategies Hold Promise, but
Effects Unknown. GAO/ T- HEHS- 00- 158. Washington, D. C.: July 20, 2000.
Welfare Reform: Improving State Automated Systems Requires
Coordinated Federal Effort. GAO/ HEHS- 00- 48. Washington, D. C.: April
27, 2000.
Foster Care: States* Early Experiences Implementing the Adoption and Safe
Families Act. GAO/ HEHS- 00- 1. Washington, D. C.: December 22, 1999.
Foster Care: HHS Could Better Facilitate the Interjurisdictional Adoption
Process. GAO/ HEHS- 00- 12. Washington, D. C.: November 19, 1999.
Foster Care: Effectiveness of Independent Living Services Unknown.
GAO/ HEHS- 00- 13. Washington, D. C.: November 5, 1999.
Foster Care: Kinship Care Quality and Permanency Issues. GAO/ HEHS99- 32.
Washington, D. C.: May 6, 1999. Related GAO Products
Related GAO Products Page 65 GAO- 03- 809 Child Welfare
Juvenile Courts: Reforms Aim to Better Serve Maltreated Children.
GAO/ HEHS- 99- 13. Washington, D. C.: January 11, 1999.
Child Welfare: Early Experiences Implementing a Managed Care Approach.
GAO/ HEHS- 99- 8. Washington, D. C.: October 21, 1998.
Foster Care: Agencies Face Challenges Securing Stable Homes for Children
of Substance Abusers. GAO/ HEHS- 98- 182. Washington, D. C.: September 30,
1998.
Managing Technology: Best Practices Can Improve Performance and Produce
Results. GAO/ T- AIMD- 97- 38, January 31, 1997.
Child Welfare: HHS Begins to Assume Leadership to Implement National and
State Systems. GAO/ AIMD- 94- 37. Washington, D. C.: June 8, 1994.
Executive Guide: Improving Mission Performance Through Strategic
Information Management and Technology. GAO/ AIMD- 94- 115. May 1, 1994.
(130163)
The General Accounting Office, the audit, evaluation and investigative arm
of Congress, exists to support Congress in meeting its constitutional
responsibilities and to help improve the performance and accountability of
the federal government for the American people. GAO examines the use of
public funds; evaluates federal programs and policies; and provides
analyses, recommendations, and other assistance to help Congress make
informed oversight, policy, and funding decisions. GAO*s commitment to
good government is reflected in its core values of accountability,
integrity, and reliability.
The fastest and easiest way to obtain copies of GAO documents at no cost
is through the Internet. GAO*s Web site (www. gao. gov) contains abstracts
and fulltext files of current reports and testimony and an expanding
archive of older products. The Web site features a search engine to help
you locate documents using key words and phrases. You can print these
documents in their entirety, including charts and other graphics.
Each day, GAO issues a list of newly released reports, testimony, and
correspondence. GAO posts this list, known as *Today*s Reports,* on its
Web site daily. The list contains links to the full- text document files.
To have GAO e- mail
this list to you every afternoon, go to www. gao. gov and select
*Subscribe to daily E- mail alert for newly released products* under the
GAO Reports heading.
The first copy of each printed report is free. Additional copies are $2
each. A check or money order should be made out to the Superintendent of
Documents. GAO also accepts VISA and Mastercard. Orders for 100 or more
copies mailed to a single address are discounted 25 percent. Orders should
be sent to: U. S. General Accounting Office 441 G Street NW, Room LM
Washington, D. C. 20548 To order by Phone: Voice: (202) 512- 6000
TDD: (202) 512- 2537 Fax: (202) 512- 6061
Contact: Web site: www. gao. gov/ fraudnet/ fraudnet. htm E- mail:
fraudnet@ gao. gov Automated answering system: (800) 424- 5454 or (202)
512- 7470 Jeff Nelligan, Managing Director, NelliganJ@ gao. gov (202) 512-
4800
U. S. General Accounting Office, 441 G Street NW, Room 7149 Washington, D.
C. 20548 GAO*s Mission Obtaining Copies of
GAO Reports and Testimony
Order by Mail or Phone To Report Fraud, Waste, and Abuse in Federal
Programs Public Affairs
*** End of document. ***