Food Assistance: Research Provides Limited Information on the
Effectiveness of Specific WIC Nutrition Services (30-MAR-01,
GAO-01-442).
Demonstration studies, despite some methodological limitations,
provide program managers and policymakers with some useful
information about the types of Special Supplemental Program for
Women, Infants and Children (WIC) nutrition service interventions
that can have positive impacts on participants. However, only one
recent demonstration study provides any information on the costs
associated with implementing various interventions. Given the
limited resources available to provide WIC nutrition services,
information about the costs to provide effective services could
play a critical role in managers' decisions to implement the
intervention and policymakers' decisions about funding the
intervention.
-------------------------Indexing Terms-------------------------
REPORTNUM: GAO-01-442
ACCNO: A00703
TITLE: Food Assistance: Research Provides Limited Information on
the Effectiveness of Specific WIC Nutrition Services
DATE: 03/30/2001
SUBJECT: Food programs for children
Nutrition research
Program evaluation
State-administered programs
Statistical data
Statistical methods
Special Supplemental Nutrition Program
for Women, Infants and Children
WIC
******************************************************************
** This file contains an ASCII representation of the text of a **
** GAO Testimony. **
** **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced. Tables are included, but **
** may not resemble those in the printed version. **
** **
** Please see the PDF (Portable Document Format) file, when **
** available, for a complete electronic file of the printed **
** document's contents. **
** **
******************************************************************
GAO-01-442
A
Report to Congressional Committees
March 2001 FOOD ASSISTANCE Research Provides Limited Information on the
Effectiveness of Specific WIC Nutrition Services
GAO- 01- 442
Letter 3 Appendixes Appendix I: Scope and Methodology 20
Appendix II: Research Funding Sources 22 Appendix III: Bibliography of
Demonstration Studies 23 Appendix IV: Bibliography of Impact Studies 25
Appendix V: Demonstration Studies: Major Findings, Scope,
and Major Limitations 26 Appendix VI: GAO Contacts and Staff Acknowledgments
28
Table Table 1: Funding for Demonstration and Impact Studies 8
Abbreviations
FNS Department of Agriculture's Food and Nutrition Service WIC Special
Supplemental Nutrition Program for Woman, Infants and
Children
Lett er
March 30, 2001 The Honorable Richard G. Lugar Chairman The Honorable Tom
Harkin Ranking Member Committee on Agriculture, Nutrition, and Forestry
United States Senate
The Honorable John A. Boehner Chairman The Honorable George Miller Ranking
Democratic Member
Committee on Education and the Workforce House of Representatives
The Special Supplemental Nutrition Program for Women, Infants and Children
(WIC) is a federally funded nutrition assistance program administered by the
U. S. Department of Agriculture's (USDA) Food and Nutrition Service (FNS).
This program provides supplemental food and nutrition services to lower-
income pregnant, breastfeeding, and postpartum women. The program also
serves infants and children up to age 5 who are at nutritional risk. The WIC
program serves a monthly average of 7.3 million individuals, including about
47 percent of all infants born in the United States. In fiscal year 2000,
the Congress appropriated $3.9 billion to
fund WIC, $2. 8 billion of which was used to provide food benefits and $1. 1
billion of which was used for nutrition services and program administration.
The nutrition services part of the program includes nutrition education,
breastfeeding promotion and support, and referrals to health and social
services.
A substantial body of research has examined the overall effectiveness of the
WIC program, but little is known about what research shows regarding the
effectiveness of specific WIC nutrition services. This report (1) identifies
the number and nature of recent studies that have examined the effectiveness
of three WIC services- nutrition education, breastfeeding promotion and
support, and referral services- and (2) summarizes what the research shows
about the effectiveness of these specific nutrition
services. This report is the fourth in a series providing information on
various aspects of WIC nutrition services as directed by the William F.
Goodling Child Nutrition Reauthorization Act of 1998 (P. L. 105- 336). 1
To identify recent studies that examine the effectiveness of each WIC
nutrition service, we searched relevant databases, such as the National
Technical Information Service, Sociological Abstracts, and Wilson Social
Science Abstracts. We also consulted with USDA WIC program staff and
other program stakeholders including officials from the National Association
of WIC Directors. Initially, we identified more than 200 published WIC
studies dealing with various aspects of the WIC program. In order to focus
on recent research, we eliminated from consideration
research published prior to 1995. We also eliminated any study that did not
specifically address the effects of at least one of the three WIC nutrition
services, whether nutrition education, breastfeeding promotion and support,
or referral services. We included research that falls in two general
categories: demonstration studies and impact studies. The demonstration
studies typically attempt to assess whether special interventions, such as
hospital visits to promote breastfeeding, more effectively achieve program
objectives than the usual WIC services. In contrast, impact studies attempt
to determine whether a WIC nutrition service, such as breastfeeding
promotion and support, improves WIC participants' outcomes compared to
similar individuals who do not participate in the WIC program. We conducted
our work between May 2000 and March 2001 in accordance with generally
accepted government auditing standards. Appendix I provides a more detailed
description of the methodology we used to conduct this work.
Results in Brief We identified 19 studies published since 1995 that examine
the effectiveness of WIC nutrition services. Twelve of the 19 are
demonstration
studies; seven are impact studies. Most of the studies- seven of the
demonstration studies and four of the impact studies- focused on
breastfeeding promotion and support. 1 Food Assistance: Financial
Information on WIC Nutrition Services and Administrative Costs (GAO/ RCED-
00- 66, Mar. 6, 2000); Food Assistance: Activities and Use of Nonprogram
Resources at Six WIC Agencies (GAO/ RCED- 00- 202, Sept. 29, 2000); Food
Assistance: Performance Measures for Assessing Three WIC Services (GAO/ 01-
339, Feb. 28, 2001).
Despite some methodological limitations, the results of the 12 demonstration
studies suggest that certain types of nutrition service interventions, such
as providing breastfeeding support services in the hospital after delivery,
have the potential to be more effective than theusual WIC interventions. Our
analysis of the demonstration studies suggests that the more effective
strategies may cost more than usual WIC approaches.
However, only one of the studies specifically addressed the potential costs
of new interventions. The results of the seven impact studies provide few,
if any, insights into the effectiveness of specific WIC nutrition services.
The results of the impact studies are severely limited by methodological
constraints, including the use of outdated and poor- quality data. We are
making a recommendation to USDA aimed at improving demonstration research by
incorporating relevant cost information. We provided a draft of this report
to USDA for its review and comment. In commenting on the draft report, USDA
generally agreed with the report and its recommendation. The agency provided
some technical comments,
which we incorporated as appropriate. Background WIC, which began as a 2-
year pilot program in 1972 and was authorized as a
permanent program in 1974, is part of the nutrition safety net available to
low- income women and their children. FNS provides annual cash grants for
food benefits and nutrition services to fund program operations at 88
statelevel WIC agencies (including agencies in all 50 states, the District
of Columbia, American Samoa, the Commonwealth of Puerto Rico, Guam, the U.
S. Virgin Islands, and 33 Indian Tribal Organizations). Some of these state-
level agencies- those that operate the program at both the state and local
levels- retain all of their federal WIC grants. Most state- level agencies,
however, retain a portion of their grants and pass the remaining
funds to over 1,800 local WIC agencies. In fiscal year 2000, about $2. 8
billion in federal program funds were used to provide food benefits to
participants. Typically food benefits are in the form of vouchers or checks
that participants can use to obtain approved foods at authorized retail food
stores. An additional $1.1 billion in federal funds were used for nutrition
services and program administration. Program administration includes, among
other things, activities related to accounting and record keeping, outreach,
monitoring and financial audits,
and general management. Nutrition services include activities related to
determining participants' eligibility and issuing food benefits, as well as
the following:
? Nutrition education: WIC offers classes, counseling, and other activities
to teach participants about proper nutrition, positive food habits, and the
prevention of nutrition- related problems. ? Breastfeeding promotion and
support: To promote breastfeeding, WIC
offers individual and group counseling sessions at WIC clinics or the
hospital. Breastfeeding support can include telephone or in- person
consultation with breastfeeding mothers.
? Referral to health care and social services: WIC agencies provide
participants with information on health care and social services and refer
them to providers including immunization clinics and the Food Stamp and
Medicaid programs.
By law, spending for nutrition education and breastfeeding promotion and
support activities combined must equal at least one- sixth of a state's
total annual expenditures for nutrition services and administration plus a
target amount for breastfeeding promotion and support that is established by
FNS
at the beginning of each fiscal year. 2 There is no minimum spending
requirement for referral activities.
Over the past 20 years, government agencies such as USDA, 3 the Centers for
Disease Control and Prevention, and GAO, 4 as well as universities and
private research organizations, have conducted a substantial body of
research on the effects of the entire WIC program. Some of the accumulated
body of WIC research and evaluations provides nationwide assessments of
WIC's effects. Most of it has focused on the effect of program participation
on birth outcomes and the nutritional status of program participants. USDA
has a review under way describing and assessing research on the diet and
health outcomes of its nutrition
programs, including WIC. The results of this review, set for release later 2
In fiscal year 2000, the state agencies' targets for breastfeeding promotion
and support were determined by multiplying $23. 92 by the average number of
pregnant and breastfeeding participants during the last 3 months for which
there were final data (MayJune 1999). 3 In 1998, the Agriculture, Rural
Development, Food and Drug Administration, and Related Agencies
Appropriations Act of 1998 (P. L. 105- 86) effectively transferred primary
responsibility for research funding on food assistance and nutrition
programs- including WIC- from FNS to USDA's Economic Research Service.
Previously, USDA had spent about
$3. 5 million annually on such research. 4 Breastfeeding: WIC's Efforts to
Promote Breastfeeding Have Increased (GAO/ HRD- 94- 13, Dec. 6, 1993); Early
Intervention: Federal Investments Like WIC Can Produce (GAO/ HRD- 9218, Apr.
7, 1992)
this year, will provide detailed information on over 70 studies, most of
which examine WIC's effects on birth outcomes or on the nutrition status of
participants. While the USDA review will not focus on the impacts of
specific nutrition services, it will include studies that examined the WIC
program's effects on the initiation and duration of breastfeeding and the
immunization status of children. These two health- related outcomes are
directly linked to two of the three nutrition services addressed in this
report- breastfeeding promotion and support and referral services.
However, because the USDA review is generally focused on overall program
impacts, its report probably will not include descriptions or assessments of
many of the demonstration studies included in this report.
Most of the Recent The 19 studies we identified included almost twice as
many demonstration Research Evaluates studies as impact studies. Of the 12
demonstration studies, 3 look at special interventions in nutrition
education, 6 look at special interventions in
Demonstrations of breastfeeding promotion and support, and 3 look at special
interventions in Special Interventions referral to health and social
services. Of the seven impact studies, one examines nutrition education,
four assess breastfeeding promotion and support services, and two evaluate
WIC health referrals. 5 Most of the studies have a relatively limited
geographic scope. Among the
12 demonstration studies, 11 are at the substate level. They generally study
multiple WIC sites and/ or multiple counties, but without sufficient
sampling rigor to draw valid statewide conclusions. The results of one
demonstration study are generalized to an entire state. Among the seven
impact studies, three are at the substate level, one is statewide, and three
are national in scope.
The 19 studies received funding from various sources. Table 1 provides
details on project funding for the 12 demonstration studies and 7 impact
studies. 5 One impact study- Fox and others- is primarily a nutrition
education evaluation, so we count it as such. However, it also contains
information on WIC breastfeeding programs and their effects on initiation
and duration.
Table 1: Funding for Demonstration and Impact Studies Number of
demonstration
Number of impact Source of funding
studies funded a studies funded a
USDA 1 2 U. S. Department of Health and Human
1 0 Services Centers for Disease Control and 5 0
Prevention State departments of health 4 1 Universities and research
institutes 1 2 Private organizations 1 1 Unspecified 1 3 a These columns do
not sum to 12 and 7, respectively, because some studies received funding
from multiple sources. Appendix II provides detailed information on the
funding sources for the 19
studies reviewed for this report. Special Interventions
While all 19 studies suffer from methodological limitations, those Improve
Participant
limitations have varying consequences. Despite their limitations, the
results of the 12 demonstration studies suggest that special interventions
have Outcomes but
some potential to improve nutrition service effectiveness over WIC Research
Says Little interventions typically used; though, based on our analysis of
these studies,
About the it appears that additional resources may have to be committed to
achieve this added effectiveness. However, the methodological limitations of
the
Effectiveness of seven impact studies enable them to provide only very
limited information
Individual Nutrition on the effects of any one nutrition service. Appendixes
III and IV contain
lists of the demonstration and impact studies, respectively, reviewed for
Services
this report. Demonstration Studies
The 12 demonstration studies evaluate a range of different special Indicate
Some Special interventions. To varying degrees, all were more effective than
usual WIC
Interventions Improve interventions. Examples of the special interventions
include the following: Participant Outcomes ? Breastfeeding promotion and
support. Gross and others evaluated
special interventions designed to encourage breastfeeding among African-
American WIC participants. 6 Mothers in the three special
intervention groups were provided a motivational video or peer counseling or
a combination of the video and counseling. Mothers in the control group
received the standard WIC service that incorporated
encouragement and support to breastfeed and brochures about breastfeeding
during discussions about infant feeding. Mothers in the special intervention
groups were twice as likely as mothers receiving the standard WIC infant
feeding education to be breastfeeding 8 weeks and 16 weeks after giving
birth, even accounting for factors that could
increase breastfeeding duration, such as prior breastfeeding experience.
? Health referrals. Birkhead and others evaluated two special interventions
designed to increase the number of WIC- eligible children who receive
measles immunizations. 7 The special interventions included having WIC staff
escort children to an on- site immunization clinic and a food voucher
incentive in which WIC staff provided only a 1- month
supply of vouchers to parents, rather than the usual 2- month supply, until
the parents provided documentation that their children's immunizations were
up- to- date. The standard WIC immunization referral consisted of notifying
parents that immunizations were due, providing information on the benefits
of immunizations, and providing the names and telephone numbers of local
health facilities that immunize children. Children at escort sites were
about five times more
likely to be immunized than children at standard referral sites; children at
voucher incentive sites were about three times more likely to receive
immunizations.
? Nutrition education. Havas and others evaluated a special intervention
designed to increase WIC participants' consumption of fruits and vegetables.
8 The special intervention- Maryland's “5- A- Day” program- was
a series of three 45- minute group sessions taught by former WIC
participants, or “peer educators,” that incorporated special
visual materials and included direct mailings to participants. The standard
6 Susan M. Gross, Laura E. Caulfield, Margaret E. Bentley, and others,
“Counseling and Motivational Videotapes Increase Duration of
Breastfeeding in African- American WIC Participants Who Initiate
Breastfeeding,” Journal of the American Dietetic Association, Vol. 98,
No. 2 (1998), p. 143.
7 Guthrie S. Birkhead, Charles W. LeBaron, Patricia Parsons, and others,
“The Immunization of Children Enrolled in the Special Supplemental
Food Program for Women, Infants, and Children (WIC),” Journal of the
American Medical Association, Vol. 274, No. 4 (1995), p. 312. 8 Steven
Havas, Jean Anliker, Dorothy Damron, and others, “Final Results of the
Maryland WIC 5- A- Day Promotion Program,” American Journal of Public
Health, Vol. 88, No. 8 (1998), p. 1161.
service generally included less than 10 minutes of nutrition education
conversation between WIC staff and participants when they picked up their
food voucher every other month. Compared to participants receiving the
standard WIC nutrition education program, participants exposed to the
special intervention displayed a significant increase in nutrition knowledge
and in the consumption of fruits and vegetables. Each of the demonstration
studies we reviewed suffers from
methodological limitations that, while not invalidating the study's
findings, should be taken into account. The limitations we identified are
common in studies that attempt to assess the extent to which social or
health program
interventions- not other factors- are responsible for changes in program
participants' behaviors or health. The major methodological limitations of
the demonstration studies we reviewed include the following:
? Lack of control group. To help isolate the effects of an intervention, an
evaluation study must compare people receiving the special intervention to
similar people receiving standard WIC services. The difference between these
groups can provide insight into whether the
special intervention is more effective than standard WIC practice. Not
having such a comparison obscures the relationship between the intervention
and participant outcomes. Four of the studies we reviewed
had a weak research design associated with a lack of control group. For
example, Hoekstra and others attempted to evaluate the effectiveness of a
new voucher incentive program in increasing WIC children's rates of
immunization. 9 However, the researchers did not compare the group receiving
the special intervention to a group receiving standard services. Instead,
they compared three special intervention groups to themselves at different
points in time over a period of 14 months. Without a control
group that does not participate in the voucher intervention, it is difficult
to attribute any changes the researchers noticed to the special
intervention. ? Inappropriate data analysis techniques. The analytic
techniques used in a study must suit the available data and the research
design- in particular, they should be selected for their ability to help
isolate the
effects of the intervention. Three of the demonstration studies we 9 Edward
J. Hoekstra, Charles W. LeBaron, Yannis Megaloeconomou, and others,
“Impact of a Large- Scale Immunization Initiative in the Special
Supplemental Nutrition Program for Women, Infants, and Children (WIC),
” Journal of the American Medical Association, Vol. 280, No. 13
(1998), p. 1143.
reviewed used questionable analytic techniques. For example, Havas and
others found that a special peer counselor program was effective in
increasing nutrition knowledge and the consumption of fruits and vegetables.
To reach this finding, Havas and others compared the average fruit and
vegetable consumption of the special “5- a- Day” group to the
average fruit and vegetable consumption of a group exposed to the standard
WIC nutrition education. The comparison, which examined
the linkage between demographic characteristics, such as race, and fruit and
vegetable consumption, did not take into account the simultaneous influence
of other characteristics, such as education level, on consumption. Without
an analysis technique, such as multiple regression, that can account for the
influence of several factors at once,
determining the extent to which the observed differences in fruit and
vegetable consumption are the result of the “5- A- Day” program
is greatly complicated.
? Selection bias. Ideally, study participants should be randomly assigned to
intervention and control groups to ensure that all participant
characteristics will, on average, be the same from one group to another. A
selection bias exists if the two groups differ in some systematic way.
Selection bias makes it more difficult to attribute an observed difference
in outcomes between the two groups to any one factor, such as the
intervention. Six of the demonstration studies we reviewed have a possible
selection bias. For example, Tuttle and Dewey examined the influence of a
new, culturally sensitive breastfeeding education intervention on the
initiation and duration of breastfeeding among
Hmong WIC participants in Northern California. 10, 11 However, the study's
research design depended on women to volunteer to participate in the study-
the women self- selected themselves to participate in the special
intervention. Thus, those choosing to participate in the special
intervention may have shared characteristics (for example, an already
existing inclination to breastfeed) that did not exist in those women who
chose not to participate. If present, selection bias could lead the
researcher to overstate the benefits of the special intervention.
? Low response rate/ missing data. To substantially reduce data reliability
concerns, experts agree that under most circumstances researchers need at
least 75 percent of the people they ask to participate in a study 10 Cynthia
Reeves Tuttle and Kathryn G. Dewey, “Impact of a Breastfeeding
Promotion Program for Hmong Women at Selected WIC Sites in Northern
California,” Journal of Nutrition Education, Vol. 27, No. 2 (1995), p.
69.
11 Most Hmong living in the United States today came from Laos.
to agree. Excessive missing data, or poor response rates, may skew research
findings. Missing data or poor response rate was a limitation in five of the
demonstration studies. For example, Ahluwalia and others evaluated five new
breastfeeding interventions, and attributed
significant improvements in breastfeeding initiation to them. 12 However,
the database employed by the study contained breastfeeding initiation data
for only 52 percent of the women in the sample. ? Measurement error. For an
analysis to produce reliable results, the
measures in the analysis must be accurate. Measurement error is the
difference between a measured value and its true value. Five of the studies
have potential measurement errors. For example, Shaw and Kaczorowski sought
to examine the effectiveness of a peer counseling program on breastfeeding
initiation and duration by asking new mothers
to recall interactions with breastfeeding peer counselors that took place at
the time of birth. 13 Mothers were interviewed 6 weeks to 6 months after
giving birth. If memory lapses occurred, new mothers may have
incorrectly recalled their dealings with peer counselors, thereby
potentially introducing measurement error into the data.
Appendix V shows the major findings and methodological limitations of each
of the 12 demonstration studies.
Our analysis suggests that the effective interventions described in the
demonstration studies may cost more than standard WIC approaches. For
example, most breastfeeding special interventions were specifically
designed to increase the amount of counseling and support provided to
prenatal and postpartum women. Although only one of the demonstration
studies provided information about the additional costs associated with such
interventions, it is reasonable to expect that such one- on- one support
will cost more than the standard WIC program. Two of the demonstration
studies help to illustrate the linkage between resource commitment and
results achieved. The first, Ahluwalia and others, which found that a
hospital- based strategy providing bedside 12 Indu B. Ahluwalia, Irene
Tessaro, Laurence M. Grummer- Strawn, and others, “Georgia's
Breastfeeding Promotion Program for Low- Income Women,” Pediatrics,
Vol. 105, No. 6 (2000), p. 85.
13 Elizabeth Shaw and Janusz Kaczorowski, “The Effect of a Peer
Counseling Program on Breastfeeding Initiation and Longevity in a Low-
Income Rural Population,” Journal of Human Lactation, Vol. 15, No. 1
(1999), p. 19.
support and counseling to women who had just given birth, was the most
effective at increasing breastfeeding initiation rates out of five new
strategies evaluated. This strategy clearly required more resources than the
standard practice of providing counseling and brochures to participants
during a visit to the WIC clinic. The second study, Weimer, which was funded
by USDA, focused on a special intervention that was similarly
resource- intensive. It reported that providing one- on- one support in the
hospital after delivery, followed by an in- home visit within 72 hours of
birth, increased breastfeeding duration. However, neither study provided
any information about the additional costs needed to implement these special
interventions. Only one of the 12 demonstration studies we reviewed,
Hutchins and others, provided any information about the costs associated
with the implementation of the special intervention. 14 This study reported
that vaccinations increased at sites providing vaccination screening and
voucher incentives (until their children are immunized, a family must visit
the clinic monthly- rather than every 3 months- to pick up WIC vouchers).
This study uses what its authors term “crude” cost-
effectiveness ratios to estimate the average cost for each additional child
with up- to- date immunizations. These costs range between $30 and $73,
depending on the number of enrolled children and rates of active
participation. Impact Research Provides The seven impact research studies we
reviewed provide few conclusive Very Limited Information on
insights into the recent effectiveness of WIC breastfeeding promotion and
the Effectiveness of WIC's
support, referral services, or nutrition education services. Individual
Nutrition Services
14 Sonja S. Hutchins, Jorge Rosenthal, Pamela Eason, and others,
“Effectiveness and CostEffectiveness of Linking the Special
Supplemental Program for Women, Infants, and Children (WIC) and Immunization
Activities,” Journal of Public Health Policy, Vol. 20, No. 4 (1999),
p. 408.
Breastfeeding Promotion Three of the four impact studies that focused on
breastfeeding promotion
and Support and support- Schwartz and others, 15 Balcazar and others, 16 and
Timbo and
others 17 -use old data from the 1988 National Maternal and Infant Health
Survey. Although, according to FNS officials, this survey represents the
most recent data available, much has changed in the program since 1988,
including the characteristics of WIC participants and the emphasis the
program places on breastfeeding. As a result, these studies' findings shed
little light on the program's current effects. Although the fourth study,
Wiemann and others, 18 uses data from the mid- 1990s, its limited scope, in
terms of geography and participants, constrains the applicability of its
findings. This study, with data collected from 684 adolescent mothers who
gave birth at a hospital in Galveston, Texas, could have some specialized
usefulness, but would have to be replicated at many other sites to provide
insights into the broader effectiveness of WIC's breastfeeding promotion and
support services. In addition, since adolescent mothers comprise only about
11 percent of all WIC mothers, the study's focus on them further compromises
its more general usefulness. 19 Taken as whole, the inconsistency in the
findings of these four studies
further limit their usefulness in assessing the effects of WIC's
breastfeeding promotion and support program. For example, Wiemann and others
and Balcazar and others find that WIC enrollment is a significant factor in
some mothers' decision to bottle- feed, while Timbo and others, and Schwartz
and
15 J. Brad Schwartz, Barry M. Popkin, Janet Tognetti, and others,
“Does WIC Participation Improve Breastfeeding Practices,”
American Journal of Public Health, Vol. 85, No. 5 (1995), p. 729.
16 Hector Balcazar, Catherine M. Trier, and Jose A. Cobas, “What
Predicts Breastfeeding Intention in Mexican- American and Non- Hispanic
White Women? Evidence from a National Survey,” Birth, Vol. 22, No. 2
(1995), p. 74. 17 Babgaleh Timbo, Sean Altekruse, Marcia Headrick, and
others, “Breastfeeding Among Black Mothers: Evidence Supporting the
Need for Prenatal Intervention,” Journal of the Society of Pediatric
Nurses, Vol. 1, No. 1 (1996), p. 35.
18 Constance M. Wiemann, Jacqueline C. DuBois, and Abbey B. Berenson,
“Racial/ Ethnic Differences in the Decision to Breastfeed Among
Adolescent Mothers,” Pediatrics, Vol. 101, No. 6 (1998), p. 11.
19 Susan Barlett, Melanie Brown- Lyons, Douglas Moore, and others, WIC
Participant and Program Characteristics 1998 (Alexandria, Va.: USDA, Food
and Nutrition Service, 2000), p. 38.
others, conclude that WIC participation increases breastfeeding. No
consistent message emerges from the studies.
Referral Services The two referral service impact studies have
methodological constraints that, to varying degrees, limit their usefulness
in assessing the effectiveness of WIC referral services. The first study,
Suarez and others, using survey data from 30 counties in Texas, found that
children who are enrolled in WIC are significantly more likely than children
who are not enrolled to be up- to- date on their immunizations, regardless
of other intervening factors such as the child's age, ethnicity, or the
family's income. 20 Although this study likely contains some measurement
error, it provides at least limited evidence that WIC referral services are
effective in increasing
immunization rates. 21 In contrast, due to major methodological problems,
the second referral study, McCunniff and others, 22 provides little useful
information on the effectiveness of WIC referral services. This study is
based on self- administered questionnaire data collected from a sample of
mothers at three WIC sites in Kansas City, Missouri. The study found that
when taking into account factors such as WIC referral, child's age,
household size, and availability of dental insurance, only the age of the
child had a significant, independent effect on the likelihood that children
will visit a dentist. There are two principal limitations to this study.
First, almost 40 percent of the sampled children were younger than 1 year
old.
Because many children less than 1 year of age do not yet have teeth, they
are much less likely to have made a dental visit, thus reducing the study's
ability to identify factors associated with dental visits other than age.
The second major limitation is the measurement error associated with the
reliance on self- reported questionnaire data about visits to the dentist.
The study did not attempt to verify questionnaire responses through a review
of dental records. The authors suggested that such reviews would have
increased the accuracy of the self- reported data. As a result of these
serious methodological problems, it is likely that McCunniff and others has
20 Lucina Suarez, Diane M. Simpson, and David R. Smith, “The Impact of
Public Assistance Factors on the Immunization Levels of Children Younger
Than 2 Years,” American Journal of Public Health, Vol. 87, No. 5
(1997), p. 845.
21 Measurement error is likely in Suarez and others because the immunization
information provided by 28 percent of those interviewed was based on
parents' recollections of their children's immunizations, rather than
medical records or other documentation. 22 Michael D. McCunniff, Peter C.
Damiano, Michael J. Kanellis, and others, “The Impact of WIC Dental
Screenings and Referrals on Utilization of Dental Services Among Low- Income
Children,” Pediatric Dentistry, Vol. 20, No. 3 (1998), p. 181.
only limited relevance to understanding the effectiveness of WIC referral
services. Nutrition Education The one study that primarily focused on the
impact of nutrition education,
Fox and others, also examined breastfeeding programs and their
effectiveness. 23 However, Fox and others was limited geographically, and
had other limitations that reduce its usefulness in assessing the
effectiveness of WIC's nutrition education. For example, its scope was
limited in that it focused on pregnant and postpartum women at six WIC
sites, in three states. Within this limited context, the study describes
program and participant characteristics; the nutrition services offered
(including breastfeeding promotion and support); participants' receipt of
and satisfaction with these services; and changes in participants' knowledge
and behaviors between the time of prenatal WIC certification
and 4 to 6 months postpartum. The study also attempted to assess the impact
of WIC nutrition education on participants' knowledge and behavior. Although
the study concluded that participants' nutrition knowledge and behavior
improved significantly over the course of the study, attributing these
changes to WIC is problematic because the study did not use a control group.
Instead, Fox and others compared intervention groups at different points in
time. The study also concluded that (1) WIC participation did not
significantly increase breastfeeding initiation or duration; and (2) women's
decisions regarding infant feeding are strongly associated with intentions
formed during pregnancy.
Conclusion Demonstration studies, despite some methodological limitations,
provide program managers and policymakers with some useful information about
the types of WIC nutrition service interventions that can have positive
impacts on participants. However, only one recent demonstration study
provides any information on the costs associated with implementing various
interventions. Given the limited resources available to provide WIC
nutrition services, information about the costs to provide effective
services could play a critical role in managers' decisions to implement the
intervention and policymakers' decisions about funding the intervention.
23 Mary Kay Fox, Nancy Burstein, Jenny Golay, and others, WIC Nutrition
Education Assessment Study: Final Report (Alexandria, Va.: U. S. Department
of Agriculture, Food and Nutrition Service, 1999), p. vii.
Recommendation In order to maximize the value of nutrition education,
breastfeeding promotion and support, and referral service demonstration and
evaluation research funded by USDA, we recommend that the Secretary of
Agriculture direct officials responsible for implementing such research to
require that this research include an assessment of the costs associated
with the special
intervention being evaluated. Agency Comments and
We provided a draft of this report to the Department of Agriculture's Food
Our Response
and Nutrition Service for review and comment. We met with Food and Nutrition
Service officials, including the Acting Associate Deputy Administrator for
Special Nutrition Programs. The agency officials generally agreed with the
report's findings and recommendation. However,
the officials questioned why our recommendation did not address actions that
WIC researchers should take to deal with some of the methodological
limitations we identified in research evaluating the effectiveness of WIC
services. We believe that USDA has a responsibility to ensure that the WIC
and other nutrition program research it funds are of high quality. However,
our review was not designed to examine USDA's policies and procedures to
ensure the quality of the research it funds or the practices it employs to
promote high- quality research in studies it does not fund. As a result, we
are not making any specific recommendations concerning how USDA might
improve the quality of WIC research at this time. The officials also
provided some technical changes and clarifications to the report, which we
incorporated as appropriate.
We are sending copies of this report to the appropriate congressional
committees; interested Members of the Congress; the Honorable Ann M.
Veneman, Secretary of Agriculture; the Honorable Mitchell E. Daniels, Jr.,
Director of the Office of Management and Budget; and other interested
parties. We will also make copies available upon request.
If you or your staff have any questions about this report, please contact me
or Thomas E. Slomba at (202) 512- 7215. Key contributors to this report are
listed in appendix VI.
Robert E. Robertson Director, Employment, Workforce, and Income Security
Issues
Appendi Appendi xes x I
Scope and Methodology To identify recent studies that examine the
effectiveness of the Special Supplemental Nutrition Program for Women,
Infants and Children (WIC) nutrition education, breastfeeding promotion and
support, and referral services, we searched relevant databases, such as
National Technical Information Service, Sociological Abstracts, and Wilson
Social Science Abstracts. We also consulted with the U. S. Department of
Agriculture (USDA) WIC program staff and other program stakeholders,
including officials from the National Association of WIC Directors. Through
this process, we identified 209 items published from 1988 through 2000
dealing with various aspects of the WIC program. To be used in our review,
individual items had to meet each of the following criteria:
? publication in a refereed medium (for example, a journal article, book or
book chapter, USDA- issued report); ? publication date of 1995 or later; ?
examination of one or more of WIC's nutrition services (breastfeeding
promotion and support, nutrition education, or health referrals); and
? original analysis of a specific nutrition service's effectiveness.
Altogether, only 19 items met all four criteria. Many- 86 of the 190 items
we rejected- were published prior to 1995, and therefore do not satisfy our
definition of recent studies. (We established 1995 as the cutoff to enable
us to better examine the program as it currently operates.) We eliminated
the remaining 104 items because they did not meet one or more of our
criteria. For example, some items appeared in our literature search as
professional papers delivered at conferences; thus, they did not undergo any
formal referee process. Other items were published as reviews or summaries
of
original research, but did not include any original research of their own.
Some items do not focus on the effectiveness of specific WIC nutrition
services. For example, one study examines the general effects of food
programs- including WIC and other food assistance programs such as food
stamps- on diet, but does not evaluate the effectiveness of specific WIC
nutrition service programs. Once we narrowed the scope of our study, we met
with staff in the USDA's Food and Nutrition Service Office of Analysis,
Nutrition and Evaluation to ensure that our methodology did not exclude any
important studies.
According to these officials, our approach successfully identified all of
the major recent evaluation studies on WIC nutrition services. We then
conducted detailed reviews of the 19 studies. These reviews entailed an
evaluation of each study's research methodology, including its
data quality, research design, and analytic techniques, as well as a summary
of its major findings and conclusions. We also assessed the extent to which
each study's data and methods support its findings and conclusions.
Appendi x II
Research Funding Sources Study Funding source( s) Demonstration studies
Ahluwalia and others, 2000 Centers for Disease Control and Prevention
Hutchins and others, 1999 Centers for Disease Control and Prevention Shaw
and Kaczorowski, 1999 Unspecified Abusabha and others, 1998 New Mexico
Department of Health Gross and others, 1998 U. S. Department of Health and
Human Services Havas and others, 1998 National Cancer Institute; Maryland
Department of Health and Mental Hygiene
Hoekstra and others, 1998 Centers for Disease Control and Prevention Weimer,
1998 USDA/ ERS Grummer- Strawn and others, 1997 Centers for Disease Control
and Prevention Reifsnider and Eckhart, 1997 Beta Delta Chapter at Large;
Sigma Theta Tau Birkhead and others, 1995 Centers for Disease Control and
Prevention Tuttle and Dewey, 1995 California State Department of Health
Impact studies
Fox and others, 1999 USDA/ FNS McCunniff and others, 1998 Unspecified
Wiemann and others, 1998 National Institute on Drug Abuse;
Hogg Foundation for Mental Health Suarez and others, 1997 Texas Department
of Health; Public Policy Research Institute, Texas A& M University
Timbo and others, 1996 Unspecified Balcazar and others, 1995 Unspecified
Schwartz and others, 1995 USDA/ FNS; Research Triangle Institute, University
of North Carolina
Appendi x II I Bibliography of Demonstration Studies Breastfeeding
Ahluwalia, Indu B., Irene Tessaro, Laurence M. Grummer- Strawn, and
Promotion and Support
others. “Georgia's Breastfeeding Promotion Program for Low- Income
Women. ” Pediatrics, Vol. 105, No. 6 (2000), pp. 85– 91.
Shaw, Elizabeth, and Janusz Kaczorowski. “The Effect of a Peer
Counseling Program on Breastfeeding Initiation and Longevity in a Low-
Income Rural Population.” Journal of Human Lactation, Vol. 15, No. 1
(1999), pp. 19– 25.
Gross, Susan M., Laura E. Caulfield, Margaret E. Bentley, and others.
“Counseling and Motivational Videotapes Increase Duration of
BreastFeeding in African- American WIC Participants Who Initiate
BreastFeeding.” Journal of the American Dietetic Association, Vol. 98,
No. 2 (1998), pp. 143– 148.
Weimer, Jon P. Breastfeeding Promotion Research: The ES/ WIC Nutrition
Education Initiative and Economic Considerations. Washington, D. C.: U. S.
Department of Agriculture, Economic Research Service, 1998.
Grummer- Strawn, Laurence M., Susan P. Rice, Kathy Dugas, and others.
“An
Evaluation of Breastfeeding Promotion Through Peer Counseling in Mississippi
WIC Clinics.” Maternal and Child Health Journal , Vol. 1, No. 1
(1997), pp. 35– 42. Reifsnider, Elizabeth, and Donna Eckhart.
“Prenatal Breastfeeding Education: Its Effect on Breastfeeding Among
WIC Participants.” Journal of Human Lactation, Vol. 13, No. 2 (1997),
pp. 121– 125.
Tuttle, Cynthia Reeves, and Kathryn G. Dewey. “Impact of a
Breastfeeding Promotion Program for Hmong Women at Selected WIC Sites in
Northern California.” Journal of Nutrition Education, Vol. 27, No. 2
(1995), pp. 69– 74.
Nutrition Education Abusabha, Rayane, Cheryl Achterberg, and Jeannie
McKenzie. “Evaluation
of Nutrition Education in WIC.” Journal of Family and Consumer
Sciences, Winter (1998), pp. 98– 104.
Havas, Stephen, Jean Anliker, Dorothy Damron, and others. “Final
Results of the Maryland WIC 5- A- Day Promotion Program.” American
Journal of Public Health, Vol. 88, No. 8 (1998), pp. 1161– 1167.
Referrals Hutchins, Sonja S., Jorge Rosenthal, Pamela Eason, and others.
“Effectiveness and Cost- Effectiveness of Linking the Special
Supplemental Program for Women, Infants, and Children (WIC) and Immunization
Activities.” Journal of Public Health Policy, Vol. 20, No. 4 (1999),
pp. 408– 426. Hoekstra, Edward J., Charles W. LeBaron, Yannis
Megaloeconomou, and others. “Impact of a Large- Scale Immunization
Initiative in the Special Supplemental Nutrition Program for Women, Infants,
and Children (WIC).” Journal of the American Medical Association, Vol.
280, No. 13 (1998), pp.
1143– 1147. Birkhead, Guthrie S., Charles W. LeBaron, Patricia
Parsons, and others. “The Immunization of Children Enrolled in the
Special Supplemental Food Program for Women, Infants, and Children
(WIC).” Journal of the American Medical Association, Vol. 274, No. 4
(1995), pp. 312– 316.
Appendi x V I Bibliography of Impact Studies Breastfeeding Fox, Mary Kay,
Nancy Burstein, Jenny Golay, and others. WIC Nutrition Promotion and Support
Education Assessment Study: Final Report. Alexandria, Va.: U. S. Department
of Agriculture, Food and Nutrition Service, 1999.
Wiemann, Constance M., Jacqueline C. DuBois, and Abbey B. Berenson.
“Racial/ Ethnic Differences in the Decision to Breastfeed Among
Adolescent Mothers.” Pediatrics, Vol. 101, No. 6 (1998), pp. 11–
23.
Timbo, Babgaleh, Sean Altekruse, Marcia Headrick, and others.
“Breastfeeding Among Black Mothers: Evidence Supporting the Need for
Prenatal Intervention.” Journal of the Society of Pediatric Nurses,
Vol. 1, No. 1 (1996), pp. 35– 46. Balcazar, Hector, Catherine M.
Trier, and Jose A. Cobas. “What Predicts Breastfeeding Intention in
Mexican- American and Non- Hispanic White Women? Evidence From a National
Survey.” Birth, Vol. 22, No. 2 (1995), pp. 74– 80. Schwartz, J.
Brad, Barry M. Popkin, Janet Tognetti, and others. “Does WIC
Participation Improve Breast- Feeding Practices?” American Journal of
Public Health, Vol. 85, No. 5 (1995), pp. 729– 731.
Referrals Fox, Mary Kay, Nancy Burstein, Jenny Golay, and others. WIC
Nutrition Education Assessment Study: Final Report. Alexandria, Va.: U. S.
Department of Agriculture, Food and Nutrition Service, 1999.
McCunniff, Michael D., Peter C. Damiano, Michael J. Kanellis, and others.
“The Impact of WIC Dental Screenings and Referrals on Utilization of
Dental Services Among Low- Income Children.” Pediatric Dentistry, Vol.
20, No. 3 (1998), pp. 181– 187. Suarez, Lucina, Diane M. Simpson, and
David R. Smith. “The Impact of Public Assistance Factors on the
Immunization Levels of Children Younger Than 2 Years.” American
Journal of Public Health, Vol. 87, No. 5 (1997), pp.
845– 848.
Demonstration Studies: Major Findings,
Appendi x V
Scope, and Major Limitations Study Major finding( s) Scope (timeframe) Major
limitation( s) Nutrition education
Abusabha and others, Nutrition education lectures and facilitated group
Seven WIC clinics in Missing data; 1998 discussions were more effective than
brochures at New Mexico (not selection bias increasing participants'
nutrition knowledge. specified)
Facilitated group discussion was more effective than brochures at increasing
participants' confidence in performing specific nutrition related behaviors.
Havas and others, 1998 Consumption of fruits and vegetables increased
Sixteen WIC clinics in Missing data; measurement after an education program
consisting of a series of Maryland (not
error; inappropriate data three 45- minute group sessions taught by paid
peer
specified) analysis techniques
educators and incorporating special visual materials and a direct mailing to
participants.
Breastfeeding
Gross and others, 1998 The duration of breastfeeding among African Four WIC
sites in Selection bias;
American WIC participants increased with peer Baltimore, Md. (1992–
measurement error counselor support or viewing promotional
1994) breastfeeding videos. Ahluwalia and others,
WIC participants increased the initiation of State of Georgia
Missing data 2000 breastfeeding when exposed to (1) an enhanced
(1992– 1996) education program with access to a hotline, (2) a free
breast- pump loan program, (3) a hospitalbased program with bedside support
and counseling after delivery, (4) community coalitions, and (5) peer
counseling provided by former
participants. Weimer, 1998 1. Breastfeeding initiation and duration rates 1.
Two communities 1. Missing data; selection bias
increased after volunteer peer counseling. in Iowa (not 2. Lack of control
group;
2. Breastfeeding initiation and duration rates specified) missing data
increased after paid peer counseling.
2. Six counties in 3. Lack of control group 3. Duration of breastfeeding
increased after
Michigan (not 4. Selection bias postdelivery contact with mother in hospital
specified) was followed up with support (including home
3. Five counties in visits) by a specially trained paraprofessional. North
Carolina 4. Breastfeeding initiation and duration rates (not specified)
increased with culturally appropriate 4. Local agency in breastfeeding
education provided in high Guam (not school or WIC clinics.
specified) Reifsnider and Eckhart,
Duration of breastfeeding increased after prenatal WIC clinics in three Lack
of control group
1997 nutrition education classes focusing on rural Oklahoma breastfeeding.
counties (1986)
Tuttle and Dewey, 1995 Breastfeeding initiation rates increased among Seven
WIC clinics in Selection bias; Hmong WIC participants after a culturally
sensitive
three California measurement error prenatal breastfeeding class and prenatal
and counties (1991– 1992) postpartum counseling.
(Continued From Previous Page)
Study Major finding( s) Scope (timeframe) Major limitation( s)
Shaw and Kaczorowski, Breastfeeding initiation and duration increased after
WIC programs in nine Measurement error
1999 counseling and support provided by WIC West Tennessee participants,
trained and paid as peer counselors. health departments (1996– 1997)
Grummer- Strawn and Clinics with paid peer counselors had higher rates
Fifty- one WIC clinics in Selection bias; others, 1997 of breastfeeding
initiation than clinics without peer
Mississippi (1989– missing data
counselors. Clinics with a lactation specialist or 1993) consultant, and
peer counselors, had higher rates of breastfeeding initiation than clinics
with only peer counselors. However, the benefits of lactation specialists
were offset when peer counselors spent at least 45 minutes with individual
participants.
Referrals
Birkhead and Children were 5.5 times more likely to be Six WIC sites in New
Lack of control group
others, immunized, and immunized more rapidly, at WIC York City (1991) 1998
sites where staff escorted children to a pediatric clinic in the same
facility for immunization. Children were almost 3 times more likely to be
immunized, and immunized more rapidly, at sites with a
voucher/ check incentive. (Until immunization, a family must visit the
clinic monthly, rather than every other month, to pick up WIC voucher/
checks.) Hoekstra and others, Immunization rates increased at sites with a
Nineteen WIC sites in Lack of control group;
1998 voucher/ check incentive (until children are Chicago (1996– 1997)
measurement
immunized, a family must visit the clinic monthly, error; inappropriate data
rather than every 3 months, to pick up WIC analysis voucher/ checks).
techniques Hutchins and others,
Vaccinations increased at sites with vaccination Seven WIC sites in
Inappropriate data analysis 1999 screening and a voucher/ check incentive
(until Chicago (1991– 1993) techniques; selection bias
immunization, a family must visit the clinic monthly, rather than every 3
months, to pick up WIC voucher/ checks).
Appendi x VI
GAO Contacts and Staff Acknowledgments GAO Contacts Robert E. Robertson,
(202) 512- 7215 Thomas E. Slomba, (202) 512- 9910 Staff
In addition to those named above, Judy Hoovler, Sara Ann Moessbauer,
Acknowledgments
Corrina Nicolaou, Judy Pagano, Debra Roush, and Eugene Wisnoski made key
contributions to this report.
(150182) Lett er
Ordering Information The first copy of each GAO report is free. Additional
copies of reports are $2 each. A check or money order should be made out to
the Superintendent of Documents. VISA and MasterCard credit cards are
accepted, also. Orders for 100 or more copies to be mailed to a single
address are discounted 25 percent.
Orders by mail: U. S. General Accounting Office P. O. Box 37050 Washington,
DC 20013
Orders by visiting: Room 1100 700 4th St. NW (corner of 4th and G Sts. NW)
U. S. General Accounting Office Washington, DC
Orders by phone: (202) 512- 6000 fax: (202) 512- 6061 TDD (202) 512- 2537
Each day, GAO issues a list of newly available reports and testimony. To
receive facsimile copies of the daily list or any list from the past 30
days, please call (202) 512- 6000 using a touchtone
phone. A recorded menu will provide information on how to obtain these
lists.
Orders by Internet: For information on how to access GAO reports on the
Internet, send an e- mail message with “info” in the body to:
info@ www. gao. gov or visit GAO's World Wide Web home page at: http:// www.
gao. gov
To Report Fraud, Contact one:
Waste, or Abuse in ? Web site: http:// www. gao. gov/ fraudnet/ fraudnet.
htm
Federal Programs
? e- mail: fraudnet@ gao. gov ? 1- 800- 424- 5454 (automated answering
system)
GAO United States General Accounting Office
Page 1 GAO- 01- 442 Research on WIC Nutrition Services
Page 2 GAO- 01- 442 Research on WIC Nutrition Services
Page 3 GAO- 01- 442 Research on WIC Nutrition Services United States General
Accounting Office
Washington, D. C. 20548 Page 3 GAO- 01- 442 Research on WIC Nutrition
Services
Page 4 GAO- 01- 442 Research on WIC Nutrition Services
Page 5 GAO- 01- 442 Research on WIC Nutrition Services
Page 6 GAO- 01- 442 Research on WIC Nutrition Services
Page 7 GAO- 01- 442 Research on WIC Nutrition Services
Page 8 GAO- 01- 442 Research on WIC Nutrition Services
Page 9 GAO- 01- 442 Research on WIC Nutrition Services
Page 10 GAO- 01- 442 Research on WIC Nutrition Services
Page 11 GAO- 01- 442 Research on WIC Nutrition Services
Page 12 GAO- 01- 442 Research on WIC Nutrition Services
Page 13 GAO- 01- 442 Research on WIC Nutrition Services
Page 14 GAO- 01- 442 Research on WIC Nutrition Services
Page 15 GAO- 01- 442 Research on WIC Nutrition Services
Page 16 GAO- 01- 442 Research on WIC Nutrition Services
Page 17 GAO- 01- 442 Research on WIC Nutrition Services
Page 18 GAO- 01- 442 Research on WIC Nutrition Services
Page 19 GAO- 01- 442 Research on WIC Nutrition Services
Page 20 GAO- 01- 442 Research on WIC Nutrition Services
Appendix I
Appendix I Scope and Methodology
Page 21 GAO- 01- 442 Research on WIC Nutrition Services
Page 22 GAO- 01- 442 Research on WIC Nutrition Services
Appendix II
Page 23 GAO- 01- 442 Research on WIC Nutrition Services
Appendix III
Appendix III Bibliography of Demonstration Studies
Page 24 GAO- 01- 442 Research on WIC Nutrition Services
Page 25 GAO- 01- 442 Research on WIC Nutrition Services
Appendix IV
Page 26 GAO- 01- 442 Research on WIC Nutrition Services
Appendix V
Appendix V Demonstration Studies: Major Findings, Scope, and Major
Limitations
Page 27 GAO- 01- 442 Research on WIC Nutrition Services
Page 28 GAO- 01- 442 Research on WIC Nutrition Services
Appendix VI
Appendix VI GAO Contacts and Staff Acknowledgments
Page 30 GAO- 01- 442 Research on WIC Nutrition Services
United States General Accounting Office Washington, D. C. 20548- 0001
Official Business Penalty for Private Use $300
Address Correction Requested Presorted Standard
Postage & Fees Paid GAO Permit No. GI00
*** End of document ***